ASTM E2855-12(2021)

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E2855 − 12 (Reapproved 2021)
Standard Test Method for
Evaluating Emergency Response Robot Capabilities: Radio
Communication: Non-Line-of-Sight Range
This standard is issued under the fixed designation E2855; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope testing robot in a non-line-of-sight environment. These links
include the command and control channel(s) and video, audio,
1.1 Purpose:
and other sensor data telemetry.
1.1.1 The purpose of this test method, as a part of a suite of
radiocommunicationtestmethods,istoquantitativelyevaluate 1.1.6 This test method is intended to apply to ground based
a teleoperated robot’s (see Terminology E2521) capability to robotic systems and small unmanned aerial systems (sUAS)
perform maneuvering and inspection tasks in a non-line-of-
capable of hovering to perform maneuvering and inspection
sight environment.
tasks down range for emergency response applications.
1.1.2 Robots shall possess a certain set of radio communi-
1.1.7 This test method specifies an apparatus that is, first of
cation capabilities, including performing maneuvering and
all, an essentially clear radio frequency channel for testing. In
inspection tasks in a non-line-of-sight environment, to suit
addition, a standard line-of-sight barrier between the testing
critical operations for emergency responses. The capability for
operator control unit (OCU) and the robot is specified. Fig. 1
arobottoperformthesetypesoftasksinobstructedareasdown
provides an illustration.
range is critical for emergency response operations. This test
method specifies a standard set of apparatuses, procedures, and
NOTE1—Frequencycoordinationandinteroperabilityarenotaddressed
metrics to evaluate the robot/operator capabilities for perform- in this standard. These issues should be resolved by the affected agencies
(Fire, Police, and Urban Search and Rescue) and written into Standard
ing these tasks.
Operating Procedures (SOPs) that guide the responses to emergency
1.1.3 Emergency response robots shall be able to operate
situations.
remotely using the equipped radios in line-of-sight
environments, in non-line-of-sight environments, and for sig-
1.1.8 The radio communication test suite quantifies elemen-
nal penetration through such impediments as buildings,
tal radio communication capabilities necessary for robots
rubbles, and tunnels. Additional capabilities include operating
intended for emergency response applications. As such, based
in the presence of electromagnetic interference and providing
on their particular capability requirements, users of this test
link security and data logging. Standard test methods are
suite can select only the applicable test methods and can
required to evaluate whether candidate robots meet these
individually weight particular test methods or particular met-
requirements.
rics within a test method. The testing results should collec-
1.1.4 ASTM E54.08.01 Task Group on Robotics specifies a
tively represent an emergency response robot’s overall radio
radio communication test suite, which consists of a set of test
communication capability. These test results can be used to
methods for evaluating these communication capabilities. This
guide procurement specifications and acceptance testing for
non-line-of-sight range test method is a part of the radio
robots intended for emergency response applications.
communication test suite. The apparatuses associated with the
test methods challenge specific robot capabilities in repeatable
NOTE 2—As robotic systems are more widely applied, emergency
ways to facilitate comparison of different robot models as well responders might identify additional or advanced robotic radio commu-
nication capability requirements to help them respond to emergency
as particular configurations of similar robot models.
situations. They might also desire to use robots with higher levels of
1.1.5 This test method establishes procedures, apparatuses,
autonomy,beyondteleoperateontohelpreducetheirworkload—seeNIST
and metrics for specifying and testing the capability of radio
Special Publication 1011-II-1.0. Further, emergency responders in ex-
(wireless) links used between the operator station and the
panded emergency response domains might also desire to apply robotic
technologies to their situations, a source for new sets of requirements.As
a result, additional standards within the suite would be developed. This
This test method is under the jurisdiction of ASTM Committee E54 on
standard is, nevertheless, standalone and complete.
Homeland Security Applications and is the direct responsibility of Subcommittee
E54.09 on Response Robots.
1.2 Performing Location—This test method shall be per-
Current edition approved Jan. 1, 2021. Published January 2021. Originally
formed in a testing laboratory or the field where the specified
approved in 2012. Last previous edition approved in 2012 as E2855 – 12. DOI:
10.1520/E2855-12R21. apparatus and environmental conditions are implemented.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E2855 − 12 (2021)
Left: The non-line-of-sight range test method uses an airstrip or flat, paved road with robot test stations placed in front of and behind a wall constructed of stacked 12
m (40 ft) International Standards Organization (ISO) shipping containers. Right: Robot test stations are prototyped behind the wall with targets on the barrels for visual
inspection tasks and circular paths for maneuvering tasks.
FIG. 1 Test Fabrication at An Air Strip
1.3 Units—The values stated in SI units shall be the stan- Terminology, Version 2.0
dard. The values given in parentheses are not precise math- NIST Special Publication 1011-II-1.0 Autonomy Levels for
ematical conversions to inch-pound units. They are close Unmanned Systems (ALFUS) Framework Volume II:
approximate equivalents for the purpose of specifying material Framework Models, Version 1.0
dimensions or quantities that are readily available to avoid
3. Terminology
excessive fabrication costs of test apparatuses while maintain-
ing repeatability and reproducibility of the test method results.
3.1 Definitions:
These values given in parentheses facilitate testing but are not
3.1.1 abstain, v—the action of the manufacturer or desig-
considered standard.
nated operator of the testing robot choosing not to enter the
test. Any decision to take such an action shall be conveyed to
1.4 This standard does not purport to address all of the
the administrator before the test begins. The test form shall be
safety concerns, if any, associated with its use. It is the
clearly marked as such, indicating that the manufacturer
responsibility of the user of this standard to establish appro-
acknowledges the omission of the performance data while the
priate safety, health, and environmental practices and deter-
test method was available at the test time.
mine the applicability of regulatory limitations prior to use.
3.1.1.1 Discussion—Abstentions may occur when the robot
1.5 This international standard was developed in accor-
configuration is neither designed nor equipped to perform the
dance with internationally recognized principles on standard-
tasks as specified in the test method. Practices within the test
ization established in the Decision on Principles for the
apparatus prior to testing should allow for establishing the
Development of International Standards, Guides and Recom-
applicability of the test method for the given robot.
mendations issued by the World Trade Organization Technical
Barriers to Trade (TBT) Committee. 3.1.2 administrator, n—person who conducts the test—The
administrator shall ensure the readiness of the apparatus, the
2. Referenced Documents
test form, and any required measuring devices such as stop-
watch and light meter; the administrator shall ensure that the
2.1 ASTM Standards:
specified or required environmental conditions are met; the
E2521 Terminology for Evaluating Response Robot Capa-
administrator shall notify the operator when the safety belay is
bilities
available and ensure that the operator has either decided not to
E2592 Practice for Evaluating Response Robot Capabilities:
use it or assigned a person to handle it properly; and the
Logistics: Packaging for Urban Search and Rescue Task
administrator shall call the operator to start the test and record
Force Equipment Caches
the performance data and any notable observations during the
2.2 Additional Documents:
test.
National Response Framework U.S. Department of Home-
3.1.3 emergency response robot, or response robot, n—a
land Security
remotely deployed device intended to perform operational
NIST Special Publication 1011-I-2.0 Autonomy Levels for
tasks at operational tempos to assist the operators to handle a
Unmanned Systems (ALFUS) Framework Volume I:
disaster.
3.1.3.1 Discussion—A response robot is designed to serve
For referenced ASTM standards, visit the ASTM website, www.astm.org, or as an extension of the operator for gaining improved remote
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website. Available from National Institute of Standards and Technology (NIST), 100
Available from Federal Emergency Management Agency (FEMA), P.O. Box Bureau Dr., Stop 1070, Gaithersburg, MD 20899-1070, http://www.nist.gov/el/isd/
10055, Hyattsville, MD 20782-8055, http://www.fema.gov/emergency/nrf/. ks/autonomy_levels.cfm.
E2855 − 12 (2021)
situational awareness and for accomplishing the tasks remotely 3.1.9 operator station, n—apparatusforhostingtheoperator
through the equipped capabilities. The use of a robot is and her/his operator control unit (OCU, see NIST Special
designed to reduce risk to the equipped capabilities. It is Publication 1011-I-2.0) to teleoperate (see Terminology
designed to reduce risk to the operator while improving E2521) the robot. The operator station shall be positioned in
effectiveness and efficiency of the mission. The desired fea- such a manner as to insulate the operator from the sights and
tures of a response robot include: rapid deployment; remote sounds generated at the test apparatuses.
operation from an appropriate standoff distance; mobile in
3.1.10 radio interference, n—adverse effect on the transfer
complex environments; sufficiently hardened against harsh
of data when unrelated external signals are received by a robot
environments; reliable and field serviceable; durable and/or
receiver or an operator station receiver.
cost effectively disposable; and equipped with operational
3.1.10.1 Discussion—In licensed frequency bands such as
safeguards.
those used by the public safety community, each radio trans-
mitter and receiver is assigned a unique frequency channel
3.1.4 fault condition, n—a certain situation or occurrence
typically with limits on power emissions. Some radio systems
during testing whereby the robot either cannot continue with-
aredesignedtoworkeffectivelywhenmultiplesystemsoperate
out human intervention or has performed some defined rules
in the same frequency band at the same time. Many of these
infraction.
systems can be found in the unlicensed Industrial, Scientific,
3.1.4.1 Discussion—Faultconditionsincluderoboticsystem
and Medical (ISM) frequency bands.
malfunctions such as de-tracking, task execution problems
such as excessive deviation from a specified path, or uncon-
3.1.11 repetition, n—robot’scompletionofthetaskasspeci-
trolled behaviors and other safety violations which require
fied in the test method and readiness for repeating the same
administrative intervention.
task when required.
3.1.11.1 Discussion—In a traversing task, the entire mobil-
3.1.5 human-scale, adj—used to indicate that the objects,
ity mechanism shall be behind the START point before the
terrains, or tasks specified in this test method are in a scale
traverse and shall pass the END point to complete a repetition.
consistent with the environments and structures typically
A test method can specify returning to the START point to
negotiated by humans, although possibly compromised or
complete the task. Multiple repetitions, performed in the same
collapsedenoughtolimithumanaccess.Also,thattheresponse
test condition, may be used to establish the tested capability to
robotsconsideredinthiscontextareinavolumetricandweight
a certain degree of statistical significance as specified by the
scale appropriate for operation within these environments.
test sponsor.
3.1.5.1 Discussion—No precise size and weight ranges are
3.1.12 test event, or event, n—a set of testing activities that
specified for this term. The test apparatus constrains the
are planned and organized by the test sponsor to be held at the
environment in which the tasks are performed. Such
one or multiple designated test site(s).
constraints, in turn, limit the types of robots to be considered
applicable to emergency response operations.
3.1.13 test form, n—a collection of data fields or graphics
used to record the testing results along with the associated
3.1.6 line-of-sight communications, n—propagating electro-
information. A single test form shall not be used to record the
magnetic energy with a direct path between a transmitting
results of multiple trials.
radio antenna and a receiving radio antenna which are in visual
contact with each other with no obstructions between them. In 3.1.14 test sponsor, n—an organization or individual that
the ideal case, the only paths that the radio waves can take in
commissions a particular test event and receives the corre-
the line-of-sight case are either the direct path between the sponding test results.
transmitter and receiver or a path that corresponds to a single
3.1.15 test suite, n—a designed collection of test methods
reflection of the radio wave off of the ground before it
that are used collectively to evaluate the performance of a
encounters the receiving antenna.
robot’s particular subsystem or functionality, including
3.1.7 non-line-of-sight communications, n—propagating mobility, manipulation, sensors, energy/power,
communications, human-system interaction (HSI), logistics,
electromagnetic energy with no direct path between a trans-
mitting radio antenna and a receiving radio antenna which are safety and operating environment, and aerial or aquatic ma-
neuvering.
not in visual contact with each other due to obstructions
between them. Radio waves propagate between the transmit-
3.1.16 testing target, or target, n—a designed physical
ting and the receiving antennas via reflections off structures,
feature to be used by the testing robotic subsystem for
diffraction around structures, and/or passage through structures
evaluating the subsystem capabilities. The feature may be an
with attenuation.
operationallyrelevantobject,anotionalobject,oronedesigned
specifically for exercising the subsystem features to its full
3.1.8 operator, n—person who controls the robot to perform
extent.
thetasksasspecifiedinthetestmethod;she/heshallensurethe
readiness of all the applicable subsystems of the robot; she/he
3.1.17 testing task, or task, n—a set of activities well
through a designated second shall be responsible for the use of
defined in a test method for testing robots and the operators to
a safety belay; and she/he shall also determine whether to
perform in order for the system’s capabilities to be evaluated
abstain the test.
according to the corresponding metric(s). A test method may
3.1.8.1 Discussion—An emergency responder would be a specify multiple tasks. A task corresponds to the associated
typical operator in emergency response situations. metric(s).
E2855 − 12 (2021)
3.1.18 trial, n—the number of repetitions to be performed 4.9 The operator is allowed to practice before the test.
for a test to reach required statistical significance. The repeti- She/he is also allowed to abstain from the test before it is
tions may be recorded on a single test form. started. Once the test begins, there shall be no verbal commu-
nication between the operator and the administrator regarding
3.2 Terminology E2521 lists additional definitions relevant
the performance of a test repetition other than describing the
to this test method.
targets as seen by the operator, instructions on when to start,
and notifications of faults and any safety related conditions.
4. Summary of Test Method
The operator shall have the full responsibility to determine
4.1 This test method uses remote maneuvering and inspec- whether and when the robot has completed a repetition and
tion tasks to measure the non-line-of-sight range of a robot
notify the administrator accordingly. However, it is the admin-
using its equipped radio communication system. This test
istrator’s authority to judge the completeness of the repetition.
methodrepresentsanobstructedpropagationenvironmentwith
NOTE 4—Practice within the test apparatus could help establish the
ground effects that will be encountered by radio linked robotic applicability of the robot for the given test method. It allows the operator
to gain familiarity with the test method and environmental conditions. It
systems.
alsohelpsthetestadministratortoestablishtheinitialapparatussettingfor
the test when applicable.
4.2 Thetestcourseshallbeaflatpavedsurfaceatleast1000
m (3300 ft) long by 20 m (65 ft) wide with a centerline robot
4.10 The test sponsor has the authority to establish the
path.Aminimumof50m(165ft)oneachsideofthecenterline
testing policy, including the robot participation, testing
robot path shall be clear of any obstructions or reflecting
schedules, test site at which this test method is implemented,
objects to minimize multi-path effects.
associated environmental conditions, the apparatus settings,
and statistical reliability and confidence levels of the testing
4.3 A stacked set of six International Standards Organiza-
results.
tion (ISO) shipping containers measuring 24 m (80 ft) wide by
7 m (24 ft) tall provides a line-of-sight obstruction abutting the
5. Significance and Use
centerline robot path allowing the robot to turn 90° from the
centerline path to perform non-line-of-sight tasks in the radio
5.1 A main purpose of using robots in emergency response
shadow of the obstruction.
operations is to enhance the safety and effectiveness of
4.4 Robot test stations shall be placed 100 m (330 ft) in
emergency responders operating in hazardous or inaccessible
front of the stacked ISO container obstruction along the environments. The testing results of the candidate robot shall
centerline path and within line-of-sight of the operator station describe, in a statistically significant way, how reliably the
to establish reference radio communication functionality. Ad- robot is able to perform the specified types of tasks and thus
ditionalteststationshallbeplacedinspecifiedlocationsbehind provide emergency responders sufficiently high levels of con-
the stacked ISO container obstruction. Each test station con- fidence to determine the applicability of the robot.
sists of eight visual and audio targets for inspection tasks along
5.2 This test method addresses robot performance require-
with circular robot paths marked on the ground for maneuver-
ments expressed by emergency responders and representatives
ing tasks.
from other interested organizations. The performance data
4.5 At each test station, the robot shall perform a maneu-
captured within this test method are indicative of the testing
vering task to follow the circular path to locate each of the
robot’s capabilities. Having available a roster of successfully
visual and audio targets.
testedrobotswithassociatedcapabilitiesdatatoguideprocure-
ment and deployment decisions for emergency responders is
4.6 The visual and audio targets shall be identified using the
consistent with the guideline of “Governments at all levels
robot’s forward facing cameras, requiring the robot to face all
have a responsibility to develop detailed, robust, all-hazards
four compass directions relative to the direction of travel to
response plans” as stated in National Response Framework.
ensure there are no directionality issues with transmitting or
receiving communication signals.
5.3 This test method is part of a test suite and is intended to
provide a capability baseline for the robotic communications
4.7 The robot’s non-line-of-sight range capability is mea-
systems based on the identified needs of the emergency
sured as the number of robot test stations successfully com-
response community. Adequate testing performance will not
pleted behind the stacked ISO container obstruction to verify
ensure successful operation in all emergency response envi-
the functionality of non-line-of-sight control, video and audio
ronments due to possible extreme communications difficulties.
transmissions.
Rather, this standard is intended to provide a common com-
4.8 Teleoperation shall be used from the operator station
parison that can aid in choosing appropriate systems. This
specified by the administrator to test the robots using an OCU
standard is also intended to encourage development of im-
provided by the operator. The operator station shall be posi-
proved and innovative communications systems for use on
tioned and implemented in such a manner as to insulate the
emergency response robots.
operator from the sights and sounds generated at the test
5.4 The standard apparatus is specified to be easily fabri-
apparatus.
cated to facilitate self-evaluation by robot developers and
NOTE 3—Separate, autonomous radio communications test methods
provide practice tasks for emergency responders to exercise
will be separately specified in the future as per community requirements.
This standard is, nevertheless, stand-alone and complete. robot actuators, sensors, and operator interfaces. The standard
E2855 − 12 (2021)
A) The non-line-of-sight range test apparatus includes a reference line-of-sight robot test station 100 m (330 ft) in front of the stacked ISO container obstruction. B) Two
additional such test stations are located behind the obstruction to provide non-line-of-sight tasks to perform (one target barrel placed away from the obstruction is not
shown).
FIG. 2 Test Station Implementation
apparatus can also be used to support operator training to 6.6 Each test station shall also have two audio sources to be
establish operator proficiency. identified through the equipped communications channel. The
center of each circular robot path shall have an audio source
5.5 Although the test method was developed first for emer-
andspeakerplayingacontinuousseriesofsingledigitnumbers
gency response robots, it may be applicable to other opera-
for the identification task. The numbers shall be articulated
tional domains, such as law enforcement and armed services.
using a computer-generated voice with a volume of at least 60
to 80 dB.
6. Apparatus
6.7 The operator station shall be placed at a distance away
6.1 The test apparatus is a straight, flat section of airstrip,
from the stacked ISO container obstruction equal to half the
roadway or other paved asphalt or concrete surface at least
maximum range performed in the associated ASTM standard
1000 m (3300 ft) long and 20 m (65 ft) wide. It shall have no
line-of-sight test method.Astart position for the robot shall be
obstructions or reflective objects within at least 50 m (165 ft)
locatednearbytheoperatorstationwiththesamedistancefrom
on either side of the centerline.
the wall.
6.2 A stacked set of six ISO containers shall form a
6.8 Antennas at the operator station shall be limited to a
reproducible line-of-sight obstruction measuring 24 m (80 ft)
maximum of 2 m (6.5 ft) elevation above the ground.
wide by 7 m (24 ft) tall abutting the centerline robot path
allowing the robot to turn 90° from the centerline path to
6.9 Since this test apparatus must be fabricated outside,
perform non-line-of-sight tasks in the radio shadow of the
control over environmental variables is not strictly possible.
obstruction (see Fig. 2). Gaps between adjacent containers
Various test conditions such as apparatus surface types and
shall be fully covered with metal.
conditions including wetn
...