ASTM E2829-11(2020)

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E2829 − 11 (Reapproved 2020)
Standard Test Method for
Evaluating Emergency Response Robot Capabilities:
Mobility: Maneuvering Tasks: Sustained Speed
This standard is issued under the fixed designation E2829; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope methoddifferentlybasedontheirspecificperformancerequire-
ments. The testing results should collectively represent an
1.1 Purpose:
emergency response ground robot’s overall mobility perfor-
1.1.1 The purpose of this test method, as a part of a suite of
mance as required. These performance data can be used to
mobility test methods, is to quantitatively evaluate a teleoper-
guide procurement specifications and acceptance testing for
ated ground robot’s (see Terminology E2521) sustained ma-
robots intended for emergency response applications.
neuvering speed on paved surfaces.
1.1.2 Robots shall possess a certain set of mobility
NOTE 1—Additional test methods within the suite are anticipated to be
developed to address additional or advanced robotic mobility capability
capabilities, including maneuvering, to suit critical operations
requirements, including newly identified requirements and even for new
such as emergency responses. The environments often pose
application domains.
constraintstoroboticmobilitytovariousdegrees.Beingableto
1.2 Performing Location—This test method shall be per-
maneuver effectively for extended distances is essential for
formed in a testing laboratory or the field where the specified
deploymentdown-rangeduringemergencyresponses.Thistest
apparatus and environmental conditions are implemented.
method specifies apparatuses to standardize this maneuvering
task for testing.
1.3 Units—The values stated in SI units are to be regarded
1.1.3 Emergency response ground robots shall be able to
as the standard.The values given in parentheses are not precise
handle many types of obstacles and terrain complexities. The
mathematical conversions to inch-pound units. They are close
required mobility capabilities include traversing gaps, hurdles,
approximate equivalents for the purpose of specifying material
stairs, slopes, various types of floor surfaces or terrains, and
dimensions or quantities that are readily available to avoid
confined passageways. Yet additional mobility requirements
excessive fabrication costs of test apparatuses while maintain-
include sustained speeds and towing capabilities. Standard test
ing repeatability and reproducibility of the test method results.
methods are required to evaluate whether candidate robots
These values given in parentheses are provided for information
meet these requirements.
only and are not considered standard.
1.1.4 ASTM Task Group E54.08.01 on Robotics specifies a
1.4 This standard does not purport to address all of the
mobility test suite, which consists of a set of test methods for
safety concerns, if any, associated with its use. It is the
evaluating these mobility capability requirements. This sus-
responsibility of the user of this standard to establish appro-
tained speed test method is a part of the mobility test suite.The
priate safety, health, and environmental practices and deter-
apparatuses associated with the test methods challenge specific
mine the applicability of regulatory limitations prior to use.
robotcapabilitiesinrepeatablewaystofacilitatecomparisonof
1.5 This international standard was developed in accor-
different robot models as well as particular configurations of
dance with internationally recognized principles on standard-
similar robot models.
ization established in the Decision on Principles for the
1.1.5 The test methods quantify elemental mobility capa-
Development of International Standards, Guides and Recom-
bilities necessary for ground robot intended for emergency
mendations issued by the World Trade Organization Technical
response applications. As such, users of this standard can use
Barriers to Trade (TBT) Committee.
either the entire suite or a subset based on their particular
2. Referenced Documents
performance requirements. Users are also allowed to weight
particular test methods or particular metrics within a test
2.1 ASTM Standards:
E2521 Terminology for Evaluating Response Robot Capa-
bilities
This test method is under the jurisdiction of ASTM Committee E54 on
Homeland Security Applications and is the direct responsibility of Subcommittee
E54.09 on Response Robots. For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Current edition approved Jan. 1, 2020. Published January 2020. Originally contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
approved in 2011. Last previous edition approved in 2011 as E2829 – 11. DOI: Standards volume information, refer to the standard’s Document Summary page on
10.1520/E2829-11R20. the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E2829 − 11 (2020)
E2592 Practice for Evaluating Response Robot Capabilities: 3.2.4 fault condition, n—during the performance of the
Logistics: Packaging for Urban Search and Rescue Task task(s) as specified by the test method, a certain condition may
Force Equipment Caches occur that renders the task execution to be failed and such a
2.2 Other Standards: condition is called a fault condition. Fault conditions result in
a loss of credit for the partially completed repetition. The test
National Response Framework U.S. Department of Home-
land Security timecontinuesuntiltheoperatordeterminesthatshe/hecannot
continueandnotifiestheadministrator.Theadministratorshall,
NIST Special Publication 1011-I-2.0 Autonomy Levels for
Unmanned Systems (ALFUS) Framework Volume I: then, pause the test time and add a time-stamped note on the
test form indicating the reason for the fault condition.
Terminology, Version 2.0
3.2.4.1 Discussion—Faultconditionsincluderoboticsystem
3. Terminology
malfunction, such as detracking, and task execution problems,
such as excessive deviation from a specified path or failure to
3.1 Terminology E2521 lists additional definitions relevant
recognize a target.
to this test method.
3.2.5 human-scale, adj—used to indicate that the objects,
3.2 Definitions:
terrains, or tasks specified in this test method are in a scale
3.2.1 abstain, v—prior to starting a particular test method,
consistent with the environments and structures typically
the robot manufacturer or designated operator shall choose to
negotiated by humans, although possibly compromised or
enter the test or abstain.Any abstention shall be granted before
collapsedenoughtolimithumanaccess.Also,thattheresponse
the test begins. The test form shall be clearly marked as such,
robotsconsideredinthiscontextareinavolumetricandweight
indicating that the manufacturer acknowledges the omission of
scale appropriate for operation within these environments.
theperformancedatawhilethetestmethodwasavailableatthe
3.2.5.1 Discussion—No precise size and weight ranges are
test time.
specified for this term. The test apparatus constrains the
3.2.1.1 Discussion—Abstentions may occur when the robot
environment in which the tasks are performed. Such
configuration is neither designed nor equipped to perform the
constraints, in turn, limit the types of robots to be considered
tasks as specified in the test method. Practice within the test
applicable to emergency response operations.
apparatus prior to testing should allow for establishing the
applicability of the test method for the given robot. 3.2.6 operator, n—person who controls the robot to perform
thetasksasspecifiedinthetestmethod;she/heshallensurethe
3.2.2 administrator, n—person who conducts the test. The
readiness of all the applicable subsystems of the robot; she/he
administrator shall ensure the readiness of the apparatus, the
through a designated second shall be responsible for the use of
test form, and any required measuring devices such as stop-
a safety belay; and she/he shall also determine whether to
watch and light meter; the administrator shall ensure that the
abstain the test.
specified or required environmental conditions are met; the
administrator shall notify the operator when the safety belay is 3.2.7 operator station, n—apparatusforhostingtheoperator
available and ensure that the operator has either decided not to
and her/his operator control unit (OCU, see NIST Special
use it or assigned a person to handle it properly; and the
Publication 1011-I-2.0) to teleoperate (see Terminology
administratorshallcalltheoperatortostartandendthetestand E2521) the robot; the operator station shall be positioned in
record the performance data and any notable observations
suchamannersoastoinsulatetheoperatorfromthesightsand
during the test. sounds generated at the test apparatuses.
3.2.3 emergency response robot, or response robot, n—a
3.2.8 repetition, n—robot’s completion of the task as speci-
robot deployed to perform operational tasks in an emergency fied in the test method and readiness for repeating the same
response situation.
task when required.
3.2.3.1 Discussion—Aresponserobotisadeployabledevice 3.2.8.1 Discussion—In a traversing task, the entire mobility
intended to perform operational tasks at operational tempos
mechanism shall be behind the START point before the
during emergency responses. It is designed to serve as an traverse and shall pass the END point to complete a repetition.
extension of the operator for gaining improved remote situ-
A test method can specify returning to the START point to
ational awareness and for projecting her/his intent through the complete the task. Multiple repetitions, performed in the same
equipped capabilities. It is designed to reduce risk to the
test condition, may be used to establish the robot performance
operator while improving effectiveness and efficiency of the of a particular test method to a certain degree of statistical
mission.Thedesiredfeaturesofaresponserobotinclude:rapid
significance as specified by the testing sponsor.
deployment; remote operation from an appropriate standoff
3.2.9 test event or event, n—asetoftestingactivitiesthatare
distance; mobility in complex environments; sufficiently hard-
planned and organized by the test sponsor and to be held at the
enedagainstharshenvironments;reliableandfieldserviceable;
one or multiple designated test site(s).
durable or cost effectively disposable, or both; and equipped
3.2.10 test form, n—form corresponding to a test method
with operational safeguards.
that contains fields for recording the testing results and the
associated information.
Available from Federal Emergency Management Agency (FEMA), P.O. Box
3.2.11 test sponsor, n—organization or individual that com-
10055, Hyattsville, MD 20782-8055, http://www.fema.gov.
missions a particular test event and receives the corresponding
Available from National Institute of Standards and Technology (NIST), 100
Bureau Dr., Stop 1070, Gaithersburg, MD 20899-1070, http://www.nist.gov. test results.
E2829 − 11 (2020)
3.2.12 test suite, n—designed collection of test methods that nication between the operator and the administrator regarding
are used, collectively, to evaluate the performance of a robot’s the performance of a test repetition other than instructions on
particular subsystem or functionality, including mobility, when to start and notifications of faults and any safety related
manipulation, sensors, energy/power, communications, conditions. The operator shall have the full responsibility to
human-robot interaction (HRI), logistics, safety, and aerial or determine whether and when the robot has completed a
aquatic maneuvering. repetitionandnotifytheadministratoraccordingly.However,it
istheadministrator’sauthoritytojudgethecompletenessofthe
3.2.13 testing task, or task, n—a set of activities well
repetition.
defined in a test method for testing robots and the operators to
performs in order for the robots’performance to be evaluated.
NOTE 2—Practice within the test apparatus could help establish the
applicability of the robot for the given test method. It allows the operator
A test method may specify multiple tasks. A task corresponds
to gain familiarity with the standard apparatus and environmental condi-
to the associated metric or metrics.
tions. It also helps the test administrator to establish the initial apparatus
setting for the test when applicable.
4. Summary of Test Method
4.5 The test sponsor has the authority to select the turning
4.1 The task for this test method, traversing a specified path
radii for the traversing task. The test sponsor also has the
while straddling a line, is defined as the robot traversing from
authority to select test methods that constitute the test event, to
the START point along the specified path, which ends back at
select one or more test site(s) at which the test methods are
the START point, thus enabling continuous repetitions. The
implemented, to determine the corresponding statistical reli-
specified path shall be a figure-eight, also known as a continu-
ability and confidence levels of the results for each of the test
ous “S,” around two pylons installed in the test course as
methods, and to establish the participation rules including the
described in Section 6. See Fig. 1 for an illustration. The robot
testing schedules and the test environmental conditions.
must straddle the line continuously throughout the path,
maintaining some part of the mobility chassis over the line at
5. Significance and Use
all times. This task is one of the maneuvering tasks that
5.1 A main purpose of using robots in emergency response
candidate response robots shall be able to perform.
operations is to enhance the safety and effectiveness of
4.2 Therobot’straversingcapabilityisdefinedastherobot’s emergency responders operating in hazardous or inaccessible
ability to complete the task and the associated effective speed.
environments. The testing results of the candidate robot shall
Further, the test sponsor can specify the statistical reliability
describe, in a statistically significant way, how reliably the
and confidence levels of such a capability and, thus, dictate the
robot is able to traverse the specified types of terrains and thus
number of successful task performance repetitions that is
provide emergency responders sufficiently high levels of con-
required. In such a case, the average effective speed shall be
fidence to determine the applicability of the robot.
used, instead, as the robot’s capability. In either case, the
5.2 This test method addresses robot performance require-
resulting effective speed is specified as the robot’s sustained
ments expressed by emergency responders and representatives
speed.
from other interested organizations. The performance data
4.3 Teleoperation shall be used from the operator station captured within this test method are indicative of the testing
specified by the administrator to test the robots using an OCU
robot’s capabilities. Having available a roster of successfully
provided by the operator. The operator station shall be posi- tested robots with associated performance data to guide pro-
tioned and implemented in such a manner so as to insulate the
curement and deployment decisions for emergency responders
operator from the sights and sounds generated at the test is consistent with the guideline of “Governments at all levels
apparatus.
have a responsibility to develop detailed, robust, all-hazards
response plans” as stated in National Response Framework.
4.4 The operator is allowed to practice before the test.
She/he is also allowed to abstain from the test before it is 5.3 The standard apparatus is specified to be easily fabri-
started. Once the test begins, there shall be no verbal commu- cated to facilitate self-evaluation by robot developers and
FIG. 1 Mobility: Maneuvering Tasks: Sustained Speed Apparatus
E2829 − 11 (2020)
would not make a difference in the cameras’functioning. It is recognized
provide practice tasks for emergency responders to exercise
that the environments in real applications may be darker than the specified
robot actuators, sensors, and operator interfaces. The standard
test condition.
apparatus can also be used to support operator training and
6.3 A stopwatch shall be provided to measure the timing
establish operator proficiency.
performance.
5.4 Although the test method was developed first for emer-
gency response robots, it may be applicable to other opera-
7. Hazards
tional domains.
7.1 Besides 1.4, which addresses the human safety and
health concerns, users of the standard shall also address the
6. Apparatus
equipment preservation concerns and human robot coexistence
6.1 This test apparatus is a flat, paved surface with two
concerns.
pylons or barrels placed 50 m (165 ft) apart. A specified path
NOTE 4—A test sponsor has the authority to decide the environmental
line is marked on the pavement forming a figure-eight or a
conditions under which this test is to be conducted. Such conditions can
continuous “S.” The turning radius of 2 m (6.5 ft) is specified
be stressful not only to the humans but also to the robots, such as high or
around each of the pylons or barrels (Fig. 2 and Fig. 3). The
low temperatures, excessive moisture, and rough terrains that can damage
path line shall be marked so as to be visible by the remote
the robotic components or cause unexpected robotic motions.
testing operator through her/his OCU, with, for example, white
8. Calibration and Standardization
or brightly colored paint. The robot must straddle the line
continuously throughout the path, maintaining some part of the
8.1 The robot configuration as tested shall be described in
mobility chassis over the line at all times. The effective
detail on the test f
...