ASTM E2804-11(2020)

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation:E2804 −11 (Reapproved 2020)
Standard Test Method for
Evaluating Emergency Response Robot Capabilities:
Mobility: Confined Area Obstacles: Stairs/Landings
This standard is issued under the fixed designation E2804; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope facilitate comparison of different robot models as well as
particular configurations of similar robot models.
1.1 Purpose:
1.1.6 The mobility test suite quantifies elemental mobility
1.1.1 The purpose of this test method is to quantitatively
capabilitiesnecessaryforgroundrobotintendedforemergency
evaluate a teleoperated ground robot’s (see Terminology
response applications. As such, users of this standard can use
E2521) capability of traversing stairs with landings in confined
either the entire suite or a subset based on their particular
areas.
performance requirements. Users are also allowed to weight
1.1.2 Robots shall possess a certain set of mobility
particular test methods or particular metrics within a test
capabilities, including negotiating obstacles, to suit critical
methoddifferentlybasedontheirspecificperformancerequire-
operations such as emergency responses. Stairs with landings
ments. The testing results should collectively represent an
are a type of obstacle that exists in emergency response and
emergency response ground robot’s overall mobility perfor-
otherenvironments.Theseenvironmentsoftenposeconstraints
mance. These performance data can be used to guide procure-
to robotic mobility to various degrees. This test method
ment specifications and acceptance testing for robots intended
specifies apparatuses, procedures, and metrics to standardize
for emergency response applications.
this testing.
1.1.3 The test apparatuses are scalable to provide a range of
NOTE 1—Additional test methods within the suite are anticipated to be
lateral dimensions to constrain the robotic mobility during task
developed to address additional or advanced robotic mobility capability
requirements, including newly identified requirements and even for new
performance. Fig. 1 shows three apparatus sizes to test robots
application domains.
intended for different emergency response scenarios.
1.1.4 Emergency response ground robots shall be able to
1.2 Performing Location—This test method shall be per-
handle many types of obstacles and terrain complexities. The
formed in a testing laboratory or the field where the specified
required mobility capabilities include traversing gaps, hurdles,
apparatus and environmental conditions are implemented.
stairs, slopes, various types of floor surfaces or terrains, and
1.3 The values stated in SI units are to be regarded as the
confined passageways. Yet additional mobility requirements
standard. The values given in parentheses are not precise
include sustained speeds and towing capabilities. Standard test
mathematical conversions to inch-pound units. They are close
methods are required to evaluate whether candidate robots
approximate equivalents for the purpose of specifying material
meet these requirements.
dimensions or quantities that are readily available to avoid
1.1.5 ASTM Task Group E54.08.01 on Robotics specifies a
excessive fabrication costs of test apparatuses while maintain-
mobility test suite, which consists of a set of test methods for
ing repeatability and reproducibility of the test method results.
evaluating these mobility capability requirements. This con-
These values given in parentheses are provided for information
fined area stairs/landings test method is a part of the mobility
only and are not considered standard.
test suite. The apparatuses associated with the test methods
challenge specific robot capabilities in repeatable ways to
1.4 This standard does not purport to address all of the
safety concerns, if any, associated with its use. It is the
responsibility of the user of this standard to establish appro-
priate safety, health, and environmental practices and deter-
This test method is under the jurisdiction of ASTM Committee E54 on
mine the applicability of regulatory limitations prior to use.
Homeland Security Applications and is the direct responsibility of Subcommittee
E54.09 on Response Robots.
1.5 This international standard was developed in accor-
Current edition approved Jan. 1, 2020. Published January 2020. Originally
dance with internationally recognized principles on standard-
approved in 2011. Last previous edition approved in 2011 as E2804 – 11. DOI:
10.1520/E2804-11R20. ization established in the Decision on Principles for the
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E2804−11 (2020)
FIG. 1 Mobility: Confined Area Obstacles: Stairs/Landings Apparatuses
Development of International Standards, Guides and Recom- 3.2.2 administrator, n—person who conducts the test—the
mendations issued by the World Trade Organization Technical administrator shall ensure the readiness of the apparatus, the
Barriers to Trade (TBT) Committee. test form, and any required measuring devices such as stop-
watch and light meter; the administrator shall ensure that the
2. Referenced Documents
specified or required environmental conditions are met; the
administrator shall notify the operator when the safety belay is
2.1 ASTM Standards:
available and ensure that the operator has either decided not to
E2521 Terminology for Evaluating Response Robot Capa-
use it or assigned a person to handle it properly; and the
bilities
administratorshallcalltheoperatortostartandendthetestand
E2592 Practice for Evaluating Response Robot Capabilities:
record the performance data and any notable observations
Logistics: Packaging for Urban Search and Rescue Task
during the test.
Force Equipment Caches
3.2.3 emergency response robot, or response robot, n—a
2.2 Additional Documents:
National Response Framework , U.S. Department of Home- robot deployed to perform operational tasks in an emergency
response situation.
land Security
NIST Special Publication 1011–I–2.0 Autonomy Levels for 3.2.3.1 Discussion—A response robot is a remotely de-
ployable device intended to perform operational tasks at
Unmanned Systems (ALFUS) Framework, Volume I:
Terminology, Version 2.0 operational tempos during emergency responses. It is designed
to serve as an extension of the operator for gaining improved
3. Terminology
remote situational awareness and for projecting her/his intent
through the equipped capabilities. It is designed to reduce risk
3.1 Terminology E2521 lists additional definitions relevant
to the operator while improving effectiveness and efficiency of
to this test method.
the mission. The desired features of a response robot include:
3.2 Definitions:
rapid deployment; remote operation from an appropriate stand-
3.2.1 abstain, v—prior to starting a particular test method,
off distance; mobility in complex environments; sufficiently
the robot manufacturer or designated operator shall choose to
hardened against harsh environments; reliable and field ser-
enter the test or abstain.Any abstention shall be granted before
viceable; durable or cost effectively disposable, or both; and
the test begins. The test form shall be clearly marked as such,
equipped with operational safeguards.
indicating that the manufacturer acknowledges the omission of
3.2.4 fault condition, n—during the performance of the
theperformancedatawhilethetestmethodwasavailableatthe
task(s) as specified by the test method, a certain condition may
test time.
occur that renders the task execution to be failed and such a
3.2.1.1 Discussion—Abstentions may occur when the robot
condition is called a fault condition. Fault conditions result in
configuration is neither designed nor equipped to perform the
a loss of credit for the partially completed repetition. The test
tasks as specified in the test method. Practice within the test
timecontinuesuntiltheoperatordeterminesthatshe/hecannot
apparatus prior to testing should allow for establishing the
continueandnotifiestheadministrator.Theadministratorshall,
applicability of the test method for the given robot.
then, pause the test time and add a time-stamped note on the
test form indicating the reason for the fault condition.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or 3.2.4.1 Discussion—Faultconditionsincluderoboticsystem
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
malfunction, such as de-tracking, and task execution problems,
Standards volume information, refer to the standard’s Document Summary page on
such as excessive deviation from a specified path or failure to
the ASTM website.
recognize a target.
Available from Federal Emergency Management Agency (FEMA), P.O. Box
10055, Hyattsville, MD 20782-8055, http://www.fema.gov/emergency/nrf/.
3.2.5 flat-floor terrain element, n—flat surface with overall
Available from National Institute of Standards and Technology (NIST), 100
dimensions of 1.2 by 1.2 m (4 by 4 ft) which is elevated by
Bureau Dr., Stop 1070, Gaithersburg, MD 20899-1070, http://www.nist.gov/
customcf/get_pdf.cfm?pub_id=824705. using 10-by 10-cm (4-by 4-in.) posts to form a 10 cm (4 in.)
E2804−11 (2020)
thick pallet. The material used to build these elements shall be manipulation, sensors, energy/power, communications,
strong enough to allow the participating robots to execute the human-robot interaction (HRI), logistics, safety, and aerial or
tasks.
aquatic maneuvering.
3.2.5.1 Discussion—The material that is typically used to
3.2.14 testing task, or task, n—a set of activities well
build these elements, oriented strand board (OSB), is com-
defined in a test method for testing robots and the operators to
monly available construction material. The frictional charac-
perform in order for the robots’performance to be evaluated.A
teristics of OSB resemble that of dust covered concrete and
test method may specify multiple tasks. A task corresponds to
other human-improved flooring surfaces often encountered in
the associated metric or metrics.
emergency responses.
3.2.6 human-scale, adj—used to indicate that the objects,
4. Summary of Test Method
terrains, or tasks specified in this test method are in a scale
4.1 The task for this test method, stair traversing, is defined
consistent with the environments and structures typically
as the entire robot traversing from the starting flat-floor terrain
negotiated by humans, although possibly compromised or
element to the landing at the top of the stairs apparatus and
collapsedenoughtolimithumanaccess.Also,thattheresponse
back. See Fig. 1 for an illustration. The test starts at the set of
robotsconsideredinthiscontextareinavolumetricandweight
stairs with the lowest slope and having dry surface. As the
scale appropriate for operation within these environments.
evaluation proceeds, the task shall be performed on all the
3.2.6.1 Discussion—No precise size and weight ranges are
other stair surfaces and on the increased stair inclines as
specifiedforthisterm.Thetestapparatusspecifiestheconfined
specified in Section 6.
areas in which to perform the tasks. Such constraints limit the
overall sizes of robots to those considered applicable to
4.2 The robot’s stair traversing capability is defined as the
emergency response operations.
steepest stair set that the robot is able to traverse for all the
3.2.7 operator, n—person who controls the robot to perform
selected surface types. Further, the test sponsor can specify the
thetasksasspecifiedinthetestmethod;she/heshallensurethe
statistical reliability and confidence levels of such a capability
readiness of all the applicable subsystems of the robot; she/he
and, thus, dictate the number of successful task performance
through a designated second shall be responsible for the use of
repetitions that is required.
a safety belay; and she/he shall also determine whether to
4.3 Teleoperation shall be used from the operator station
abstain the test.
specified by the administrator to test the robots using an OCU
3.2.8 operator station, n—apparatusforhostingtheoperator
provided by the operator. The operator station shall be posi-
and her/his operator control unit (OCU, see ALFUS Frame-
tioned and implemented in such a manner so as to insulate the
work Volume I: Terminology) to teleoperate (see Terminology
operator from the sights and sounds generated at the test
E2521) the robot. The operator station shall be positioned in
apparatus.
suchamannersoastoinsulatetheoperatorfromthesightsand
sounds generated at the test apparatuses. 4.4 The operator is allowed to practice before the test.
She/he is also allowed to abstain from the test before it is
3.2.9 repetition, n—robot’s completion of the task as speci-
started. Once the test begins, there shall be no verbal commu-
fied in the test method and readiness for repeating the same
nication between the operator and the administrator regarding
task when required.
the performance of a test repetition other than instructions on
3.2.9.1 Discussion—In a traversing task, the entire mobility
when to start and notifications of faults and any safety related
mechanism shall be behind the START point before the
conditions. The operator shall have the full responsibility to
traverse and shall pass the END point to complete a repetition.
determine whether and when the robot has completed a
Atest method can also specify returning to the STARTpoint to
repetitionandnotifytheadministratoraccordingly.However,it
complete the task. Multiple repetitions, performed in the same
istheadministrator’sauthoritytojudgethecompletenessofthe
test condition, may be used to establish the robot performance
repetition.
of a particular test method to a certain degree of statistical
significance as specified by the testing sponsor.
NOTE 2—Practice within the test apparatus could help establish the
applicability of the robot for the given test method. It allows the operator
3.2.10 test event or event, n—a set of testing activities that
to gain familiarity with the standard apparatus and environmental condi-
are planned and organized by the test sponsor and to be held at
tions. It also helps the test administrator to establish the initial apparatus
the designated test site(s).
setting for the test when applicable.
3.2.11 test form, n—form corresponding to a test method
4.5 The test sponsor has the authority to select the size of
that contains fields for recording the testing results and the
the lateral clearance for the specified confined area apparatus.
associated information.
The test sponsor also has the authority to select the test
3.2.12 test sponsor, n—an organization or individual that
methodsthatconstitutethetestevent,toselectoneormoretest
commissions a particular test event and receives the corre-
site(s)atwhichthetestmethodsareimplemented,todetermine
sponding test results.
thecorrespondingstatisticalreliabilityandconfidencelevelsof
3.2.13 test suite, n—designed collection of test methods that the results for each of the test methods, and to establish the
participation rules including the testing schedules and the test
are used, collectively, to evaluate the performance of a robot’s
particular subsystem or functionality, including mobility, environmental conditions.
E2804−11 (2020)
5. Significance and Use apparatus can also be used to support operator training and
establish operator proficiency.
5.1 A main purpose of using robots in emergency response
operations is to enhance the safety and effectiveness of 5.5 Although the test method was developed first for emer-
emergency responders operating in hazardous or inaccessible gency response robots, it may be applicable to other opera-
environments. The testing results of the candidate robot shall tional domains.
describe, in a statistically significant way, how reliably the
6. Apparatus
robot is able to negotiate various types of obstacles, including
the specified one, and thus provide emergency responders
6.1 Four different stair inclines (30°, 35°, 40°, and 45°) are
sufficiently high levels of confidence to determine the applica-
specified with two different stair tread surfaces, wood and
bility of the robot.
“diamond plate” steel (Figs. 2 and 3). Each surface type is
further specified with both the wet and dry conditions. Each
5.2 This test method addresses robot performance require-
stairs set contains five stairs with 20-cm (8-in.) risers. The
ments expressed by emergency responders and representatives
depths of the treads are determined from the stair inclines and
from other interested organizations. The performance data
the riser heights. A safety rope belay shall be provided as an
captured within this test method are indicative of the testing
option for the operator.
robot’s capabilities. Having available a roster of successfully
tested robots with associated performance data to guide pro-
6.2 The test apparatuses specify three lateral clearances,
curement and deployment decisions for emergency responders
which are 2.4 m (8 ft), 1.2 m (4 ft), or 0.6 m (2 ft) wide, to be
is consistent with the guideline of “Governments at all levels
determined by the test sponsor. All three scales have 2.4 m (8
have a responsibility to develop detailed, robust, all-hazards
ft) long launch and landing areas as their default setting. The
response plans” as stated in National Response Framework.
apparatuses shall be strong enough to allow the participating
robots to execute the testing tasks.
5.3 The test apparatuses are scalable to constrain robot
maneuverability during task performance for a range of robot
6.3 The test sponsor has the authority to implement further
sizes in confined areas associated with emergency response
confined launch and landing areas, which are square to match
operations. Variants of the apparatus provide minimum lateral
the selected lateral clearance. Removable containment walls
clearance of 2.4 m (8 ft) for robots expected to operate around
shall be placed accordingly.
environments such as cluttered city streets, parking lots, and
NOTE 3—The material that is typically used to build this test
building lobbies; minimum lateral clearance of 1.2 m (4 ft) for
apparatus, OSB plywood, is a commonly available construction material.
robots expected to operate in and around environments such as
The frictional characteristics of OSB plywood resemble that of dust
large buildings, stairwells, and urban sidewalks; minimum covered concrete floors and other improved flooring surfaces often
encountered in emergency responses.
lateral clearance of 0.6 m (2 ft) for robots expected to operate
within environments such as dwellings and work spaces, buses
6.4 Various test conditions such as apparatus surface types
and airplanes, and semi-collapsed structures; minimum lateral
and conditions, including wetness and friction levels,
clearance of less than 0.6 m (2 ft) with a minimum vertical temperature, types of lighting, smoke, humidity, and rain shall
clearanceadjustablefrom0.6m(2ft)to10cm(4in)forrobots
be facilitated when the test sponsor requires. For example, for
expected to deploy through breeches and operate within
a test run in the dark environment, a light meter shall be used
sub-human size confined spaces voids in collapsed structures.
toread0.1luxorless.Thedarknessshallbere-measuredwhen
the
...