ASTM F3269-21

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: F3269 − 21
Standard Practice for
Methods to Safely Bound Behavior of Aircraft Systems
Containing Complex Functions Using Run-Time Assurance
This standard is issued under the fixed designation F3269; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
INTRODUCTION
This practice defines an architecture using Run-Time Assurance (RTA) in conjunction with
unassured functions or commercial off-the-shelf (COTS) functions that have not been developed to
traditional aerospace standards and processes. This section provides the scope, applicability, and
intended use for the understanding of this practice.
Thepracticeisorganizedasfollows: (1)Anintroduction,background,andscopetoprovidecontext
for applying the capabilities defined in this practice to unmanned aircraft system (UAS) certification,
or operational approval, or both. (2) Definitions of key terms and abbreviations. (3) Description of a
Run-Time Assurance (RTA) architecture. (4) Appendixes that contain Examples of RTA in systems
and supplemental information. (a) Ground CollisionAvoidance System (GCAS) as an Example RTA.
(b) Machine Learning AI Autopilot (MLAA). (c) Run-Time Assurance for a Neural Network-Based
AdaptiveFlightControlofanUnmannedAircraft. (d)Run-TimeAssuranceforRisk-BasedOperation.
(e) Example Implementation of Timing and Latency Requirement. (5)Alist of documents referenced
herein.
BACKGROUND
There is significant interest from industry and civil aviation authorities (CAA) to have a standard
practicetoenablenewandnoveltechnologiesusedinUASoperationscontainingunassuredorCOTS
functions/systems, or both, to be used on certified aircraft and aviation systems. From this point
forward, “functions/systems” will be referenced as “functions.” Developing a certification path for
these technologies may also introduce greater safety to aviation.
In this practice, the term Complex Function (CF) may be any function, algorithm, component, or
system that has not been subject to accepted CAA or aerospace design assurance practices, or both
(DO-178C, DO-254,ARP4754A, etc.). Motivations to use such an unassured function arise from the
need or desire to use commercial, off-the-shelf systems or parts that have algorithmic complexity,
probabilistic algorithms, fuzzy logic, environmental uncertainties, or no pedigree. The complexity
may also come from factors associated with new and novel technologies such as sensor measurement
precision, nondeterministic algorithms, data-driven algorithms, or artificial intelligence (for example,
machine learning, genetic algorithms). A complex function may be any combination of software or
hardware.
Traditional approaches to digital avionics design begin with the assumption that each software and
hardware component on an aircraft contribute independently to the safe operation of the platform.At
the core of this process is an assessment of the risks associated with the functional failure of each
system, assembly, or component to ensure that the aircraft meets the required safety objectives. This
is known as design-time assurance.
This practice describes a run-time assurance method, which may be used as an alternative means
to or in combination with design-time assurance. RTA mitigates the risk of complex function
misbehaviorbymanagingthesystem’suseoftheComplexFunctionoutput.TheRTAincludesasafety
monitor, which monitors the complex function or the behavior the complex function has on the
system, or both, at run-time. In the event the safety monitor determines that the complex function is
not operating correctly, or is driving the system to an unsafe state, it disengages the complex function
and initiates a recovery function.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
F3269 − 21
This practice provides an RTAarchitecture and best practices that provide guidance to an applicant
for ensuring that the behavior of an unmanned aircraft system (UAS) containing complex functions
maintains the acceptable level of safety.
At the time of this practice’s development, there is no accepted formal guidance material for
certifying commercial UAS containing complex functions. Emerging CAA certification guidance,
processes, and concepts have been considered in the development of this practice.
ThispracticeisunderthejurisdictionofASTMCommitteeF38onUnmannedAircraftSystemsandisthedirectresponsibilityofSubcommitteeF38.01onAirworthiness.
Current edition approved July 15, 2021. Published November 2021. Originally approved in 2017. Last previous edition approved in 2017 as F3269–17. DOI:
10.1520/F3269-21.
1. Scope
Title Section
RTA System Coverage 5.4.2
1.1 The scope of this practice includes the following:
RTA Scenarios 5.4.3
1.1.1 A set of components that comprise an RTA system. Event Sequencing and Timing 5.4.3.8
Best Practices 5.4.4
1.1.2 Requirements and best practices to determine safe
Requirements 5.4.5
boundaries and RTA system coverage.
RTA Interfaces 5.5
Input Manager 5.6
1.1.3 Requirements and best practices for an RTA system
Description 5.6.1
and RTA components, as applicable.
Requirements 5.6.2
1.1.4 Appendixes with examples that demonstrate key RTA
Safety Monitor 5.7
Requirements 5.7.2
system concepts.
RTA Switch 5.8
1.2 RTAcomponents are required to meet the design assur-
Description 5.8.1
Requirements 5.8.2
ance level dictated by a safety assessment process. Guidance
Recovery Function 5.9
for the safety assessment process may be found in references
Description 5.9.1
appropriate for the intended operations (ARP4754A,
Best Practices 5.9.2
Requirements 5.9.3
ARP4761, Practice F3178, etc.).
Keywords 6
Ground Collision Avoidance System (GCAS) as an Example Appendix X1
1.3 This practice was developed with UAS in mind. It may
RTA
be applicable for aspects of manned aircraft certification/
Introduction
approval,aswellasaviationgroundsystems.Thescopeofthis
Unassured Function X1.1
practice is also envisioned to allow a variety of aircraft RTA Required Inputs X1.2
RTA Input Manager X1.3
implementationswhereahumanmayperformtheroleofeither
Safety Monitor X1.4
the Complex Function or a Recovery Function.
Recovery Function X1.5
RTA Switch X1.6
1.4 The scope of this practice does not cover aspects of
Vehicle Management System X1.7
hardware/software integration. These should be considered
Machine Learning AI Autopilot (MLAA) Appendix X2
Introduction
separately during the development process.
Assured and Unassured Data X2.1
Input Manager X2.2
NOTE 1—This practice does not suggest a one-size-fits-all strategy
Complex Function X2.3
knowing that not all use cases may fit well into this architecture. There
Safety Monitors X2.4
may exist additional components required to satisfy specific applications
Recovery Control Function X2.5
to the practice.
RTA Switch X2.6
Summary X2.7
1.5 The values stated in inch-pound units are to be regarded
Run-Time Assurance for a Neural Network-Based Adaptive Appendix X3
asstandard.Nootherunitsofmeasurementareincludedinthis
Flight Control of an Unmanned Aircraft
standard.
Visual Line-of-Sight Operations X3.1
Beyond Visual Line-of-Sight Operation X3.2
1.6 Table of Contents:
Run-Time Assurance for Risk-Based Operation Appendix X4
Example Implementation of Timing and Latency Requirement Appendix X5
Title Section
References
Introduction
Background
1.7 This standard does not purport to address all of the
Scope 1
safety concerns, if any, associated with its use. It is the
Referenced Documents 2
ASTM Standards 2.1
responsibility of the user of this standard to establish appro-
FAAAdvisory Circular 2.2
priate safety, health, and environmental practices and deter-
RTCA Standards 2.3
SAE Standards 2.4 mine the applicability of regulatory limitations prior to use.
Terminology 3
1.8 This international standard was developed in accor-
Unique and Common Terminology 3.3
dance with internationally recognized principles on standard-
Definitions of Terms Specific to This Standard 3.4
Abbreviations 3.5
ization established in the Decision on Principles for the
Significance and Use 4
Development of International Standards, Guides and Recom-
RTA Functional Architecture 5
Overall Architecture 5.4 mendations issued by the World Trade Organization Technical
Components and Interfaces 5.4.1
Barriers to Trade (TBT) Committee.
F3269 − 21
2. Referenced Documents 3.4.2 Complex Function, n—the unassured function for
2 which run-time-assurance is being used. For examples, see
2.1 ASTM Standards:
Background.
F3060Terminology for Aircraft
F3178Practice for Operational Risk Assessment of Small 3.4.3 Designer, n—thepersonororganizationthatisrespon-
Unmanned Aircraft Systems (sUAS)
sible for the design, development, and/or integration of the
F3341/F3341MTerminology for Unmanned Aircraft Sys- RTA System.
tems
3.4.4 Dynamic Consistency, n—independently measured
ASTM AC377 TR2-EBDevelopmental Pillars of Increased
variables are checked for consistency using known models of
Autonomy for Aircraft Systems
behavior. For further detail, reference ASTMAC377 TR2-EB,
2.2 FAA Advisory Circular:
Section 5, Dynamic Consistency.
AC 23.1309-1ESystem SafetyAnalysis andAssessment for
3.4.5 Input Manager, n—an assured RTA function that
Part 23 Airplanes
accepts assured and unassured data and conditions, validates
2.3 RTCA Standards:
andperformsconsistencychecking,andoutputsassureddatato
RTCA DO-178CSoftware Considerations in Airborne Sys-
RTA Components.
tems and Equipment Certification
3.4.6 Larger System, n—the system within which the RTA
RTCA DO-254Design Assurance Guidance for Airborne
System exists. It provides external RTA data and inputs and
Electric Hardware
consumes the RTA Output. Example of Larger Systems are
2.4 SAE Standards:
avionics system/subsystem, air vehicle, or UAS, which may
SAE ARP4754AGuidelines for Development of Civil Air-
contain multiple RTA Systems.
craft and Systems
SAE ARP4761Guidelines and Methods for Conducting the
3.4.7 Larger System Specification, n—The collection of all
SafetyAssessmentProcessonCivilAirborneSystemsand
requirements used to specify the design of the Larger System.
Equipment
A subset of the Larger System Specification contains require-
ments derived from this architecture standard specifying the
3. Terminology
design and implementation of the RTA System.
3.1 Thissectiondefineskeytermsandabbreviationsforthis
3.4.8 Monitor Coverage, n—unionofthecoverageprovided
practice.
by each Monitor Subfunction within the RTA system.
3.2 Note on terminology: shall versus should versus may—
3.4.9 Predefined Bounds, n—acceptable limits to maintain
useoftheword“shall”impliesthataprocedureorstatementis
the Larger System in a Safe State.Any violation of this bound
mandatory and must be followed to comply with this practice,
is a failure of the RTA System. Predefined Bounds may be
“should”impliesrecommended,and“may”impliesoptionalat
static or dynamic and are determined during design.
the discretion of the supplier, manufacturer, or operator. Since
3.4.10 Recovery Function, n—an assured RTAfunction that
“shall” statements are requirements, they include sufficient
generatesoutputsintendedtokeeptheLargerSysteminaSafe
detail needed to define compliance (for example, threshold
State. Recovery Function may provide “fail safe” or “fail
values, test methods, oversight, and references to acceptable
functional” capabilities in order to allow for graceful degra-
industry standards). “Should” statements represent best prac-
dation of functionality.
tices to guide in the development of RTA Systems. “May”
statements are provided to clarify acceptability of a specific
3.4.11 Recovery Function Coverage, n—union of the cov-
item or practice and offer options for satisfying requirements.
erage provided by each Recovery Function within the RTA
System.
3.3 Unique and Common Terminology—Terminology used
in multiple standards is defined in F3341/F3341M, UAS
3.4.12 RTA Components, n—the set of assured functions
Terminology Standard, and F3060,AircraftTerminology Stan-
defined by RTA architecture; includes Input Manager, Safety
dard.
Monitor, RTA Switch, Recovery Function(s).
3.4 Definitions of Terms Specific to This Standard:
3.4.13 RTA Output, n—the output of the RTA Switch.
3.4.1 Assured, adj—Attribute of an entity for which suffi-
3.4.14 RTA Switch, n—anassuredRTAfunctionthataccepts
cientevidenceexiststodemonstratethatanacceptablelevelof
the source selection for the RTA Output from the Safety
rigor has been met.
Monitor and provides that one output to the Larger System.
3.4.15 RTA System, n—the system containing RTACompo-
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
nents and the Complex Function.
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Standards volume information, refer to the standard’s Document Summary page on
3.4.16 RTA System Coverage, n—the RTA System’s opera-
the ASTM website.
tional domain where both Monitor Coverage and Recovery
Available from Federal Aviation Administration (FAA), 800 Independence
Coverage exists.
Ave., SW, Washington, DC 20591, http://www.faa.gov.
Available from Radio Technical Commission for Aeronautics (RTCA), 1150
3.4.17 Run-Time Assurance, n—a method that uses RTA
18th NW, Suite 910, Washington, DC 20036, https://www.rtca.org.
systemstoensurethataLargerSystem’sbehaviorremainsina
AvailablefromSAEInternational(SAE),400CommonwealthDr.,Warrendale,
PA 15096, https://www.sae.org. Safe State.
F3269 − 21
3.4.18 Run-Time Assurance Architecture, n—a system of 4.2 The following three-step process is used to derive
assured components that implements monitoring, prediction, verifiable design requirements using this architecture standard:
and fail-safe recovery mechanisms that bounds the behavior of
4.2.1 Create RTA System requirements using the guidance
a system containing a Complex Function. RTA Components provided by this architecture standard.
are: Input Manager, Safety Monitor, RTA Switch, and Recov-
4.2.2 Capture RTA System requirements in the Larger
ery Function(s). System Specification.
4.2.3 Perform verification and validation on the RTA Sys-
3.4.19 Safe State, n—a condition where the Larger System
tem requirements in the Larger System Specification.
is within acceptable limits.
4.3 The RTAarchitecture can be applied to all sizes, levels,
3.4.20 Safety Assessment Process, n—the set of activities
and classes of UAS. Using run-time assurance can provide
applied during the design of the Larger System to generate
systems with the following benefits:
safety objectives and determine the necessary level of assur-
4.3.1 Theabilitytomitigatehazardsrelatedtonondetermin-
ance for the RTA Components.
istic or unexpected behavior from unassured functions that
3.4.21 Safety Monitor, n—an assured RTA function that
employ advanced software methods or algorithmic complexity
continuously evaluates Larger System and/or Complex Func-
that cannot be certified using traditional certification practices.
tion behaviors, with the intent of discovering misbehavior of
4.3.2 The ability to use functions for which it may not be
the Complex Function. When necessary, the monitor selects
possibletoobtainartifactsofconventionalDO-178orDO-254
andcommandstheRTASwitchtoaRecoveryFunctionorback
assurance processes.
to the Complex Function. The Safety Monitor is composed of
4.3.3 The ability to use COTS hardware or software, or
one or more Monitor Subfunctions.
both, for the unassured function.
3.4.22 Safety Monitor Trigger Threshold (SMTT), n—limits
4.3.3.1 For example, automotive components, thereby le-
derivedfromthePredefinedBoundsthatareusedbytheSafety
veraging mature software with extensive service history that
MonitortodeterminethesourceoftheRTAOutput. The SMTT
wasdevelopedforothersafety-criticalindustries,butcannotbe
may be static or dynamic and is determined during design.
shown to comply with aviation development assurance prac-
3.4.23 Unassured, adj—Attribute of an entity that is not
tices.
assured and, hence, may not be directly used and trusted by
4.3.3.2 For example, industry components where source
RTA components.
code or other associated engineering artifacts are unavailable.
4.3.4 Areduction in cost and schedule burdens by allowing
3.5 Abbreviations:
3.5.1 CAA—Civil Aviation Authority rapid design iterations of the unassured or complex function
duringandafterinitialcertification.Thisupdateofthestandard
3.5.2 CF—Complex Function
allows unassured or complex function upgrades after initial
3.5.3 IM—Input Manager
certification to minimize subsequent modifications to the
3.5.4 RF—Recovery Function
certification or approval.
3.5.5 RS—RTA Switch
5. RTA Functional Architecture
3.5.6 RTA—Run-time assurance
5.1 This section defines key attributes of the overall RTA
3.5.7 SM—Safety Monitor
architecture and its components that meet the intent described
3.5.8 SMTT—Safety Monitor Trigger Threshold
inthispractice.Minimumrequirementsaredefinedforvarious
3.5.9 UAS—Unmanned Aircraft System
intended uses of this reference architecture (see Fig. 1). It is
expected that the reference architecture will be tailored by the
4. Significance and Use
users to their specific application.
4.1 This practice provides an architectural framework for
5.2 Subsections5.3through5.9arewrittentosolelyprovide
developinganRTAsystem,whichprovidesrun-timeassurance
the functional characteristics of an RTA System. The RTA
as an alternative to design-time assurance to fulfill safety
components with their attributes and their relationships are
requirements for an unassured or complex function. The
described in 5.3 through 5.9. Implementations may distribute
standard provides best practices and guidelines to assist in the
RTA functionality across hardware and software modules, as
RTA system’s development. Further, it describes the architec-
desired.
tural components and requirements for designing the RTA
system. Compliance to this practice is achieved by deriving 5.3 This practice is written with a single RTA System in
RTA System requirements from the standard and capturing mind,thatis,whereaLargerSystem’sbehaviorisboundusing
them in the Larger System Specification. The system design a single RTA implementation. However, complex systems
containing multiple independent RTA systems are envisioned.
requirements can then be validated and verified using accept-
able engineering practices. It is anticipated that this practice This practice is applicable to multiple, composable RTA
will provide a means to accept complex automation/autonomy systems as long as their respective RTA Outputs are indepen-
aircraft functions that have been difficult to certify using dent (that is, RTA Systems do not contend to output the same
traditional methods. data).
F3269 − 21
FIG. 1 RTA Architecture
5.4 Overall Architecture: then verified for dynamic consistency. Assured External Data
5.4.1 Components and Interfaces: doesnotrequireconditioning.OutputsfromtheInputManager
5.4.1.1 TheRTASystemarchitecture(Fig.1)consistsofthe
canbesafelyusedwithintheRTASystem.TheSafetyMonitor
following Components and Interfaces:
ensures that the RTA Output or Larger System behavior, or
(1)RTA Components
both, is within Predefined Bounds. When the Safety Monitor
(a)Input Manager
determines that the RTAOutput or Larger System behavior, or
– Data Conditioning
both,exceedstheSafetyMonitorTriggerThreshold,theSafety
– Dynamic Consistency Checking
Monitor selects an appropriate Recovery Function and passes
(b)Safety Monitor
the selection to the RTASwitch.The RTASwitch then ensures
(c)RTA Switch
that the RTA Output is sourced from the selected Recovery
(d)Recovery Function(s)
Function. The Safety Monitor may return control to the
(2)RTA Interfaces
Complex Function when it is determined that the Complex
(a)Assured External Data
Function’s behavior has become acceptable.
(b)Unassured External Data
(c)Safety Monitor Inputs NOTE2—WhenmonitoringtheComplexFunctionbehaviordirectly(as
opposed to monitoring the behavior of the Larger System), the Complex
(d)Recovery Function Inputs
Function Output is routed to the Safety Monitor through the Input
(e)Recovery Function Outputs
Manager. This allows data conditioning and dynamic consistency check-
(f)Complex Function Outputs
ing of the Complex Function Output while maintaining its behavioral
(g)RTA Source Selection
attributes.
(h)RTA Output
5.4.2 RTA System Coverage:
(3)Complex Function
5.4.2.1 This section describes one process for defining RTA
5.4.1.2 This section provides guidance to the developer for
System Coverage. Determining coverage for an RTASystem is
implementing an RTA System. The architecture ensures that
highly dependent on the use-case and pertinent variables. Fig.
“RTAOutput” in Fig. 1 is always bounded with the intent that
2 denotes an abstract 2-D version of coverage for illustrative
theLargerSystemcontainingtheComplexFunctionremainsin
aSafeState.First,theUnassuredExternalDataisconditioned, purposes.
F3269 − 21
FIG. 2 RTA System Coverage
5.4.2.2 The RTA System Coverage may be determined as theLargerSystemoutsidetheRTASystemCoverageisoutside
follows: the scope of this practice.
(1)Denote D as the set of all reachable, nominal, or
LS
5.4.3 RTA Scenarios:
otherwise, operational points where the Larger System is
5.4.3.1 Four possible scenarios are identified in Fig. 3. All
expected to operate.
scenarios start with (1) the Complex Function is the source of
(2)For every available Recovery Function, RF, compute
i
the RTA Output, (2) the RTA Output is within the Safety
thesetofallpointsthatcanbedesign-timeassuredtomaintain
Monitor Trigger Threshold, which is defined as the RTA
Larger System safety or recover functionality, or both, pro-
System being within the Nominal Region.
vided by the Complex Function at those points. Denote this
5.4.3.2 When the RTA Output exceeds the Safety Monitor
computed set as D .
RF
i
Trigger Threshold and is within the Predefined Bounds, the
(3)For every available Monitor Function, M, compute the
j
RTA System is defined as being in the Recovery Region. The
set of all points that can be design-time assured to detect
RTAOutput is sourced from one of the Recovery Functions. If
Complex Function or Larger System misbehavior. Denote this
the RTAOutput exits the Predefined Bounds, the RTASystem
computed set as D .
M
j
has failed.
5.4.2.3 Then the RTASystem Coverage may be defined as:
5.4.3.3 Scenario a)—RTAOutput remains within the Safety
n n
RF M
D 5 ¯ D ˘ ¯ D 5 D ˘D (1)
S D S D Monitor Trigger Threshold, and the RTA System remains
RTA RF M RF M
i j
i51 j51
within the Nominal Region. The Complex Function remains
where:
the source of the RTA Output.
n ,n = the number of available recovery functions and
RF M 5.4.3.4 Scenario b)—RTAOutput exceeds the Safety Moni-
monitor functions, respectively; and
torTriggerThreshold, thus, the RTASystem exits the Nominal
Q and P = the union and intersection operators, respectively.
Region.ARecovery Function becomes the source of the RTA
Note that Eq 1 shows that RTA System Coverage can be
Output. When the Complex Function Output returns to within
composed of disconnected sets.
the Safety Monitor Trigger Threshold, the RTA System re-
enters the Nominal Region, and the Complex Function Output
5.4.2.4 LargerSystembehaviorisonlyboundedbytheRTA
System when operating within D . Therefore, operation of again becomes the source of RTA Output.
RTA
F3269 − 21
FIG. 3 RTA System Operational Scenarios
5.4.3.5 Scenario c)—RTAOutput exceeds the Safety Moni- occur for each of the scenarios are ordered according to t
start
torTriggerThreshold, thus, the RTASystem exits the Nominal #t #t #t . All events can occur at the same time step,
exit entry fail
Region.ARecovery Function becomes the source of the RTA
but in the order afforded above.
Output. The Recovery Function is unable to return the RTA
(2)Even in Scenario Case a) where the RTA System is
System to the Nominal Region, or the Complex Function
within the Nominal Region, an estimate of when and whether
Output continues to exceed the Safety Monitor Trigger
the a and a mayoccurshouldbeusedtocomputebuffers,
exit fail
Threshold, or both.The RTAOutput remains sourced from the
margins, and other quantities that may be required to make
RecoveryFunction,andtheRTASystemoperatessafelywithin
decisions to maintain the RTA System within the Nominal
the Recovery Region.
Region.
5.4.3.6 Scenario d)—RTAOutput exceeds the Safety Moni-
5.4.4 Best Practices:
torTriggerThreshold, thus, the RTASystem exits the Nominal
5.4.4.1 Safety Assessment Process—The designer should
Region.ARecovery Function becomes the source of the RTA
Output. The Recovery Function is unable to return the RTA apply a SafetyAssessment Process to determine the functional
OutputtowithintheSafetyMonitorTriggerThresholdoreven hazard associated with a failure of the Complex Function. The
keep it within the Predefined Bounds. The RTA System exits hazardous conditions associated with a failure of the Complex
the Recovery Region, thus, the RTA System has failed.
Function are used to determine the necessary levels of design
5.4.3.7 Forallscenarios,theSafetyMonitorTriggerThresh-
assurance imposed on the RTA components.
old and Predefined Bounds may be either static or dynamic.
5.4.4.2 Robust Design—The designer should engage in
Dynamic bounds and thresholds require continual recomputa-
appropriate design, test, and validation activities to enable the
tion to determine the actual boundary of the Recovery Region.
complexfunctionandRTAcomponentstoperformasintended,
Dynamically changing bounds and thresholds may be impre-
including establishing required resources, bandwidth, etc.
cise and may benefit from or even require implementation of a
5.4.4.3 Poor Complex Function Design—The designer
safety margin to ensure the RTASystem does not accidentally
should not use RTA as a substitute for poor complex function
exit the Recovery Region.
design or implementation, or both.
5.4.3.8 Event Sequencing and Timing:
5.4.4.4 False Recovery Activation—The designer should
(1)For each of the scenarios in Fig. 3, the following
definetheSafetyMonitorTriggerThresholdsuchthattheRTA
sequence of events are possible:
System has an acceptable false alarm rate.The false alarm rate
a
start
should be defined in the Larger System Specification.
b →b →b
start exit entry
c →c 5.4.4.5 Chattering—The designer should prevent the occur-
start exit
rence of frequent switching between Complex Function and
d →d →d
start exit fail
where the times at which the start, exit, entry, and fail events Recovery Functions.
F3269 − 21
5.4.4.6 Partitioning and Modularity—The designer should 5.5.4 Recovery Function Inputs—Interface from the Input
leverage the concepts of partitioning and modularity when ManagertotheRecoveryFunctionthatincludesaminimumset
developing the RTA System. of assured data needed to perform its intended function.
5.4.4.7 Safety Margins—The designer should ensure RTA 5.5.5 Complex Function Output—Interface from the Com-
activatesinatimelymanner.SafetyMarginsshouldbeselected plex Function to the RTASwitch and the Input Manager when
to ensure boundaries are not violated. For example, margins
monitoring the behavior of the Complex Function. It is the
may be considered in the definitions of Predefined Bounds and desired source for the RTA Output.
Safety Monitor Trigger Threshold.
5.5.6 Recovery Function Outputs—Interface from the Re-
5.4.4.8 Recovery Function Monitoring—The designer covery Function to the RTASwitch that contains only assured
should consider availability of Recovery Functions in RTA data. It is an alternate source for the RTA Output.
System design.
5.5.7 RTA Source Selection—Interface from the Safety
5.4.5 Requirements: Monitor to the RTA Switch that contains only assured data,
which specifies the source of the RTA Output.
5.4.5.1 F3269-RTAS-001—The designer shall develop RTA
Components to the necessary level of assurance as determined 5.5.8 RTA Output—Interface from the RTA Switch to the
in the Larger System Specification.
LargerSystem.TheRTAOutputhasbeenboundedbytheRTA
System.
5.4.5.2 F3269-RTAS-002—The designer shall develop all
RTA components and RTA interfaces listed in 5.4.1.
5.6 Input Manager:
5.4.5.3 F3269-RTAS-003—The designer shall quantify and
5.6.1 Description:
address the impact of timing and latency in the RTA System
5.6.1.1 The Input Manager’s role is to ensure that down-
andtheLargerSystemoneachRTAComponent. This includes
stream RTA components receive assured data. This is accom-
all timing considerations such as latency, time constants, and
plished using two distinct capabilities—data conditioning and
event sequencing from sensor input to detecting misbehavior
dynamicconsistencychecking.Embeddedsystemsdatacanbe
through completion of the recovery process.
invalid for a variety of reasons. It is important to perform data
5.4.5.4 F3269-RTAS-004—The designer shall ensure that
validity checking (or data conditioning and dynamic consis-
adverse transients do not occur while switching between
tency checking). In the following paragraphs, we discuss the
different sources of RTA Output.
means to show that data is assured.
5.4.5.5 F3269-RTAS-005—The designer shall determine
(1) Data Conditioning—The Input Manager “conditions”
RTASystem Coverage in accordance with the process defined
data to ensure inputs lie within predefined domains by drop-
in 5.4.2.
ping invalid data, conditioning data to be within range or
5.4.5.6 F3269-RTAS-006—The designer shall ensure that
marking data as out of range. For example, physical signals
theRTASystemisonlyusedwithinthecomputedRTASystem
can be limited in voltage, current, etc., while software data
Coverage.
streams might include checks for not-a-number, protocol syn-
5.4.5.7 F3269-RTAS-007—Thedesignershalldefinebounds tax compliance, etc. These are akin to syntactic data type
checks.
to ensure that the Larger System remains in a Safe State.
(2) Dynamic Consistency Checking—Ensuresthatdatathat
5.4.5.8 F3269-RTAS-008—The designer shall define a
passes a set of predefined checks and criteria to increase
threshold beyond which the Safety Monitor activates a Recov-
confidence to a level that the data is valid. An example is the
ery Function that ensures the RTA Output or Larger System
data from a magnetometer represented as a 32-bit unsigned
behavior,orboth,ismaintainedwithinPredefinedBounds. The
integer. A 32-bit unsigned integer can contain a numeric range
overall recovery process includes all activities and their
well beyond 0 to 360; range checking can be performed to
timing, from exit of the region enclosed by the SMTT through
validate acceptable values between 0 degrees to 360 degrees.
re-entry; this may include determination that the trigger
Any values outside this range can be considered bad or invalid
threshold has been exceeded, time to select a recovery function,
data and should be marked as invalid to not cause any
time to initialize the Recovery Function, time to switch to the
unintended system behaviors. Another example is, if data is
Recovery Function, time to perform recovery, and time to
received on a bus from an independent upstream system, a risk
re-enter the region enclosed by the SMTT, when possible.
of data corruption exists if no validity checking is performed.
5.4.5.9 F3269-RTAS-009—The designer shall ensure RTA
Current microcontrollers have built-in communication checks
Component failures are detected and handled.
to help detect failures in protocols. One such criteria is to
5.5 RTA Interfaces:
check for communication errors, that is, invalid message
5.5.1 Assured External Data—InterfacefromLargerSystem frames, incorrect parity, register overflows, and timely periodic
rates.
to the RTA System that contains only assured data.
5.5.2 Unassured External Data—Interface from the Larger 5.6.1.2 Furthermore, multiple redundant data variables and
System to the RTA System that may contain unassured data. their time histories may be considered together (for example,
voting, estimation). Multiple low-quality sensors may be fused
5.5.3 Safety Monitor Inputs—Interface from the Input Man-
to provide higher data quality.
ager to the Safety Monitor that includes the minimum set of
assured data needed to monitor Larger System or Complex 5.6.1.3 The Input Manager may derive or estimate, or both,
Function behavior, or both. parameters from input data to be used by RTAcomponents as
F3269 − 21
additional “sanity check.” These parameters can be used for Larger System may leave the Safe State. This is accomplished
dissimilar and redundant voting schemes providing additional by establishing Predefined Bounds to maintain the Larger
data assurance. System in a Safe State.ASafety Monitor Trigger Threshold is
5.6.2 Requirements: then defined to ensure that the Predefined Bounds are not
5.6.2.1 F3269-IM-001—The IM shall accept as inputs, As- violated by the RTA System. It has the goal of returning the
sured External Data, Unassured External Data and, when RTA System to within the Safety Monitor Trigger Threshold,
monitoring the Complex Function behavior directly, Complex and eventually returning control to the Complex Function, if
Function Outputs. possible.
5.6.2.2 F3269-IM-002—The IM shall perform Data Condi-
5.7.1.2 The Safety Monitor Coverage fully implements the
tioning on all unassured data. defined RTA System Coverage. The Safety Monitor Function
5.6.2.3 F3269-IM-003—The IM shall perform Dynamic
may consist of multiple monitoring functional capabilities.
Consistency Checking on all data. When multiple monitoring subfunctions are required, then a
5.6.2.4 F3269-IM-004—TheIMshalloutputSafetyMonitor
monitor arbitrator (such as a priority selector) is required.
Inputs.
5.7.1.3 Multiple monitoring subfunctions (Fig. 4) may be
5.6.2.5 F3269-IM-005—The IM shall output Recovery
used to meet the necessary Monitor Coverage or achieve the
Functions Inputs.
required level of assurance, or both. The monitors may use
5.6.2.6 F3269-IM-006—The IM shall compute and output
functionally dissimilar methods and input parameters.
data quality attributes for all IM outputs, such as update rate,
5.7.1.4 When more than one monitoring subfunction is
accuracy, precision, and latency.
implementedandpotentialexistsforambiguityinsourceatany
given time, the overall RTA Safety Monitor, an Arbitrator
5.7 Safety Monitor:
function, is necessary to disambiguate conflicting monitor
5.7.1 Description:
subfunction recommendations. This subfunction ensures that
5.7.1.1 The Safety Monitor evaluates Larger System behav-
exactly one RTA Source Selection is made from the available
ior by directly monitoring the Complex Function Output or
RTA Output sources.
Larger System behavior, or both. When directly monitoring
5.7.2 Requirements:
ComplexFunctionbehavior,theSafetyMonitorInputsinclude
Complex Function Output routed through the Input Manager. 5.7.2.1 F3269-SM-001—The Safety Monitor shall accept
The Safety Monitor detects misbehaviors that indicate that the Safety Monitor Inputs from the Input Manager.
FIG. 4 Multiple Monitoring Subfunctions
F3269 − 21
5.7.2.2 F3269-SM-002—The Safety Monitor shall output 5.8.2.2 F3269-RS-002—The RS shall ensure that the only
exactly one RTA Source Selection to the RTA Switch at any sourceoftheRTAOutputisthefunction(ComplexFunctionor
point in time. Recovery Function) selected by the Safety Monitor.
5.7.2.3 F3269-SM-003—The Safety Monitor shall continu- 5.8.2.3 F3269-RS-003—The RS shall ensure that there is
ously evaluate the Larger System behavior by monitoring:
always exactly one source of RTA Output at all times.
(1)the Complex Function Output; or
5.9 Recovery Function:
(2)the RTA Output; or
5.9.1 Description:
(3)Larger System behavior; or
5.9.1.1 A Recovery Function’s purpose is to provide a
(4)any combination of the above.
known output that will maintain safe operation should the
5.7.2.4 F3269-SM-004—The Safety Monitor shall detect
complex function misbehave. Recovery Function objectives
behaviors that indicate that the Larger System will leave the
are to maintain a Safe State and, when possible, return control
Safe State.
to the Complex Function by re-entering the Nominal Region.
5.7.2.5 F3269-SM-005—When the Safety Monitor Trigger
5.9.1.2 A Recovery Function provides an assured, alterna-
Threshold is exceeded, the Safety Monitor shall select and
tive output to the Complex Function Output. When the Safety
activate a suitable Recovery Function to ensure that the
Monitor determines the RTA System has reached the Safety
PredefinedBoundsarenotviolated. It has the goal of returning
MonitorTriggerThreshold,therecoveryfunctionisselected.A
the RTA System to the Nominal Region, and eventually return-
RecoveryFunctionisselectablebytheSafetyMonitorthrough
ing control to the Complex Function, if possible.
the RTA Switch.
5.7.2.6 F3269-SM-006—The Safety Monitor shall support
5.9.2 Best Practices:
the following transitions:
5.9.2.1 Re-enter Nominal Region—The Recovery Function
(1)Complex Function to Recovery Function.
should be designed to return the RTA System to within the
(2)Recovery Function to a different Recovery Function.
SMTT, to allow the Safety Monitor to reactivate the Complex
(3)Recovery Function to the Complex Function.
Function. In some cases, re-entering the SMTT and reactivat-
5.7.2.7 F3269-SM-007—When more than one Recovery
ing the Complex Function may not be possible or desirable.
Function is available, the Safety Monitor shall have a priority
5.9.3 Requirements:
scheme for selecting the appropriate Recovery Function. A
5.9.3.1 F3269-RF-001—The Recovery Function shall ac-
Recovery Function is considered available when the function is
cept only assured data from the Input Manager.
operational and applicable. A priority scheme could be to
5.9.3.2 F3269-RF-002—The Recovery Function shall en-
choose the Recovery Function with the lowest risk outcome.
sure Recovery Function Output is available at time of selec-
5.7.2.8 F3269-SM-008—WhenmorethanoneMonitorSub-
tion.
functionisimplemented,anArbitratorshallbeimplementedto
5.9.3.3 F3269-RF-003—The Recovery Function should in-
selectfrompotentiallyconflictingMonitorSubfunctionrecom-
formtheSafetyMonitorwhentheRecoveryFunctionisunable
mendations.
to provide Recovery Function Output.
5.7.2.9 F3269-SM-009—The Arbitrator shall evaluate each
5.9.3.4 F3269-RF-004—The Recovery Function shall send
Monitor Subfunction recommendation and select the appropri-
Recovery Function Output to the RTA Switch.
ate Recovery Function.
5.9.3.5 F3269-RF-005—TheRecoveryFunctionshallmain-
5.8 RTA Switch:
tain the RTA Output within the Predefined Bounds.
5.8.1 Description:
5.8.1.1 The RTA Switch (RS) is the means to switch the
6. Keywords
source of the RTA Output between functions (Complex
Function, Recovery Function). Typically, the Complex Func- 6.1 adaptive; airworthiness; artificial intelligence; assur-
tion is the source of the RTA Output, but the Safety Monitor ance; automated; autonomous software; autonomy; certifica-
can command an alternate source. tion; complex; control systems; deep neural networks; fuzzy
5.8.2 Requirements: logic; machine learning; online verification; reinforcement
5.8.2.1 F3269-RS-001—The RS shall only accept RTA learning; run-time assurance; safety; safety monitor; security;
Source Selection from the Safety Monitor. software; unmanned aircraft system; validation; verification
F3269 − 21
APPENDIXES
(Nonmandatory Information)
X1. GROUND COLLISION AVOIDANCE SYSTEM (GCAS) AS AN EXAMPLE RTA
INTRODUCTION
The intent of this appendix is to provide context for portions of this practice by discussing an
exampleRTAsystem.Theexamplesystemusedhereisagroundcollisionavoidancesystem(GCAS).
The functional purpose of a GCAS is to reduce the probability of controlled flight into terrain
(CFIT) by avoiding ground impact while the aircraft is under controlled flight. The GCAS monitors
aircraftstateallowingtheaircrafttobecontrolledbythepilot,groundcontroloperator,orautonomous
guidancesystemuntilgroundimpactisimminent.Atthatpoint,theGCAStakescontrolfromthepilot
and gives it to an autopilot to perform a recovery maneuver to avoid the ground. The GCAS returns
control to the pilot when the aircraft velocity vector is clear of near terrain.The concept of operations
for GCAS is to support flight at low altitude over terrain for both a piloted or unmanned aircraft.
An automatic GCAS has been fielded on U.S.Air Force F-16s (1). GCAS has also been adapted
tootheraircraftthatincludegeneralaviationfixedwing,aswellasbothsmallandlargeUAS (2).The
examplesandreferenceimplementationprovideddonotreflectanyonespecificimplementation.(See
Fig. X1.1.)
FIG. X1.1 Reference GCAS Implementation for Piloted Aircraft
The boldface numbers in parentheses refer to the list of references at the end of this standard.
X1.1 Unassured Function Althoughthepilotislicensed, onrareoccasionsthepilotmay
be disoriented, incapacitated, or otherwise incapable of react-
X1.1.1 The unassured function for GCAS is the pilot in the
ing to imminent ground impact.
case of a piloted aircraft. The pilot, being an unassured
function, controls the aircraft the vast majority of the time.
While a pilot license is a form of pedigree, human performance includes
behaviors and failure modes that are not fully addressed in the pedigree.
F3269 − 21
X1.1.2 If the unassured function of a UAS is an automatic sponse time are used to predict one or more escape trajectories
guidance system, and it erroneously guided the UAS toward through 3-D space.The digital terrain map is then interrogated
imminent ground collision or known obstacle collision, the and compared to the trajectories to determine if ground impact
GCAS would override the guidance system to conduct an is imminent (that is, Recovery Function trigger threshold).
avoidance maneuver. When the Safety Monitor Trigger Threshold is reached or
exceeded, an escape maneuver (that is, a Recovery Function)
X1.2 RTA Required Inputs
as determined by the safety monitor is requested and passed to
X1.2.1 To perform the functions of a ground collision the RTA switch. The RTA switch is then set to the recovery
function that corresponds to the requested maneuver. The
avoidance system as an RTA, GCAS requires information to
sense its own state sufficient to support trajectory estimation safetymonitorcontinuestoevaluatetheviabilityofthevarious
escapeoptions.Arequesttoterminatetheescapemaneuverand
andtosensepotentialgroundcollisionthreatsufficienttoavoid
the threat without nuisance activations.These inputs consist of restore control to the pilot is activated when the monitor
determines that the terrain will be cleared by appropriate
information such as aircraft position (latitude, longitude,
altitude),bankangleandgroundtrack,velocities(forexample, margins (for example, Recovery Function complete). This is
calibratedairspeed,trueairspeed,verticalairspeed),winds(for accomplished by commanding the RTA switch back to the
unassured function (the pilot in this example).
slower aircraft), and a digital terrain map of sufficient resolu-
tion and accuracy.
X1.4.2 A trade space exists with regard to establishing the
SMTT.IftheSMTTistooconservative,thepilotwillconsider
X1.3 RTA Input Manager
the system a nuisance as it is impeding the pilot’s ability to
X1.3.1 Managing the inputs used by GCAS is vital to its
accomplish the mission. If the SMTT is not conservative
integrity as an RTA. Parameters that are needed but not
enough, the possibility exists that system errors could lead to
provided by the available sensors are estimated or derived, or
violation of a safety threshold. A more detailed discussion of
both. The integrity/validity of the data coming into the GCAS
this trade space may be found in (3).
aretrackedandreportedtothesafetymonitorandtherecovery
X1.5 Recovery Function
controlfunction.Reportedhealthismonitoredfromtheaird
...