DOT/FAA/TC-xx/xx Qualification of Tools for Airborne

184 Pages·2013·3.53 MB·English

NOT FAA POLICY OR GUIDANCE
LIMITED RELEASE DOCUMENT
24 SEPTEMBER 2013
DOT/FAA/TC-xx/xx Qualification of Tools for Airborne

Federal Aviation Administration Electronic Hardware
William J. Hughes Technical Center
Aviation Research Division

Atlantic City International Airport
New Jersey 08405

DISCLAIMER
This draft document is being made available as a “Limited
Release” document by the FAA Software and Digital
Systems (SDS) Program and does not constitute FAA
policy or guidance. This document is being distributed by
permission by the Contracting Officer’s Representative
(COR). The research information in this document
represents only the viewpoint of its subject matter expert
authors.

The FAA is concerned that its research is not released to
the public before full editorial review is completed.
However, a Limited Release distribution does allow
exchange of research knowledge in a way that will benefit
the parties receiving the documentation and, at the same
time, not damage perceptions about the quality of FAA
research.

NOT FAA POLICY OR GUIDANCE
LIMITE D RELEASE DOCUMENT

24 SEPTEMBER 2013
NOTICE

This document is disseminated under the sponsorship of the U.S.
Department of Transportation in the interest of information exchange.
The United States Government assumes no liability for the contents or
use thereof. The United States Government does not endorse products
or manufacturers. Trade or manufacturer's names appear herein solely
because they are considered essential to the objective of this report. The
findings and conclusions in this report are those of the author(s) and do
not necessarily represent the views of the funding agency. This document
does not constitute FAA policy. Consult the FAA sponsoring organization
listed on the Technical Documentation page as to its use.
Technical Report Documentation Page
1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.

DOT/FAA/TC-xx/xx

4. Title and Subtitle 5. Report Date

QUALIFICATION OF TOOLS FOR AIRBORNE ELECTRONIC HARDWARE May 2012
6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.

Brian Butka1, Andrew J. Kornecki1, and Janusz Zalewski2
9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

1 Embry Riddle Aeronautical University 2 Florida Gulf Coast University
600 S. Clyde Morris Blvd. 10501 FGCU Blvd S 11. Contract or Grant No.

Daytona Beach, FL 32114 Fort Myers, Florida 33965

12. Sponsoring Agency Name and Address 13. Type of Report and Period Covered

U.S. Department of Transportation Final Report
Federal Aviation Administration
William J. Hughes Technical Center
Aviation Research Division
Advanced Aircraft Systems & Avionics
Atlantic City International Airport, NJ 08405 14. Sponsoring Agency Code

AIR-120

15. Supplementary Notes

The Federal Aviation Administration Aviation Research Division COTRs were Charles Kilgore and Emmanuel Papadopoulos.
16. Abstract

The objective of this research was to study the use and qualification of tools used in developing airborne electronic hardware
(AEH) for aircraft. The AEH are custom, microcoded components or devices used as part of the airborne system. The primary
technologies include Programmable Logic Devices, Field Programmable Gate Arrays, Application-Specific Integrated Circuits,
and similar circuits used as components of programmable electronic hardware.

The avionics standard RTCA DO-254 provides design assurance guidance on project conception, planning, design,
implementation, testing, and supporting processes in the hardware design life cycle. In particular, details on the processes that
must be followed in respective tools’ assessment and qualification are discussed. This study seeks to identify and address
potential safety issues in qualifying both hardware design tools and hardware verification tools.

The study was conducted in several steps, including literature search, industry survey, identification of primary safety,
performance, and certification concerns, developing a plan for validating these concerns, conducting experiments with the tools,
evaluating the experiments, and producing the final report. The results of this study were aimed at the determination of the major
issues related to using tools that support AEH design and verification and recommendations for addressing these issues in the
assessment and qualification process.

17. Key Words 18. Distribution Statement
This document is available to the U.S. public through the
Programmable Logic, Tools, Qualification, Airborne
National Technical Information Service (NTIS), Springfield,
electronic hardware
Virginia 22161. This document is also available from the
Federal Aviation Administration William J. Hughes Technical
Center at actlibrary.tc.faa.gov.

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price
Unclassified Unclassified 183
Form DOT F 1700.7 (8-72) Reproduction of completed page authorized
TABLE OF CONTENTS
Page

EXECUTIVE SUMMARY x

1. INTRODUCTION 1
1.1 Objectives 2
1.2 Problem Statement 2
1.3 Research Method 3
1.4 Audience 3
1.5 Results 4
1.6 Document Structure 4

2. BACKGROUND 5
2.1 Software and Hardware Relationship 6
2.2 Programmable Logic History 7
2.3 A Typical Airborne Electronic Hardware Development Flow 8
2.4 The Airborne Electronic Hardware Design 10
2.5 Verification of Airborne Electronic Hardware 13
2.6 Simple vs Complex Electronic Hardware 14
2.7 Airborne Electronic Hardware Tool Categories 14
2.8 The Airborne Electronic Hardware Tools in DO-254 Framework 16

2.8.1 The DO-254 Design Assurance Guidance 17
2.8.2 The DO-254 Tool Guidance 18

2.9 What is a Tool? 19
2.10 When is Tool Qualification Required? 20
2.11 Tools Disclaimers 21

3. ALTERNATIVES TO TOOL ASSESSMENT AND QUALIFICATION 22
3.1 Independent Assessment of the Tool’s Outputs 23

3.1.1 What Does Independent Assessment Mean? 24
3.1.2 Independent Processes at all Phases of the Design 24
3.1.3 What Happens if the Independent Assessment Results do not Agree? 25
3.1.4 Independent Assessment is a Process, not an Event 25

3.2 Service History 26

3.2.1 Service History Case Studies 26
3.2.2 Service History Guidance for Hardware 27
iii
3.2.3 Service History for Design Tools 28
3.2.4 Service History Versus the Latest Technology 28
3.2.5 Tool Service History is not Sufficient 28
3.2.6 Testing Maturity Model 29

4. DESIGN ASSURANCE 29
4.1 Constrained Random Verification 29
4.2 Observability 30
4.3 Derived Requirements 31

5. SURVEY OF TOOL USERS 32
5.1 Aviation Community Survey 32

5.1.1 Survey Population 32
5.1.2 Multiple Choice Answers 33
5.1.3 Narrative Answers 35

5.2 Semiconductor Industry Viewpoint 35

6. LITERATURE OVERVIEW 37
7. CASE STUDIES 39
8. SAFETY ISSUES 41
8.1 Hardware Design Error Characterization 41
8.2 The FPGA’s Environment 44
8.3 Timing Issues 44

8.3.1 Synchronous Design 45
8.3.2 Synchronous Design—Multiple Clock Domains 45
8.3.3 Asynchronous Designs 46

8.4 Wide Data Busses and Data Pattern Dependent Errors 46
8.5 Combinational Feedback/Quasi-Digital Circuits 47
8.6 Synthesis Issues—What did the Tool Really Build? 48

8.6.1 Getting Less Than Expected 48
8.6.2 Getting More Than Expected 49

iv
8.7 Hardware That is Nonfunctional in Normal Operation 49

8.7.1 Synthesizer Optimizations 49
8.7.2 Gate-Level Verification 50
8.7.3 Adding Test Circuitry 50

8.8 Radiation Effects and FPGA Architectures 50
8.9 Radiation—DO-254 and DO-160 51
8.10 What Circuit is Being Generated? 51
8.11 Unused Inputs and Outputs 52
8.12 Other Considerations 52
8.13 Power up/Reset Issues 53
8.14 What can be Done to Prevent Problems? 53
8.15 Design Issues Summary 53

9. FINDINGS AND RECOMMENDATIONS 55
10. REFERENCES 57
11. GLOSSARY OF TERMS 63
APPENDICES

A—Survey Questionnaire
B—Survey Results
C—Test Procedure
D—Annotated Bibliography
E—Hardware Case Study Experiments
F—Evaluation Report for Hardware Design Tools
v
LIST OF FIGURES
Figure Page

1 The AEH Stakeholders 4
2 Hardware and Software Boundary 8
3 A Typical AEH Design and Verification Flow 9
4 Generic Design Flow for the PLD Tool 12
5 The DO-254 Tool Assessment and Qualification Process 21
6 Survey Population—Type of Organization 33
7 Role of Respondents in DO-254 Projects 34
8 Factors Affecting System Safety 40
9 Macroevaluation Model of the Tool Evaluation Process 40
10 Functional Flaws Requiring Design Re-Spins 42
11 Communication Barriers That can Prevent Clear Design Specifications 43
12 Ring Oscillator 47
13 A TRM With Three Multipliers 48
14 Flip-Flop Replication 49
15 An SEU in an FPGA Using a CMOS Process 50

vi
LIST OF TABLES
Table Page

1 Tool Feature Comparison 16
2 The SEU Mean Time to Error for a Large FPGA in Geosynchronous Orbit 51

vii
LIST OF ACRONYMS
AC Advisory Circular
AEH Airborne electronic hardware
ASC A Stream Compiler
ASIC Application-specific integrated circuit
BNC Bayonet Neill-Concelman connector
CAST Certification Authorities Software Team
CDC Clock domain crossing
CMOS Complementry metal-oxide-semiconductors
COTS Commercial off-the-shelf
CPLD Complex programmable logic device
CPU Central processing unit
CVS Concurrent Version System
DAL Design assurance level
DC Direct current
Def Stan Defence Standard (UK)
DER Designated Engineering Representative
DSP Digital signal processing
DUT Device under test
EDA Electronic design automation
FAA Federal Aviation Administration
FFPA Functional failure path analysis
FPGA Field-programmable gate array
FSM Finite state machine
FVI Formal verification interface
GAL Generic array logic
GCLK Global clock resource
GND Ground
GUI Graphical user interface
HDL Hardware description language
IC Integrated circuit
ILA Internal Logic Analyzer
I/O Input/Output
IP Intellectual property
JAA Joint Aviation Authority
LED Light-emitting diode
MBD Model-based design/development
NASA National Aeronautics and Space Administration
NI National Instruments
PAL Programmable array logic
PBD Platform-based design
PCB Printed circuit board (also circuit board assemblies)
PCI Peripheral component interconnect
PLA Programmable logic array
viii