Qualification of Tools for Airborne Electronic Hardware

185 Pages·2014·3.11 MB·English

DOT/FAA/TC-12/21 Qualification of Tools for Airborne

Federal Aviation Administration Electronic Hardware
William J. Hughes Technical Center
Aviation Research Division
Atlantic City International Airport
New Jersey 08405

June 2014

Final Report

This document is available to the U.S. public
through the National Technical Information
Services (NTIS), Springfield, Virginia 22161.

This document is also available from the
Federal Aviation Administration William J. Hughes
Technical Center at actlibrary.tc.faa.gov.

U.S. Department of Transportation
Federal Aviation Administration
NOTICE

This document is disseminated under the sponsorship of the U.S.
Department of Transportation in the interest of information exchange.
The U.S. Government assumes no liability for the contents or use thereof.
The U.S. Government does not endorse products or manufacturers.
Trade or manufacturers’ names appear herein solely because they are
considered essential to the objective of this report. The findings and
conclusions in this report are those of the author(s) and do not necessarily
represent the views of the funding agency. This document does not
constitute FAA policy. Consult the FAA sponsoring organization listed on
the Technical Documentation page as to its use.

This report is available at the Federal Aviation Administration William J.
Hughes Technical Center’s Full-Text Technical Reports page:
actlibrary.tc.faa.gov in Adobe Acrobat portable document format (PDF).
Technical Report Documentation Page
1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.

DOT/FAA/TC-12/21

4. Title and Subtitle 5. Report Date

QUALIFICATION OF TOOLS FOR AIRBORNE ELECTRONIC HARDWARE June 2014
6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.

Brian Butka1, Andrew J. Kornecki1, and Janusz Zalewski2
9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

1 Embry Riddle Aeronautical University 2 Florida Gulf Coast University
600 S. Clyde Morris Blvd. 10501 FGCU Blvd S 11. Contract or Grant No.

Daytona Beach, FL 32114 Fort Myers, FL 33965

12. Sponsoring Agency Name and Address 13. Type of Report and Period Covered

U.S. Department of Transportation Final Report
Federal Aviation Administration
Aircraft Certification Service—Design, Manufacturing, and Airworthiness
FAA National Headquarters
950 L’Enfant Plaza, S.W.
Washington DC 20024 14. Sponsoring Agency Code

AIR-134

15. Supplementary Notes

The Federal Aviation Administration Aviation Research Division CORs were Charles Kilgore and Emmanuel Papadopoulos.
16. Abstract

The objective of this research was to study the use and qualification of tools used in developing airborne electronic hardware
(AEH) for aircraft. The AEH are custom, microcoded components, or devices used as part of the airborne system. The primary
technologies include Programmable Logic Devices, Field Programmable Gate Arrays, Application-Specific Integrated Circuits,
and similar circuits used as components of programmable electronic hardware.

The avionics standard RTCA DO-254 provides design assurance guidance on project conception, planning, design,
implementation, testing, and supporting processes in the hardware design life cycle. In particular, details are discussed regarding
the processes that must be followed in respective tools’ assessment and qualification. This study seeks to identify and address
potential safety issues in qualifying both hardware design tools and hardware verification tools.

The study was conducted in several steps, which included: literature search; industry survey; identification of primary safety,
performance, and certification concerns; developing a plan for validating these concerns; conducting experiments with the tools;
evaluating the experiments; and producing the final report. The results of this study were aimed at the determination of the major
issues related to using tools that support AEH design and verification and recommendations for addressing these issues in the
assessment and qualification process.

17. Key Words 18. Distribution Statement

Programmable logic, Tools, Qualification, Airborne electronic This document is available to the U.S. public through the
hardware National Technical Information Service (NTIS), Springfield,
Virginia 22161. This document is also available from the
Federal Aviation Administration William J. Hughes Technical
Center at actlibrary.tc.faa.gov.

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price
Unclassified Unclassified 185

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized
TABLE OF CONTENTS
Page

EXECUTIVE SUMMARY x

1. INTRODUCTION 1
1.1 Objectives 2
1.2 Problem Statement 2
1.3 Research Method 3
1.4 Audience 3
1.5 Results 4
1.6 Document Structure 4

2. BACKGROUND 5
2.1 Software and Hardware Relationship 6
2.2 Programmable Logic History 7
2.3 A Typical Airborne Electronic Hardware Development Flow 8
2.4 The AEH Design 10
2.5 Verification of AEH 13
2.6 Simple vs. Complex Electronic Hardware 14
2.7 The AEH Tool Categories 14
2.8 The AEH Tools in DO-254 Framework 17

2.8.1 The DO-254 Design Assurance Guidance 17
2.8.2 The DO-254 Tool Guidance 18

2.9 What Is a Tool? 20
2.10 When Tool Qualification Is Required 20
2.11 Tools’ Disclaimers 22

3. ALTERNATIVES TO TOOL ASSESSMENT AND QUALIFICATION 23
3.1 Independent Assessment of the Tool’s Outputs 23

3.1.1 The Meaning of Independent Assessment 24

3.1.2 Independent Processes at All Phases of the Design 25

3.1.3 What Happens When the Independent Assessment Results Do Not
Agree? 25

3.1.4 Independent Assessment Is a Process, Not an Event 25

iii
3.2 Service History 26

3.2.1 Service History Case Studies 26
3.2.2 Service History Guidance for Hardware 27
3.2.3 Service History for Design Tools 28
3.2.4 Service History vs. the Latest Technology 28
3.2.5 Tool Service History Is Not Sufficient 28
3.2.6 Testing Maturity Model 29

4. DESIGN ASSURANCE 29
4.1 Constrained Random Verification 29
4.2 Observability 30
4.3 Derived Requirements 31

5. SURVEY OF TOOL USERS 32
5.1 Aviation Community Survey 32

5.1.1 Survey Population 32
5.1.2 Multiple Choice Answers 33
5.1.3 Narrative Answers 35

5.2 Semiconductor Industry Viewpoint 35

6. LITERATURE OVERVIEW 37
7. CASE STUDIES 39
8. SAFETY ISSUES 41
8.1 Hardware Design Error Characterization 41
8.2 The FPGA’s Environment 44
8.3 Timing Issues 44

8.3.1 Synchronous Design 45
8.3.2 Synchronous Design—Multiple Clock Domains 45
8.3.3 Asynchronous Designs 46

8.4 Wide Data Buses and Data Pattern-Dependent Errors 46
8.5 Combinational Feedback/Quasi-Digital Circuits 47
8.6 Synthesis Issues—What the Tool Really Built 47

8.6.1 Getting Less Than Expected 48
8.6.2 Getting More Than Expected 48

iv
8.7 Hardware That Is Nonfunctional in Normal Operation 49

8.7.1 Synthesizer Optimizations 49
8.7.2 Gate-Level Verification 49
8.7.3 Adding Test Circuitry 50

8.8 Radiation Effects and FPGA Architectures 50
8.9 Radiation—DO-254 and DO-160 51
8.10 What Circuit Is Being Generated 51
8.11 Unused Inputs and Outputs 52
8.12 Other Considerations 52
8.13 Power Up/Reset Issues 53
8.14 What Can Be Done to Prevent Problems 53
8.15 Design Issues Summary 53

9. FINDINGS AND RECOMMENDATIONS 55
10. REFERENCES 57
11. GLOSSARY OF TERMS 63
APPENDICES

A—Survey Questionnaire
B—Survey Results
C—Test Procedure
D—Annotated Bibliography
E—Hardware Case Study Experiments
F—Evaluation Report for Hardware Design Tools
v
LIST OF FIGURES
Figure Page

1 The AEH Stakeholders 4
2 Hardware and Software Boundary 8
3 A Typical AEH Design and Verification Flow 9
4 Generic Design Flow for the PLD Tool 12
5 The DO-254 Tool Assessment and Qualification Process 21
6 Survey Population—Type of Organization 33
7 Role of Respondents in DO-254 Projects 34
8 Factors Affecting System Safety 40
9 Macroevaluation Model of the Tool Evaluation Process 40
10 Functional Flaws Requiring Design Re-Spins 42
11 Communication Barriers That Can Prevent Clear Design Specifications 43
12 Ring Oscillator 47
13 A TRM With Three Multipliers 48
14 Flip-Flop Replication 49
15 An SEU in an FPGA Using a CMOS Process 50

vi
LIST OF TABLES
Table Page

1 Tool Feature Comparison 16
2 The SEU Mean Time to Error for a Large FPGA in Geosynchronous Orbit 51

vii
LIST OF ABBREVIATIONS, ACRONYMS, AND LABELS
AC Advisory Circular
AEH Airborne electronic hardware
ASIC Application-specific integrated circuit
BNC Bayonet Neill-Concelman connector
CAST Certification Authorities Software Team
CDC Clock domain crossing
CMOS Complementry metal-oxide-semiconductors
COTS Commercial off-the-shelf
CPLD Complex programmable logic device
CPU Central processing unit
CVS Concurrent Version System
DAL Design assurance level
DC Direct current
DER Designated Engineering Representative
DSP Digital signal processing
DUT Device under test
EDA Electronic design automation
FAA Federal Aviation Administration
FFPA Functional failure path analysis
FPGA Field-programmable gate array
FSM Finite state machine
FVI Formal verification interface
GAL Generic array logic
GCLK Global clock
GND Ground
GUI Graphical user interface
HDL Hardware description language
IC Integrated circuit
ILA Integrated Logic Analyzer
I/O Input/Output
IP Intellectual property
JAA Joint Aviation Authority
LED Light-emitting diode
LVTTL Low voltage transistor-transistor logic
MBD Model-based design/development
NASA National Aeronautics and Space Administration
NI National Instruments™
PAL Programmable array logic
PBD Platform-based design
PCB Printed circuit board
PCI Peripheral component interconnect
PLA Programmable logic array
PLB Programmable logic block
viii
PLD Programmable logic device
QA Quality assurance
RF Radio frequency
RTL Register transfer language
SCADE Safety-Critical Application Development Environment
SEFI Single event functional interrupt
SEU Single-event upset
SLD System-level design
SMA Subminiature version A
SoC System-on-chip
SPLD Simple programmable logic device
SRAM Static random access memory
SRPT Synchronous Receptive Process Theory
SSN Simultaneous switching noise
TRM Triple-redundant module
TTL Transistor/transistor logic
UAV Unmanned aerial vehicle
UCF User Constraint File
V&V Validation and verification
VDD Label for IC power supply pin
VHDL Very high-speed integrated circuit HDL
VIL Maximum voltage for low input
VIH Minimum voltage for high input
Vin_dc Voltage input, dc
Vterm Termination voltage

ix