User:IveGoneAway/sandbox/AC 20-189

AC 20-189 and AMC 20-189, both entitled Management of Open Problem Reports (OPRs), are harmonized advisory publications of the Federal Aviation Administration (FAA) and European Union Aviation Safety Agency (EASA), respectively.

These orders cover the importance of reporting and taking care of aircraft design problems known during certification or discovered any time after certification. They suggest that establishing effective problem reporting and correction methods early in development improves confidence in problem management during and after certification.

Applicability
These AC/AMCs describe acceptable means for assisting both design approval applicants and holders of design approvals as they demonstrate compliance with airworthiness regulations for the management of open problem reports. Their intent is to provide consistent problem report guidance across the domains of systems (ARP4754), software (DO-178/ED-12), and hardware (DO-254/ED-80).

Structure
The two publications have nearly identical content; however, the EASA AMC has an appended section entitled "Guidance Material", regarding problem report lifecycle management, open problem report classification, and additional guidance related to one of the problem report classifications. The FAA did not include this appended material in AC 20-189 but has published it separately in AC 00-71, Best Practices for Management of Open Problem Reports (OPRs). For some background to the distinction between "Guidance Material" and "Best Practices", refer to the RTCA DO-178C clarification between guidance and guidelines.

Overview
Businesses developing equipment and system designs for aircraft should record evidence that assures the FAA and EASA that their designs are safe enough to fly in public airspace. Whenever a mistake or hazard is discovered or suspected in such designs, problem reports (PR) may be issued and managed until the problem is completely fixed and the problem report is "closed". Any time between starting a problem report and closing it, the problem report is usually said to be "open". Ideally, all problems are fixed and closed before the FAA or EASA certifies a design, but these authorities give specific considerations to problem reports that have not been closed at the time of certification, designating them as Open Problem Reports (OPR).

By communicating how the FAA and EASA want to see problem reports handled during and after certification, they hope to inform developers how to best prepare and manage their problem report systems for successful and effective certification.

Problems and reporting
Problem reporting is an important activity in the creation of safe equipment and systems for hazardous industries. As a discipline, problem reporting conducts a life cycle of activities; beginning by recording the problem,  hazard in the design of a product or system,

Open problem reports (OPR)
Broadly, problem reports are open from the moment they are started to the moment they are closed. More narrowly, OPR can refer to problem reports that are identified as open at the time of a major milestone or certification review. The intention is for the OPR to remain open past completion of the review to allow work and certification effort to continue before the issues are closed. The FAA is particularly concerned that OPR at the time of certification are adequately managed so that they represent no unacceptable flight hazard.

History
In 1987, the International Organization for Standardization introduced problem reporting to quality system management for the development of new products, through the first edition of their standard ISO 9001, Model for quality assurance in design, development, production, installation, and servicing. From this origin, problem reporting entered aviation software safety considerations through DO-178B:
 * 1992: The RTCA defined airborne software certification objectives for problem reporting in DO-178B.
 * 1998: The SAE established problem reporting and change control for aircraft system certification through publication of ARP4754, Guidelines For Development Of Civil Aircraft and Systems.
 * 2001: The RTCA's Discussion Paper #9 published in DO-248B, Supporting Information for DO-178C and DO-278A, discussed how to proceed through software certification with "Known Software Problems" (soon to be called Open Problem Reports in following publications). The focus was clarification on how to report known problems in the Software Accomplishment Summary (SAS). The discussion cited DO-178B objectives and activities for normal life cycle problem reporting, suggesting that problem reporting in the SAS should come from the continuation of effective life cycle problem reporting.


 * 2001: The Certification Authorities Software Team published 6 page CAST-7 Open Problem Report Management for Certification.
 * clarification of the role of manufacturers and suppliers to managed and communicate known problems at the time of certification and the use of the Software Accomplishment Summary as a means of the communication.
 * proposes classification levels: From "3" where the software is presumed defective because a requirement is wrong, to "1" where there is no evidence of software defect, but some level of assurance of absence of error has been lost (the numerical rank is reversed from later numerical type classifications).
 * added term "defect" (defect is not picked up in the following problem reporting publications)


 * 2010: The FAA's Notice 8110.110 Chapter 2 (3 pages) provided guidance to applicants and certification authorities in management and oversight software problem reporting down through all supplier levels
 * states that the problem reports should be categorized by risk according to potential impact on 5 categories (safety, functionality, performance, operation, or design assurance) and how they will be communicated from suppliers to certification authorities.


 * 2010: In CM-SWCEH-002 Software Aspects of Certification, which is effectively EASA's version of Order 8110.49, EASA makes many recognizable problem reporting provisions, but its guidance is worded and structured quite differently from Notice 8110.110 and Order 8110.49.


 * 2011: Revision 1 of the FAA's Mega Order 8110.49 directly incorporated the content of Notice 8110.110 Chapter 2 paragraph-by-paragraph.


 * 2011: The RTCA's updated Discussion Paper #9 published in DO-248C defines a complete replacement of the prior DP #9 clarifications for problem reporting. The discussion's 4 pages are similar but less-developed compared to 8110.110. However, paper notably introduces the "5 types" classifications of OPR with more details on expectations of reporting each problem report's life cycle. These types are assigned numbers 0-4 broadly following the classifications introduced in Notice 8110.110 above (Type 0 for safety impact, Type 1 for functional defects, Type 2 for non-functional documentation defects, Type 3 for design confidence issues, and Type 4 for issues having "no functional consequence").


 * 2012: DO-178C clarified and reinforced applicant objectives for problem reporting.


 * 2013: Citing Order 8110.49A and CM-SWCEH-002 her book Developing Safety-Critical Software, Leanna Rierson provides pages of recommendations for problem reporting, including the "5 types" classifications of DO-248C DP #9.


 * 2017: The FAA removed all software developer guidance from Order 8110.49. Most of the removed guidance was then covered in greatly reduced form as "Best Practices" in AC 20-115D and AC 00-69. However, the problem reporting guidance was the only removed topic not covered by any new best practice advisory.


 * 2018: Lion Air Flight 610 crashed on 28 October 2018, killing 189 passengers and crew. The root cause was determined to be an Open Problem Report that had not been effectively communicated to the FAA and aircraft operators.

The present FAA/EASA guidelines for problem reporting are now found in these later advisories:
 * Present publications


 * 2020: EASA AMC 20-189 Management of Open Problem Reports (OPRs)
 * 2022: FAA AC 20-189 Management of Open Problem Reports (OPRs)
 * These ACs emphasize both the requirements for suppliers of compliance data and, when the applicant is not the supplier, the requirements for applicants to flow those requirements to suppliers.
 * Problem Reports should have managed life cycles (i.e., recorded, classified, resolved, closed) with extent of assessment, disposition, and reporting influenced by classification according to 'Significant', 'Functional', 'Process', 'Life Cycle Data', or any 'other' tailored classification.


 * 2022: FAA AC 00-71 Best Practices for Management Open Problem Reports
 * The classifications ‘Significant’, ‘Functional’, ‘Process’, and ‘Life Cycle Data’ are correlated to the prior "5 types" classifications of DO-248 DP #9..

"Advice"
4.2 States of PRs/OPRs
 * Recorded – A problem that has been documented using the problem reporting process.
 * Classified – A problem report that has been categorized in accordance with an established classification scheme.
 * Resolved – A problem report that has been corrected or fully mitigated, for which resolution of the problem has been verified but not formally reviewed and confirmed.
 * Closed – A resolved problem report that underwent a formal review and confirmation of an effective resolution of the problem.

4.3 Classification of PRs/OPRs
 * Significant – assessed at the product, system, or equipment level, a PR that has an actual or potential effect on the product, system, or equipment function that may lead to a Catastrophic, Hazardous or Major failure condition, or may affect compliance with the operating rules.
 * Functional – a PR that has an actual or a potential effect on a function at the product, system, or equipment level.
 * Process – a PR that records a process non-compliance or deficiency that cannot result in a potential safety, nor a potential functional, effect.
 * Life Cycle Data – a PR that is linked to a deficiency in a life cycle data item but not linked to a process non-compliance or process deficiency