General

A Civil Aviation Authority (CAA) advisory circular (AC) contains guidance and information about standards, practices, and procedures that the Director has found to be an acceptable means of compliance with the associated rules and legislation.

However, the information in the AC does not replace the requirement for participants to comply with their own obligations under the Civil Aviation Rules, the Civil Aviation Act 1990 and other legislation.

An AC reflects the Director’s view on the rules and legislation. It expresses CAA policy on the relevant matter. It is not intended to be definitive. Consideration will be given to other methods of compliance that may be presented to the Director. When new standards, practices, or procedures are found to be acceptable they will be added to the appropriate AC. Should there be any inconsistency between this information and the rules or legislation, the rules and legislation take precedence.

An AC may also include guidance material generally, including guidance on best practice as well as guidance to facilitate compliance with the rule requirements. However, guidance material should not be regarded as an acceptable means of compliance.

An AC may also include technical information that is relevant to the standards or requirements.

Purpose

This AC provides material to assist in managing flight test activities to support the certification of an aviation product or part.

Related Rules

This AC relates to Civil Aviation Rule Part 21 Certification of Products and Parts, Part 19 Transition Rules and Part 91 General Operating and Flight Rules. Specifically, the advisory circular relates to rules 21.35, 21.39, 21.193, 19.405, 91.101 and 91.105.

Change Notice

This is the initial issue of this AC.

Cancellation

There was no previous issue, so there is no cancellation.

Version History

History Log

1.   Introduction

1.1         This AC describes the acceptable means of compliance for rule 21.39 and rule 21.35 in relation to flight tests. It outlines the key interactions with the CAA that should be followed to conduct a successful flight test programme.

1.2         While this AC is aimed at flight testing, the principles herein are equally applicable to ground testing aircraft. It should not be assumed that a lower level of rigor is required for ground testing. Depending on the ground test to be conducted, the level of risk and uncertainty can be just as high as flight testing.

2.   Scope of Flight Test

2.1         Within the context of this AC, flight test relates to flight activities which seek to gather information about an aircraft, its associated equipment or procedures. The information gathered can have a variety of uses, including but not limited to, developing a new or modified aircraft and demonstrating its airworthiness and/or performance.

2.2         The focus of this AC relates to experimental flight test including certification flight test conducted in New Zealand. While evaluation test flying would benefit from the mindset and general process that this document illustrates, the guidance is not aimed at this category of test flying. Detailed definitions of test flying terminology is available in section 4 of AC19-1.

3.   Outline Process

3.1         Figure 1 shows a generic flight test process flow chart within the context of the New Zealand aviation regulatory system. It describes the major steps expected and highlights (in red text) the specific CAA expectations.

3.2         Each of the parts of the flowchart are expanded later in this AC, including the typical key interfaces with the CAA.

3.3         The process and associated detail within this AC form the outline of acceptable flight test procedures required by rule 21.39(b)(2)(ii) for certification flight test.

3.4         It is important to note that the process is generic and may be tailored to suit different projects and organisations. It is recommended that organisations that conduct flight testing on a regular basis generate their own flight test standard operating procedures.

Figure 1: Generic Flight Test Process

4.   Requirement for Test

4.1         Any test flying should be driven by a clear requirement. For development or research flight testing, this may be to validate models and increase understanding. For certification testing, the requirement will be to show that the aircraft complies with applicable airworthiness rules in alignment with rule 21.39 and rule 21.35. The requirement for testing can be for reasons as diverse as a check of the functionality of a simple system new to the aircraft to research of a novel technology. A thorough understanding of the requirement is needed to ensure that the correct data is collected during the test and that no unnecessary risks are taken. The personnel planning, conducting and analysing the test need to understand the what, how and why of each requirement to ensure the test is completed safely and effectively.

4.2         For projects associated with new or amended type certificates (TC), supplemental type certificates (STC) or modifications to an aircraft, the CAA’s prime focus related to those requirements is set out in the project specific certification plan (PSCP). The PSCP is the document which describes the certification basis of a project and thus what needs to be demonstrated in certification flight testing. Further information regarding PSCPs is described in section 1.4 of AC21-8.

4.3         For testing other than for certification, the CAA will be looking to ensure that the requirement for the data outweighs the risk associated with gathering the data. The risk considered is not only related to the personnel and equipment involved in the test, but also risk to other forms of aviation and the general public.

5.   Flight Test Plan

5.1         The flight test plan (may also be known as a flight test schedule) is a key piece of documentation for a safe and effective flight test. The success of the flight test is highly dependent on having a comprehensive and well thought out plan to collect the required data. The document serves a number of purposes including but not limited to:

• (a) providing a method of authorisation for the flight test
• (b) outlining all the facilities or equipment needed for the test such that everything needed is available at the correct time
• (c) agreement of the techniques and methods used to gather the data required
• (d) helping to underpin the issue of a test pilot approval and a special category - experimental airworthiness certificate.

5.2         A comprehensive flight test plan sets the scene for how the test will be conducted and will typically allow the testing to be completed quicker and to a higher standard than if the plan is ill considered.

5.3         In the context of certification, the need for a flight test plan is outlined in AC21-8. AC21-8 details the information that should be included in the flight test plan. The information within AC21-8 is expanded on in this guidance to provide further information when writing a flight test plan.

5.4         The following sections outline the considerations necessary when planning a flight test programme. Not all considerations will be applicable to all projects. The flight test plan should address the relevant considerations but does not need to detail why certain considerations have not been covered. However, when considerations have been omitted, it may be beneficial to document the decision making where it was not obvious.

5.5         The level of detail of the flight test plan should be proportional to the risk of the test conduct and the reliance on the test data, and should be sufficient for an experienced test team not involved in the testing to replicate in the future.

5.6         For organisations that conduct flight test on a regular basis it may be more effective to lodge some of the common elements of test plans to a single document (e.g. a flight test procedures document or an overarching test plan).

5.7         The following sections provide detail on the considerations that may need to be addressed in a flight test plan.

5.7.1      Introduction

5.7.1.1    The introduction should provide all the background information needed for the reader to understand the testing planned. It should include:

• (a) Scope of the test – An outline of the testing to be conducted and, if applicable, how it fits in with the rest of a test programme.
• (b) Description of the system under test – The description should contain enough information that the reader is able to understand what is being tested. It is not necessary to go to a part by part description. In the case of the flight test of a modification, the description should concentrate on the modification rather than the aircraft itself. However key interfaces between the modification and aircraft should be described.
• (c) Objective – Objective(s) to the test should be defined to steer the testing and ensure that data being collected supports the objective of the testing. In the case of certification, the plan should include a list of airworthiness requirements which it is anticipated that the testing will show compliance against. These should flow from the PSCP. The PSCP should also be referenced in the plan.
• (d) Category of test – The test plan should state the experimental flight testing category appropriate to the testing as defined by AC19-1.
• (e) Other relevant background – Other background may be needed to allow the reader to understand the context behind the testing. For example, if the project has been through a number of iterations then it may be necessary to provide an explanation of the previous testing.

5.7.2      Test requirements

5.7.2.1    The particulars of the test should be detailed in the flight test plan including consideration of the following aspects:

• (a) Envelope to be tested – Details of the envelopes which are to be tested should be provided. In the majority of cases, this should align with the target envelope that will be substantiated. These envelopes may include but are not limited to: weight, centre of gravity, temperature, speed, and altitude. There is no need to replicate the complete aircraft flight manual (AFM), however limitations that are pertinent to the testing should be included. Consideration should be given to how to present the envelopes. Often this information is most easily absorbed by the reader in a diagram or similar format. In cases where there is extensive change to the limitations or operating procedures for the aircraft, the draft AFM/supplement should also be provided for the flight test pilot.
• (b) Test conditions – It is recognised that is it impractical to test at every condition, therefore it is important to define what test conditions will be used in the flight test. This may include the configurations of the aircraft (e.g. gear position or modification state) or a particular point in the envelope. By identifying the test conditions, the reader should be able to relate the test coverage with the envelope to be characterised. There are a number of ways of presenting this information. Often tables or diagrams allow for the information to be interpreted easily, e.g. a weight vs centre of gravity plot with areas being tested shaded. Test conditions are also not limited to part of an envelope: any parameter that has a significant effect on the test should be defined. Examples include:
• the position of the sun, as this is of huge importance when assessing the readability of a display
• the mode of a particular system, as this is often critically important to the outcome of a test on that system
• weather minima including wind maximums/direction, outside air temperature.
• (c) Test techniques – The test techniques should be described such that the reader knows how the test will be conducted. References should be made to material used to inform the test techniques used. If the technique is new, novel or unconventional, a detailed description of the technique will be required. FAA ACs or CAA accepted industry consensus standards[1] offer guidance on test techniques depending on the rule part that is being used for certification.

Typically, test techniques fall into two categories:

• Qualitative – Testing the behaviour of the aircraft or a system often requires a subjective judgement to be made. There are numerous examples where airworthiness regulations are subjective in nature. However, in order to add weight to a judgement, it is recommended that quantitative and objective measures are used where possible. Examples include:
  • (i) When evaluating the usability of a system, recording the number of button presses or number of mistakes the tester makes in attempting to carry out a task.
  • (ii) When evaluating workload, using a suitable workload scale including NASA TLX(external link) or Bedford(external link) to add meaning to how high or low the workload was.
  • (iii) When evaluating handling qualities, considering the use of the cooper harper scale to demonstrate how suitable the handling qualities were.

Note: The context of the evaluation is also important. The tester needs to put themselves in the position of the end user of the item under test. This may mean, if the tester is far more experienced than the end user, that allowances need to be made for this. Equally, if the tester has little experience with the test article (likely to be the case if the tester is evaluating a new system) allowances will need to be made for that too.  The tester needs to make allowances for their learning curve and that of the end user, before assessing it formally.

• Quantitative – Quantitative test techniques should be used where possible and are a powerful tool to evaluate an aircraft. Any test should be set up so it can be repeated at a later date by a different test team or regulator. The relevant FAA ACs1 have advice on how to carry out test techniques to generate repeatable data used for certification. Typically, the test team will be trying to vary one parameter while attempting to keep all others constant. The test team should ensure that they know what parameters are of primary importance, and suitable tolerances for the parameters should be captured in the test plan and/or test cards. In the case of techniques that are used for certification, it is very important that the technique is carried out correctly. Misinterpreting the technique will often cause the data gathered to be of little value or, worse still, misleading. The test team need to carefully plan what measurements can be practically taken in the time and environment of the test.
• (d) Progression criteria and predictive data – The test team should consider if a cautious build-up in testing is required. The build-up approach is very dependent on the test (or set of tests) being performed. The test team should work from a known safe part of the envelope and move toward the edges of the target envelope. If possible, predictive data should be calculated and presented before conducting a test flight. Knowing what to expect is a huge advantage for the test team, however uncertain the prediction may be. It allows the team to track progress and may provide an early warning of an unexpected event. The test plan should outline the data available and discuss how that data will be used to monitor the test. Clear criteria should be detailed in order that the test team can terminate a test point or the test program at a suitably safe point. A pause in testing may be required to allow time for suitable data analysis against criteria to be completed. Like test techniques, quantitative progression criteria are preferred as they make decision making less subjective. Examples of progression criteria include: being within 10% of the predictive data, steady cyclic positions leaving greater than 10% control margin, the workload being acceptable before events are injected to increase the demand on the operator, and all stall performance tests being conducted before take-off/landing performance tests.
• (e) Pass/fail criteria – The test plan should set out clear criteria regarding what characteristics and/or functionality are ultimately acceptable for the aircraft. Typically, achieving compliance with the certification basis as per rule 21.39(a)(1) will be one criteria. There may, however, also be performance or customer requirements which need to be met. These criteria are key when deciding when to finish the test program outlined in the plan.
• (f) Go/no go criteria – The critical items or conditions that need to be in place before testing can commence should be identified in the plan. This removes the temptation to undertake the test prematurely or in inappropriate conditions. The criteria may be identified individually as each test or support requirement is outlined, but the criteria should always be unambiguous to prevent conflict concerning whether or not the criteria have been met. As a minimum, rule 21.39(b)(1) requires that the aircraft complies with the structural requirements of the applicable airworthiness design standards and has undergone the necessary ground inspections and tests.

5.7.3      Support requirements

5.7.3.1    All test flying will require some level of support to ensure that it is conducted safely, accurately and effectively. The support requirements will vary in scope depending on the nature of the testing. While planning the supporting activities may appear trivial, a good plan for the necessary support allows the test team to concentrate on the testing itself rather than dealing with emergent activities needed to facilitate the test flying.

• (a) Aircraft requirements – The test aircraft will frequently need to be in a particular configuration for the test. Requirements need to be effectively communicated to the aircraft maintenance team. While some of the required configurations may be very obvious, e.g., the system under test needing to be fitted, some configurations may be less obvious. Consideration should be given to whether items on the aircraft need to be specifically rigged to the worst case tolerances or whether modifications are required to enable the aircraft to better facilitate the test: e.g.,, an easy to locate switch to turn off a system under test or markings to assist the pilot in meeting specific targets. In certification testing, rule 21.39(b)(1)(iii) requires that aircraft conforms to the type design in order to gain certification of the product. Therefore, it is vital to track the conformity of the aircraft configuration so that any differences between the test aircraft and production design can be substantiated.
• (b) Data requirements – After the flight test, the data collected will need to be analysed. Before undertaking the test activity, the analysis techniques should be identified in the test plan such that the data requirements to undertake the analysis are fully understood. Failure to capture the data requirements before the test taking place may require the test to be repeated.
• (c) Instrumentation requirements – Instrumentation is available in a number of forms and levels of complexity. The test team should prepare a list of the data needed, the method by which it will be collected, and the level of accuracy required. It is preferable that the measurements are taken with a calibrated instrumentation system. Such systems allow precise data to be captured easily. Equally, such systems tend to record a whole flight which allows unexpected events to be captured. However, it is also recognised that instrumentation systems can be expensive and beyond the reach of small test programmes. In this case, manual data collection should be undertaken in order to collect quantitative data from the test. The test team need to consider how the manual data will be collected, and the methodology needs to be described to a point where someone outside the test team could set up the instrumentation and get the same result. For example, if measuring stick position with a tape, a suitable datum or reference point to measure from needs to be identified. Cameras can also provide valuable data allowing the test team to replay events from the flight. Calibration of the instrumentation is critical to assure the flight test data quality, and the plan should detail how this will be done. Guidance on the calibration of instrumentation is available in the FAA ACs referenced in section 5.7.2 of this document.
• (d) Documentation requirements – The documentation required for any test will vary according to the nature of the test. For experimental, developmental or certification flight test, the following documents need to be considered:
• Special category - experimental airworthiness certificate. In order to carry out an experimental, developmental or certification flight test, a valid special category - experimental airworthiness certificate is required (rule 21.193) for the test aircraft. Applications for such a certificate must be made using CAA form 24021/06 (Application for a Special Category Airworthiness Certificate). Guidance for the issue of a special category - experimental airworthiness certificate is contained in AC21-3.
• Test pilot approval. In order to carry out an experimental, developmental or certification flight test, a valid test pilot approval is required (rule 19.405). Application for a test pilot approval should be made using CAA form 24019/04. Guidance on the process for the approval is contained within AC19-1.
• Conformity. For a certification flight test, the test aircraft must be suitably conformed to the design data (rule 21.39(b)(1)(ii) and rule 21.35(a)(3)). The method by which the conformity is proven and who is responsible should be outlined in the test plan. Guidance on how to conduct the conformance is contained in AC21-8 para 2.3.
• Flight manual amendments/supplements. Suitable information to safely operate the aircraft and guide the test should be in place, including relevant limitations and procedures. For smaller test programmes, it is acceptable to include this information with the flight test plan. For larger test programmes or where there is a significant change in the operating procedures, a flight manual or supplement should be generated.
• How the testing will be reported should be described such that the reader knows what outputs to expect from the testing. Further information regarding reporting is contained at section 7 of this AC.
• Other documents. Other documents may be required to conduct the flight testing safely and effectively. These may include: specific maintenance documentation for the test article, predictions of the aircraft/system behaviour, manuals of items fitted to the aircraft, weight and balance sheets, bench/ground testing that supports the flight test and/or engineering analysis documents.
• (e) Personnel requirements – The key personnel associated with the test should be identified along with their responsibilities. This should include the minimum crew members needed to undertake the testing and who is required at the briefings. At a minimum, a test team will typically consist of two individuals; the test engineer and test pilot. It is important that the team works together to achieve the test objectives. Frequently, it is beneficial for the designer to interface with the test team early in the project to discuss the testing, thus reducing risk in the project. At a minimum, both the test engineer and test pilot should contribute to and review the test plan to ensure that the testing is methodical, safe and executable.
• (f) Facility/other equipment requirements – It is often the case that flight tests need to be supported by other facilities and/or equipment requirements. For the testing to be a success, it is important that all parties have aligned expectations of testing. Early coordination with stakeholders is recommended to inform the test planning process and to help smooth test conduct. The test plan should list the facilities and other equipment to be used in the testing. It may also be advantageous to document contact details of key individuals associated with the facilities/equipment. The facilities/equipment are highly dependent on the testing to be conducted and, as such, no exhaustive list can be provided. However, some examples are below.
• Airspace/airfield – Coordination for priority over other traffic and checks for local weather conditions.
• Emergency provisions – Prior coordination is beneficial to increase readiness.
• Targets – Sensor testing may require specific visual or radio targets in known conditions to evaluate the sensor’s detection ability.
• Ground support – An enhanced level of ground and maintenance support may be required due to the installation of the test article. This may be complex or may be as simple as ensuring that ground support is ready to efficiently turn around the aircraft for another test.
• Ballast – Testing may require the aircraft to be loaded to a particular weight and/or centre of gravity. If this is the case, consideration should be given to how the ballast will be fitted, how it will be secured and whether it will interfere with other aircraft systems. It is not acceptable to use people purely for ballast reasons as this places those people in a situation of unnecessary risk.
• (g) Weather requirements – Weather may play a critical part in a test programme. Smooth stable air is required for performance assessments and gusty conditions are required for an assessment of handling in turbulent conditions. The test plan should set out the boundaries of the weather conditions that are required by the test. These boundaries should help in the decision making for whether to test or not on a particular day. Equally it may support the decision to deploy to an alternative location to increase the likelihood of encountering the correct weather conditions.
• (h) Non test flying requirements – It is acknowledged that sometimes it is essential to carry out activities that are not directly related to completing a test, but are essential to the test programme, such as post maintenance checks not related to the system under test and ferry flying to support test activity. These activities must support the purposes stipulated by rule 91.105(e) assuming that the aircraft is being operated under a Special Category Airworthiness Certificate. If such activity is required by the test programme, then it should be identified in the test plan. Suitable and reasonable justification for why an activity is essential during the test programme should be documented. These activities should be treated as if they were tests in themselves and hence the risk assessment should consider the hazards of conducting the activity and any additional restrictions which will apply during the activity. In addition, consideration should be given to any effects the activity might have on the flight testing programme. For example, maintenance may require a modification or repair to be embodied, which could have an effect on the validity of flight testing if embodied during a flight test programme.

5.7.4      Risk management

5.7.4.1    The correct management of flight test risk is vital to safely conducting flight test and aligns with Safety Management System principles. The management of risk should be undertaken in accordance with an organisation’s Safety Management System. However, as flight test is typically not routine and often concerns the evaluation of something new, care needs to be taken to ensure that the risks are managed appropriately using the correct tools. An assessment of the planned test activity should be undertaken to identify hazards, the risk of the hazard occurring, and mitigations/recovery procedures that minimise potential effects of the hazard. The hazards identified should focus on those introduced or affected by the system under test, test instrumentation, and/or test methods. Normal hazards associated with operating the aircraft which are not affected by the test do not need to be addressed.

5.7.4.2    The assessment should be specific to the testing planned and be documented in the test plan. Once the assessment has been completed by the test team, it should be reviewed by individuals who are suitably experienced and qualified to provide an independent view of the assessment: this activity is typically called a Safety Review Board.

5.7.4.3    There are various approaches to the assessment. One method is laid out in FAA order 4040.26B.(external link) This order details the internal FAA approach to the management of risk for its staff conducting certification test activity, but the principles within the order may be used as guidance for any flight test programme. CASA has also issued guidance on flight test risk management in CASA AC21-47(external link). This CASA advisory circular provides both guidance on what to consider when looking at flight test risk management and a number of typical examples of high, medium and low tests for general guidance.

5.7.4.4    Both FAA order 4040.26B(external link) and CASA AC21-47(external link) outline a number of strategies that may be employed to mitigate the risks involved in testing, including but not limited to: appropriate work-up (e.g. practice technique in a simulator or another aircraft or preparatory training) to the test, appropriate ‘knock it off’ criteria to be respected, allowance for an adequate build-up approach and use of safety equipment. The mitigations that will be adhered to in the test should be clearly identified in the flight test plan.  

5.7.4.5    If knock it off criteria are identified within the risk mitigations, the criteria set should be unambiguous. The ideal knock it off criteria will be measurable and hence there is no doubt that the criteria has been met. For example, more than a 30 degree bank angle following a wings level stall. Good criteria should reduce the pressure and workload on the test team. 

5.7.5      Schedule

5.7.5.1    Test programmes are notoriously difficult to schedule with a high degree of confidence. Despite this, the test plan should contain a schedule to outline the projected cadence of the programme, such that stakeholders can make appropriate provisions at the correct time. The schedule should include a summary of the test programme, its important milestones, the projected number of flights, their duration and their high level configuration (weight, centre of gravity and modification status).

5.7.5.2    A schedule which is too aggressive may increase the perceived pressure on the test team and cause safety issues to arise.  The schedule should be realistic and fit in with the overall programme schedule. The test plan schedule should make allowances for rest if needed, probability of suitable weather conditions, aircraft unserviceability, regular maintenance and other factors that the test team cannot control.

5.7.6      Test/compliance matrix

5.7.6.1    The value of generating a test/compliance matrix is dependent on the complexity of the test and number of requirements that the test plan is to satisfy. For simple tests with low numbers of requirements/tests, a matrix may not be necessary. However, for most flight test activities, a matrix which links test points and their conditions to specific requirements (or rules in the case of certification tests) is hugely beneficial. The process of generating the matrix will help the test team ensure that all the objectives of the testing are satisfied.

5.7.6.2    For certification testing, this matrix aids alignment between the test team and CAA by making clear unambiguous links between tests planned and how they will satisfy each applicable airworthiness rule. Breaking down the relevant rules into sub elements and addressing them separately provides confidence that the detail in each rule has been appropriately planned for, thus showing compliance with rule 21.39(a)(1). One method to link requirements with tests is shown in Table 1. This example that shows one rule which is partially completed.

Table 1: Example Test/Compliance Matrix

6.   Conduct of the Test

6.1         Test readiness review

6.1.1      After the test plan has been issued and before the test programme begins, it is typical to carry out a test readiness review to ensure that everything is in place for the testing to proceed. The requirements of what needs to be in place should be drawn from the test plan. Consideration should be given to the readiness of documentation, test equipment, test facilities, test article, the test team (e.g. licences, currency and work-up), risk mitigations and any other necessary preparations for the test.

6.2         Test cards

6.2.1      A test card is a mechanism to help the team conduct the test safely and efficiently during the activity itself. Typically, these take the form of a piece of paper with notes regarding the test conditions, test technique, data to be collected and any other important information (e.g. ‘knock it off’ criteria or relevant limitations). Test cards are produced to break down individual test points from the plan into a more digestible form that can be carried on the aircraft. The test cards can vary hugely in detail and complexity: this will depend on the task and the test team’s preferences.

6.3         Briefing

6.3.1      Before each set of test points (e.g. a test flight) the personnel involved in the test should brief together. All personnel with an active role in the test should be present during the brief. Following the brief, all attendees should be aligned and have a clear understanding of what the test is, their role within it, emergency procedures, and methods for communicating during the test. FAA order 4040.26B(external link) Appendix B has a guide of what should be covered in a briefing.

6.4         Test/flight test

6.4.1    During the test itself, the team should only conduct activities that are within the scope of the test plan and that have been briefed i.e. Plan the flight and fly the plan. Test teams should not conduct any testing which has not been briefed, even if the testing goes better than expected. It is important not to rush the test and to carry it out appropriately. Mistakes during the test mean that, at best the test point will have to be re-flown, at worst the safety of the test flight may be under threat. Knowing what to expect from each test is important as it allows the test team to recognise poor behaviour and make informed decisions about whether continuing the test is appropriate. If unexpected behaviour is observed, the team must not be afraid to discontinue the testing. The team should maintain a critical watch on the test article and the techniques being used. The techniques need to be flown in accordance with the tolerances in the test plan (e.g. within 2 knots of the target airspeed). It is also important for the test team to maintain an open mind. If the test team is having difficulty completing the test, it could be that the test article is at fault. Remember the objective of the test is not to evaluate the test team, it is to evaluate the test article.

6.5         Debrief

6.5.1    Immediately following the test flight, the test team should gather to debrief the testing conducted. The team should consider what went well, what went badly and what data was collected. Any anomalies should be recorded whilst they are fresh in the test team’s minds. The debrief should be reflective in nature, but also inform what needs to happen next. Similar to the brief, all attendees should have a common understanding of the debrief and hence the test outcome. The key points and issues from the test should be agreed and recorded. Again, FAA order 4040.26B(external link) Appendix B contains a guide of what should be covered in a debrief.

7.   Reporting

7.1         The final phase of the test process is reporting. A variety of reports may be produced to document the testing carried out. The goal should be that a complete record of the test activity is maintained. Each organisation will have its own processes for reporting, but ultimately the documentation for the test should allow the test to be re-flown and obtain the same result, i.e. the tests should be repeatable. Typically, reporting falls into two categories: those reports which are completed immediately after the test and those reports which are produced to satisfy the test requirements. For smaller projects, it may be appropriate to go straight to compliance reporting. If this is the case, the report should combine the requirements of both the immediate reporting and compliance reporting.

• (a) Immediate reporting – These reports are completed as soon as possible after the test flight. They should include or make reference to all the data collected during the test. Typically, this is also where the test team will document their qualitative comments. The emphasis of the report should be to gather as much information as possible, not to analyse the data. Deficiencies found in the test article should be documented. A deficiency is any issue with the test article that is noted, but it is important to bear in mind that a deficiency doesn’t necessarily mean that the test article is unsuitable, non-compliant or unsafe.
• (b) Compliance reporting – These reports collect the information from the reports generated immediately after the test and analyse the data presented within them. Such reports are typically aimed at substantiating the test article’s performance and demonstrating whether or not it met requirements. For certification testing, ultimately this type of report will provide evidence that verifies that design change meets applicable airworthiness rules as per rule 21.39(a)(1) and rule 21.35(a)(1). AC21-8 requires the test pilot and any nominated test witnesses to sign flight test reports before they are submitted to the CAA.

8.   CAA Flight Test Audit

8.1         For TC or STC projects, the CAA may elect to perform a flight test audit in accordance with AC21-8(external link). The audit is typically flown after the applicant completes their testing and is satisfied that the design is compliant. However, it is acceptable to complete it during the applicant test programme where there is a good reason to do so, e.g., if there is a temporary piece of instrumentation/equipment installed for the test.

8.2         Should the flight test audit be required, the CAA will typically verify a cross-section of test points which the applicant has made a statement of compliance against. The plan for the flight test audit will be agreed with the applicant.

9.   Useful Links

9.1         The following links are provided to more information and guidance:

• (a) Australian Civil Aviation Safety Authority flight test and evaluation website(external link) provides a number of documents that may be useful in planning a flight test:
• CASA AC21-47 – Flight Test Safety(external link) provides information on how to plan safe flight tests, resources that may be needed for flight test, risk management and operational considerations.
• (b) EASA FTOM guidance(external link) - Useful for organisations who conduct flight test on a regular basis and would like to develop organisation specific procedures for flight test.
• (c) Flight Test Safety Committee(external link) – provides a large amount of information relating to flight test safety includes videos from previous conferences and recommended practices.
• (d) Society of Flight Test Engineers(external link) - a fraternity of engineers, whose principal professional interest is the flight testing of aerospace vehicles.
• (e) Society of Experimental Test Pilots(external link) – aims to be the recognized world leader in promoting safety, communication and education in the design & flight test of aerospace vehicles and their related systems.

[1] Guidance documents may include:

• Part 23 aircraft - AC 23-8C Flight Test Guide for Certification of Part 23                 Airplanes(external link) or CAA accepted industry consensus standards e.g. ASTM(external link)
• Part 25 aircraft - AC 25-7D Flight Test Guide for the Certification of Transport                 Category Airplanes(external link)
• Part 27 aircraft - AC 27-1B Certification of Normal Category Rotorcraft(external link)
• Part 29 aircraft - AC 29-2C Certification of Transport Category Rotorcraft(external link)
  • Other more focused ACs are also available, on topics such as minimum flight crew/workload, icing and flutter, from the FAA website(external link).