By CAA Flight Examiner Guy Brooking

It’s no secret that pilots who’ve had to make multiple attempts to pass flight tests, later feature disproportionally in accidents¹.

The issue

The connection between multiple attempts to pass a flight test and experiencing an accident, has been identified as a global phenomenon. In New Zealand, it applies not only to checks in an airline, but also to the initial issue flight test for a pilot licence or rating.

An ASPEQ review of flight test pass rates, carried out on behalf of the CAA, reveals an increasing number of candidates failing multiple flight tests, and in particular, that for the CPL.

Some of the worst cases have tried six times to pass their test. Feedback from examiners reveals that remedial training between test attempts has lacked focus or has been entirely absent.

This is particularly concerning with professional pilot licence tests.

While it’s not unusual for a candidate to fail their first test at a professional licensing level, a second failed attempt needs to be viewed by the training organisation, the instructor, and the candidate themselves, as serious.

Changes to AC61-1

To address these concerns, Advisory Circular AC61-1 Pilot licences and ratings – general has been amended. Where a candidate has failed a flight test, remedial training needs to take place. This must be recorded in the logbook and will be checked by the flight examiner.

In more serious cases, where a candidate has failed a flight test three times, the training organisation should develop a remedial training plan and submit it to the CAA before proceeding.

If the CAA is satisfied the plan addresses the cause of the flight test failures, the training organisation can go ahead with arranging the pilot’s next attempt.

Planning remedial training

Due to the serious nature of multiple failures in flight tests, the CAA expects that remedial training – addressing a third failed flight test – is planned, and submitted to us (email licensing@caa.govt.nz) by the CFI, training manager, or a senior instructor.

The first step in planning remedial training is to gather information from various sources. When a candidate is being debriefed by a flight examiner after a test, the instructor should be present. Where the test has been unsuccessful, the CFI or a senior instructor should also be present.

Relying on the candidate to explain what was said by the examiner when the candidate is probably in an emotional state, will likely result in missing key information relevant to subsequent training.

So talk with the candidate, once they’ve had time to calm down, and speak with other instructors who’ve flown with them.

Training records can be particularly helpful in tracking the progress of the student, and where they may have had difficulties in the past. The aim of gathering this information is to look for any patterns or trends through the training and flight tests that might point to an underlying cause.

A simple root cause analysis tool such as the Five Whys can also provide good insight. The tool simply repeats the question, ‘Why?’ at least five times, by which point you’re likely to have reached the root cause of the problem.

From here, a remedial training plan can be built, beginning with the root cause, and working upwards to the specific issues causing the flight tests to be failed.

Send this plan to us, clearly showing how many flights are planned and what the objectives are of each of the flights. Include copies of all previously failed flight tests.

Plan from the root cause

A root cause analysis tool such as the Five Whys can provide good insight into the root cause of the problem.

A remedial training plan can be built, beginning with the root cause, and working upwards to the specific issues causing the flight tests to be failed.

Avoiding failure in the first place

Training organisations and instructors should ensure training is closely managed as a pilot approaches a flight test, and they should deal with any concerns arising during this time.

Many recent tests have been unsuccessful due to poor groundwork or eligibility problems identified in the logbook. Groundwork should be practised multiple times as there’s no cost to the student, and instructors must review practice groundwork to ensure students are competent in their flight planning and preparation.

Logbooks need to be ‘forensically’ checked by the organisation, to prevent a wasted test result due to eligibility errors.

We recommend a complete mock test is carried out by the CFI or senior instructor to provide the candidate with an experience close to the real thing. This mock test should include all the paperwork, logbook review, and oral questions to be expected on a real test.

Instructors conducting these mock tests should avoid providing tips or instruction, as well as avoid giving any type of affirmation during the mock test. This will help the candidate know what to expect, and it would provide a good assessment of their performance under test conditions.

Before the student goes up for a test, make certain they’re very current, and if there are any doubts as to their preparedness, delay the test.

With the change to the advisory circular, together with advice provided in this article, we expect that the number of multiple flight test failures will reduce, and that failures at the third attempt will become very rare.

A reduction in the frequency of attempts at professional-level licence and rating tests will provide a better barrier inside the ‘Swiss cheese model of accident causation’², and it’s hoped it will break at least one link in the chain to a potential accident.

Footnotes

¹ For instance, the captain of Colgan Air 3407, which crashed in the US in 2009, was identified as having a history of multiple failures of flight tests and checks.

Source: NTSB/AAR-10/01(external link)

² SKYbrary.aero(external link) and search on ‘Swiss cheese model of accident causation’.

Photo: iStock.com/AzmanL