Friday, November 26, 2010

Blame Game

When I heard that AOPA was researching why 70 to 80% of people who start flight training end up quitting and that they were to release the results at their trade show in Long Beach, I wondered what was up. At the heart of this debate are two basic claims: The pilot population is shrinking and 70 to 80% of those who start flight training end up dropping out without ever earning a pilot's certificate. An additional claim is that an 80% drop-out rate is unreasonable, but it's not clear that any of these claims are supported by the new "research." Still a public debate of these issues is reasonable and since I wasn't in Long Beach, I'll observe that it seems AOPA has an agenda and they went in search of evidence to support that agenda.

A few weeks before pilots began gathering for the AOPA Summit, press releases informed us of the high drop out rate for student pilots. Those who know me heard my prediction: The AOPA study would blame the high drop out rate on shoddy instruction offered by inept and unmotivated flight instructors because ... wait for it ... CFIs are easy targets. Largely powerless, underpaid, and seldom recognized, being a professional flight instructor isn't a career path for the faint-of-heart.

In point of fact, shoddy flight training is exactly how some people have summarized what was presented in Long Beach. I won't go into the issues of time-building CFIs and pilots who just don't have the interest or ability to be effective instructors, but if you watch the entire 30 minute on-line video of the AOPA presentation you'll hear that flight instructors (and the conditions under which they work) were only two of the contributing factors identified in these opinion polls. Paradoxically, the opinion polls (which is what this research really is) indicate that people seem to think that:
The flight training industry is, in fact, well aligned; doing the important things well.
Furthermore, the industry is reportedly providing good value, good instructor support, effective instruction, and good information sharing. One thing that seems to have been glossed over is that the poll revealed that 40% of people who actually solo and become student pilots never continue to earn a pilot certificate. The 70 to 80% failure rate that keeps being reported includes those who drop out before soloing (never become official student pilots) as well as those that never earn a pilot certificate.

I'm not an expert on research surveys nor on the techniques used to gather such data, but if we want to truly understand the state of flight training there seems to be a bunch of useful data that is missing. Early in the presentation, Mark Benson, chairman of an opinion research firm called APCO Insight, explained that they didn't want to do a conventional customer service survey into the flight training problem. Unfortunately, the polling methods don't seem to have been published and so we don't know how their procedures might compare with, say, this NASA study.

Some questions I'd like to see answered include the following. What percentage of private pilots pass their check ride on the first attempt? How many hours of flight time does the average pilot have when they take their check ride and how are those hours distributed across the age range of the pilots? How many times on average does a student pilot change instructors and was the change voluntary or involuntary? How many pilots begin flight training with no intention of ever soloing or earning a certificate? How many pilots who hold a current medical certificate are flying at least 25 hour a year? How many flight instructors recommend three or more candidates per year? How many instructors train under 14 CFR part 61 versus part 141? In short, some measurable evidence about the overall effectiveness and health of flight training rather than just soliciting the opinions and perceptions of those trained by said instructors.

Back in 2008, I offered some ideas on the shrinking pilot population along with some modest suggestions on making the process of learning to fly a bit easier based on thousands of hours of dual instruction I have given to all types of pilots. I just took for granted the assertion that the pilot population was indeed shrinking, but FAA data covering the years from 1999 to 2009 seems to show a stable pilot population. This is remarkable considering the unemployment rate of the last few years. Could it be that AOPA is assuming a premise that isn't supported by facts?

Chart from FAA data on pilots, 1999 through 2009

When considering whether or not a 70-80% drop-out rate for students is reasonable, we have to consider the skills a person must acquire to pass a check ride. Based on my teaching experience, I'd estimate that about 20% of the population is simply not equipped to be pilot-in-command. First are medical certification issues that preclude someone from getting a third class medical certificate, though these folks may still be able to fly under sport pilot rules (don't get me started on sport pilot and aircraft regulations).

Then there more complicated issues. Flying involves high workload moments that require a combination of skills involving language, listening, spatial awareness, visualization, motor planning, prioritizing, and multi-tasking. Flying an airplane can be a lot like riding a mountain bike while simultaneously playing chess: Most people can do a subset of these tasks, but fewer people are able to do all of them.

Consider what is sometimes called "reaction to flight." A person may be afraid of flying or they may become motion sick or anxious. No matter how hard they try, these may be insurmountable problems. This doesn't make them bad or defective people, it just means they aren't cut out to be pilots. Heck, as a young adult I dreamed of being another Adolph Herseth, but wishing doesn't make it so. Occasionally a flight instructor has to sit down with a student and have the difficult conversation. It's not something that I look forward to, but it is an important part of the job.

Without the basic skills and abilities, there is no amount of training, simulator experience, advanced avionics, customer service, instructor charisma, or good will that will keep these people from dropping out or get them through a check ride. This gets to the crux of the problem with AOPA's initiative: A better process, better customer focus, and happier customers will not guarantee that more people will end up earning a pilot certificate. More money may end up being spent, but that's another goal that is not necessarily aligned with helping GA grown. And as one seasoned instructor I know quipped "General aviation doesn't need any more substandard pilots."

It seems reasonable to assume that attempts to increase the pilot population would likely increase AOPA's membership and perhaps that is one of their goals. Frankly, some if this "research" seems to border on truthiness.  Don't get me wrong, AOPA has done some great things for GA and the Air Safety Foundation is probably the most valuable resource available to pilots and instructors. While AOPA works on their initiatives, dedicated flight instructors will continue to do the hard work on the front lines, giving instruction, helping students solve problems, helping them succeed, endorsing their logbooks and, occasionally, having the difficult conversations that must be had.

Tuesday, November 23, 2010

A Few Photos ...

I continue to be impressed with the iPhone4's built-in camera. I mean, it's not a Leica M9, but it's a heck of a lot cheaper and pretty simple to use. Plus, I always seem to have it with me. Some of these photos capture a bit of the joy of being in flight, others have sentimental value, and others are simply ... utilitarian.

Enjoy ...


Between Layers
A beautiful sight on a rare, solo flight ...

More layers ...

Odd shaft of light, near Danville

Strong pressure gradient, near Mt. Diablo

Who says GPS never fails?
Where did all the satellites go?
All better a few minutes later.
New Commercial Multi Pilot

Sunset over SF Bay

Somewhere on the SCK RNAV 11L approach
Parade of Clouds

Friday, November 19, 2010

The Problem with Checklists

Having finished reading The Checklist Manifesto, which deals with how checklist procedures can and have improved the delivery of medical care, I found some thoughts on aviation checklist procedures bubbled to the surface. The author, Atul Gawande, credits aviation as the inspiration for implementing checklist procedures in medicine. While there's no doubt that checklists help pilots complete complex procedures, the mere existence of a checklist doesn't immunize us from disaster. Pilot and aircrew performance is tightly linked to the quality and appropriateness of the checklist as well as checklist discipline. And let's not sidestep the whole issue of having to face a situation for which no checklist exists.

Appropriate and Useful

A fellow freight pilot once observed that there are just two kinds of mistakes that pilots can make: Those that embarrass us and those that can damage the airplane and/or kill us. An airline pilot friend once confided that he realized he could make errors in the cockpit at any time. His goal was to uncover any errors and correct them before they became a safety issue. For checklists to be an effective first line of defense against fatal mistakes, they must contain the crucial tasks and actions that apply to the situation at hand. Once a good checklist has been made available, pilots have to follow it. Here's a video a horrific accident during the test flight of a turbine conversion for the Caribou. The crew apparately neglected to remove the control lock, with deadly results.




One of Gawande's central themes is that the development of useful and appropriate checklists is aided by organizations that exhibit teamwork, continuous enhancement, and decentralized control. Gawande cites a study where a checklist was developed for the medical procedure of inserting central line. The steps were simple but it turns out that some of the crucial ones were often skipped, resulting in a high rate of infection, complications, and even patient death. When a checklist was developed for central lines and adhered to, the rate of infection dropped dramatically, survival rates improved and (here's what got a lot of attention) hospitals saved a ton of money. Interestingly, it was nurses who often reminded doctors when they were about to make a mistake, like forgetting to don a surgical mask or use a sterile drape.

The sort of cooperation being suggested by Gawande, where decision making, process improvement and double-checks are performed by people on the front lines, is not always what happens in aviation. Top-down organizational structure is often the rule in aviation, with the FAA being at the top of the heap. When I flew freight, I was astonished to learn that the checklists for our aircrafts' optional equipment weren't included in the regular manufacturer's checklist. Checks that needed to be done every day were strewn throughout the Supplements section of the Approved Aircraft Flight Manual. Creating a company checklist seemed like the answer, but that would have required a lot of time, effort and money since the checklist would need to be ... wait for it ... approved by the FAA. So there were procedures in the manufacturer's checklist that were incorrect or missing and the FAA's regulations (the ones supposed to ensure safety) condoned an environment where required equipment checks were easily forgotten or skipped altogether. At least this gave us something to talk about during recurrent training.

Gawande's glowing view of aviation checklists aside, I'd be remiss if I didn't point out that many aircraft manufacturer checklists are woefully inadequate, incomplete, and in some cases they even contain incorrect information. One manufacturer's After Landing checklist for a late-model aircraft in which I instruct contains just one task: FLAPS UP. Don't mistake this simplicity for elegance, because in point of fact it is inadequate. Checklist procedures continue to be spread throughout Approved Aircraft Flight Manuals, due to the manner in which aircraft and their components are certificated, and this may lead a pilot to incorrectly conclude that all of the manufacturer's checklist content is irrelevant. 

The good news here is that student pilots (and certificated pilots, too) flying under 14 CFR parts 61 and 91 can develop their own checklists. Just ensure that whatever checklists you develop contain, at their core, the manufacturer's checklist items. Creating your own checklist can be a great learning experience, but borrow an idea from Gawande's book and have some other pilots and instructors check your work for accuracy and completeness.

Familiarity Breeds ... Complacency

Assuming you have correct and complete checklists, there is another issue to address. Do the same routine countless times and you'll find you tend to skip using the checklist, do the tasks from memory, or use a flow check or mnemonic. Variety is the spice of life and I think it might even be the key to safety. There's emerging research that indicates what we have always suspected; endless routine is booooring and it can actually keep our brains from performing well.

On the ground, I still recommend using the checklist as a do-list, but that doesn't mean you can't have a passenger participate by reading the checklist to you. In the air you can mix things up by alternating between the checklist and a flow check or a mnemonic backed up with the checklist. Mix up your procedural routine and you may find checklist procedures to be less onerous, just be sure you don't skip any steps in the process.

I'm a big fan of Foreflight's Checklist app for the iPhone (it works on the iPad, too). What's nice about this app is that it's hard to lose your place as you check off items. Should you miss something, the app takes you right back to the first thing you skipped. If you discover a missing task or want to change the order of tasks, you can edit a checklist on the spot; no need to print out a new version, laminate it, etc. I fly a lot of different aircraft types and the checklist app really helps me keep it all straight without carrying a gazillion paper checklists.




Infrequently Used = Easily Forgotten

With abnormal or emergency checklists, we're faced with the opposite problem of routine checklists. Under stress and time pressure, these seldom-used checklists can make us feel confused and clumsy. The answer is to review these checklists by doing some arm chair flying from time to time, imagine an abnormal or emergency situation, then work you way through the checklist. You can also practice this in a simulator, just like the airlines do. Review emergency and abnormal checklists once a month and you'll be less likely to be flummoxed should a real emergency occur in flight or on the ground.

Designers and engineers can't anticipate every possible situation that a pilot or flight crew might face, so there isn't a checklist for all possible abnormal or emergency situations. Several years ago, flying in the wee hours of the morning of course, I heard a single, loud chirp every few seconds combined with annunciator lights flashing on and off. After a few confusing minutes (it didn't help that I'd barely gotten 6 hours of sleep in my 8 hour rest period), I determined the generator was being tripped off-line and then coming back on-line, all on its own, over and over. A quick review of the checklists showed there was no defined procedure for this problem. I followed the "generator offline" checklist as best I could and thankfully there was a standby electrical system. When I got on the ground and described the problem to maintenance, their first reaction was "That can't be!" Eventually the root cause was found and it did turn out to be an oddball failure.

I've witnessed numerous landing gear system problems for which there was no checklist. Most aircraft have emergency landing gear extension procedures, but many do not have checklist for situations like the landing gear failing to retract or only two out of three landing gear being extended. One of the most potentially dangerous situations you can face in single-pilot operations is an abnormality or emergency for which there is no checklist. This is where pilots try to use their knowledge of the aircraft systems to decide the correct course of action, in effect creating their own checklist in the moment.

The key in these situations is don't be in a hurry. Think very carefully and avoid impulsively jumping to any conclusions or simple explanations. If you have another pilot or a passenger on board, involve them in the process even if that only means you talking out loud and them listening to your thought process. You can learn a lot by listening to yourself talk.

Pessimistic or Realistic

It's been said that a good pilot is a pessimist, but I think being a realist may be better. Avoid an overly optimistic or inflated view of your skills, your knowledge, or your currency. Remember that there are mistakes that can embarrass you and mistakes that can kill you. The only thing standing between you and a fatal error just might be an open mind and a good checklist.

Monday, November 08, 2010

Stardust

Anyone who's been involved in GA or flight instruction at Hayward or Oakland, California has undoubtedly met or heard of Mal Raff. Mal passed away last week at his home, a mere three months after being diagnosed with a rare form of cancer.

I didn't know Mal well, yet I considered him a friend. When I lost my medical in 2008, Mal was both sanguine and supportive. He recognized the fragile nature of aviation and human health. To him the most important thing seemed to be teaching and he encouraged me to continue sharing my knowledge in whatever capacity I could while I waited for my medical issue to be resolved.

Mal was an accomplished fixed-wing pilot and flight instructor, but he was also a helicopter pilot. Though he flew them infrequently, he once confided to me that he enjoyed the particular challenge of doing his flight review in a helicopter.

Mal was famous for holding strong opinions on many topics, especially those that involved aviation. In spite of his strongly held beliefs and opinions, I knew Mal to be a person who would listen, and I mean actually listen to people who held a position different from his own. He may not have agreed with you after hearing you out, but I didn't know him to be a person who would reject other people's ideas out of hand. In current American culture where everyone seems obligated to annihilate those who disagree with us, Mal was a gentleman.

Mal was straightforward and up-front, almost to the point of being ingenuous. In a society marked by double-speak, duplicity, and self-interest, Mal was unique because what you saw was mostly who he was. I say mostly because he once revealed to me that his original training was in astrophysics.

I did two aircraft check-outs with Mal and enjoyed flying with him because, like all true instructors, he loved to fly and it showed. Pilots who where thoughtful, who were trying to do their best, who weren't full of false bravado, who wanted to learn, those pilots were likely to get a fair shake from Mal. The others? Well let's just say he didn't suffer fools gladly.

I imagine that Mal has returned to the stars in the heavens that so fascinated him. Godspeed my friend.