Friday, November 19, 2010

The Problem with Checklists

Having finished reading The Checklist Manifesto, which deals with how checklist procedures can and have improved the delivery of medical care, I found some thoughts on aviation checklist procedures bubbled to the surface. The author, Atul Gawande, credits aviation as the inspiration for implementing checklist procedures in medicine. While there's no doubt that checklists help pilots complete complex procedures, the mere existence of a checklist doesn't immunize us from disaster. Pilot and aircrew performance is tightly linked to the quality and appropriateness of the checklist as well as checklist discipline. And let's not sidestep the whole issue of having to face a situation for which no checklist exists.

Appropriate and Useful

A fellow freight pilot once observed that there are just two kinds of mistakes that pilots can make: Those that embarrass us and those that can damage the airplane and/or kill us. An airline pilot friend once confided that he realized he could make errors in the cockpit at any time. His goal was to uncover any errors and correct them before they became a safety issue. For checklists to be an effective first line of defense against fatal mistakes, they must contain the crucial tasks and actions that apply to the situation at hand. Once a good checklist has been made available, pilots have to follow it. Here's a video a horrific accident during the test flight of a turbine conversion for the Caribou. The crew apparately neglected to remove the control lock, with deadly results.

One of Gawande's central themes is that the development of useful and appropriate checklists is aided by organizations that exhibit teamwork, continuous enhancement, and decentralized control. Gawande cites a study where a checklist was developed for the medical procedure of inserting central line. The steps were simple but it turns out that some of the crucial ones were often skipped, resulting in a high rate of infection, complications, and even patient death. When a checklist was developed for central lines and adhered to, the rate of infection dropped dramatically, survival rates improved and (here's what got a lot of attention) hospitals saved a ton of money. Interestingly, it was nurses who often reminded doctors when they were about to make a mistake, like forgetting to don a surgical mask or use a sterile drape.

The sort of cooperation being suggested by Gawande, where decision making, process improvement and double-checks are performed by people on the front lines, is not always what happens in aviation. Top-down organizational structure is often the rule in aviation, with the FAA being at the top of the heap. When I flew freight, I was astonished to learn that the checklists for our aircrafts' optional equipment weren't included in the regular manufacturer's checklist. Checks that needed to be done every day were strewn throughout the Supplements section of the Approved Aircraft Flight Manual. Creating a company checklist seemed like the answer, but that would have required a lot of time, effort and money since the checklist would need to be ... wait for it ... approved by the FAA. So there were procedures in the manufacturer's checklist that were incorrect or missing and the FAA's regulations (the ones supposed to ensure safety) condoned an environment where required equipment checks were easily forgotten or skipped altogether. At least this gave us something to talk about during recurrent training.

Gawande's glowing view of aviation checklists aside, I'd be remiss if I didn't point out that many aircraft manufacturer checklists are woefully inadequate, incomplete, and in some cases they even contain incorrect information. One manufacturer's After Landing checklist for a late-model aircraft in which I instruct contains just one task: FLAPS UP. Don't mistake this simplicity for elegance, because in point of fact it is inadequate. Checklist procedures continue to be spread throughout Approved Aircraft Flight Manuals, due to the manner in which aircraft and their components are certificated, and this may lead a pilot to incorrectly conclude that all of the manufacturer's checklist content is irrelevant. 

The good news here is that student pilots (and certificated pilots, too) flying under 14 CFR parts 61 and 91 can develop their own checklists. Just ensure that whatever checklists you develop contain, at their core, the manufacturer's checklist items. Creating your own checklist can be a great learning experience, but borrow an idea from Gawande's book and have some other pilots and instructors check your work for accuracy and completeness.

Familiarity Breeds ... Complacency

Assuming you have correct and complete checklists, there is another issue to address. Do the same routine countless times and you'll find you tend to skip using the checklist, do the tasks from memory, or use a flow check or mnemonic. Variety is the spice of life and I think it might even be the key to safety. There's emerging research that indicates what we have always suspected; endless routine is booooring and it can actually keep our brains from performing well.

On the ground, I still recommend using the checklist as a do-list, but that doesn't mean you can't have a passenger participate by reading the checklist to you. In the air you can mix things up by alternating between the checklist and a flow check or a mnemonic backed up with the checklist. Mix up your procedural routine and you may find checklist procedures to be less onerous, just be sure you don't skip any steps in the process.

I'm a big fan of Foreflight's Checklist app for the iPhone (it works on the iPad, too). What's nice about this app is that it's hard to lose your place as you check off items. Should you miss something, the app takes you right back to the first thing you skipped. If you discover a missing task or want to change the order of tasks, you can edit a checklist on the spot; no need to print out a new version, laminate it, etc. I fly a lot of different aircraft types and the checklist app really helps me keep it all straight without carrying a gazillion paper checklists.

Infrequently Used = Easily Forgotten

With abnormal or emergency checklists, we're faced with the opposite problem of routine checklists. Under stress and time pressure, these seldom-used checklists can make us feel confused and clumsy. The answer is to review these checklists by doing some arm chair flying from time to time, imagine an abnormal or emergency situation, then work you way through the checklist. You can also practice this in a simulator, just like the airlines do. Review emergency and abnormal checklists once a month and you'll be less likely to be flummoxed should a real emergency occur in flight or on the ground.

Designers and engineers can't anticipate every possible situation that a pilot or flight crew might face, so there isn't a checklist for all possible abnormal or emergency situations. Several years ago, flying in the wee hours of the morning of course, I heard a single, loud chirp every few seconds combined with annunciator lights flashing on and off. After a few confusing minutes (it didn't help that I'd barely gotten 6 hours of sleep in my 8 hour rest period), I determined the generator was being tripped off-line and then coming back on-line, all on its own, over and over. A quick review of the checklists showed there was no defined procedure for this problem. I followed the "generator offline" checklist as best I could and thankfully there was a standby electrical system. When I got on the ground and described the problem to maintenance, their first reaction was "That can't be!" Eventually the root cause was found and it did turn out to be an oddball failure.

I've witnessed numerous landing gear system problems for which there was no checklist. Most aircraft have emergency landing gear extension procedures, but many do not have checklist for situations like the landing gear failing to retract or only two out of three landing gear being extended. One of the most potentially dangerous situations you can face in single-pilot operations is an abnormality or emergency for which there is no checklist. This is where pilots try to use their knowledge of the aircraft systems to decide the correct course of action, in effect creating their own checklist in the moment.

The key in these situations is don't be in a hurry. Think very carefully and avoid impulsively jumping to any conclusions or simple explanations. If you have another pilot or a passenger on board, involve them in the process even if that only means you talking out loud and them listening to your thought process. You can learn a lot by listening to yourself talk.

Pessimistic or Realistic

It's been said that a good pilot is a pessimist, but I think being a realist may be better. Avoid an overly optimistic or inflated view of your skills, your knowledge, or your currency. Remember that there are mistakes that can embarrass you and mistakes that can kill you. The only thing standing between you and a fatal error just might be an open mind and a good checklist.

No comments: