Anyone who has ever sat in a cockpit will know how mentally challenging it is to pilot an aircraft. During a flight, pilots and copilots have to process an enormous quantity of visual, acoustic and spatial information. Keeping a constant eye on the numerous instruments in the cockpit is a strenuous task, as pilots must check the correct indicators during a manoeuvre – often in a specific order.
This process of “scanning” the flight systems is something that pilots internalise during their training. But even for experienced instructors, it is hard to judge whether a student pilot is looking at the right instruments at the crucial moment. Now, in collaboration with Swiss International Air Lines, researchers led by ETH Zurich Professor Martin Raubal have used eye-tracking technology for the first time to understand how pilots monitor the automatic systems of a modern passenger aircraft.
Seeing what the pilot sees
Camera-based eye-tracking technology allows precise monitoring of a person’s eye movements. “Since eye movements allow conclusions to be drawn about a person’s thought processes, Swiss came to us with the idea of using eye-tracking in pilot training,” says Martin Raubal, Professor of Geoinformation Engineering at ETH Zurich.
The idea developed into a several-year economic partnership involving NASA, Lufthansa Aviation Training and the University of Oregon in addition to ETH Zurich. Here, the common goal was to improve flight simulator training and thereby cockpit safety. Raubal’s team developed software by the name of “iAssyst” that assists flight instructors as they train budding pilots. The researchers recently wrote about their work in the journal
Ergonomics
.
Reducing the burden on instructors
“iAssyst” stands for “Instructor Assistant System”. The program integrates video, audio and simulator recordings while simultaneously displaying the pilots’ gaze patterns. To avoid distracting the pilots, an eye-tracking system consisting of fixed cameras and infrared sensors was specially installed in the cockpit of an A320 flight simulator. “Setting up and calibrating the system for each trainee pilot is more laborious than with eye-tracking glasses, but it provided us with better results,” explains David Rudi, who implemented the application as part of his doctorate at the Chair of Geoinformation Engineering’s Geogaze Lab.
The ETH Zurich researchers designed their software in close cooperation with aviation experts from the project partners before evaluating it with the help of seven active instructors from Swiss. During a training flight, the instructor sits in the rear of the cockpit, from where they operate the simulator and play the role of flight controller while also keeping a close eye on the pilot. “As a result, instructors sometimes miss – or misjudge – relevant information that is vital for analysing the pilot’s training session,” says Rudi.