How Microsoft Made Its Flight Simulator Seem So Real
Last August, Microsoft released the latest version of its Flight Simulator, extending the run of that franchise to 38 years and making it the longest-running product line in Microsoft’s history. Published by the technology giant’s Xbox Game Studios, the new Flight Simulator treats gamers to vastly greater detail and texture in both environment and aircraft, far better lighting, and much more realistic flight characteristics than in previous versions. The precise renderings of all 20 airplanes (which include the Boeing 787 Dreamliner, Cessna 172, and Beechcraft B350) and the particulars of individual airports are stunning.
Aircraft cockpits are functional down to the position of almost every switch. Air traffic (both actual and that of other users) is reproduced in real time, as is the weather: If Denver International Airport’s weather station is recording snow and FlightAware is showing 40 airplanes in the Denver airspace, then users will fly their simulated approach through a virtual snowstorm with 40 other aircraft mimicking those actually in the sky.
One of the great joys of aviation is seeing Earth from above, so Flight Simulator reproduces the entire planet, from pole to pole, in high resolution. Stars appear as they would in real time, including the correct phase of the moon. The system even reproduces the effects of atmospheric refraction, which “notches” the rising or setting sun, causes stars to twinkle, and produces distortion generated by engine exhaust.
In this virtual world, users can adjust myriad criteria, including flight control sensitivity, time of day, weather conditions, and views of aircraft and surroundings.
Achieving this feat required coalescing and processing massive troves of data in disciplines ranging from aerodynamics to geodesy and photogrammetry to optics.
Asobo Studio, a video game company based in Bordeaux, France, led the program’s development. Asobo worked with a host of specialized firms and organizations around the globe, including geospatial company Bing Maps, atmospheric data specialist Meteoblue, machine-learning system builder Blackshark.ai, FlightAware for real-time global air traffic, and many others. One reason I know how Microsoft achieved its verisimilitude is that I worked as a contractor on the project, researching airplanes and airports.
Fly Like an Eagle
To create Flight Simulator’s high fidelity, Asobo scanned actual airframes, inside and out, relying on an Artec Leo 3D scanner, which provides half-a-millimeter resolution. Gamers see renditions of these actual aircraft, down to scuff marks on windows and nicks on levers. Asobo also captured the full spectrum of an airplane’s interior and exterior acoustics, including the cockpit echoes caused by the pelting of rain or snow. The team carefully studied each airplane’s aerodynamics by simulating how the air interacts with 1,306 airplane surfaces and developed a flight control regime. “All aircraft have been reviewed with professional pilots who have many hours flying these aircraft or test pilots from the manufacturers to make sure they are controlled exactly like they should,” stated Asobo Studio founder and CEO Sebastian Wloch in a Flight Simulator “Discovery Series” episode about virtual aircraft aerodynamics. In other words, when sim-pilots try to yank-and-bank an A320, they will get the exact same response as if they were flying the real thing.
Simulating the World
Just as Asobo scanned individual aircraft, the team used high-resolution satellite imagery, aerial photography, and in many cases detailed photogrammetric data from Bing Maps to re-create the globe. The size of this data set, which is continuously updated and expanded, is now at three petabytes (and growing). Most of the data are vertical (looking straight down), but some are oblique, and all are in two dimensions. In order to present this information in three dimensions and ensure visual continuity, the Flight Simulator team decided to tap the services of Austria-based Blackshark.ai, a company with 50 computer specialists who developed proprietary machine-learning algorithms they fed into the Microsoft Azure cloud computing service. Initially helped along with human input, the Blackshark systems eventually learned to render and color all aspects of the planet, including determining the heights of trees (all 1.5 trillion of them), removing shadows of clouds from aerial imagery when appropriate, and ensuring accurate color. The end result is Earth, a planet rendered to a resolution of at least three feet, with the most populated 300,000 square miles rendered even finer, with a resolution of between three and 10 inches, including 58,000 square miles (mostly cities) rendered in such detail you can practically walk the streets.
Flying Conditions
What would a flight simulator be without the sky? Meteoblue, a Switzerland-based graphical weather model pioneer, divided the Earth’s atmosphere from ground level to stratosphere into 250 million spatial compartments. “We do this by storing temperature, wind speed, humidity, pressure, rain, snow, and pretty much everything that defines the weather,” explains Mathias D. Müller, co-founder of Meteoblue. Its system coalesces massive amounts of historical and real-time weather data from ground stations and satellites and then re-creates the sky and all of its conditions at any given time based on algorithms that take into account molecular interaction. The system even renders how the weather interacts with aircraft, including icing and streaks of rainwater on a canopy.
To handle air traffic, the system relies on aviation information company FlightAware to stream real-time data of aircraft in the sky using its continuously updated servers. What Microsoft Flight Simulator pilots virtually experience is actually occurring, from the pink coloration of a towering thunderstorm at sunset to a nearby airliner making an approach into an airport and (if desired) the nearby virtual aircraft of other Flight Simulator users. And this visualization, from within the cockpit to an external view, is rendered cinematically as if seen through a camera lens, with the experience streamed seamlessly from Microsoft’s servers.
Flight Simulator attracted over two million players within weeks of its release—the game is already a classic in the making.