Wednesday, September 10, 2008

Flying Real UAVs From a Virtual World

By now, we all have a mental image of how unmanned aerial vehicles are flown. Some young military service member sits closeted in a small, dark room with lots of camera and data feeds displayed on glowing screens and guides the Predator or other bird with a flight stick from thousands of miles away. But that's not quite right and omits perhaps the most important part of the experience: the intense, continuous attention to a sometimes unchanging scene over many hours and the degree of concentration and patience and visual acuity needed to "see" whatever it is the UAV is looking for and/or at.

So much information and attention are involved, in fact, that it takes not one, but two pilots to operate each drone.

Robert Kaplan captured some of the grim reality in a terrific September 2006 story in The Atlantic, "Hunting the Taliban in Las Vegas." Here's a look from his visit to a camouflaged trailer at Nellis Air Force Base in Nevada where two Air Force pilots were flying a Predator over Afghanistan:
"There were grim, colorless computer bays in freezing, pulsing darkness—a three-dimensional world of flashing digits from light-emitting diodes. Like sub drivers, Pred pilots fly blind, using only the visual depiction of their location on a map and math—numerical readouts indicating latitude, longitude, height, wind speeds, ground elevation, nearby planes, and so forth. The camera in the rotating ball focuses only on the object under surveillance. The crew’s situational awareness is restricted to the enemy on the ground. Much of the time during a stakeout, the Pred flies a preprogrammed hexagon, racetrack, bow tie, or some other circular-type holding pattern."

Soon, this scene will change. The trailers likely will be abandoned, probably in favor of one or more 10-foot-by-10-foot-10-foot rooms with projectors illuminating all four walls, the ceiling and floor. Pilots likely will wear special glasses to render the projected images into a 3D immersive virtual environment. And they will control not one but several, possibly as many as eight, UAVs at the same time.

An Iowa State University team is developing the virtual environment in hopes of reducing the mental fatigue pilots suffer while monitoring a tangle of information on a welter of screens. Inside the room, pilots will see their UAVs, the airspace around them and the terrain below, as well as feeds from instruments, cameras, radar and weapons systems.

The 3-D audiovisual stereoscopic facility lives at ISU, where lead researcher on the project, James Oliver, heads the Virtual Reality Applications Center. The C6, as the wrap-around virtual display is known, was refurbished over the last several years thanks to Air Force funding. But it's used by many other federal agencies, as well.

The National Science Foundation is sponsoring Meta!Blast, a research application to let people walk around in plant cells to see what happens when researchers make molecular changes.

The National Guard wants to use C6 to let soldiers do battlefield walk-throughs before missions or deployments.

A combined Air Force and Iowa National Guard project, Virtual Battlespace, pulls together data about about land- and air-based forces and targets and sensor feeds and permits views from multiple perspectives. VRAC is working on a multi-touch table to serve as a controller for UAVs in the 3D space.

C6 has myriad commercial applications as well, but what lights up my imagination is its potential for helping people visualize data and the effect of human interventions. Given a complete enough simulation of the real world, walking through it should let users see relationships and consequences the might not otherwise imagine in urban planning, medicine, ecology, and who knows what else. Embedded Technology magazine has a fuller treatment of current uses here.

Nota Bene: You can find the articles I used in writing this post on the left column of the blog under the heading, "Recommended Reading." From now on, I'll be posting there links to stories and Web pages I find intriguing, informative and useful in my explorations of virtual government. The widget is courtesy of Publish2, a terrific journalism project by my friend Scott Karp, the wizard of Web 2.0 and beyond. By all means, have a look at P2.

No comments: