You’re creeping around the side of a dusty building, knees bent, the butt of your M4 tucked tight into your shoulder. The muzzle’s pointed down — for now — and your finger rests on the grip and not the trigger.
Before you round the corner you look back. There’s a line of four guys behind you in matching uniforms. Each one is stamped by a patch bearing the American flag.
When you take your next step an explosion shatters the air.
Your rifle whips up against your cheek, your eyes searching for the perpetrator, the cold metal of the sight burning into your skin. Right as you’re about to squeeze the trigger, your headset is turned off.
This is an example of a combat simulation — and an exciting new method of training — made possible by technology called virtual reality.
Virtual reality (VR) and augmented reality (AR) have become cornerstones of military training and operations. Applied in flight simulators, weapons training, and even immersive virtual combat, these new tools have completely changed the way our military functions.
What most people don’t realize is that we’ve been using them since the ‘60s.
First, what exactly is VR?
Virtual reality is defined as:
- “the computer-generated simulation of a three-dimensional image or environment that can be interacted with in a seemingly real or physical way by a person using special electronic equipment, such as a helmet with a screen inside or gloves fitted with sensors.”
Augmented reality, on the other hand, combines digital elements with what we already see in the real world; an example would be a digital filter overlayed on a picture, which can be done with apps like Snapchat and Pokemon Go.
This is different from VR, which is meant to be a totally immersive experience, cutting out the “real world” altogether.
The invention of virtual reality
While we may imagine VR to be relatively new-age tech, it can be traced back as far as 1966.
Ivan Sutherland, a professor at the University of Utah, was working on “The Sword of Damocles”, a head-mounted virtual display and one of the first VR systems known to man. Similarly, in the Air Force, Lt. Tom Furness was building a system of his own.
“I started working on VR 53 years ago as a means of solving some problems with fighter cockpits,” Furness said.
Furness, who had originally planned on becoming an astronaut, entered the Air Force with a degree in electrical engineering from Duke University. As a flight test engineer, he noticed limitations within the cockpit of American fighter jets.
- You couldn’t aim the weapons system without aiming the entire aircraft.
- There wasn’t enough room in the cockpit for new and important tools.
- There wasn’t a practical way to get sensory information to the pilot.
All of this was compounded by the fact that the U.S. was waging war over 7,000 miles away, in the communist nation of Vietnam.
“We were trying to solve some very difficult problems,” Furness explained.
“We had the intelligence reports; we knew what the bad guys were doing, and we had to come up with methods of countering that. How do we build a system that would counter these threats?”
The Helmet-Mounted Display
The first problem addressed was with the weapons system.
“What I started working on was a way to track head position, in order to aim the weapons on the airplane and designate targets on the ground,” Furness said.
This was called the Head Tracking System. Quickly, though, Furness realized he must also solve the issue of space in the cockpit. At the time, there were already 75 displays, 300 switches, 11 switches on the control stick, and nine switches on the throttle. There simply wasn’t any room for more instruments.
“I began working on another device called a Helmet-Mounted Display, where we would basically take a miniature television picture but put it on the helmet,” Furness expressed.
“You’d get a big screen display without taking up any cockpit space.”
Not only did the Helmet-Mounted Display save space, but it allowed pilots to tap into an entirely new resource: night vision.
“The Vietnam War put a lot of emphasis on these problems — especially of a night interdiction. How do you find the targets, when you’re flying really fast and low to the ground? The government was spending a lot of money on building these sensors that let us see at night, but how in the world do you get it to the pilot?”
By combining the Head Tracking System and the Helmet-Mounted Display, pilots could aim these sensors with their head movement; this, in turn, was fed to them through a display inside their helmets, letting them see straight through the cockpit and into the night.
The Super Cockpit
In the early ‘70s, Furness made the decision to leave the Air Force as a captain.
“After running the [VR] program for five years, I switched over to civil service: doing the same job, same place, for another 18 years.”
Near the end of his DoD career, he created the concept of the “Super Cockpit”. It was the ultimate space-saving cockpit, because it was completely virtual.
“The Super Cockpit is a cockpit that you wear. You put on a magic helmet, magic light suit, magic gloves, and you plug into the airplane and have the whole cockpit presented to you in a panoramic, three-dimensional image, suspended in space,” Furness said.
“What you see now in the F-35 aircraft are some of the things we worked on.”
VR expands beyond the military
“About the ‘85, ‘86 time period, I was called by a general officer in the Pentagon, saying, ‘We need some positive publicity. It’s just come out in the news that the Army was spending $500 for hammers, and the Navy was spending $800 for toilet seats,’” Furness recounted.
In a bid to impress Americans with their budget allocation, Furness was told to unveil the Super Cockpit Program to the public.
“I ended up on the CBS Evening News. Then after that, ABC had to come. And NBC, and CNN, and BBC, Australian television and the New York Times. So it made a big splash,” Furness said.
“The interesting thing that happened were the phone calls I started getting.”
First, a mother called, seeking a high-tech solution to her son’s cerebral palsy. Then, a thoracic surgeon, wondering how to get a virtual view inside a patient’s body. Firefighting companies were calling for a means to see through smoke-filled rooms and better locate victims.
“I was getting three or four phone calls a week from people like this, and my answer to their questions was always: ‘Yes, you could do that. As a matter of fact, that would be sort of easy compared to what I’m trying to do.’”
“That’s when I realized we were onto something really big.”
The Human Interface Technology Lab
Furness was intrigued by how he could be applying VR technology outside of the Air Force. In 1987, he went on a year-long sabbatical to explore these options.
“I went everywhere. Hospitals, toy companies, aerospace companies, computer companies, kindergartens…”
“I realized we were going to have a huge explosion of telecommunication, but no one was working on how to get that information to the human,” Furness remarked. “It was all a competition of telecommunications, with no bandwidth to the brain.”
“So I decided: I know how to do that with virtual reality and augmented reality.”
Officially leaving the Department of Defense in 1989, Furness came to the University of Washington (UW) as a full professor with tenure. This is where he opened the HIT lab — the Human Interface Technology lab.
“The whole idea was to take what I’d learned in the Air Force and use it for other applications, with medicine, education, and scientific visualization, where we can help mankind.”
Now, the engineering professor has three running HIT labs: one at UW, one in Christchurch, New Zealand, and one in Tasmania, Australia.
27 companies have formed from these labs, with an overall net worth of $10 billion. VR in and of itself is forecasted to be a $120 billion industry, worth even more as it makes the transition into homes and schools.
“In 2014 we formed a nonprofit called the Virtual World Society to work on inspiring technology development for humanitarian purposes,” Furness said. “Basically to lift mankind — to become the ‘Peace Corps of VR.’”
How is VR used in today’s military?
As outlined by Furness, the military has three main uses for VR:
“It’s clear that VR is pervasive in the military. You can do war game planning and training, where you don’t have to physically bring people together in the same place… you can operate virtual tanks and airplanes… and so you create a very powerful training tool,” he described.
As all of these elements come together, you may be hard-pressed to find an area that VR has not touched in the military.
Furness’s early work on the Helmet-Mounted Display and Super Cockpit is still being used, and evolving further every day; drone operations are made possible through VR, by linking the mind of the pilot with the remotely-powered vehicle.
Even the way we teach service members to fight is changing, in a world where we can place them on a lifelike battlefield with the click of a button.
“We’re training people to kill. That’s what the military does. And I have mixed emotions about that,” Furness stated.
“I’ve been a warrior for all these years, but you look back on it and all these lives are lost, and you say: ‘What did we do? What came out of that?’ What we really want to have is a strong military that doesn’t have to fight. And the reason they don’t have to fight is because they’re strong and nobody wants to engage with them.”
Can VR treat PTSD?
Through research at the HIT lab, Furness has discovered that VR can alleviate both acute and chronic pain. And through trials with 9/11 survivors, he’s found that it treats PTSD as well.
“In terms of acute pain, VR provides a distraction,” Furness supplied.
“The mind is a powerful tool, but it’s directed; if you’re busy doing something else that’s driving your attention, you may not be paying attention to your pain receptors.”
When he ran trials on subjects with chronic pain, he found the same positive results. One 30-minute to an hour session of VR would relieve pain for weeks. Even when subjects were not in VR, they could escape the pain again by remembering their time in virtual reality.
Treating PTSD, on the other hand, is very similar to treating a phobia.
“In the case of PTSD, what happens is a fear of your fear — it keeps enforcing itself. So what we try to do is expose the soldiers, or others, back to that setting where they were frightened to begin with,” Furness said.
They started by subtracting certain parts of the situation that caused trauma. For the 9/11 survivors, they took out the police sirens and changed the appearance of the buildings. Then, as the subjects became more comfortable, they would gradually reintroduce those elements.
“It’s not so much a safe space as one they can cope with. They don’t have the fear or the terror that they had before.”
What is the future of VR?
“We’re going to see a great improvement in the performance. The technology is getting better and better, and we’ll have more photo-realistic capability,” Furness supposed.
Right now, VR hasn’t fully breached the consumer market. While it’s available, it requires a lot of (usually expensive) equipment to run it. Furness sees it becoming much more consumer-driven and accessible as new systems are released, like the Oculus Quest.
“In the next five to ten years, most of the dramatic improvements will be made in actually building new tools to make content, to be able to synthesize worlds in 3-D, and to do this in a much more efficient way.”
In the long term, Furness predicts that VR will completely change the way we use the physical world, and even how we connect with one another.
“We often think about the world in which we live as being limited to the physical things we interact with. Especially, for example, your home — your physical home.”
He continues: “Right now we build these mansions with all these picture windows, three stories, and all that. But now, do you really need to do that? If you are able to occupy a virtual space, it’s infinite. You could make the biggest mansion you want. You could have picture windows looking out over the Pacific Ocean. Or you could be on the Serengeti. You could be anywhere, and it’s like you’re there. So the whole notion of physical space will change.”
Will virtual reality disconnect us from the real world?
A big problem most people have with modern technology is that it leads to human disconnection.
When we’re busier staring at screens than actually facing one another, it can dull our sense of reality, and make everything a bit more two-dimensional. Furness believes the opposite is true for virtual reality.
“I think that instead of further separating people, it’s going to be a super-glue to bring people together. Better than we ever have before,” he predicted.
“I just did a conference talk on Mars,” he began, “using what is called Engage.”
Engage is a virtual reality platform made for corporate connection and education.
“There were people from all around the world. We had this little conference room with picture windows, and we’re on Mars. I would see each of them, look around, interact with them… and it was all about education and VR.”
Let’s get one thing straight: virtual reality is not the same as physical reality.
VR is a tool we can use to add value to our lives — and not one that detracts from the physical world we’ve known for thousands of years.
“VR will never take the place of human touch. It will never take the place of human embrace, or of looking into another person’s eyes. That is a part of physical reality that cannot be replaced, nor do we ever want to.”
What is the real purpose of VR?
Virtual reality can be used for any multitude of things. From our surgical tools, military hardware, and deep-space probes, to gaming and home entertainment.
“For many years we have been looking for the driving application for VR, and I think it has always been education,” Furness said.
Whether we’re teaching pilots how to fly, or students how to create their own virtual spaces, there are no limits to what VR can turn into, or how powerful this application can be.
“My work reinforces why I want to stay away from violence,” Furness said emphatically.
“When you give kids the chance to build their own worlds, there’s no violence in them. They just want to create. That’s what most people want to do. And that’s the paradigm shift that I want to see: people getting out of entertaining themselves by destroying, and switching over to the excitement and exhilaration of creating.”