Hands-on Use

While Elon, and his teams, were working to make cars using Machine Learning to look at the road ahead, Andy Wilson was working in his lab inside Microsoft Research to come up with better technology tools, not for our roads, but for our hands.

Wilson was the first to show us how a computer could watch your hands for gestures. He demonstrated how a computer could see that you had pinched your thumb and finger together.

His first demo showed how you could "grab" a map in midair and zoom it and twist it. This gesture survives even more than a decade later on Microsoft's HoloLens.

Wilson has been trusted for decades by Microsoft. Back when Bill Gates was giving speeches, Wilson built the demos for showing off the most advanced technology. That earned him his own Augmented Reality lab. Once when we visited, he had a set of projectors and sensors hung from tresses overhead. He demonstrated how he could put computing on every part of your body, and even the room. "Here you go," he said, while dragging a virtual photo out of his pocket and handing it to us. It was the first time we had experienced Spatial Computing where we were "inside" the computer and it was interacting with us.

That research, along with others, came together to build the basis of HoloLens where, now, while wearing a $3,500 headset (instead of standing in a room with thousands of dollars of projectors and sensors), you can do the same thing. It's amazing to see how long it takes some things to get commercialized. We first started visiting Wilson's lab back in 2005, but there's a guy who has been working on this stuff for far longer: Tom Furness.

His work in the 1960s was as an airplane cockpit developer, and that's when he built new virtual interfaces that we think of today as Virtual Reality or Spatial Computing. Now, he is a professor at the University of Washington and founder of the Human Interface Technology Lab. That school did seminal work on studying how Virtual Reality could be used to cure pain. Its results, at http://vrpain.com, show that Virtual Reality, when used with burn victims, is better at removing pain than morphine is. Something we hope becomes more adopted as we try to keep people from being addicted to opiates.

Photo Credit: Robert Scoble. Tom Furness, right, and Robert Scoble hang out at Digital Raign's Reality Summit.

He told us that he and his students are developing Spatial Computing to engage, enlighten, and even entertain people in new ways. We include him here not just because he is seen as the grandfather of Virtual Reality, but because he is a great example of the pattern of how innovation frequently happens: first for military uses, later moving to consumer applications. Many of the industry's leaders follow this pattern and is why technology centers tend to cluster around strong military centers, whether in Tel Aviv, Silicon Valley, or in Las Vegas, where the military tests out new kinds of airplanes and continues Furness' early work today. We visited Nellis Airforce Base, where we met several pilots who fly F-35s. The pilots in them use a big, expensive, Augmented Reality headset. One of them said something that stuck: "I'll never lose to an F-16. Why? Because I can see you and you can't see me, and I can stop." The F-35 not only has those magic headsets, which let pilots see the airspace in great detail, but are designed to be stealthy against radar, so they can't be detected by other planes. They are equipped with engines that can direct the flow from the jets to literally stop in mid-air, which the F-16s can't do.

It isn't lost on us that recently there've been reports that newer F-35s get rid of the pilots altogether, and that the AI flying usually beats the pilots because the computers flying can perform maneuvers that humans just can't handle due to the forces involved.

Wars handled by machines is a controversial thing to be certain, and one we won't take sides on here, but that means fewer pilots coming home in a box due to accidents or being shot down, and the military sees that as just as good a thing as seeing fewer deaths on the road. No matter what way you look at these new technologies and their role, you can't deny that most of these technologies were first designed with a military use in mind.