Many of my maker friends occasionally talk about what it would be like to build Tony Stark’s Jarvis computer. It’s a fantastic vision for computing. Mr. Stark can invoke commands, find information and request analysis of situations simply by having conversations with the Jarvis computer. Using the Jarvis heads up display, Mr. Stark sees an enhanced view of the world showing the status of his systems, targets, or reference data.
In my work, my team and I have been given the opportunity to review the best devices in augmented reality heads up displays. And it feels like we are building Jarvis. Last week, we received our Hololens by Microsoft. And it’s amazing! The Microsoft designed Hololens to usher in a new paradigm of computing where devices enhance the vision of the user. Using the Hololens glasses, the user can see digital content or Holograms drawn on top of reality. This form of computing creates many innovations where the user perspective is important. If you’re connected to a Hololens user over Skype, you will be able to see their perspective as they move around an environment. In this blog post, I wanted to share a few first impressions.
- Location and mapping features are well executed: Robert Scoble wrote an insightful book entitled the “Age of Context.” In this work, he asserts that computing will become more impactful when devices become aware of user context and use the data tastefully. The Hololens does an exceptional job of building a map of your space, tracking the camera pose, and location. Using this context, the user can place digital content, windows, or apps around your office space. They feel like you’ve hung TV panels on walls or in “mid air.” We’re looking forward to using the location and mapping features of this device to build new types of experiences and user interactions. When you fix a web browser to the wall and walk around it, it feels like a physical object.
- Making work spaces fun with Holograms: Many of our team members have had fun decorating our lab with Holograms. Each time I put on the Hololens glasses, I see fun experiments all over our workspace. Some of our favorite apps include the following: Hololens Skype, a cool game where you shoot robots, the 3D model designer, and placing 3D models in space. Our workspace starts looking like an absurd “high tech” museum. It’s very fun.
- Hololens senses your physical space: It’s amazing to see the impact this kind of augmented reality will have on gaming. Any indoor space can become a playground or new level. As you start the “robot shooting” game, you’ll notice the Hololens scanning the 3D structure of the room. While using it the other day, I noticed a robot crawling toward me behind a computer screen. The game experience correctly rendered the view of the robot with the computer screen in the foreground. This is a subtle, but important rendering detail that helps the user feel like these robot players have a physicality in my space.
- Holographic computing is a new paradigm: Our team sees great potential in exploring Microsoft Hololens and other technologies like it. This is a new paradigm of computing. Computers work best when they augment human senses. The Hololens offers a capability for enhancing the user view of the world. As natural language processing and bot technology improves, we will have the ability to collaborate with our machines in delightful and useful ways. I believe that the location and mapping technology in Hololens provides a new context that will inspire a new generation of innovations for all markets: engineering, medical, entertainment, art, and business.
Keep watching this blog for new updates on our augmented reality project. I hope to post weekly updates on building content for Hololens and other AR devices. What’s the coolest application of Microsoft Hololens that you’ve seen so far? What ideas for apps do you have?