Update #1: Man and Machine

I’ve recently come into possession of a developer’s edition of the Microsoft HoloLens, an Augmented Reality headset that serves as a fully functional untethered Windows 10 computer. This is one of the leading devices in the Augmented Reality space, and using it is truly an eye-opening experience that I hope to share with the community when I arrive back on campus in the Fall. While it is seemingly magical, the development edition (which has been released since 2016) is purposefully not a consumer product because of its newness as a platform. Unfortunately, my attempts at developing on this Generation-1 device have made this newness very apparent –

  • There was a LOT of software that needed to be installed and set up for developing on the HoloLens, which was a time and technical barrier to learn. (VisualStudio, Unity, HoloLens Emulator, many special configurations)
  • After getting a little used to the workflow there was a Unity update which broke a lot of normal development workflow. I had to spend a whole day debugging and troubleshooting, but eventually all was back to good again, and I’ve gained some more familiarity and understanding with the workflow.
  • The documentation and tutorials created to help developers get started are frequently out of date, sparse, and generally confusing or unclear.

Fortunately I am persistent enough at debugging and solving the initial technical hurdles to get everything set up, but this is certainly not a simple and hacker-friendly process. This is also apparent by the limited number and quality of apps publicly launched so far. On the plus side, a new and improved (and hopefully more stable) version 2 will be coming in early 2019, and I’ll have significant experience working with AR by then.

How does Augmented Reality work?

Tracking: The HoloLens is equipped with an array of cameras and sensors that excel at tracking the headsets position in space as well as the real world around it. It’s constantly sensing the environment to build a 3D understanding of the walls and objects in a room.

See-through display: Unlike Virtual Reality headsets that cover your sight of the real world, Augmented Reality allows you to bring the digital world into the real world, without obstructing your vision. A popular example who’s development was discontinued for a variety of reasons is Google Glass. The challenge of these headsets is to be able to overlay digital content into our eyes and trick our brains into believing that the digital content exists in the real world. This might look like a projection of Netflix on your wall, a holographic ball bouncing around your house, or an AI fitness instructor jogging next to you.

Intelligent Interactions: In order to create more natural interactions with digital content projected into the real world, the device also needs a way to track the user’s interactions. In the HoloLens, this means using advancements in computer vision and speech recognition to react appropriately when a user’s hand gesture or voice is recognized. This is a very advanced feature of the device but I believe there is the most potential and room for growth here.

I’ll be giving more frequent updates soon! 🙂 Here’s a few fascinating TED talks to get a sense of what AR is like:




Speak Your Mind