Augmented reality platform makers have not only made real-world cameras commonplace through headsets that offer practical applications such as remote assistance, but have also demonstrated how to benefit from the power of immersive computing across industries.
From design and engineering to medicine and education, Microsoft’s untethered mixed reality HoloLens 2 is a case in point. The head-mounted, self-contained computer unit with Wi-Fi connectivity projects 3D images from its projectors onto the glasses, while see-through lenses allow the user to see the real world in their field of vision, so the holograms meld with the real world.
Microsoft stepped up processing power with the second iteration of the HoloLens by including a CPU (central processing unit), GPU (graphics processing unit) and HPU (holographic processing unit) for a real-time ability to spatially map the world around the user. Along with a tracking camera, the unit houses sensors that capture information in the environment to make sense of how the user’s head is moving.
This is done through the inertial measurement unit (IMU), which includes an accelerometer, gyroscope and magnetometer. The unit houses binaural speakers to create a 3D stereo sound experience; it is able to detect hand-tracking gestures so the user can grab virtual objects and move them.
Transforming Triage Services
HoloLens was intentionally designed to be collaborative for such applications as servicing and maintenance, complex assembly, training and visualization across markets. In a pilot test group, Microsoft partnered with elevator company Thyssenkrupp, using the technology to support field repair experts. While wearing a HoloLens, a novice repair technician can be coached by an experienced technician.
A coach or colleague, looking at a computer screen, can see what the field technician sees, and is able to talk trainees through a repair. The coach can annotate in their field of view, draw a circle around the right part to replace, bring up documents and show them in their field of view. “It’s almost as if that person was literally standing over the shoulder of the trainee technician. And when you have that technology connection, it’s hard to make a mistake,” said Greg Sullivan, director of mixed reality at Microsoft.
Complex Assembly, Training and Visualization
Remote assistance through mixed reality is earmarked for scaling expertise across industries. “When you put the HoloLens on, it does a digital overlay, walking you through the process of installing or maintaining or repairing the equipment,” said Sullivan.
The potential of HoloLens has been explored in case studies with NASA, Japan Airlines, Volvo and the Dubai Electricity and Water Authority, as well as Case Western Reserve University. The device could be used to take a very complicated task or piece of machinery or process to guide someone who is not familiar with the procedure or task using a digital overlay.
Japan Airlines, for instance, is using HoloLens for training flight crews (pilots and cabin crew), as well as training mechanics. As Sullivan explained: “They realized they could create a digital, three-dimensional jet engine. And, instead of taking an engine off a plane and flying mechanics to its location, they could send a HoloLens to every airport in the world, where mechanics wear the device. They can see a hologram of the engine, and peel away layer by layer of the complicated piece of machinery, and start to isolate the subsystems to see how they interact with each other.”
The idea of bringing the digital world into our three-dimensional world has profound implications for learning and training and remote assistance, he said.
But while hands-free procedural guidance and instruction are key value drivers, not all analysts are keen to declare HoloLens 2 the next go-to device. Critics flag that early adopters will still need to overcome change management and onboarding challenges, and the price tag ($3,500) remains a possible deterrent.