Use of HoloLens in production and logistics

Published on: February 15, 2017 / update from: October 27, 2021 - Author: Konrad Wolfenstein

HoloLens: Also increasingly interesting for use by companies (Source: Microsoft)

Welcome to the virtual world

Microsoft's HoloLens on everyone's lips in Germany since it was delivered at the end of 2016 In the entertainment and gaming sector, it is already exploring ways of three-dimensional 3D representation that differ fundamentally from previous viewing and gaming habits. The virtual experience is made possible by innovative technology: Kinect sensors and cameras are installed on the front part of the HoloLens, which display virtual images in the wearer's field of vision and constantly scan the space in front of them. A second virtual level is placed over the viewing area for the user, which, in addition to the virtual impression, also allows a good perception of the surroundings. The HoloLens has an innovative operating concept in which the programs are controlled with gestures from fingers and hands.

It is the sum of all of its features that make the HoloLens so interesting to use beyond private use. Even in the professional environment, a whole new working world opens up to the user thanks to the additional information and control elements. The device thus enables interactive levels of action and communication, with the help of which previous physical boundaries are removed. It is therefore not surprising that service providers such as Xpert.Digital have already specialized in the creation of augmented reality applications.

Use of HoloLens in logistics

The use of data glasses in logistics is still controversial. With Google-type smart glasses, warehouse workers can be shown valuable additional information for their orders. But since the display is very small, there is a risk that the user's eyes will tire too quickly or that they will experience symptoms such as headaches. AR or VR glasses could be a solution here, because these systems have much larger and therefore more user-friendly displays. Since the closed VR-type glasses hide the user's real surroundings and therefore the respective warehouse environment would always have to be recreated virtually in a laborious manner, these systems are not suitable for regular use in logistics for the time being. This problem does not occur with the transparent HoloLens solution because the user still has their usual field of vision. Here, relevant information is provided depending on the location, time and viewing angle, which is integrated into the field of vision. This makes navigation and item selection much easier for the order picker.

Order picking with AR glasses (Source: Technical University of Munich)

HoloLens recognizes the storage location of every item you are looking for and always shows the picker the optimal route. With the help of the integrated tracking system, virtual objects can be displayed in addition to static information depending on the viewing direction. In addition to navigation arrows, this can be a colored border around the storage compartment with the item to be removed as soon as the warehouse worker stands in front of it. Sophisticated software ensures that real objects are not covered by virtual objects and thus impair the employee's view. This modern type of order picking is certainly suitable to replace the tried and tested pick by light process in the future, as with similar precision there is no need for the complex implementation of pick by light technology in the warehouse.

In this context, Phaenom GmbH, the technical partner of For this purpose, the data glasses will be equipped with specially developed picking software, the user-friendliness of which will be tested and optimized during the project. This is also interesting from the point of view of ergonomics, as there is hardly any experience with long-term use of this system.

The use of the HoloLens is also helpful in packaging logistics; especially for bulky parts for which there are no standard containers. While it previously had to be laborious to take measurements and then look for suitable packaging, in the future this can be done in a single virtual step: Thanks to its spatial cameras, the HoloLens independently measures the dimensions of the item and compares them with the available packaging the order to produce a new box. In contrast to the laborious recording with paper and pen or tablet, this happens in real time, while the order picker is still standing in front of the goods. This saves additional valuable time.

First successes in production and assembly

The first deployments in production are already successful. paint manufacturer Bergolin, wear chemical-resistant protective gloves when working, which means they cannot use keyboards. So far, each

Documentation of work steps (source: Daenet/Bergolin)

After half an hour of work on terminals, the production steps processed during this time are reported. The heavy gloves had to be taken off with difficulty. In addition, the terminals were only distributed sporadically throughout production, which sometimes led to long walking distances. This changes with the HoloLens, as a virtual 3D dashboard now appears in the employee's field of vision, where the work steps are displayed and the necessary entries are controlled using gestures. These can be easily done even with heavy gloves. In this way, all processes to be documented are recorded directly and in real time. Bergolin wants to replace computer and paper-based documentation in production. Since the employee is only shown the current production steps, the workload can be relieved and quality can be increased at the same time. A positive side aspect is that the new technology leads to an increase in safety in the workplace, as employees now have to confirm with a gesture that they have put on their protective clothing correctly.

ThyssenKrupp also relies on HoloLens technology . The service technicians can use the HoloLens to visualize the specific characteristics of an elevator that needs to be serviced before they are deployed. On site, the system then gives employees access to all technical information about the elevator at any time. In addition, they can optionally secure the support of a company expert via image transmission so that they always have both hands free while working. According to the company, this leads to a significant reduction in maintenance time. Accordingly, the system has been successfully tested and has already been used in buildings such as One World Trade Center in New York.

AR glasses are increasingly being used at trade fairs

Virtual trade fair presentation (Source: Xpert.Digital)

However, their realistic representation of complex issues is not only used in the production and warehouse environment. Logistics manufacturers are increasingly using AR and VR technologies to present their offerings at trade fairs . This clear form of virtual representation allows manufacturers to create completely new ways of representation, for example by allowing interested parties to create three-dimensional models of storage systems and view them from all sides or even from the inside. In this way, visitors gain completely new, detailed insights into the storage processes of the devices. Compared to conventional trade fair stands, more products can be presented more clearly in a smaller space.

Due to its technical properties and the diverse possibilities for expanding the working environment with virtual elements, one can assume that developers will invest a lot of energy in the future to provide companies with a wide range of helpful apps for the HoloLens with which they can use their Further streamlining and optimizing processes.