Surgical procedures with augmented reality data glasses – The app combines MRI images with the real surgical situation

Medicine is advancing inexorably, and a particularly exciting field is the integration of augmented reality (AR) into the operating room. This is not about science fiction, but about a reality that is within reach and has the potential to make surgical procedures safer, more precise and gentler for patients. Neurosurgery in particular, a field that requires maximum precision due to the complexity of the brain, benefits significantly from these technological advances.

Medicine is advancing inexorably, and a particularly exciting field is the integration of augmented reality (AR) into the operating room. This is not about science fiction, but about a reality that is within reach and has the potential to make surgical procedures safer, more precise and gentler for patients. Neurosurgery in particular, a field that requires maximum precision due to the complexity of the brain, benefits significantly from these technological advances.

A promising approach is the use of data glasses that overlap in real time preoperative image data, such as from magnetic resonance imaging (MRI). This technology enables the surgeon to see the patient's body “through the body” and thus gain a more detailed insight into the region to be operated. In contrast to conventional navigation systems, which are often bulky and expensive, AR data glasses open up a new dimension of spatial orientation in the operating room. This is particularly relevant for interventions in which access to the surgical field is limited, such as the operations of brain tumors that take place over the nose. The minimally invasive method is supported by the improved visualization by AR glasses, which can potentially lead to lower tissue damage, shorter recovery times and reduced complications.

The app as the key to precise navigation

The heart of this innovative technology is a specially developed app that synchronizes the MRI image created before the operation with the real-time view of the operating field. This development is the result of years of research and development work of collaboration between the research group Legend of the Clinic and Polyclinic for Neurosurgery of the University Hospital Leipzig (UKL) and the Fraunhofer Institute for Machine Tools and Reforming Technology IWU in Zittau. The app acts as a “GPS system” for the surgeon by not only indicating the position of the operating goal, but also the optimal, i.e. gentlest, access path. This form of navigation aid is a significant improvement compared to previous methods that rely on static image data and the spatial imagination of the surgeon.

Another major advantage of this new technology is the possibility of integrating surgical instruments into the navigation system. By precisely recording the position of the instruments in real time and their representation in data glasses, the surgeon can lead them even more specifically and safely. This real -time representation, which is made possible by the app, minimizes the risk of errors and enables the surgeon to plan and implement its movements precisely. In addition, important additional information, such as the distance to the target area, is shown directly in the surgeon's field of vision, which optimizes the flow of information and enables a quick and safe reaction. This not only increases the safety of the intervention, but also potentially shortens the operation time, which leads to relief for the patient and medical staff.

Precision in real time: a quantum leap in neurosurgery

One of the most remarkable achievements in the research group is the almost delay -free operational readiness of the system. "With a fully automated registration at the neurosurgical Spatial Computing Navigation, our team has succeeded in a worldwide novelty," said PD Dr. Habil. Ronny Grunert, the research assistant at Fraunhofer IWU and head of the research group »Legend« at the UKL, reports enthusiastically. “The calibration and registration is complete within a second and navigation to real-time detection of the instrument position is ready to go. The developed system is very intuitive and comes very close to the use of GPS assistance. ” This speed and intuition are crucial in order not to disturb the work flow in the operating room and offer the surgeons an intuitive and reliable navigation aid.

Another important detail is the user -friendliness of the system. The user interface was developed by doctors for doctors and is limited to the display of the essential information, which minimizes operating errors. An example of the clear display logic is a green crosshair that shows the position of the instrument tip and is perfectly integrated into the MRI image, which is shown in the data glasses. This clear and clear presentation is a decisive factor for acceptance and success of the system in the operating room, since the surgeons can concentrate on the essentials - the successful implementation of the operation.

Economy and accessibility: a revolution for everyone

In addition to the technical innovation, another central aspect of this project is the economy and the associated accessibility of the technology. While conventional navigation systems suitable for use in clinics often cost several hundred thousand euros in neurosurgery, the development group relies on standard data glasses whose prices are located in the consumer area. "These glasses cost a fraction of computer -aided navigation systems for neurosurgery," explains Grunert. This cost reduction is a crucial step in order to not only make the technology accessible to financially strong health systems and institutions, but also in countries and regions with limited resources. This democratization of technology can benefit from the advantages of precise and gentle neurosurgical treatment.

The development of the handpiece that takes up the instruments and enables its exact position determination is also an important aspect of the project. Special markers were developed at Fraunhofer IWU, the geometries and patterns of which are recognized by the data glasses. These markers can occur in different shapes such as balls, cuboids or other bodies and serve to capture the position of instruments in the three -dimensional space. The plastic handpieces are manufactured in Zittau and Leipzig in the 3D printing process, which enables high flexibility and adaptability to the needs of the surgeons.

The way to clinical practice: an outlook

The first pilot course for training on the anatomical model took place in autumn 2024 at the UKL. This step is an important milestone on the way to the clinical application of the technology. In the next step, the team focuses on the completion of the prototype, which then has to go through the approval process in accordance with the Medical Device Regulation (Medical Device Ordinance) for the European Market or in accordance with the provisions of the American Health Authority FDA for the USA. The aim is to be able to use the system on the patient in about two years.

The development of AR-based navigation systems for neurosurgery is not only a technological progress, but also a paradigm shift in medical practice. The integration of real-time image data, precise instrument management and economic solutions has the potential to fundamentally change the neurosurgical treatment and to make them more secure, precise and accessible. The vision of a future in which innovative technologies such as AR data glasses are part of the standard in the operating room is approaching with every step of this development. This not only opens up new perspectives for doctors and surgeons, but especially for patients who can benefit from improved quality of treatment and more gentle interventions. With this technology, the future of neurosurgery has taken an exciting direction in which precision and innovation go hand in hand.

Suitable for:

• Germany is the leader here: 'Iteration Metaverse' solutions with XR technologies such as extended, augmented, virtual and mixed reality
• Augmented and virtual reality – into a new dimension