The complete guide to Mixed Reality (MR) in Surgery

3D visualization of medical organ images is becoming more important in surgery. Understanding the patient-specific 3D anatomy helps surgeons better prepare for surgery, providing better results for patients.

Currently, conventional volumetric images such as MRI and CT or even ultrasound images are used to schedule surgery.

Mixed reality, which is the fusion of real and virtual worlds where physical and digital objects coexist and interact in real time, has already managed to provide holographic 3D images to surgeons.

What is mixed reality?

Mixed reality is the merging of real and virtual worlds where physical and digital objects co-exist and interact in real-time.

Mixed reality is the possibility to do both, virtual reality and augmented reality in the same app or on the same device.

The user views the real world while manipulating the digital content generated by the device.

Applications of mixed reality in surgery:

The 5 applications of mixed reality in surgery are:

1. Preoperative planning

2. Documentation of the intervention

3. Holographic 3D visualization of any imaging of the patient during surgery with the possibility to interact with it: immersion and interaction with the two real and virtual worlds

4. Collaboration of surgeons in real-time and collaboration of remote surgeons

5. Education of medical students and training of surgical residents

1. Preoperative planning

Mixed reality can be really useful for planning surgery. A surgical plan performed on a patient’s X-ray may be used during the operation and even after the operation for resident training in surgery or the patient.

Mixed reality has already been successfully applied as a planning tool in various surgical scenarios, including urology, thoracic surgery, neurosurgery, colorectal surgery, and bariatric surgery...

Mixed reality glasses such as HoloLens 2, and Holoeyes, facilitate the spatial localization of anatomical structures and improve mental alignment, which simplifies preoperative planning.

These smart glasses can also improve patient comfort and understanding with Virtual Surgery Intelligence for HoloLens 2.

2. Documentation of the operation

With mixed reality, any surgeon can document the procedure and then use images, videos, or audio to train residents or students.

3. Holographic 3D visualization during the operation

Mixed reality allows the process of reconstructive surgery. With reconstructive surgery, surgeons can locate and reconnect major blood vessels.

Holographic overlays help surgeons see the bones and identify the evolution of the blood vessels that help them with their surgery, improving the outcome for the patient.

Mixed reality helps healthcare professionals access 3D medical information and anatomical images of the patient during surgery.

The use of 3D models to estimate size and shape before performing surgery has been effectively implemented for almost a decade.

Mixed reality allows you to generate customized 3D models for each patient and visualize the internal anatomy in a fully immersive environment.

This opens up new possibilities, such as preoperative simulations, to determine optimal procedures and predict final surgical outcomes, it is a good method for any surgeon to refine his gestures and simulate with the surgical team the virtual operation before the day of the operation.

4. Collaboration between surgeons

Mixed reality also helps build a bridge between surgeons who work in different parts of the world to collaborate or even surgeons in the same operating room or hospital.

5. Education and training of students and residents

Mixed reality surgical simulators are becoming an important part of the training process for medical students and residents, as they allow an appropriate training environment to recreate realistic and reproducible scenarios without putting the patient at risk.

Mixed reality has already been evaluated as an aid tool during endoscopic procedures, spinal surgery, interventional radiology procedures, and orthopedic surgery.

Mixed reality allows a better understanding of the specific anatomy of each patient and thus improves surgical planning for more precise, safer, and less invasive surgical operations.

Mixed reality can include complex information such as holographic images or 3D objects that can be placed in the surgeon’s field of vision, avoid using other displays in the operating room, and facilitate more accurate alignment between virtual information and physical objects.

This would reduce the need for awkward postures for the surgeon and provide new interactive experiences in surgical planning.