How to plan Surgery using Extended Reality

Visualization of the patient's 3D anatomy helps surgeons better prepare for surgery, providing better results for patients.

Currently, conventional images such as MRI and CT or even ultrasound images are used in 2D format to schedule surgery.

Mixed reality, which is the fusion of real and virtual worlds where physical and digital objects coexist and interact in real-time, has already managed to provide holographic 3D images to surgeons.

What is extended reality?

Source: Microsoft

Extended or mixed reality is the merging of real and virtual worlds, where physical and digital objects co-exist and interact in real-time.

Mixed reality is the possibility to do both, virtual reality and augmented reality in the same app or on the same device.

The user views the real world while manipulating the digital content generated by the device.

How to use mixed reality in surgery planning?

In practice, there are 5 steps to using mixed reality in surgical planning:

1. 2D CT/MRI images are transformed into 3D images.

2. A specific model for each patient and organ is then created.

3. Projection of this holographic 3D model through smart glasses with the possibility to interact with each hologram.

4. The surgeon can use this model to plan surgery, collaborate with other surgeons, and educate residents.

5. These 3D images can be registered after surgery and kept for the education and training of residents and medical students, or even to inform the patient about his pathology and the surgical act in detail.

The time taken for the first 3 steps generally can range from 15 (Holoeyes) to 60 minutes (Hololens 2).

Mixed reality allows a better understanding of the specific anatomy of each patient and thus improves surgical planning for more precise, safer, and less invasive surgical operations.