Virtual reality, augmented reality and mixed reality all have useful applications in the field of medicine. When it comes to using these spatial computing technologies intraoperatively, mixed reality has the greatest potential because it allows for interaction with both real and digital objects.
Their Differences and Potential Impact on Healthcare
An architect shows you what it will be like to stand in your new home before ground is even broken. A virtual character appears in front of you, turning the real world around you into a game space. You try on an outfit in a store without even picking it up.
Technologies like these that alter our perception of the world around us, such as augmented reality (AR), mixed reality (MR) and virtual reality (VR), are becoming a reality of everyday life. Spatial computing, the catch-all term for these types of perception altering systems, allows us to experience more than ever before. But how will these technologies affect the healthcare industry? And how are they evolving to become the future of healthcare data visualization?
Before we can understand how these technologies will go about improving health and healthcare with interactive visualization methods, we have to understand the terms themselves. What is the difference between AR, MR and VR? And how can data visualization improve healthcare decision making?
How can clinicians use Virtual Reality?
VR allows the user to experience a whole new, entirely computer generated world. The user is immersed in this three dimensional world and they can move around in and interact with it. The user wears a VR headset, which can be as simple as a smartphone mounted in a cardboard visor or as advanced as complete wireless systems with integrated sound, cameras and microphones. These applications have a range of uses but for the most part today consist of gaming, training and education. But what about virtual reality in medicine?
“Virtual simulation (and serious gaming) presents a viable, cost-effective training option for community health nursing trainees, providing the opportunity to practice within an interactive, engaging, and safe environment.”
Using VR in healthcare could see an upswing in the future, especially when it comes to virtual reality medical training simulation. These “serious games” — games used for educational purposes — are an important teaching tool for areas like neurosurgical procedures, EMS training, and patient teaching modules [1]. VR is even being used in neuropsychology to improve the traditional assessment of executive functions [2]. VR training in healthcare can successfully bridge the gap between theory and practice.
When it comes to the operating room, virtual reality surgery is limited by the immersive nature of the system. It may be quite some time before virtual reality used in hospitals is standard practice. In terms of surgery, VR may never be the ideal intraoperative solution, since the clinician can’t see the real patient while wearing the headset. Mixed and augmented reality surgery seem to be more suited to the O.R. since they combine the real and virtual to assist during surgical procedures.
How does Augmented Reality fit into healthcare?
With augmented reality technology, the real world is visible and digital objects are overlaid onto it. For gaming and shopping applications, this is achieved with a mobile device’s cameras. The image on screen shows the camera’s perspective but also the digital content. These types of spatial computing applications are by far the most common today. Other applications use special glasses, like Google Glass, to allow the user to see both the real and virtual environments with a see-through headset.
In terms of augmented reality use cases in healthcare, augmented reality assisted surgery already exists. With today’s technology, digital objects can be overlaid with a microscope. While looking into it, or viewing the feed from the microscope on an external display, digital structures can be seen on top of the real patient. These structures could represent delicate parts of the brain or a tumor and help guide the surgeon. By superimposing these images on the patient, the surgeon knows where they are going and what to avoid.
So how do virtual reality and augmented reality in healthcare compare? In terms of use during procedures, AR technologies have the clear advantage. Since the clinician isn’t immersed in a virtual world, they can keep their eyes on the patient. The lack of interaction with augmented objects, however, limits the depth of this technology for the medical field. Regardless, AR healthcare applications will no doubt increase as this technology continues to develop in the future.
“[…] surgical practice and education can derive significant benefits from the implementation of AR and MR tools in daily practice.”
Surgery guided by mixed reality: presentation of a proof of concept
Is Mixed Reality the best of both digital worlds?
AR and MR are sometimes challenging to differentiate. For some, there is no difference between the two. In terms of our technology, the main difference is the interaction between the real and virtual worlds. In AR, objects do not interact with the real world but are simply overlaid on top of it. An example of this would be a filter on a social media platform. The filter is visible to the user, but they can’t manipulate the objects on screen. With MR, however, users can interact with digital objects as if they were objects in the real world.
In the case of MR in healthcare, medical schools could use these systems to teach their students about patient anatomy from the safety of the classroom. They could even create a library of exemplary cases for students to interact with and learn from. Even patients would be able to scroll through their own preoperative scans to see how the doctor will approach their case to ease their minds about their treatment.
For MR assisted surgery, several companies are currently developing applications and medical devices that enable clinicians to view and interact with patient data, even in the operating room. With the mapped out surgical plan overlaid onto the real patient, the surgeon could be precisely guided during surgery. When multiple experts join the session, even remotely, the surgeon has the advantage of immediate support for surgical decision-making. The future of MR in the hospital and specifically in the operating room is endless. We are just beginning to understand where this technology could go.
When comparing medical virtual reality, augmented reality and mixed reality, all three have their own distinct differences and advantages. In terms of future development of 3D visualization and augmented reality for surgery, mixed reality with its added interactive capabilities, has the potential to change both the digital and real landscape of medicine.
[1] Ribaupierre, Sandrine de, Bill Kapralos, Faizal Haji, Eleni Stroulia, Adam Dubrowski, and Roy Eagleson. “Healthcare Training Enhancement Through Virtual Reality and Serious Games.” https://link.springer.com/chapter/10.1007/978-3-642-54816-1_2. Springer, Berlin, Heidelberg, April 26, 2014.
[2] Serino, Silvia, Elisa Pedroli, Pietro Cipresso, Federica Pallavicini, Giovanni Albani, Alessandro Mauro, and Giuseppe Riva. “The Role of Virtual Reality in Neuropsychology: The Virtual Multiple Errands Test for the Assessment of Executive Functions in Parkinson’s Disease.” https://link.springer.com/chapter/10.1007/978-3-642-54816-1_14. Springer, Berlin, Heidelberg, April 26, 2014.