3D Printing vs. Augmented Reality: Comparing the physical and virtual for surgical planning

February 16, 2023

3D printing and augmented reality are gaining pace in the world of surgery. Although different in design and output – physical vs. virtual – these complementary, next-generation data visualization tools have a lot to offer when it comes to producing patient-specific 3D renderings, supporting surgeons with surgical planning and more.

Introducing the next generation of data visualization

In the field of medicine, 3D printing and augmented reality represent the next-generation of data visualization tools that enable surgeons to interact with highly-detailed, anatomically accurate renderings of patient-specific data. These tools, while different in design and output, both have the potential to improve preoperative planning, surgical training and patient education, thereby facilitating better treatment outcomes.

As technology advances, so does the healthcare industry. Taking a closer look at surgery specifically, the current standard of surgical planning still primarily relies on medical imaging, prior surgical experience and the skill and judgment of surgeons.

In comparison, relatively new technologies like 3D printing and augmented reality visualization extend past current industry standards and can help surgeons with spatial orientation and guidance during the planning and implementation of complex surgical procedures.

But what exactly are the benefits and limitations of 3D printing and augmented reality in the context of surgery? How can these two technologies work synergistically? And what does the future hold for preoperative planning, surgical procedures and more?

3D printing and surgery

“Three-dimensional printing of models for preoperative planning enhances the 3D perception of the planned operation, either as a visual-tactile aid or for performing mock surgeries. It allows for preadaptation of surgical instruments such as fixation plates and thus shortens the operation and improves precision.” – Printing the Future—Updates in 3D Printing for Surgical Applications.”

3D printing is a manufacturing process in which solid objects are created by fusing or depositing layers of material based on input from computer-aided digital files. While 3D printing was first mainly used by the manufacturing industry, as the technology developed, it was clear that the customizable features of 3D printing had immense potential in the field of medicine. To date, 3D printing in healthcare has been used for tissue and organ fabrication, the creation of customized prosthetics, implants and anatomical models as well as for pharmaceutical research [1]. In the context of surgery, 3D printing is currently being used for preoperative planning purposes such as producing 3D surgical models, surgical guides and implants [2].

One case study published in the Journal of Surgical Case Reports highlights the main advantages of using 3D printing for the presurgical planning of a revision hip arthroplasty, a surgical procedure that often requires highly experienced medical staff depending on the severity of anatomical defects. In this case, surgeons used 3D printing to produce an anatomically accurate and life-size 3D representation of the patient’s hemipelvis, which had severe defects. This approach allowed for preoperative surgical drill and position simulation as well as close examination of potentially fragile and complicated areas, resulting in a successful surgery and positive clinical outcome for the patient.

While this case features only one use of 3D printing for preoperative planning, this technology has been used to plan surgeries in several specialties including cardiovascular surgery, neurosurgery, craniomaxillofacial surgery, orthopedic surgery and interventional radiology [3].

Nonetheless, medical 3D modeling requires expensive in-house equipment—which not all hospitals have access to—extensive expertise in computer-based design and additional time to produce each individual anatomical model. Although the benefits for this technology in the context of surgery are clear, 3D printing’s limitations have yet to be overcome.

Augmented reality and surgery

“With the help of AR in medicine, a surgeon can see hidden organs inside a body and improve the perception of treatment procedure by interacting with the real world.” – Grand Adventure of Augmented Reality in Landscape of Surgery

Augmented reality refers to a technique in which virtual, digitally generated objects are combined with or superimposed upon the physical world. This technology can be accessed using tools like tablets, smartphones and smart glasses. The next step past augmented reality is mixed reality, which enables users to actively interact with the digital objects as if they were real. The terms “augmented reality” and “mixed reality” are sometimes used interchangeably [4]. For a full breakdown of spatial computing technologies and how they differ from one another, read our comparison article here.

Put simply, augmented reality in preoperative planning works by collecting key patient imaging data from sources like computed tomography (CT), ultrasound imaging, positron emission tomography (PET) and magnetic resonance imaging (MRI) [5], [6]. Then, using surgical planning software, anatomical structures and surgical targets are identified and an initial surgical plan is outlined. After that, the patient data is calibrated, transferred and projected which ultimately enables the digital 3D rendering to be displayed in the real world and further assessed, allowing for the continued definition and optimization of a surgical plan [7].

While medical 3D printing materials offer surgeons the benefit of planning and practicing for surgery on a physical object that can provide useful haptic feedback, augmented reality offers the benefits of a quick-to-produce, interactive virtual environment and thereby counters some of 3D printing’s limitations.

For example, for surgical planning that requires large anatomical models, 3D printing may often require a lot more time and material to produce the 3D model [8]. In addition, surgeons using a 3D anatomical model are limited to the actual model itself as well as the physical space in which the object is present. In contrast, the creation of digital 3D models is faster than using 3D printing and, when using mixed reality, enables surgeons to collaborate from different locations. Using this approach, surgeons can also resize, zoom in on and even walk through specific parts of a patient’s anatomy, enabling them to gain important additional insights that can influence planning and beyond.

Augmented reality can also extend past the planning phase, helping surgeons ‘see inside’ the patient intraoperatively by providing a digital overlay—with all relevant anatomical and pathology information [9].

A review article from the publication, Hepatobiliary & Pancreatic Diseases International, provides an overview of how augmented reality can be used both preoperatively and intraoperatively for hepatobiliary surgery—a complex surgical procedure that treats the liver, bile duct and pancreas. In the preoperative stage, augmented reality is used to carefully examine the patient-specific liver anatomy and help predict surgical outcomes. It is also used to generate surgical navigation visual aids for use during surgery. Intraoperatively, surgeons used augmented reality to visualize internal liver structures as either superimposed images on the surgical field or in the surgical field of view facilitating greater accuracy during the procedure.

Complex surgical procedures in other clinical specialties like neurosurgery, also benefit from augmented reality as seen in the study, Application of Augmented Reality in Percutaneous Procedures—Rhizotomy of the Gasserian Ganglion.

While the advantages of augmented reality in surgical planning and navigation are substantial, the technology is still evolving to address complete data management and patient privacy, interoperability and price sensitivity [10]. In addition, the quality of patient-specific augmented and mixed reality models is highly reliant on the segmentation and rendering techniques used. However, as these advanced technologies become more widespread and further developed, these limitations will improve over time.

From theory to practice: combining 3D printing and augmented reality in clinical environments

3D printing and augmented reality have considerable overlap in terms of potential use cases for surgery. The core differentiating factor is physical versus virtual. When combined, these two technologies are complementary and can enable surgeons to have a complete and thorough understanding of an individual patient’s anatomy and the related surgical plan.

One extraordinary example of the powerful fusion between these two data visualization technologies comes from UC Davis Children’s Hospital. In October 2020, after months of meticulous planning, surgeons at the hospital successfully separated nine-month-old, conjoined twins.

“3-D printing allowed generation of multiple models of the twins’ fused skulls, useful for precision planning and practice. Surgeons were also able to explore inside their heads with the use of mixed reality goggles, which offered an augmented view of the complex network of blood vessels to be detangled and separated.” – A first-ever separation of conjoined twins at UC Davis Children’s Hospital

Other use cases of this combined approach include a study in orthopedic oncology, a case report and literature review for hard plate adenoid cystic carcinoma and a published article titled, Augmented Reality, Surgical Navigation, and 3D Printing for Transcanal Endoscopic Approach to the Petrous Apex.

Future outlook: what can we expect as these innovative tools become more widely integrated?

While this article focuses on the use of next-generation data visualization tools in the context of surgery, these new technologies, particularly augmented reality, are useful to more than just surgeons.

In the future, as these tools become more advanced, less expensive and more integrated into medical institutions and routine clinical diagnosis, they will also become more accessible to medical students and patients for educational purposes [11].

“The use of three-dimensional (3D) representations of these data in immersive settings provides new ways to explore the data and to further enhance the tools available to medical professionals in several areas including medical training, surgical planning, and intraoperative guidance.” – Application of Mixed Reality in Medical Training and Surgical Planning Focused on Minimally Invasive Surgery

For medical students, the shift from traditional surgical training that typically offers limited hands-on scenarios to training that requires more frequent and realistic simulations can increase and enhance their skills and thereby, the safety of patients [12].

Patients are also increasingly interested in gaining a more comprehensive understanding of their disease diagnosis and related treatment options. 3D visualization techniques are therefore applicable as educational materials and, as demonstrated in this study of kidney and prostate cancer patients, can contribute to increased comfort levels in relation to surgical planning.

All in all, with these major technological advancements, standards in healthcare can more rapidly transform to better serve patients as well as medical professionals. Studies assessing 3D visualization tools have already outlined an initial snapshot of how preoperative planning and surgery will evolve. As we look toward hospitals of the future, we see how the physical and virtual will be further merged, manipulated and leveraged to support more streamlined surgical procedures and improved treatment outcomes.

1. C. Lee Ventola. Medical Applications for 3D Printing: Current and Projected Uses. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4189697/. Published October 2014.

2. Shilo D, Emodi O, Blanc O, Noy D, Rachmiel A. Printing the Future—Updates in 3D Printing for Surgical Applications. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6115481/. Published 30 Jul 2018.

3. Segaran N, Saini G, Mayer J, Naidu S, Patel I, Alzubaidi S, Oklu R. Application of 3D Printing in Preoperative Planning. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7956651/. Published 26 Feb 2021.

4. Greenwood L. Mixed Reality in Medicine: Providing More Insights to Advance Surgery. https://www.brainlab.com/journal/mixed-reality-in-medicine-providing-more-insights-to-advance-surgery/. Published 16 Nov 2020.

5. Halabi O, Balakrishnan S, Dakua S, Navab N, Warfa M. Virtual and Augmented Reality in Surgery. https://link.springer.com/chapter/10.1007/978-3-030-48230-5_11. Published 14 Jul 2020.

6. Moreta-Martinez R, Garcia-Mato D, García-Mato D, García-Sevilla M, Pérez-Mañanes R, Calvo-Haro J, Pascau J. Augmented reality in computer-assisted interventions based on patient-specific 3D printed reference. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6222179/. Published 14 Sep 2018.

7. Augmented Reality 101: What It is and How It Works. https://www.magicleap.com/en-us/news/news/augmented-reality-101-what-it-is-and-how-it-works. Published 19 Jan 2022.

8. Moreta-Martinez R, Pose-Díez-de-la-Lastra A, Calvo-Haro J, Mediavilla-Santo L, Pérez-Mañanes R, Pascau J. Combining Augmented Reality and 3D Printing to Improve Surgical Workflows in Orthopedic Oncology: Smartphone Application and Clinical Evaluation. https://pubmed.ncbi.nlm.nih.gov/33672053/. Published 15 Feb 2021.

9. Greenwood L. Mixed Reality in Medicine: Providing More Insights to Advance Surgery. https://www.brainlab.com/journal/mixed-reality-in-medicine-providing-more-insights-to-advance-surgery/. Published 16 Nov 2020.

10. Khor W, Baker B, Amin K, Chan A, Patel K, Wong J. Augmented and virtual reality in surgery—the digital surgical environment: applications, limitations and legal pitfalls. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5220044/. Published 23 Dec 2016.

11. Madison D. The future of augmented reality in healthcare. https://healthmanagement.org/c/healthmanagement/issuearticle/the-future-of-augmented-reality-in-healthcare. Published 1 Jan 2018.