Project Details
Project description 

Since fall 2017, a multidisciplinary team worked on an augmented reality (AR) solution for surgical navigation for orthopedic and neurosurgery in the context of the imec.icon SARA. Unique technical building blocks have been developed, which allowed for the development of clinical applications. Currently, phantom trials have proven a clear benefit of using our solution when compared to standard clinical practice. Patient trials are ongoing. A recent imec trajectory has revealed technological uniqueness, clear valorization potential, but also weaknesses in the current maturity of the complete product. Imec venture track funding has been applied for. IOF-POC funding serves to further mature the demonstration software, in particular the desktop side of the AR solution.

AR in surgical applications centers around accurately displaying preoperative imaging in 3D on top of the actual patient’s anatomy. This requires the headset to continuously know its location and orientation in space, relative to the patient (this is called tracking), such that 3D holograms can be projected correctly. Out of the box, the Microsoft Hololens – the most suited device for this kind of applications – reaches a tracking accuracy in the order of magnitude of a few centimeters, whereas the surgical applications require an accuracy 2mm. The uniqueness of the SARA work is that this accuracy was obtained by using only the onboard Hololens sensors (i.e. inside-out tracking), in combination with infrared markers placed on or near the patient. This makes that the SARA solution can provide similar functionality as the very expensive professional neuro-navigation systems, which are due to their size and complexity restricted to operating room (OR) use. Thanks to SARA, a large variety of applications inside and outside the OR become feasible. The unique tracking method is combined with a method to calibrate the display for every individual user (related to eye position vs. the glasses). In phantom trials investigating intracranial drain placement with AR based guidance, the solution has proven to significantly increase accuracy in untrained participants in comparison to the current practice. In directional drilling experiments, errors were reduced by half when using AR compared to proprioception-guided or blind drilling.

In this POC application, the AR solution will be complemented with a web-based application for preoperative image segmentation. Such applications allow the clinician to segment anatomical structures in the images, to perform measurements based on the images and to perform surgical planning, including enriching the images with various annotations that will be relied on during the surgical procedure. Based on all this information, 3D holograms can then be generated suited for visualization on the AR device. Medical image segmentation is performed semi-automatically, i.e., computer algorithms perform the task, but review by an expert is required, as certain pathology or deformations can cause these methods to fail. In case such a failure is observed, the software must provide the functionality to manually correct these segmentations. The POC funding will be used to develop exactly this interactive software application, such that we can offer the complete workflow needed for surgical augmented reality. Without developing this critical missing part in the complete pipeline, our offering will remain dependent on external software partners, lowering our business value significantly.

Runtime: 2021 - 2021