In May 2026, the UK's National Health Service (NHS) performed its first clinical surgery using a technology called "AI Color Vision Navigation." At a teaching hospital in London, surgeons wearing augmented reality headsets saw a "color-coded" surgical field — different tissue types (arteries, veins, nerves, tumors, healthy tissue) were labeled in real time with distinct colors by AI, helping doctors make more precise decisions during surgery. This technology represents a pivotal shift from AI-assisted diagnosis toward AI-surgeon symbiosis in the operating room.
The Technology: How AI Colors the Surgical Field
Visual identification during surgery has always been a core challenge for surgeons. The human eye struggles to distinguish certain tissue boundaries under natural light — such as the transition zone between tumor edges and surrounding healthy tissue, the difference between minute nerve fibers and connective tissue, and the "sentinel" status of various lymph nodes. Traditionally, surgeons rely on "mental spatial mapping" from preoperative imaging (CT, MRI) — memorizing the positions on scans before surgery and then correlating them to actual tissue through experience during the operation.
The AI color vision navigation system addresses this challenge through the following pipeline:
1. Multimodal Data Fusion: Before surgery, the system ingests the patient's preoperative CT, MRI, and PET scan data to build a personalized three-dimensional anatomical model. These data are used to train a foundational "tissue semantic map."
2. Real-time Optical Feature Analysis: During surgery, hyperspectral cameras (covering visible and near-infrared wavelengths) capture multispectral images of the surgical field. Different tissue types possess unique "spectral fingerprints" — for instance, oxygenated hemoglobin in arteries and deoxygenated hemoglobin in veins have different spectral absorption characteristics, and tumor tissue has higher microvascular density than healthy tissue, resulting in distinct scattering properties.
3. Deep Learning Inference: The AI model performs pixel-level registration of real-time spectral data with preoperative images, outputting a tissue-type probability distribution for each pixel in milliseconds. The model is based on an improved version of the U-Net architecture, trained on data from over 50,000 surgical recordings and corresponding postoperative pathology gold standards.
4. Augmented Reality Overlay: The processed results are overlaid onto the surgeon's field of view through augmented reality headsets (such as Microsoft HoloLens or Magic Leap) — arteries are marked in red, veins in blue, nerves in yellow, tumor boundaries in green dashed lines, and the tumor body in purple. Color coding and transparency can be adjusted by the surgeon in real time.
Results of the First Clinical Application
In the NHS's initial deployment, the research team selected head and neck tumor resection — one of the most anatomically complex surgery types with the highest density of critical tissues. The head and neck region contains the carotid artery, internal jugular vein, vagus nerve, hypoglossal nerve, and many other vital structures, damage to which can be fatal or cause permanent functional impairment.
Specific Case Data:
- Patient: 62-year-old, T3 stage oral squamous cell carcinoma
- Procedure: Selective neck lymph node dissection + wide local excision of primary lesion
- System: AI Color Vision Navigation + HoloLens 3 AR Headset
Outcomes:
- The AI system correctly identified 11 out of 12 cervical lymph nodes (sensitivity 91.7%), including 2 micrometastases undetectable by naked eye or palpation
- All critical nerves and blood vessels were successfully labeled, with zero nerve damage during surgery
- Operative time was extended by only approximately 7 minutes compared to standard surgery (primarily for system configuration and calibration)
Surgeon feedback: "Being able to 'see' tissue boundaries that are invisible to the naked eye is like switching from black-and-white TV to color TV. The biggest benefit is the increased confidence in decision-making — especially when determining tumor resection margins."
Progress in Other Regions
Beyond the UK, medical institutions in several countries are advancing this technology:
United States: The Johns Hopkins University School of Medicine launched the "AI-SIGHT" clinical trial in January 2026, aiming to enroll 500 patients covering liver, pancreatic, and brain surgeries. Preliminary results (120 cases) show an R0 resection rate (microscopically negative margins) of 93% in the AI-navigated group versus 78% in the control group.
Japan: Tokyo Medical and Dental University has developed an endoscope-based AI color navigation system specifically for endoscopic submucosal dissection (ESD) of gastric cancer. AI assistance increased the curative resection rate for early gastric cancer from 82% to 94%.
Germany: A research team at the Technical University of Munich has extended the technology to robot-assisted surgery (da Vinci Xi system), solving the registration problem from 2D endoscopic images to 3D AR overlay — with even more significant implications for precision surgery.
Key Breakthrough: Clinical Deployment of Hyperspectral Imaging
The maturity of AI color navigation systems owes much to the miniaturization and cost reduction of hyperspectral imaging hardware. In 2022, a surgical-grade hyperspectral camera system cost approximately $250,000–$300,000 and was about the size of a shoebox. By 2026, a system with comparable performance costs approximately $80,000–$120,000 and has shrunk to the size of a smartphone, enabling direct integration into surgical microscopes or endoscopes.
This decline is driven by three technological advances:
- Snapshot Hyperspectral Sensors: Replacing traditional scanning spectrometers, these capture complete spectral data in a single exposure
- Edge AI Processors: NVIDIA's Orin architecture chips enable real-time inference of spectral data on-site, eliminating the need for cloud processing
- Standardized Spectral Databases: A multi-center collaborative human tissue spectral database (containing spectral curves from over 1 million tissue samples) has dramatically improved AI model generalization
Challenges and Limitations
Anatomical Depth Limitations: Current systems cannot penetrate deep tissue (beyond 5 mm depth). For vital structures beneath the surface (such as the ureter encased in fatty tissue), current near-infrared spectroscopy cannot provide useful information. Researchers are exploring fusion with intraoperative ultrasound.
Tissue Shift: During surgery, tissue deforms due to cutting, retraction, and edema, causing registration errors between preoperative images and the real-time surgical field. This is especially pronounced in brain surgery — "brain shift" can displace a tumor's location from preoperative MRI by more than 1 cm. Adaptive registration algorithms are a major focus of current research.
Surgeon Learning Curve: Although AI color navigation is highly intuitive, surgeons need training to use AR overlay information effectively. Excessive visual information may cause "information overload," particularly among less experienced surgeons. Current training protocols recommend that surgeons complete at least 20 simulation sessions before being cleared for live surgery.
Regulatory Pathway: AI color navigation systems are classified as Class IIb (EU MDR) or Class II (US FDA) medical devices in most countries. However, as systems expand from "assistive visualization" to "automated tissue classification" and "margin decision recommendations," their regulatory classification may escalate to Class III, requiring longer prospective clinical trial data.
Forward Outlook
AI surgical vision navigation represents the evolution of surgery from "experience-based operation" toward "data-driven precision intervention." In the next 3-5 years, we may witness:
- Color navigation becoming the "standard configuration" for complex surgeries — analogous to the widespread adoption of intraoperative navigation in neurosurgery
- AI expanding from visual assistance to decision assistance — not only telling you "what tissue this is" but also suggesting "the optimal resection path"
- Personalized surgical prediction — predicting postoperative functional outcomes of different resection plans based on a patient's specific anatomical and pathological characteristics before surgery
- Visual enhancement for remote surgery — remote surgeons can more accurately understand the surgical field through color navigation systems
Just as minimally invasive surgery reshaped the surgical landscape of the late 20th century, AI-enhanced color vision may define the surgical standard of the mid-21st century. When surgeons can "see the invisible," surgical precision and safety will reach unprecedented levels.
Disclaimer: This article is for informational purposes only and does not constitute investment advice or a basis for business decisions. Data and time-sensitive information are accurate as of the publication date and may change with subsequent developments. Neither the author nor POC.HK assumes any liability for losses arising from the use of this information.