SurgLAT: Learning Latent Surgical Attention for Depth-Aware Robotic Laparoscope Control

Abstract

Autonomous laparoscopic camera control requires continuous understanding of the surgeon's operative intent in dynamic surgical scenes, where the target operative region is not a stable physical object but a latent and temporally evolving attention state. In this work, we present SurgLAT, a causal online framework for latent surgical attention modeling and autonomous laparoscopic view control.

SurgLAT leverages a frozen DINOv3 encoder and a state-conditioned spatial token mixer to extract operative evidence under a memory-guided spatial prior, while a selective causal latent memory module jointly models short-term motion continuity and long-horizon surgical intent evolution through dynamic retrieval of current, recent, and historical latent states. The learned latent surgical attention state is decoded into a probabilistic attention heatmap and operative region for downstream endoscope guidance.

Beyond perception, we further introduce a robotic deployment framework with explicit laparoscopic Remote Center of Motion constrained control based on virtual-axis formulation, together with redundancy-aware null-space initialization for stable and smooth manipulator motion. Experiments on real laparoscopic videos and a physical robotic laparoscope platform demonstrate robust online tracking and stable autonomous endoscopy adjustment under occlusion, rapid motion, and target transitions.

The pipeline combines depth-aware operative scale estimation, motion-conditioned depth setpoint prediction, memory-conditioned spatial priors, spatial token mixing, selective latent memory, and a state-conditioned attention decoder. The decoded operative region and depth target are then sent to an RCM-constrained robotic FoV executor for stable autonomous laparoscope motion.