From video understanding to surgical accountability
Surgery is full of decisive visual events that happen quickly, disappear from view, and only become clinically meaningful much later.
A sponge enters the abdomen.
A needle is passed into the field.
A clip is deployed.
A gauze swab is moved, hidden, retrieved, or possibly forgotten.
For a human surgical team, the safety-critical question is not simply: “What is visible in this frame?”
The real question is:
What entered the surgical field, where did it go, was it retrieved, and can we prove that it was accounted for before closure?
That is exactly the challenge posed by ORena SAVE FOCUS — Foreign Object Contextual Understanding for Safe Surgical AI.
ORena describes SAVE FOCUS as a challenge designed to test whether AI can truly comprehend full-length surgeries in order to solve clinically relevant safety problems, specifically the tracking, contextualisation, counting, localisation, and retrieval verification of foreign objects such as sponges and needles during minimally invasive surgery.
At MAIVAN.ai, we believe this is more than a benchmark. It is a test of the architecture required for the next generation of surgical intelligence.
And our thesis is simple:
SurgiCorder® with NVIDIA VSS could become the backbone of a winning ORena SAVE FOCUS solution — but the winning architecture is not “a VLM watching a video.” It is VSS plus a domain-specific surgical foreign-object ledger.
In other words: the winning system should not behave like a chatbot.
It should behave like a surgical item accountant.
The ORena SAVE FOCUS challenge
SAVE FOCUS is organised around three levels of increasing difficulty:
- Frame Track — single-image surgical scene understanding.
- Segment Track — short video understanding over clips of up to five minutes.
- Procedure Track — long-context reasoning over extended or full-length laparoscopic procedures.
The clinical goal is to help prevent retained surgical items by enabling AI systems to track, count, localise, and verify retrieval of foreign objects throughout surgery. The technical goal is to benchmark long-context video understanding, from single-frame perception to full-procedure reasoning.
This distinction matters.
Many current medical AI systems are good at static recognition. They can detect anatomy, tools, lesions, implants, or workflow phases. But SAVE FOCUS asks something deeper:
Can an AI system maintain a reliable memory of safety-critical objects across time?
That is the hard problem.
And it is precisely why NVIDIA’s Video Search and Summarisation, or VSS, is such an interesting foundation.
Why NVIDIA VSS is the right backbone
NVIDIA VSS is a reference architecture for building visual agents that can ingest stored or streamed video, generate captions, index video semantically, search across video, summarise events, and answer natural-language questions using vision-language models, large language models, NIM microservices, and hybrid retrieval.
In NVIDIA’s own documentation, VSS includes real-time computer vision, object detection, tracking, embedding, VLM captioning, downstream analytics, and alert-style workflows. It can process video, images, and live streams, generate semantic embeddings, and use VLMs such as Cosmos Reason or Qwen-based models to caption and reason over visual content.
That makes VSS highly relevant to ORena SAVE FOCUS because the problem is fundamentally a long-video retrieval and reasoning problem.
A retained surgical item is not detected by looking at one image. It is detected by reconstructing a chain of events:
object inserted
→ object manipulated
→ object moved
→ object occluded
→ object reappears
→ object removed or not removed
→ procedure closesVSS can provide the scalable video intelligence layer. But VSS alone is not enough.
The missing layer is surgical accountability.
The key idea: a surgical foreign-object ledger
The winning architecture should transform surgical video into a structured, timestamped ledger of foreign objects.
Every relevant object receives an identity, a timeline, a location history, and a retrieval status.
For example:
Object ID: sponge_02
Class: sponge / gauze
First seen: 00:18:04
Last seen: 01:12:55
Current status: unresolved
Last known location: lower-right abdomen
Evidence clips: 00:18:04 insertion, 00:43:18 manipulation, 01:12:55 last visible
Removal evidence: none detected
Confidence: 0.82This is the architectural shift.
Instead of asking a vision-language model to “remember” an hour-long procedure, the system externalises memory into a structured object ledger.
The VLM sees.
VSS retrieves.
NemoClaw orchestrates.
The ledger remembers.
That is the essence of surgical foreign-object accounting.
How this architecture wins the Frame Track
The Frame Track tests whether the system can answer clinically relevant questions from a single image: object detection, object identification, attribute recognition, counting, and localisation.
A VSS-based system can handle the frame as a single visual input, but the winning solution should add a specialised surgical perception layer:
image preprocessing
→ foreign-object detector
→ segmentation mask
→ VLM verification
→ spatial localisation
→ answer generationThe model must distinguish between:
- a sponge and tissue,
- a needle and a shiny instrument edge,
- a clip and a reflection,
- a foreign object and a connected surgical instrument.
For the Frame Track, the surgical ledger is not yet long-range memory. It is a structured perception output.
Example:
{
"foreign_objects": [
{
"class": "sponge",
"count": 1,
"location": "lower-left quadrant",
"visibility": "partially visible",
"interaction": "held by grasper",
"confidence": 0.91
}
]
}The winning system should not merely answer:
“There is a sponge.”
It should answer:
“One partially visible sponge is present in the lower-left field, held by a grasper, and should be counted as a foreign object.”
That difference matters.
How this architecture wins the Segment Track
The Segment Track tests short-term temporal reasoning over video clips of up to five minutes. This is where static perception becomes event understanding.
In a short segment, the system must determine whether an object was:
- inserted,
- removed,
- manipulated,
- dropped,
- occluded,
- still present,
- or merely moved out of view.
This is a classic failure mode for naive video AI.
An object disappearing from view does not mean it was removed.
The segment-level architecture should therefore look like this:
VSS clip ingestion
→ adaptive keyframe extraction
→ foreign-object detection per frame
→ object tracking
→ event classification
→ local object ledger
→ VQA answerThe most important addition is event classification.
For example:
00:00:12 — sponge appears, held by grasper
00:00:19 — sponge is released into abdominal cavity
00:01:47 — sponge is moved to lower-centre field
00:02:31 — sponge is occluded behind tissue
00:03:02 — sponge reappears
00:04:12 — sponge is grasped and removed through trocarThis allows the model to answer questions such as:
- “Was a sponge removed during this clip?”
- “How many foreign objects were introduced?”
- “Where was the needle last seen?”
- “Did the object remain in the abdomen at the end of the segment?”
VSS provides the searchable video intelligence.
The object ledger provides temporal accountability.
How this architecture wins the Procedure Track
The Procedure Track is the decisive one.
This is where the challenge moves from visual recognition to surgical memory.
A full procedure may contain many object events separated by long intervals. A sponge inserted early may be moved many times, hidden behind tissue, disappear, reappear, and only become relevant when the system is asked whether all items were retrieved before closure.
This is not a “long prompt” problem.
It is a memory architecture problem.
The Procedure Track should therefore be solved through a full-procedure object ledger.
The pipeline:
full surgical video
→ VSS ingestion and chunking
→ dense captioning and embeddings
→ object-event extraction
→ cross-chunk identity resolution
→ procedure-level foreign-object ledger
→ retrieval-aware answer generationThe critical step is cross-chunk identity resolution.
The system must decide whether a sponge seen at 00:42:11 is the same sponge first inserted at 00:18:04, or a different sponge introduced later. This requires a combination of:
- visual appearance,
- object class,
- location continuity,
- trajectory,
- instrument interaction,
- occlusion reasoning,
- insertion/removal event logic,
- and temporal consistency.
The ledger then becomes the source of truth:
Object Type First seen Last seen Status
sponge_01 sponge 00:07:32 00:41:19 retrieved
sponge_02 sponge 00:18:04 01:12:55 unresolved
needle_01 needle 00:52:11 00:54:08 retrieved
bag_01 specimen 01:03:20 01:15:44 retrievedNow the system can answer the clinically decisive question:
Were all foreign objects accounted for before closure?
Not by guessing.
Not by hallucinating.
But by querying the ledger and retrieving the exact evidence clips.
That is the difference between video understanding and surgical accountability.
VSS + NemoClaw + object ledger
For MAIVAN.ai, the most interesting architecture is not VSS alone. It is:
NVIDIA VSS
+ NemoClaw orchestration
+ domain-specific surgical object ledgerEach component has a clear role.
NVIDIA VSS provides the video intelligence backbone:
- video ingestion,
- chunking,
- embeddings,
- dense captions,
- visual Q&A,
- semantic search,
- evidence retrieval.
NemoClaw provides the agentic orchestration layer:
- calls the right perception tools,
- manages track-specific workflows,
- routes questions,
- invokes object-detection and tracking tools,
- queries the ledger,
- composes answers,
- preserves auditability.
The surgical object ledger provides the accountable memory:
- object identity,
- object state,
- event history,
- last-seen timestamp,
- retrieval status,
- unresolved-item warning,
- evidence trace.
Together, this becomes more than a competition stack.
It becomes a general-purpose surgical intelligence infrastructure.
Why this matters for MAIVAN, SurgiCorder®, and ORTHO-X
For MAIVAN, ORTHO-X and SurgiCorder®, a strong ORena result on the Procedure Track would not merely be a competition win.
It would be a proof of architecture for the entire SurgiCorder®/ORTHO-X intra-operative documentation thesis.
Why?
Because the core problem is the same.
In ORena SAVE FOCUS, the objects are sponges, needles, clips, and other foreign bodies in laparoscopic surgery.
In ORTHO-X and SurgiCorder®, the equivalent objects may be:
- guidewires,
- drills,
- screws,
- plates,
- nails,
- K-wires,
- implants,
- trial components,
- definitive components,
- broken instrument fragments,
- cement,
- surgical steps,
- fluoroscopy events,
- intra-operative decisions.
The architecture generalises:
laparoscopic foreign-object accounting
→ orthopaedic implant and instrument accounting
→ intra-operative case documentation
→ structured surgical evidence
→ training, safety, registry, MDR/PMCF workflowsA system that can maintain a reliable ledger of sponges and needles across a full laparoscopic procedure can be adapted to maintain a reliable ledger of implants, instruments, decisions, and fluoroscopy events across orthopaedic trauma surgery.
This is the heart of the SurgiCorder® thesis:
Surgery should not remain an unstructured video file.
Surgery should become a structured, timestamped, searchable, auditable clinical record.
From competition benchmark to surgical intelligence infrastructure
The long-term opportunity is not just to answer VQA questions.
It is to create a new class of intra-operative systems:
live surgical video
→ multimodal perception
→ object and event ledger
→ surgeon-supervised confirmation
→ structured case record
→ training asset
→ outcome-linked evidence
→ regulatory-grade documentationIn laparoscopy, this could support retained-item prevention.
In orthopaedics, it could support:
- implant-use documentation,
- guidewire and screw-position workflows,
- surgical step recognition,
- case-based training,
- complication analysis,
- post-market clinical follow-up,
- implant registry enrichment,
- and real-world evidence generation.
This is where ORena SAVE FOCUS becomes strategically important for MAIVAN.
It is not only a benchmark for surgical video understanding.
It is a proving ground for trusted surgical memory.
The core architectural principle
The lesson from SAVE FOCUS is clear:
The future of surgical AI is not just better perception.
It is accountable perception over time.
A frame model sees.
A segment model follows.
A procedure model remembers.
But in surgery, remembering is not enough. The system must remember in a way that can be audited, queried, verified, and corrected by the surgical team.
That is why the right architecture is not simply:
video → VLM → answerThe right architecture is:
video → VSS → surgical agents → object ledger → evidence-backed answerThis is the architecture that could win ORena SAVE FOCUS.
And more importantly, it is the architecture that could become the foundation for SurgiCorder®, OrthoClaw, and ORTHO-X.
Conclusion: from SAVE FOCUS to SurgiCorder®
ORena SAVE FOCUS asks a deceptively simple question:
Can AI understand a full surgery well enough to help prevent retained foreign objects?
Our answer is:
Yes — but only if we stop treating surgery as a long video and start treating it as a structured, evolving clinical state.
NVIDIA VSS provides the scalable long-video intelligence layer.
NemoClaw can orchestrate the surgical reasoning workflow.
A domain-specific object ledger provides the accountable memory.
Together, they form a generalisable surgical intelligence infrastructure:
VSS + NemoClaw + surgical object ledger
= accountable intra-operative intelligenceFor MAIVAN.ai, a strong ORena Procedure Track result would therefore prove something much larger than benchmark performance.
It would validate the core SurgiCorder®/ORTHO-X thesis:
The operating room can become a structured, searchable, auditable source of clinical intelligence — first for safety, then for training, evidence, outcomes, and better surgical care.
Leave a Reply