An Egocentric Life-Saving Interventional Procedure Dataset of Actions, Medical Questions, Maneuvers and Tools

Yupeng Zhuo; Eddie Zhang; Xiangchen Yu; Aditya Pachpande; Wenqu Fang; Xiyue Chen; Andrew W. Kirkpatrick; Kyle Couperus; Christopher Colombo; Oanh Tran; Jonah Beck; De Anna DeVane; Jessica McKee; Chad Gorbatkin; Ross Candelore; Eleanor Birch; Juan Wachs

doi:10.1038/s41597-025-06365-y

An Egocentric Life-Saving Interventional Procedure Dataset of Actions, Medical Questions, Maneuvers and Tools

Yupeng Zhuo
, Eddie Zhang
, Xiangchen Yu
, Aditya Pachpande
, Wenqu Fang
, Xiyue Chen
, Andrew W. Kirkpatrick
, Kyle Couperus
, Christopher Colombo
, Oanh Tran
, Jonah Beck
, De Anna DeVane
, Jessica McKee
, Chad Gorbatkin
, Ross Candelore
, Eleanor Birch
, Juan Wachs^*

^*Corresponding author for this work

Medicine

Research output: Contribution to journal › Article › peer-review

Abstract

This paper introduces the Trauma THOMPSON dataset, designed to advance AI-driven decision support for life-saving interventions (LSIs) in emergency care, particularly in resource constrained humanitarian settings. The dataset comprises 3,717 high resolution and egocentric video clips of both regular and just-in-time (JIT) procedures. The JIT procedures consists of videos of the same LSI procedures, but with makeshift tools, and is useful for studying human medical commonsense. Each clip is annotated by medical professionals with verb-noun format, such as “take scalpel” and “make incision”. In addition to action segments, the dataset includes annotations for medical visual question answering (MVQA), hand maneuvers, and object detection. Eventually, these rich annotations and dataset can be used to train an AI agent to advise first-responders in the field about what to do next with the resources at hand. We provide benchmarks for action recognition, anticipation, and MVQA using state-of-the-art machine learning models.

Original language	English
Article number	51
Journal	Scientific Data
Volume	13
Issue number	1
DOIs	https://doi.org/10.1038/s41597-025-06365-y
State	Published - Dec 2026

Access to Document

10.1038/s41597-025-06365-y

Cite this

Zhuo, Y., Zhang, E., Yu, X., Pachpande, A., Fang, W., Chen, X., Kirkpatrick, A. W., Couperus, K., Colombo, C., Tran, O., Beck, J., DeVane, D. A., McKee, J., Gorbatkin, C., Candelore, R., Birch, E., & Wachs, J. (2026). An Egocentric Life-Saving Interventional Procedure Dataset of Actions, Medical Questions, Maneuvers and Tools. Scientific Data, 13(1), Article 51. https://doi.org/10.1038/s41597-025-06365-y

@article{bcc3ae1982704c1389827f2eaef409cf,

title = "An Egocentric Life-Saving Interventional Procedure Dataset of Actions, Medical Questions, Maneuvers and Tools",

abstract = "This paper introduces the Trauma THOMPSON dataset, designed to advance AI-driven decision support for life-saving interventions (LSIs) in emergency care, particularly in resource constrained humanitarian settings. The dataset comprises 3,717 high resolution and egocentric video clips of both regular and just-in-time (JIT) procedures. The JIT procedures consists of videos of the same LSI procedures, but with makeshift tools, and is useful for studying human medical commonsense. Each clip is annotated by medical professionals with verb-noun format, such as {\textquotedblleft}take scalpel{\textquotedblright} and {\textquotedblleft}make incision{\textquotedblright}. In addition to action segments, the dataset includes annotations for medical visual question answering (MVQA), hand maneuvers, and object detection. Eventually, these rich annotations and dataset can be used to train an AI agent to advise first-responders in the field about what to do next with the resources at hand. We provide benchmarks for action recognition, anticipation, and MVQA using state-of-the-art machine learning models.",

author = "Yupeng Zhuo and Eddie Zhang and Xiangchen Yu and Aditya Pachpande and Wenqu Fang and Xiyue Chen and Kirkpatrick, \{Andrew W.\} and Kyle Couperus and Christopher Colombo and Oanh Tran and Jonah Beck and DeVane, \{De Anna\} and Jessica McKee and Chad Gorbatkin and Ross Candelore and Eleanor Birch and Juan Wachs",

note = "Publisher Copyright: {\textcopyright} The Author(s) 2025.",

year = "2026",

month = dec,

doi = "10.1038/s41597-025-06365-y",

language = "English",

volume = "13",

journal = "Scientific Data",

issn = "2052-4463",

publisher = "Nature Research",

number = "1",

}

Zhuo, Y, Zhang, E, Yu, X, Pachpande, A, Fang, W, Chen, X, Kirkpatrick, AW, Couperus, K, Colombo, C, Tran, O, Beck, J, DeVane, DA, McKee, J, Gorbatkin, C, Candelore, R, Birch, E & Wachs, J 2026, 'An Egocentric Life-Saving Interventional Procedure Dataset of Actions, Medical Questions, Maneuvers and Tools', Scientific Data, vol. 13, no. 1, 51. https://doi.org/10.1038/s41597-025-06365-y

TY - JOUR

T1 - An Egocentric Life-Saving Interventional Procedure Dataset of Actions, Medical Questions, Maneuvers and Tools

AU - Zhuo, Yupeng

AU - Zhang, Eddie

AU - Yu, Xiangchen

AU - Pachpande, Aditya

AU - Fang, Wenqu

AU - Chen, Xiyue

AU - Kirkpatrick, Andrew W.

AU - Couperus, Kyle

AU - Colombo, Christopher

AU - Tran, Oanh

AU - Beck, Jonah

AU - DeVane, De Anna

AU - McKee, Jessica

AU - Gorbatkin, Chad

AU - Candelore, Ross

AU - Birch, Eleanor

AU - Wachs, Juan

PY - 2026/12

Y1 - 2026/12

N2 - This paper introduces the Trauma THOMPSON dataset, designed to advance AI-driven decision support for life-saving interventions (LSIs) in emergency care, particularly in resource constrained humanitarian settings. The dataset comprises 3,717 high resolution and egocentric video clips of both regular and just-in-time (JIT) procedures. The JIT procedures consists of videos of the same LSI procedures, but with makeshift tools, and is useful for studying human medical commonsense. Each clip is annotated by medical professionals with verb-noun format, such as “take scalpel” and “make incision”. In addition to action segments, the dataset includes annotations for medical visual question answering (MVQA), hand maneuvers, and object detection. Eventually, these rich annotations and dataset can be used to train an AI agent to advise first-responders in the field about what to do next with the resources at hand. We provide benchmarks for action recognition, anticipation, and MVQA using state-of-the-art machine learning models.

AB - This paper introduces the Trauma THOMPSON dataset, designed to advance AI-driven decision support for life-saving interventions (LSIs) in emergency care, particularly in resource constrained humanitarian settings. The dataset comprises 3,717 high resolution and egocentric video clips of both regular and just-in-time (JIT) procedures. The JIT procedures consists of videos of the same LSI procedures, but with makeshift tools, and is useful for studying human medical commonsense. Each clip is annotated by medical professionals with verb-noun format, such as “take scalpel” and “make incision”. In addition to action segments, the dataset includes annotations for medical visual question answering (MVQA), hand maneuvers, and object detection. Eventually, these rich annotations and dataset can be used to train an AI agent to advise first-responders in the field about what to do next with the resources at hand. We provide benchmarks for action recognition, anticipation, and MVQA using state-of-the-art machine learning models.

UR - https://www.scopus.com/pages/publications/105027907517

U2 - 10.1038/s41597-025-06365-y

DO - 10.1038/s41597-025-06365-y

M3 - Article

C2 - 41413427

AN - SCOPUS:105027907517

SN - 2052-4463

VL - 13

JO - Scientific Data

JF - Scientific Data

IS - 1

M1 - 51

ER -

An Egocentric Life-Saving Interventional Procedure Dataset of Actions, Medical Questions, Maneuvers and Tools

Abstract

Access to Document

Fingerprint

Cite this