TY - GEN
T1 - TON-ViT
T2 - 27th Annual Conference on Medical Image Understanding and Analysis, MIUA 2023
AU - Zhuo, Yupeng
AU - Jiang, Nina
AU - Kirkpatrick, Andrew W.
AU - Couperus, Kyle
AU - Tran, Oanh
AU - Beck, Jonah
AU - Devane, Deanna
AU - Candelore, Ross
AU - McKee, Jessica
AU - Gorbatkin, Chad
AU - Birch, Eleanor
AU - Colombo, Christopher
AU - Duerstock, Bradley
AU - Wachs, Juan
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
PY - 2024
Y1 - 2024
N2 - The objective of this paper is to present a neuro-symbolic AI based technique to represent field-medicine knowledge, referred as to TON-ViT. TON-ViT integrates a Deep Learning Model with an explicit symbolic manipulation, a task graph. This task graph describes the steps of each trauma resuscitation as denoted by a verb and noun pair. Through this representation, symbolic processing and manipulation on task graphs, we can find stereotypical procedures, regardless of style of the performer. Furthermore, we can use this technique to find differences in styles, errors, shortcuts and generate procedures never seen before. When used in combination with a transformer, it can help recognize actions in egocentric vision datasets. Last, through symbolic manipulations on the graph, it is possible to generate medical knowledge which the model has not seen before. We present preliminary results after testing the TON-ViT with the Trauma Thompson Dataset.
AB - The objective of this paper is to present a neuro-symbolic AI based technique to represent field-medicine knowledge, referred as to TON-ViT. TON-ViT integrates a Deep Learning Model with an explicit symbolic manipulation, a task graph. This task graph describes the steps of each trauma resuscitation as denoted by a verb and noun pair. Through this representation, symbolic processing and manipulation on task graphs, we can find stereotypical procedures, regardless of style of the performer. Furthermore, we can use this technique to find differences in styles, errors, shortcuts and generate procedures never seen before. When used in combination with a transformer, it can help recognize actions in egocentric vision datasets. Last, through symbolic manipulations on the graph, it is possible to generate medical knowledge which the model has not seen before. We present preliminary results after testing the TON-ViT with the Trauma Thompson Dataset.
KW - knowledge graph
KW - medical procedures
KW - neuro-symbolic AI
KW - semantic understanding
KW - task graph
KW - vision transformer
UR - http://www.scopus.com/inward/record.url?scp=85185554630&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-48593-0_12
DO - 10.1007/978-3-031-48593-0_12
M3 - Conference contribution
AN - SCOPUS:85185554630
SN - 9783031485923
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 157
EP - 170
BT - Medical Image Understanding and Analysis - 27th Annual Conference, MIUA 2023, Proceedings
A2 - Waiter, Gordon
A2 - Leontidis, Georgios
A2 - Morris, Teresa
A2 - Lambrou, Tryphon
A2 - Oren, Nir
A2 - Gordon, Sharon
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 19 July 2023 through 21 July 2023
ER -