{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T12:46:25Z","timestamp":1775825185181,"version":"3.50.1"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"8","license":[{"start":{"date-parts":[[2021,4,22]],"date-time":"2021-04-22T00:00:00Z","timestamp":1619049600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,22]],"date-time":"2021-04-22T00:00:00Z","timestamp":1619049600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["42671"],"award-info":[{"award-number":["42671"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"published-print":{"date-parts":[[2021,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>The automatic extraction of knowledge about intervention execution from surgical manuals would be of the utmost importance to develop expert surgical systems and assistants. In this work we assess the feasibility of automatically identifying the sentences of a surgical intervention text containing procedural information, a subtask of the broader goal of extracting intervention workflows from surgical manuals.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We frame the problem as a binary classification task. We first introduce a new public dataset of 1958 sentences from robotic surgery texts, manually annotated as procedural or non-procedural. We then apply different classification methods, from classical machine learning algorithms, to more recent neural-network approaches and classification methods exploiting transformers (e.g., BERT, ClinicalBERT). We also analyze the benefits of applying balancing techniques to the dataset.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The architectures based on neural-networks fed with FastText\u2019s embeddings and the one based on ClinicalBERT outperform all the tested methods, empirically confirming the feasibility of the task. Adopting balancing techniques does not lead to substantial improvements in classification.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>This is the first work experimenting with machine \/ deep learning algorithms for automatically identifying procedural sentences in surgical texts. It also introduces the first public dataset that can be used for benchmarking different classification methods for the task.\n<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s11548-021-02370-9","type":"journal-article","created":{"date-parts":[[2021,4,22]],"date-time":"2021-04-22T07:03:09Z","timestamp":1619074989000},"page":"1287-1295","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Automatic detection of procedural knowledge in robotic-assisted surgical texts"],"prefix":"10.1007","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8607-8495","authenticated-orcid":false,"given":"Marco","family":"Bombieri","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9391-3201","authenticated-orcid":false,"given":"Marco","family":"Rospocher","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7300-5975","authenticated-orcid":false,"given":"Diego","family":"Dall\u2019Alba","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0711-8605","authenticated-orcid":false,"given":"Paolo","family":"Fiorini","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,4,22]]},"reference":[{"issue":"3","key":"2370_CR1","first-page":"62","volume":"19","author":"M Abbas","year":"2019","unstructured":"Abbas M, Ali K, Memon S, Jamali A, Memon S, Ahmed A (2019) Multinomial Naive Bayes classification model for sentiment analysis. IJCSNS Int J Comput Sci Netw Secur 19(3):62\u201367","journal-title":"IJCSNS Int J Comput Sci Netw Secur"},{"key":"2370_CR2","unstructured":"Agarwal S, Atreja, S, Agarwal V (2020) Extracting procedural knowledge from technical documents. arXiv preprint arXiv:2010.10156"},{"key":"2370_CR3","doi-asserted-by":"crossref","unstructured":"Alsentzer E, Murphy J, Boag W, Weng WH, Jin D, Naumann T, McDermott M (2019) Publicly available clinical BERT embeddings. In: Proceedings of the 2nd clinical natural language processing workshop, Association for Computational Linguistics, Minneapolis, Minnesota, USA, pp 72\u201378","DOI":"10.18653\/v1\/W19-1909"},{"key":"2370_CR4","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1145\/1007730.1007735","volume":"6","author":"G Batista","year":"2004","unstructured":"Batista G, Prati R, Monard MC (2004) A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explor 6:20\u201329","journal-title":"SIGKDD Explor"},{"issue":"70","key":"2370_CR5","first-page":"2079","volume":"11","author":"GC Cawley","year":"2010","unstructured":"Cawley GC, Talbot NLC (2010) On over-fitting in model selection and subsequent selection bias in performance evaluation. J Mach Learn Res 11(70):2079\u20132107","journal-title":"J Mach Learn Res"},{"key":"2370_CR6","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)"},{"key":"2370_CR7","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-91045-1","volume-title":"The SAGES atlas of robotic surgery","author":"Y Fong","year":"2018","unstructured":"Fong Y, Woo Y, Hyung W, Lau C, Strong V (2018) The SAGES atlas of robotic surgery. Springer, Berlin"},{"key":"2370_CR8","unstructured":"Gupta A, Khosla A, Singh G, Dasgupta G (2018) Mining procedures from technical support documents. arXiv:1805.09780"},{"issue":"05","key":"2370_CR9","first-page":"8018","volume":"34","author":"D Jin","year":"2020","unstructured":"Jin D, Jin Z, Zhou JT, Szolovits P (2020) Is Bert really robust? A strong baseline for natural language attack on text classification and entailment. Proc AAAI Conf Artif Intell 34(05):8018\u20138025","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"2370_CR10","doi-asserted-by":"crossref","unstructured":"Joulin A, Grave E, Bojanowski P, Mikolov T (2016) Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759","DOI":"10.18653\/v1\/E17-2068"},{"key":"2370_CR11","doi-asserted-by":"crossref","unstructured":"Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1746\u20131751. Association for Computational Linguistics, Doha, Qatar","DOI":"10.3115\/v1\/D14-1181"},{"issue":"1","key":"2370_CR12","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","volume":"33","author":"JR Landis","year":"1977","unstructured":"Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159\u2013174","journal-title":"Biometrics"},{"issue":"185","key":"2370_CR13","first-page":"1","volume":"18","author":"L Li","year":"2018","unstructured":"Li L, Jamieson K, DeSalvo G, Rostamizadeh A, Talwalkar A (2018) Hyperband: a novel bandit-based approach to hyperparameter optimization. J Mach Learn Res 18(185):1\u201352","journal-title":"J Mach Learn Res"},{"key":"2370_CR14","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1016\/j.neucom.2019.01.078","volume":"337","author":"G Liu","year":"2019","unstructured":"Liu G, Guo J (2019) Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing 337:325\u2013338","journal-title":"Neurocomputing"},{"key":"2370_CR15","doi-asserted-by":"publisher","first-page":"476","DOI":"10.21037\/jtd.2017.12.36","volume":"10","author":"A Pardolesi","year":"2018","unstructured":"Pardolesi A, Bertolaccini L, Brandolini J, Gallina F, Novellis P, Veronesi G, Solli P (2018) Four arms robotic-assisted pulmonary resection-right lower\/middle lobectomy: how to do it. J Thorac Dis 10:476\u2013481","journal-title":"J Thorac Dis"},{"key":"2370_CR16","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"key":"2370_CR17","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1007\/978-3-030-49435-3_17","volume-title":"Advanced information systems engineering","author":"C Qian","year":"2020","unstructured":"Qian C, Wen L, Kumar A, Lin L, Lin L, Zong Z, Li S, Wang J (2020) An approach for process model extraction by multi-grained text classification. In: Dustdar S, Yu E, Salinesi C, Rieu D, Pant V (eds) Advanced information systems engineering. Springer, Cham, pp 268\u2013282"},{"key":"2370_CR18","doi-asserted-by":"crossref","unstructured":"Ramadhan WP, Astri Novianty STMT, Casi Setianingsih STMT (2017) Sentiment analysis using multinomial logistic regression. In: 2017 International conference on control. electronics, renewable energy and communications (ICCREC), pp 46\u201349","DOI":"10.1109\/ICCEREC.2017.8226700"},{"key":"2370_CR19","doi-asserted-by":"publisher","first-page":"113749","DOI":"10.1016\/j.eswa.2020.113749","volume":"163","author":"M Rospocher","year":"2020","unstructured":"Rospocher M (2020) Explicit song lyrics detection with subword-enriched word embeddings. Expert Syst Appl 163:113749","journal-title":"Expert Syst Appl"},{"key":"2370_CR20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s42452-020-2266-6","volume":"2","author":"P Saigal","year":"2020","unstructured":"Saigal P, Khanna V (2020) Multi-category news classification using support vector machine based classifiers. SN Appl Sci 2:1\u201312","journal-title":"SN Appl Sci"},{"issue":"2","key":"2370_CR21","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/j.thorsurg.2014.02.010","volume":"24","author":"IS Sarkaria","year":"2014","unstructured":"Sarkaria IS, Rizk NP (2014) Robotic-assisted minimally invasive esophagectomy: the IVOR Lewis approach. Thorac Surg Clin 24(2):211\u2013222","journal-title":"Thorac Surg Clin"},{"issue":"2","key":"2370_CR22","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1016\/j.athoracsur.2004.07.022","volume":"79","author":"MA Savitt","year":"2005","unstructured":"Savitt MA, Gao G, Furnary AP, Swanson J, Gately HL, Handy JR (2005) Application of robotic-assisted techniques to the surgical evaluation and treatment of the anterior mediastinum. Ann Thorac Surg 79(2):450\u2013455","journal-title":"Ann Thorac Surg"},{"key":"2370_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s41133-019-0017-2","volume":"5","author":"K Shah","year":"2020","unstructured":"Shah K, Patel H, Sanghvi D, Shah M (2020) A comparative analysis of logistic regression, random forest and KNN models for the text classification. Augment Hum Res 5:1\u201316","journal-title":"Augment Hum Res"},{"key":"2370_CR24","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1007\/978-3-642-23620-4_36","volume-title":"Active media technology","author":"S Song","year":"2011","unstructured":"Song S, Oh H, Myaeng SH, Choi S, Chun H, Choi Y, Jeong C (2011) Procedural knowledge extraction on medline abstracts. In: Zhong N, Callaghan V, Ghorbani AA, Hu B (eds) Active media technology. Springer, Berlin, pp 345\u2013354"},{"key":"2370_CR25","doi-asserted-by":"crossref","unstructured":"Tahsin R, Mozumder MH, Shahriyar SA, Salim Mollah MA (2020) A novel approach for e-mail classification using fasttext. In: 2020 IEEE region 10 symposium (TENSYMP), pp 1392\u20131395","DOI":"10.1109\/TENSYMP50017.2020.9230961"},{"key":"2370_CR26","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A.N, Kaiser L.u, Polosukhin (2017) I.: attention is all you need. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30, Curran Associates, Inc"},{"key":"2370_CR27","doi-asserted-by":"crossref","unstructured":"Wambsganss T, Fromm H (2019) Mining user-generated repair instructions from automotive web communities. In: HICSS","DOI":"10.24251\/HICSS.2019.144"},{"key":"2370_CR28","doi-asserted-by":"crossref","unstructured":"Yang H, Aguirre CA, De La Torre MF, Christensen D, Bobadilla L, Davich E, Roth J, Luo L, Theis Y, Lam A, Han TY, Buttler D, Hsu WH (2019) Pipelines for procedural information extraction from scientific literature: towards recipes using machine learning and data science. In: 2019 International conference on document analysis and recognition workshops (ICDARW), vol 2, pp 41\u201346","DOI":"10.1109\/ICDARW.2019.10037"},{"issue":"5","key":"2370_CR29","doi-asserted-by":"publisher","first-page":"4242","DOI":"10.4249\/scholarpedia.4242","volume":"3","author":"Y Yang","year":"2008","unstructured":"Yang Y, Joachims T (2008) Text categorization. Scholarpedia 3(5):4242","journal-title":"Scholarpedia"},{"key":"2370_CR30","unstructured":"Zhang Z, Webster P, Uren V, Varga A, Ciravegna F (2012) Automatically extracting procedural knowledge from instructional texts using natural language processing. In: Proceedings of the eighth international conference on language resources and evaluation (LREC\u201912), European Language Resources Association (ELRA), Istanbul, Turkey, pp 520\u2013527"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-021-02370-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-021-02370-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-021-02370-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,21]],"date-time":"2021-07-21T12:34:40Z","timestamp":1626870880000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-021-02370-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,22]]},"references-count":30,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2021,8]]}},"alternative-id":["2370"],"URL":"https:\/\/doi.org\/10.1007\/s11548-021-02370-9","relation":{},"ISSN":["1861-6410","1861-6429"],"issn-type":[{"value":"1861-6410","type":"print"},{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,22]]},"assertion":[{"value":"13 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 April 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 April 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"M. Bombieri, M. Rospocher, D. Dall\u2019Alba and P. Fiorini declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This article does not contain studies with human participants or animals.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Statement of informed consent is not applicable since the manuscript does not contain any patient data.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}}]}}