{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T00:07:56Z","timestamp":1790208476149,"version":"4.1.0"},"reference-count":42,"publisher":"SPIE-Intl Soc Optical Eng","issue":"09","license":[{"start":{"date-parts":[[2024,7,24]],"date-time":"2024-07-24T00:00:00Z","timestamp":1721779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Biomed. Opt."],"published-print":{"date-parts":[[2024,7,24]]},"abstract":"<jats:p>SignificanceHyperspectral imaging (HSI) is an emerging imaging modality for oncological applications and can improve cancer detection with digital pathology.AimThe study aims to highlight the increased accuracy and sensitivity of detecting the margin of thyroid carcinoma in hematoxylin and eosin (H&amp;E)-stained histological slides using HSI and data augmentation methods.ApproachUsing an automated microscopic imaging system, we captured 2599 hyperspectral images from 65 H&amp;E-stained human thyroid slides. Images were then preprocessed into 153,906 image patches of dimension 250\u00d7250\u00d784 pixels. We modified the TimeSformer network architecture, which used alternating spectral attention and spatial attention layers. We implemented several data augmentation methods for HSI based on the RandAugment algorithm. We compared the performances of TimeSformer on HSI against the performances of pretrained ConvNext and pretrained vision transformers (ViT) networks on red, green, and blue (RGB) images. Finally, we applied attention unrolling techniques on the trained TimeSformer network to identify the biological features to which the network paid attention.ResultsIn the testing dataset, TimeSformer achieved an accuracy of 90.87%, a weighted F1 score of 89.79%, a sensitivity of 91.50%, and an area under the receiving operator characteristic curve (AU-ROC) score of 97.04%. Additionally, TimeSformer produced thyroid carcinoma tumor margins with an average Jaccard score of 0.76\u2009mm. Without data augmentation, TimeSformer achieved an accuracy of 88.23%, a weighted F1 score of 86.46%, a sensitivity of 85.53%, and an AU-ROC score of 94.94%. In comparison, the ViT network achieved an 89.98% accuracy, an 88.14% weighted F1 score, an 84.77% sensitivity, and a 96.17% AU-ROC. Our visualization results showed that the network paid attention to biological features.ConclusionsThe TimeSformer model trained with hyperspectral histological data consistently outperformed conventional RGB-based models, highlighting the superiority of HSI in this context. Our proposed augmentation methods improved the accuracy, the F1 score, and the sensitivity score.<\/jats:p>","DOI":"10.1117\/1.jbo.29.9.093505","type":"journal-article","created":{"date-parts":[[2024,7,24]],"date-time":"2024-07-24T10:07:48Z","timestamp":1721815668000},"page":"1-16","source":"Crossref","is-referenced-by-count":10,"title":["Detection and margin assessment of thyroid carcinoma with microscopic hyperspectral imaging using transformer networks"],"prefix":"10.1117","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9936-9654","authenticated-orcid":false,"given":"Minh Ha","family":"Tran","sequence":"first","affiliation":[{"name":"The University of Texas at Dallas"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4352-5697","authenticated-orcid":false,"given":"Ling","family":"Ma","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1141-3471","authenticated-orcid":false,"given":"Hasan","family":"Mubarak","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ofelia","family":"Gomez","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8398-1919","authenticated-orcid":false,"given":"James","family":"Yu","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-3739-1314","authenticated-orcid":false,"given":"Michelle","family":"Bryarly","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9123-9484","authenticated-orcid":false,"given":"Baowei","family":"Fei","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"189","reference":[{"key":"r1","doi-asserted-by":"publisher","DOI":"10.3322\/caac.21763"},{"key":"r2","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(16)30172-6"},{"key":"r3","doi-asserted-by":"publisher","DOI":"10.1016\/j.jamcollsurg.2021.06.011"},{"key":"r4","doi-asserted-by":"publisher","DOI":"10.1002\/hed.24320"},{"key":"r5","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2023.958310"},{"key":"r6","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-50313-x"},{"key":"r7","doi-asserted-by":"publisher","DOI":"10.1002\/cncy.22238"},{"key":"r8","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2020.101814"},{"key":"r9","doi-asserted-by":"publisher","DOI":"10.1117\/12.2548609"},{"key":"r10","doi-asserted-by":"publisher","DOI":"10.3390\/cancers11091367"},{"key":"r11","doi-asserted-by":"publisher","DOI":"10.1117\/12.2612963"},{"key":"r12","doi-asserted-by":"publisher","DOI":"10.1364\/BOE.386338"},{"key":"r13","doi-asserted-by":"publisher","DOI":"10.1117\/12.2581046"},{"key":"r14","doi-asserted-by":"publisher","DOI":"10.1117\/1.JBO.27.4.046501"},{"key":"r15","doi-asserted-by":"publisher","DOI":"10.1117\/1.JBO.27.5.056502"},{"key":"r16","doi-asserted-by":"publisher","DOI":"10.1117\/1.JBO.28.4.040901"},{"key":"r17","doi-asserted-by":"publisher","DOI":"10.1016\/j.ymeth.2021.04.005"},{"key":"r18","doi-asserted-by":"publisher","DOI":"10.1117\/1.JBO.29.1.016005"},{"key":"r19","article-title":"An image is worth 16x16 words: transformers for image recognition at scale","author":"Dosovitskiy","year":"2020"},{"key":"r20","doi-asserted-by":"publisher","DOI":"10.3390\/rs13030498"},{"key":"r21","doi-asserted-by":"publisher","DOI":"10.3390\/rs13112216"},{"key":"r22","article-title":"Spectr: spectral transformer for hyperspectral pathology image segmentation","author":"Yun","year":"2021"},{"key":"r23","doi-asserted-by":"publisher","DOI":"10.1145\/3512388.3512419"},{"key":"r24","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2023.3253722"},{"key":"r25","article-title":"How to train your VIT? Data, augmentation, and regularization in vision transformers","author":"Steiner","year":"2021"},{"key":"r26","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2019.101544"},{"key":"r27","first-page":"168","article-title":"Tailoring automated data augmentation to H&E-stained histopathology","author":"Faryna","year":"2021"},{"key":"r28","first-page":"66","author":"\u00d6rnberg","year":"2020"},{"key":"r29","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW50498.2020.00359"},{"key":"r30","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2102.05095"},{"key":"r31","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2017.8297014"},{"key":"r32","doi-asserted-by":"publisher","DOI":"10.1117\/12.2650815"},{"key":"r33","doi-asserted-by":"publisher","DOI":"10.1117\/12.2253614"},{"key":"r34","doi-asserted-by":"publisher","DOI":"10.1117\/12.2042350"},{"key":"r35","doi-asserted-by":"publisher","DOI":"10.1117\/12.2654851"},{"key":"r36","doi-asserted-by":"publisher","DOI":"10.1117\/12.2082299"},{"key":"r37","author":"Gonzalez","year":"2006"},{"key":"r38","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2201.03545"},{"key":"r39","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2016.06.008"},{"key":"r40","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-021-06687-z"},{"key":"r41","doi-asserted-by":"publisher","DOI":"10.1109\/IPTA59101.2023.10320014"},{"key":"r42","article-title":"Attention is not explanation","author":"Jain","year":"2019"}],"container-title":["Journal of Biomedical Optics"],"original-title":[],"link":[{"URL":"https:\/\/www.spiedigitallibrary.org\/journals\/journal-of-biomedical-optics\/volume-29\/issue-09\/093505\/Detection-and-margin-assessment-of-thyroid-carcinoma-with-microscopic-hyperspectral\/10.1117\/1.JBO.29.9.093505.full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T23:52:39Z","timestamp":1790207559000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.spiedigitallibrary.org\/journals\/journal-of-biomedical-optics\/volume-29\/issue-09\/093505\/Detection-and-margin-assessment-of-thyroid-carcinoma-with-microscopic-hyperspectral\/10.1117\/1.JBO.29.9.093505.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,24]]},"references-count":42,"journal-issue":{"issue":"09","published-online":{"date-parts":[[2024,9,1]]}},"URL":"https:\/\/doi.org\/10.1117\/1.jbo.29.9.093505","relation":{},"ISSN":["1083-3668","1560-2281"],"issn-type":[{"value":"1083-3668","type":"print"},{"value":"1560-2281","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,24]]}}}