{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T21:22:09Z","timestamp":1779312129542,"version":"3.51.4"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,8,18]],"date-time":"2023-08-18T00:00:00Z","timestamp":1692316800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,18]],"date-time":"2023-08-18T00:00:00Z","timestamp":1692316800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000069","name":"U.S. Department of Health & Human Services | NIH | National Institute of Arthritis and Musculoskeletal and Skin Diseases","doi-asserted-by":"publisher","award":["T32 5T32AR007422-38"],"award-info":[{"award-number":["T32 5T32AR007422-38"]}],"id":[{"id":"10.13039\/100000069","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["NSF CAREER 1942926"],"award-info":[{"award-number":["NSF CAREER 1942926"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Images depicting dark skin tones are significantly underrepresented in the educational materials used to teach primary care physicians and dermatologists to recognize skin diseases. This could contribute to disparities in skin disease diagnosis across different racial groups. Previously, domain experts have manually assessed textbooks to estimate the diversity in skin images. Manual assessment does not scale to many educational materials and introduces human errors. To automate this process, we present the Skin Tone Analysis for Representation in EDucational materials (STAR-ED) framework, which assesses skin tone representation in medical education materials using machine learning. Given a document (e.g., a textbook in .pdf), STAR-ED applies content parsing to extract text, images, and table entities in a structured format. Next, it identifies images containing skin, segments the skin-containing portions of those images, and estimates the skin tone using machine learning. STAR-ED was developed using the Fitzpatrick17k dataset. We then externally tested STAR-ED on four commonly used medical textbooks. Results show strong performance in detecting skin images (0.96\u2009\u00b1\u20090.02 AUROC and 0.90\u2009\u00b1\u20090.06 F<jats:sub>1<\/jats:sub>score) and classifying skin tones (0.87\u2009\u00b1\u20090.01 AUROC and 0.91\u2009\u00b1\u20090.00 F<jats:sub>1<\/jats:sub>score). STAR-ED quantifies the imbalanced representation of skin tones in four medical textbooks: brown and black skin tones (Fitzpatrick V-VI) images constitute only 10.5% of all skin images. We envision this technology as a tool for medical educators, publishers, and practitioners to assess skin tone diversity in their educational materials.<\/jats:p>","DOI":"10.1038\/s41746-023-00881-0","type":"journal-article","created":{"date-parts":[[2023,8,18]],"date-time":"2023-08-18T08:24:36Z","timestamp":1692347076000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Skin Tone Analysis for Representation in Educational Materials (STAR-ED) using machine learning"],"prefix":"10.1038","volume":"6","author":[{"given":"Girmaw Abebe","family":"Tadesse","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Celia","family":"Cintas","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kush R.","family":"Varshney","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Staar","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chinyere","family":"Agunwa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Skyler","family":"Speakman","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Justin","family":"Jia","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elizabeth E.","family":"Bailey","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ademide","family":"Adelekun","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jules B.","family":"Lipoff","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ginikanwa","family":"Onyekaba","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jenna C.","family":"Lester","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0639-2677","authenticated-orcid":false,"given":"Veronica","family":"Rotemberg","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8880-4764","authenticated-orcid":false,"given":"James","family":"Zou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7988-9356","authenticated-orcid":false,"given":"Roxana","family":"Daneshjou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,8,18]]},"reference":[{"key":"881_CR1","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1016\/j.socscimed.2018.02.023","volume":"202","author":"P Louie","year":"2018","unstructured":"Louie, P. & Wilkes, R. Representations of race and skin tone in medical textbook imagery. Soc. Sci. Med. 202, 38\u201342 (2018).","journal-title":"Soc. Sci. Med."},{"key":"881_CR2","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1016\/j.jaad.2020.04.084","volume":"84","author":"A Adelekun","year":"2021","unstructured":"Adelekun, A., Onyekaba, G. & Lipoff, J. B. Skin color in dermatology textbooks: an updated evaluation and analysis. J. Am. Acad. Dermatol. 84, 194\u2013196 (2021).","journal-title":"J. Am. Acad. Dermatol."},{"key":"881_CR3","first-page":"88","volume":"113","author":"JP Massie","year":"2021","unstructured":"Massie, J. P. et al. A picture of modern medicine: race and visual representation in medical literature. J. Natl Med. Assoc. 113, 88\u201394 (2021).","journal-title":"J. Natl Med. Assoc."},{"key":"881_CR4","doi-asserted-by":"publisher","first-page":"593","DOI":"10.1111\/bjd.19258","volume":"183","author":"JC Lester","year":"2020","unstructured":"Lester, J. C., Jia, J. L., Zhang, L., Okoye, G. A. & Linos, E. Absence of images of skin of colour in publications of COVID-19 skin manifestations. Br. J. Dermatol. 183, 593\u2013595 (2020).","journal-title":"Br. J. Dermatol."},{"key":"881_CR5","doi-asserted-by":"publisher","first-page":"e2563","DOI":"10.1097\/GOX.0000000000002563","volume":"7","author":"JP Massie","year":"2019","unstructured":"Massie, J. P. et al. Patient representation in medical literature: are we appropriately depicting diversity. Plast. Reconstr. Surg. Glob. Open 7, e2563 (2019).","journal-title":"Plast. Reconstr. Surg. Glob. Open"},{"key":"881_CR6","doi-asserted-by":"publisher","first-page":"1907","DOI":"10.1001\/archinte.166.17.1907","volume":"166","author":"JN Cormier","year":"2006","unstructured":"Cormier, J. N. et al. Ethnic differences among patients with cutaneous melanoma. Arch. Intern Med. 166, 1907\u20131914 (2006).","journal-title":"Arch. Intern Med."},{"key":"881_CR7","unstructured":"Codella, N. C. F. et al. Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the International Skin Imaging Collaboration. arXiv. Preprint at https:\/\/arxiv.org\/abs\/1902.03368 (2019)."},{"key":"881_CR8","doi-asserted-by":"crossref","unstructured":"Sun, X., Yang, J., Sun, M., Wang, K. A benchmark for automatic visual classification of clinical skin disease images. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds) Computer Vision \u2013 ECCV 2016. Lecture Notes in Computer Science, vol 9910. (Springer: Cham, 2016).","DOI":"10.1007\/978-3-319-46466-4_13"},{"key":"881_CR9","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1111\/bjd.12529","volume":"169","author":"S Del Bino","year":"2013","unstructured":"Del Bino, S. & Bernerd, F. Variations in skin colour and the biological consequences of ultraviolet radiation exposure. Br. J. Dermatol 169, 33\u201340 (2013).","journal-title":"Br. J. Dermatol"},{"key":"881_CR10","doi-asserted-by":"crossref","unstructured":"Kinyanjui, N. M., Odonga, T., Cintas, C., Codella, N. C., Panda, R., Sattigeri, P., & Varshney, K. R. Fairness of classifiers across skin tones in dermatology. In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention 320\u2013329 (Springer International Publishing: Cham, 2020).","DOI":"10.1007\/978-3-030-59725-2_31"},{"key":"881_CR11","doi-asserted-by":"crossref","unstructured":"Groh, M. et al. Evaluating deep neural networks trained on clinical images in dermatology with the Fitzpatrick 17k dataset. In: Proc. IEEE\/CVF Conference on Computer Vision and Pattern Recognition. (2021).","DOI":"10.1109\/CVPRW53098.2021.00201"},{"key":"881_CR12","doi-asserted-by":"publisher","first-page":"902","DOI":"10.1001\/jamadermatol.2015.0351","volume":"151","author":"M Wilkes","year":"2015","unstructured":"Wilkes, M., Wright, C. Y., du Plessis, J. L. & Reeder, A. Fitzpatrick skin type, individual typology angle, and melanin index in an African population: steps toward universally applicable skin photosensitivity assessments. JAMA Dermatol. 151, 902\u2013903 (2015).","journal-title":"JAMA Dermatol."},{"key":"881_CR13","doi-asserted-by":"crossref","unstructured":"Chang, C-C, et al. Robust skin type classification using convolutional neural networks. In: Proc. 13th IEEE Conference on Industrial Electronics and Applications (ICIEA). 2011\u20132014 (2018).","DOI":"10.1109\/ICIEA.2018.8398040"},{"key":"881_CR14","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1016\/j.jaad.2005.10.068","volume":"55","author":"T Ebede","year":"2006","unstructured":"Ebede, T. & Papier, A. Disparities in dermatology educational resources. J. Am. Acad. Dermatol. 55, 687\u2013690 (2006).","journal-title":"J. Am. Acad. Dermatol."},{"key":"881_CR15","doi-asserted-by":"publisher","first-page":"1565","DOI":"10.1038\/nbt1206-1565","volume":"24","author":"WS Noble","year":"2006","unstructured":"Noble, W. S. What is a support vector machine. Nat. Biotechnol. 24, 1565\u20131567 (2006).","journal-title":"Nat. Biotechnol."},{"key":"881_CR16","doi-asserted-by":"crossref","unstructured":"Chen, T., Guestrin, C. XGBoost: a scalable tree boosting system. In Proc. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 785\u2013794 (2016).","DOI":"10.1145\/2939672.2939785"},{"key":"881_CR17","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., Sun, J. Deep residual learning for image recognition. 770\u2013778 (2016).","DOI":"10.1109\/CVPR.2016.90"},{"key":"881_CR18","doi-asserted-by":"crossref","unstructured":"Deng, J. et al. Imagenet: a large-scale hierarchical image database. In Proc. IEEE Conference on Computer Vision and Pattern Recognition. 248\u2013255 (2009).","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"881_CR19","unstructured":"Chen, C., Liaw, A., Breiman, L. Using random forest to learn imbalanced data. University of California, Berkeley; 110(24) (2004)."},{"key":"881_CR20","doi-asserted-by":"crossref","unstructured":"Staar, P., Dolfi, M., Auer, C., Bekas, C. Corpus conversion service: a machine learning platform to ingest documents at scale. In Proc. International Conference on Knowledge Discovery & Data Mining. 774\u2013782 (2018).","DOI":"10.1145\/3219819.3219834"},{"key":"881_CR21","doi-asserted-by":"crossref","unstructured":"Javaid, A., Sadiq M., Akram, F. Skin cancer classification using image processing and machine learning. In: Proc. International Bhurban Conference on Applied Sciences and Technologies (IBCAST). 439\u2013444 (2021).","DOI":"10.1109\/IBCAST51254.2021.9393198"},{"key":"881_CR22","doi-asserted-by":"crossref","unstructured":"Montenegro, J., G\u00f3mez, W., S\u00e1nchez-Orellana, P. A comparative study of color spaces in skin-based face segmentation. In: Proc. 10th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE). 313\u2013317 (2013).","DOI":"10.1109\/ICEEE.2013.6676048"},{"key":"881_CR23","doi-asserted-by":"publisher","unstructured":"Lester, J. C., Clark, L., Linos, E., Daneshjou, R. Clinical photography in skin of colour: tips and best practices. Br. J. Dermatol. https:\/\/doi.org\/10.1111\/bjd.19811 (2021).","DOI":"10.1111\/bjd.19811"},{"key":"881_CR24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3555634","volume":"6","author":"M Groh","year":"2022","unstructured":"Groh, M., Harris, C., Daneshjou, R., Badri, O. & Koocheck, A. Towards transparency in dermatology image datasets with skin tone annotations by experts, crowds, and an algorithm. Proc. ACM Hum.Comput. Interact. 6, 1\u201326 (2022).","journal-title":"Proc. ACM Hum.Comput. Interact."},{"key":"881_CR25","doi-asserted-by":"publisher","unstructured":"Okoji, U. K., Taylor, S. C., Lipoff, J. B. Equity in skin typing: why it is time to replace the Fitzpatrick scale. Br. J. Dermatol. https:\/\/doi.org\/10.1111\/bjd.19932 (2021).","DOI":"10.1111\/bjd.19932"},{"key":"881_CR26","unstructured":"Burns, T., Breathnach, S., Cox, N., Griffiths C. Rook\u2019s Textbook of Dermatology. (John Wiley & Sons, 2008)."},{"key":"881_CR27","unstructured":"Bolognia, J., Schaffer, J. V., Cerroni, L. Dermatology. 4th ed. 138\u2013160 (Elsevier, 2018)."},{"key":"881_CR28","unstructured":"Wolff, K., Johnson, R., Saavedra, A., Roh, E. Fitzpatrick\u2019s Color Atlas and Synopsis of Clinical Dermatology (McGraw Hill, 2017)."},{"key":"881_CR29","unstructured":"Kang, S. et al. Fitzpatrick\u2019s Dermatology (McGraw Hill, 2019)."},{"key":"881_CR30","doi-asserted-by":"crossref","unstructured":"Livathinos, N. et al. Robust pdf document conversion using recurrent neural networks. In Proc. 32nd Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21) 35, 15137\u201315145 (2021).","DOI":"10.1609\/aaai.v35i17.17777"},{"key":"881_CR31","first-page":"6","volume":"45","author":"C Severance","year":"2012","unstructured":"Severance, C. Discovering javascript object notation. Computer 45, 6\u20138 (2012).","journal-title":"Computer"},{"key":"881_CR32","doi-asserted-by":"crossref","unstructured":"Dalal, N. & Triggs, B. Histograms of oriented gradients for human detection. In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905) 1, 886\u2013893 (IEEE, 2005).","DOI":"10.1109\/CVPR.2005.177"},{"key":"881_CR33","doi-asserted-by":"crossref","unstructured":"Saxen, F., Al-Hamadi, A. Color-based skin segmentation: an evaluation of the state of the art. In Proc. IEEE International Conference on Image Processing (ICIP). 4467\u20134471 (IEEE, 2014).","DOI":"10.1109\/ICIP.2014.7025906"},{"key":"881_CR34","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman, L. Random forests. Mach. Learn. 45, 5\u201332 (2001).","journal-title":"Mach. Learn."},{"key":"881_CR35","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s10994-006-6226-1","volume":"63","author":"P Geurts","year":"2006","unstructured":"Geurts, P., Ernst, D. & Wehenkel, L. Extremely randomized trees. Mach. Learn. 63, 3\u201342 (2006).","journal-title":"Mach. Learn."},{"key":"881_CR36","doi-asserted-by":"publisher","first-page":"349","DOI":"10.4310\/SII.2009.v2.n3.a8","volume":"2","author":"T Hastie","year":"2009","unstructured":"Hastie, T., Rosset, S., Zhu, J. & Zou, H. Multi-class adaboost. Stat. Interface 2, 349\u2013360 (2009).","journal-title":"Stat. Interface"},{"key":"881_CR37","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F. et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 12, 2825\u20132830 (2011).","journal-title":"J. Mach. Learn. Res."},{"key":"881_CR38","first-page":"1","volume":"18","author":"G Lema\u00eetre","year":"2017","unstructured":"Lema\u00eetre, G., Nogueira, F. & Aridas, C. Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 18, 1\u20135 (2017).","journal-title":"J. Mach. Learn. Res."},{"issue":"Sep","key":"881_CR39","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1038\/s41586-020-2649-2","volume":"585","author":"CR Harris","year":"2020","unstructured":"Harris, C. R. et al. Array programming with NumPy. Nature 585(Sep), 357\u2013362 (2020).","journal-title":"Nature"},{"key":"881_CR40","unstructured":"Paszke, A. et al. Pytorch: an imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems (2019)."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00881-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00881-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00881-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,26]],"date-time":"2024-10-26T09:53:39Z","timestamp":1729936419000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00881-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,18]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["881"],"URL":"https:\/\/doi.org\/10.1038\/s41746-023-00881-0","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,18]]},"assertion":[{"value":"5 November 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 July 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 August 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing non-financial interests but the following competing financial interests: C.C., K.R.V., P.S., C.A., S.S. are employed by IBM. G.A.T. was employed by IBM when the work was completed and is now employed by Microsoft.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"151"}}