{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T15:31:31Z","timestamp":1782487891036,"version":"3.54.5"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,3,13]],"date-time":"2025-03-13T00:00:00Z","timestamp":1741824000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Manage. Inf. Syst."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>The remarkable advancements in large language models (LLMs) and large-scale visual encoders have laid a solid foundation for enhancing the capabilities of artificial intelligence (AI) in various application scenarios. In this study, we introduce MOSS-MED, a suite of models designed for biomedical vision-language understandings. MOSS-MED includes two models currently: MOSS-MED-2.5B and MOSS-MED-LLaMA, with different scale of their underlying LLMs. We employ a two-stage training pipeline that involves visual-text alignment and visual-involved instruction fine-tuning from both general domains and the medical domain. We evaluate the performance of MOSS-MED family on medical visual question answering (VQA) benchmarks. The results and cases demonstrate the impressive proficiency in medical expertise that MOSS-MED embodies.<\/jats:p>","DOI":"10.1145\/3688005","type":"journal-article","created":{"date-parts":[[2024,8,22]],"date-time":"2024-08-22T11:35:20Z","timestamp":1724326520000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["MOSS-MED: A Family of Multimodal Models Serving Medical Image Analysis"],"prefix":"10.1145","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-7427-0357","authenticated-orcid":false,"given":"Junqi","family":"Dai","sequence":"first","affiliation":[{"name":"Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-9412-2022","authenticated-orcid":false,"given":"Qin","family":"Zhu","sequence":"additional","affiliation":[{"name":"Fudan University, Shanghai China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-2181-9154","authenticated-orcid":false,"given":"Jun","family":"Zhan","sequence":"additional","affiliation":[{"name":"Fudan University, Shanghai China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0526-0533","authenticated-orcid":false,"given":"Bo","family":"Wang","sequence":"additional","affiliation":[{"name":"Obstetrics and Gynecology Hospital of Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7163-5247","authenticated-orcid":false,"given":"Xipeng","family":"Qiu","sequence":"additional","affiliation":[{"name":"Fudan University, Shanghai China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,3,13]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Proceedings of the 36th Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022","author":"Alayrac Jean-Baptiste","year":"2022","unstructured":"Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L. Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Kar\u00e9n Simonyan. 2022. Flamingo: A visual language model for few-shot learning. In Proceedings of the 36th Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022. Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (Eds.). Curran Associates Inc.57 Morehouse LaneRed HookNYUnited States."},{"key":"e_1_3_2_3_2","volume-title":"Proceedings of the 10th International Conference on Learning Representations, ICLR 2022","author":"Bao Hangbo","year":"2022","unstructured":"Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. 2022. BEiT: BERT pre-training of image transformers. In Proceedings of the 10th International Conference on Learning Representations, ICLR 2022. OpenReview.net."},{"key":"e_1_3_2_4_2","unstructured":"Zhijie Bao Wei Chen Shengze Xiao Kuang Ren Jiaao Wu Cheng Zhong Jiajie Peng Xuanjing Huang and Zhongyu Wei. 2023. DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation. (2023)."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.3390\/bioengineering10030380"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","unstructured":"Victor Ion Butoi Jose Javier Gonzalez Ortiz Tianyu Ma Mert R. Sabuncu John Guttag and Adrian V. Dalca. 2023. Universeg: Universal medical image segmentation. In Proceedings of the IEEE\/CVF International Conference on Computer Vision. 21438\u201321451.","DOI":"10.1109\/ICCV51070.2023.01960"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00356"},{"key":"e_1_3_2_8_2","unstructured":"Junlong Cheng Jin Ye Zhongying Deng Jianpin Chen Tianbin Li Haoyu Wang Yanzhou Su Ziyan Huang Jilong Chen Lei Jiangand Hui Sun Junjun He Shaoting Zhang Min Zhu and Yu Qiao. 2023. SAM-Med2D. (2023)."},{"key":"e_1_3_2_9_2","volume-title":"Proceedings of the 37th Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023","author":"Dai Wenliang","year":"2023","unstructured":"Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. 2023. InstructBLIP: Towards general-purpose vision-language models with instruction tuning. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023. Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Curran Associates Inc.57 Morehouse LaneRed HookNYUnited States."},{"key":"e_1_3_2_10_2","volume-title":"Proceedings of the 9th International Conference on Learning Representations, ICLR 2021","author":"Dosovitskiy Alexey","year":"2021","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of the 9th International Conference on Learning Representations, ICLR 2021. OpenReview.net."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-eacl.88"},{"key":"e_1_3_2_12_2","unstructured":"Hiroki Furuta Kuang-Huei Lee Ofir Nachum Yutaka Matsuo Aleksandra Faust Shixiang Shane Gu and Izzeddin Gur. 2024. Multimodal web navigation with instruction-finetuned foundation models. In The Twelfth International Conference on Learning Representations (ICLR\u201924) Vienna Austria OpenReview.net. https:\/\/openreview.net\/forum?id=efFmBWioSc"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","unstructured":"Tianyu Han Lisa C. Adams Jens-Michalis Papaioannou Paul Grundmann Tom Oberhauser Alexander L\u00f6ser Daniel Truhn and Keno K. Bressem. 2023. MedAlpaca - an open-source collection of medical conversational AI models and training data. (2023). arXiv:2304.08247 10.48550\/ARXIV.2304.08247","DOI":"10.48550\/ARXIV.2304.08247"},{"key":"e_1_3_2_14_2","unstructured":"Jinlong He Pengfei Li Gang Liu Zixu Zhao and Shenjun Zhong. 2024. PeFoMed: Parameter efficient fine-tuning on multimodal large language models for medical visual question answering. .arXiv:2401.02797. Retrieved from https:\/\/arxiv.org\/abs\/2401.02797"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"e_1_3_2_16_2","unstructured":"Xuehai He Yichen Zhang Luntian Mou Eric P. Xing and Pengtao Xie. 2020. PathVQA: 30000+ questions for medical visual question answering. CoRR abs\/2003.10286 (2020). arXiv:2003.10286 https:\/\/arxiv.org\/abs\/2003.10286"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i21.30570"},{"key":"e_1_3_2_18_2","volume-title":"Proceedings of the 37th Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023","author":"Huang Shaohan","year":"2023","unstructured":"Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, Qiang Liu, Kriti Aggarwal, Zewen Chi, Nils Johan Bertil Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, and Furu Wei. 2023. Language is not all you need: Aligning perception with language models. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023. Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Curran Associates Inc.57 Morehouse LaneRed HookNYUnited States."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","unstructured":"Ziyan Huang Haoyu Wang Zhongying Deng Jin Ye Yanzhou Su Hui Sun Junjun He Yun Gu Lixu Gu Shaoting Zhang and Yu Qiao. 2023. STU-Net: scalable and transferable medical image segmentation models empowered by large-scale supervised Pre-training. CoRR abs\/2304.06716 (2023). arXiv:2304.06716 10.48550\/ARXIV.2304.06716","DOI":"10.48550\/ARXIV.2304.06716"},{"key":"e_1_3_2_20_2","unstructured":"Wisdom Oluchi Ikezogwo Mehmet Saygin Seyfioglu Fatemeh Ghezloo Dylan Stefan Chan Geva Fatwir Sheikh Mohammed Pavan Kumar Anand Ranjay Krishna and Linda Shapiro. 2023. Quilt-1M: One Million Image-Text Pairs for Histopathology. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023 (NeurIPS\u201923)."},{"key":"e_1_3_2_21_2","unstructured":"Jason J. Lau Soumya Gayen Asma Ben Abacha and Dina Demner-Fushman. 2018. Descriptor : A dataset of clinically generated visual questions and answers about radiology images. Scientific Data."},{"key":"e_1_3_2_22_2","volume-title":"Proceedings of the 37th Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023","author":"Li Chunyuan","year":"2023","unstructured":"Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2023. LLaVA-Med: Training a large language-and-vision assistant for biomedicine in one day. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023. Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Curran Associates Inc.57 Morehouse LaneRed HookNYUnited States."},{"key":"e_1_3_2_23_2","volume-title":"Proceedings of the International Conference on Machine Learning, ICML 2023","author":"Li Junnan","year":"2023","unstructured":"Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. 2023. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the International Conference on Machine Learning, ICML 2023. Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.), Proceedings of Machine Learning Research, PMLR."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISBI53787.2023.10230743"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.40895"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","DOI":"10.1109\/ISBI48211.2021.9434010","article-title":"Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering","author":"Liu Bo","year":"2021","unstructured":"Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Fang Yang, and Xiao-Ming Wu. 2021. Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering. In Proceedings of the 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) (2021).","journal-title":"Proceedings of the 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI)"},{"key":"e_1_3_2_27_2","volume-title":"Proceedings of the 37th Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023","author":"Liu Haotian","year":"2023","unstructured":"Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023. Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Curran Associates Inc.57 Morehouse LaneRed HookNYUnited States."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-34048-2_34"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","unstructured":"Zhengliang Liu Yiwei Li Peng Shu Aoxiao Zhong Longtao Yang Chao Ju Zihao Wu Chong Ma Jie Luo Cheng Chen Sekeun Kim Jiang Hu Haixing Dai Lin Zhao Dajiang Zhu Jun Liu Wei Liu Dinggang Shen Tianming Liu Quanzheng Li and Xiang Li. 2023. Radiology-Llama2: Best-in-Class large language model for radiology. CoRR abs\/2309.06419 (2023). arXiv:2309.06419 10.48550\/ARXIV.2309.06419","DOI":"10.48550\/ARXIV.2309.06419"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","unstructured":"Jun Ma and Bo Wang. 2023. Segment anything in medical images. CoRR abs\/2304.12306 (2023). arXiv:2304.12306 10.48550\/ARXIV.2304.12306","DOI":"10.48550\/ARXIV.2304.12306"},{"key":"e_1_3_2_31_2","volume-title":"Proceedings of the 37th Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023","author":"Mu Yao","year":"2023","unstructured":"Yao Mu, Qinglong Zhang, Mengkang Hu, Wenhai Wang, Mingyu Ding, Jun Jin, Bin Wang, Jifeng Dai, Yu Qiao, and Ping Luo. 2023. EmbodiedGPT: Vision-language pre-training via embodied chain of thought. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023. Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Curran Associates Inc.57 Morehouse LaneRed HookNYUnited States."},{"key":"e_1_3_2_32_2","unstructured":"OpenAI. 2023. GPT-4V(ision) System Card."},{"key":"e_1_3_2_33_2","unstructured":"Maxime Oquab Timoth\u00e9e Darcet Theo Moutakanni Huy V. Vo Marc Szafraniec Vasil Khalidov Pierre Fernandez Daniel Haziza Francisco Massa Alaaeldin El-Nouby Russell Howes Po-Yao Huang Hu Xu Vasu Sharma Shang-Wen Li Wojciech Galuba Mike Rabbat Mido Assran Nicolas Ballas Gabriel Synnaeve Ishan Misra Herve Jegou Julien Mairal Patrick Labatut Armand Joulin and Piotr Bojanowski. 2023. DINOv2: Learning Robust Visual Features without Supervision. Transactions on Machine Learning Research."},{"key":"e_1_3_2_34_2","volume-title":"Proceedings of the 25th Annual Conference on Neural Information Processing Systems 2011","author":"Ordonez Vicente","year":"2011","unstructured":"Vicente Ordonez, Girish Kulkarni, and Tamara L. Berg. 2011. Im2Text: Describing images using 1 million captioned photographs. In Proceedings of the 25th Annual Conference on Neural Information Processing Systems 2011. John Shawe-Taylor, Richard S. Zemel, Peter L. Bartlett, Fernando C. N. Pereira, and Kilian Q. Weinberger (Eds.). Curran Associates Inc.57 Morehouse LaneRed HookNYUnited States."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","unstructured":"Jay N. Paranjape Nithin Gopalakrishnan Nair Shameema Sikder S. Swaroop Vedula and Vishal M. Patel. 2024. AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentation. 14860 (2024) 187\u2013201. 10.1007\/978-3-031-66958-3_14","DOI":"10.1007\/978-3-031-66958-3_14"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00958-w"},{"key":"e_1_3_2_37_2","volume-title":"Proceedings of the 38th International Conference on Machine Learning, ICML 2021","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021. Marina Meila and Tong Zhang (Eds.), Proceedings of Machine Learning Research, PMLR."},{"key":"e_1_3_2_38_2","article-title":"ZeRO: Memory optimizations toward training trillion parameter models","author":"Rajbhandari Samyam","year":"2019","unstructured":"Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2019. ZeRO: Memory optimizations toward training trillion parameter models. In SC20: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis .","journal-title":"SC20: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis"},{"key":"e_1_3_2_39_2","doi-asserted-by":"crossref","unstructured":"Johannes R\u00fcckert Christoph Balada Raphael Br\u00fcngel Ahmad Idrissi-Yaghir Henning Sch\u00e4fer Cynthia S. Schmidt Sven Koitka Obioma Pelka Asma Ben Abacha Alba Garcia Seco de Herrera Henning M\u00fcller Peter A. Horn Felix Nensa and Christoph M. Friedrich. 2023. ROCOv2: Radiology Objects in COntext Version 2 An Updated Multimodal Image Dataset (Version 2.0). (Nov.2023).","DOI":"10.1038\/s41597-024-03496-6"},{"key":"e_1_3_2_40_2","doi-asserted-by":"crossref","unstructured":"Mehmet Saygin Seyfioglu Wisdom O. Ikezogwo Fatemeh Ghezloo Ranjay Krishna and Linda Shapiro. 2023. Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videos. (2023).","DOI":"10.1109\/CVPR52733.2024.01252"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1238"},{"key":"e_1_3_2_42_2","unstructured":"Chang Shu Baian Chen Fangyu Liu Zihao Fu Ehsan Shareghi and Nigel Collier. 2023. Visual Med-alpaca: A Parameter-efficient Biomedical llm with Visual Capabilities. (2023)."},{"key":"e_1_3_2_43_2","article-title":"MOSS: An open conversational large language model","author":"Sun Tianxiang","year":"2024","unstructured":"Tianxiang Sun, Xiaotian Zhang, Zhengfu He, Peng Li, Qinyuan Cheng, Xiangyang Liu, Hang Yan, Yunfan Shao, Qiong Tang, Shiduo Zhang, Xingjian Zhao, Ke Chen, Yining Zheng, Zhejian Zhou, Ruixiao Li, Jun Zhan, Yunhua Zhou, Linyang Li, Xiaogui Yang, Lingling Wu, Zhangyue Yin, Xuanjing Huang, Yu-Gang Jiang, and Xipeng Qiu. 2024. MOSS: An open conversational large language model. Machine Intelligence Research 21 (2024), 1\u201318.","journal-title":"Machine Intelligence Research"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","unstructured":"Omkar Thawakar Abdelrahman M. Shaker Sahal Shaji Mullappilly Hisham Cholakkal Rao Muhammad Anwer Salman H. Khan Jorma Laaksonen and Fahad Shahbaz Khan. 2023. XrayGPT: Chest radiographs summarization using medical vision-language models. CoRR abs\/2306.07971 (2023). arXiv:2306.07971 10.48550\/ARXIV.2306.07971","DOI":"10.48550\/ARXIV.2306.07971"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale Dan Bikel Lukas Blecher Cristian Canton-Ferrer Moya Chen Guillem Cucurull David Esiobu Jude Fernandes Jeremy Fu Wenyin Fu Brian Fuller Cynthia Gao Vedanuj Goswami Naman Goyal Anthony Hartshorn Saghar Hosseini Rui Hou Hakan Inan Marcin Kardas Viktor Kerkez Madian Khabsa Isabel Kloumann Artem Korenev Punit Singh Koura Marie-Anne Lachaux Thibaut Lavril Jenya Lee Diana Liskovich Yinghai Lu Yuning Mao Xavier Martinet Todor Mihaylov Pushkar Mishra Igor Molybog Yixin Nie Andrew Poulton Jeremy Reizenstein Rashi Rungta Kalyan Saladi Alan Schelten Ruan Silva Eric Michael Smith Ranjan Subramanian Xiaoqing Ellen Tan Binh Tang Ross Taylor Adina Williams Jian Xiang Kuan Puxin Xu Zheng Yan Iliyan Zarov Yuchen Zhang Angela Fan Melanie Kambadur Sharan Narang Aur\u00e9lien Rodriguez Robert Stojnic Sergey Edunov and Thomas Scialom. 2023. Llama 2: Open foundation and fine-tuned chat models. Technical Report. arXiv:2307.09288 10.48550\/ARXIV.2307.09288","DOI":"10.48550\/ARXIV.2307.09288"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-43904-9_70"},{"key":"e_1_3_2_47_2","volume-title":"Proceedings of the 31st Annual Conference on Neural Information Processing Systems 2017","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems 2017. Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). Curran Associates Inc.57 Morehouse LaneRed HookNYUnited States."},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","unstructured":"Junke Wang Lingchen Meng Zejia Weng Bo He Zuxuan Wu and Yu-Gang Jiang. 2023. To see is to believe: Prompting GPT-4V for better visual instruction tuning. CoRR abs\/2311.07574 (2023). arXiv:2311.07574 10.48550\/ARXIV.2311.07574","DOI":"10.48550\/ARXIV.2311.07574"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","unstructured":"Chaoyi Wu Jiayu Lei Qiaoyu Zheng Weike Zhao Weixiong Lin Xiaoman Zhang Xiao Zhou Ziheng Zhao Ya Zhang Yanfeng Wang and Weidi Xie. 2023. Can GPT-4V(ision) serve medical applications? Case studies on GPT- 4V for multimodal medical diagnosis. CoRR abs\/2310.09909 (2023). arXiv:2310.09909 10.48550\/ARXIV.2310.09909","DOI":"10.48550\/ARXIV.2310.09909"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocae045"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","unstructured":"Chaoyi Wu Xiaoman Zhang Ya Zhang Yanfeng Wang and Weidi Xie. 2023. Towards generalist foundation model for radiology. CoRR abs\/2308.02463 (2023). arXiv:2308.02463 10.48550\/ARXIV.2308.02463","DOI":"10.48550\/ARXIV.2308.02463"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","unstructured":"Zhiheng Xi Wenxiang Chen Xin Guo Wei He Yiwen Ding Boyang Hong Ming Zhang Junzhe Wang Senjie Jin Enyu Zhou Rui Zheng Xiaoran Fan Xiao Wang Limao Xiong Yuhao Zhou Weiran Wang Changhao Jiang Yicheng Zou Xiangyang Liu Zhangyue Yin Shihan Dou Rongxiang Weng Wensen Cheng Qi Zhang Wenjuan Qin Yongyan Zheng Xipeng Qiu Xuanjing Huang and Tao Gui. 2023. The rise and potential of large language model based agents: A survey. CoRR abs\/2309.07864 (2023). arXiv:2309.07864 10.48550\/ARXIV.2309.07864","DOI":"10.48550\/ARXIV.2309.07864"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","unstructured":"Zhiling Yan Kai Zhang Rong Zhou Lifang He Xiang Li and Lichao Sun. 2023. Multimodal ChatGPT for medical applications: an experimental study of GPT-4V. CoRR abs\/2310.19061 (2023). arXiv:2310.19061 10.48550\/ARXIV.2310.19061","DOI":"10.48550\/ARXIV.2310.19061"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.1055"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","unstructured":"Kai Zhang Jun Yu Zhiling Yan Yixin Liu Eashan Adhikarla Sunyang Fu Xun Chen Chen Chen Yuyin Zhou Xiang Li Lifang He Brian D. Davison Quanzheng Li Yong Chen Hongfang Liu and Lichao Sun. 2023. BiomedGPT: A unified and generalist biomedical generative pre-trained transformer for vision language and multimodal tasks. CoRR abs\/2305.17100 (2023). arXiv:2305.17100 10.48550\/ARXIV.2305.17100","DOI":"10.48550\/ARXIV.2305.17100"},{"key":"e_1_3_2_56_2","article-title":"On the challenges and perspectives of foundation models for medical image analysis","author":"Zhang Shaoting","year":"2023","unstructured":"Shaoting Zhang and Dimitris N. Metaxas. 2023. On the challenges and perspectives of foundation models for medical image analysis. Medical Image Analysis 91 (2023), 102996.","journal-title":"Medical Image Analysis"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","unstructured":"Sheng Zhang Yanbo Xu Naoto Usuyama Jaspreet Bagga Robert Tinn Sam Preston Rajesh Rao Mu Wei Naveen Valluri Cliff Wong Matthew P. Lungren Tristan Naumann and Hoifung Poon. 2023. Large-scale domain-specific pretraining for biomedical vision-language processing. CoRR abs\/2303.00915 (2023). arXiv:2303.00915 10.48550\/ARXIV.2303.00915","DOI":"10.48550\/ARXIV.2303.00915"},{"key":"e_1_3_2_58_2","unstructured":"Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li and Mohamed Elhoseiny. 2024. MiniGPT-4: Enhancing vision- language understanding with advanced large language models. In The Twelfth International Conference on Learning Representations ICLR 2024 Vienna Austria May 7-11 2024. OpenReview.net. https:\/\/openreview.net\/forum?id=1tZbq88f27"}],"container-title":["ACM Transactions on Management Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3688005","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3688005","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:05:55Z","timestamp":1750291555000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3688005"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,13]]},"references-count":57,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3688005"],"URL":"https:\/\/doi.org\/10.1145\/3688005","relation":{},"ISSN":["2158-656X","2158-6578"],"issn-type":[{"value":"2158-656X","type":"print"},{"value":"2158-6578","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,13]]},"assertion":[{"value":"2024-01-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-23","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}