{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T00:00:35Z","timestamp":1781740835426,"version":"3.54.5"},"reference-count":54,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,3,27]],"date-time":"2024-03-27T00:00:00Z","timestamp":1711497600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Recent advances in natural language processing enable more intelligent ways to support knowledge sharing in factories. In manufacturing, operating production lines has become increasingly knowledge-intensive, putting strain on a factory's capacity to train and support new operators. This paper introduces a Large Language Model (LLM)-based system designed to retrieve information from the extensive knowledge contained in factory documentation and knowledge shared by expert operators. The system aims to efficiently answer queries from operators and facilitate the sharing of new knowledge. We conducted a user study at a factory to assess its potential impact and adoption, eliciting several perceived benefits, namely, enabling quicker information retrieval and more efficient resolution of issues. However, the study also highlighted a preference for learning from a human expert when such an option is available. Furthermore, we benchmarked several commercial and open-sourced LLMs for this system. The current state-of-the-art model, GPT-4, consistently outperformed its counterparts, with open-source models trailing closely, presenting an attractive option given their data privacy and customization benefits. In summary, this work offers preliminary insights and a system design for factories considering using LLM tools for knowledge management.<\/jats:p>","DOI":"10.3389\/frai.2024.1293084","type":"journal-article","created":{"date-parts":[[2024,3,27]],"date-time":"2024-03-27T04:50:30Z","timestamp":1711515030000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":60,"title":["Knowledge sharing in manufacturing using LLM-powered tools: user study and model benchmarking"],"prefix":"10.3389","volume":"7","author":[{"given":"Samuel","family":"Kernan Freire","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chaofan","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mina","family":"Foosherian","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefan","family":"Wellsandt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Santiago","family":"Ruiz-Arenas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Evangelos","family":"Niforatos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,3,27]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1906.02569","author":"Abid","year":"2019","journal-title":"Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild"},{"key":"B2","doi-asserted-by":"publisher","first-page":"e35179","DOI":"10.7759\/cureus.35179","article-title":"Artificial hallucinations in chatgpt: implications in scientific writing","volume":"15","author":"Alkaissi","year":"2023","journal-title":"Cureus"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.3390\/pr11010193","article-title":"Is industry 5.0 a human-centred approach? A systematic review","author":"Alves","year":"2023","journal-title":"Processes"},{"key":"B4","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1016\/j.aiepr.2023.03.003","article-title":"Assessing the capabilities of chatgpt to improve additive manufacturing troubleshooting","volume":"6","author":"Badini","year":"2023","journal-title":"Adv. Ind. Eng. Polym. Res"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.ijcnlp-main.45","article-title":"A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity","author":"Bang","year":"2023","journal-title":"arXiv"},{"key":"B6","first-page":"1877","article-title":"\u201cLanguage models are few-shot learners,\u201d","volume-title":"Advances in Neural Information Processing Systems, Vol. 33","author":"Brown","year":"2020"},{"key":"B7","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-3-031-35989-7_1","article-title":"\u201cHuman-centered hci practices leading the path to industry 5.0: a systematic literature review,\u201d","volume-title":"HCI International 2023 Posters","author":"Br\u00fcckner","year":"2023"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.14314","author":"Dettmers","year":"2023","journal-title":"Qlora: Efficient Finetuning of Quantized Llms"},{"key":"B9","first-page":"193","article-title":"Clustering and classification of maintenance logs using text data mining","volume":"87","author":"Edwards","year":"2008","journal-title":"Data Mining Anal"},{"key":"B10","doi-asserted-by":"publisher","first-page":"105058","DOI":"10.1016\/j.cie.2018.01.025","article-title":"Placing the operator at the centre of industry 4.0 design: modelling and assessing human activities within cyber-physical systems","volume":"139","author":"Fantini","year":"2020","journal-title":"Comp. Ind. Eng"},{"key":"B11","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1115\/1.4037178","article-title":"Toward knowledge management for smart manufacturing","volume":"17","author":"Feng","year":"2017","journal-title":"J. Comp. Inf. Sci. Eng"},{"key":"B12","first-page":"3816","article-title":"\u201cMaking pre-trained language models better few-shot learners,\u201d","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Gao","year":"2021"},{"key":"B13","first-page":"39","article-title":"\u201cThe manufacturing knowledge repository - consolidating knowledge to enable holistic process knowledge management in manufacturing,\u201d","volume-title":"Proceedings of the 16th International Conference on Enterprise Information Systems","author":"Gr\u00f6ger","year":"2014"},{"key":"B14","volume-title":"Applied","author":"Guest","year":"2011"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.418","article-title":"Temporalwiki: a lifelong benchmark for training and evaluating ever-evolving language models","author":"Jang","year":"2022","journal-title":"arXiv"},{"key":"B16","doi-asserted-by":"crossref","first-page":"3651","DOI":"10.18653\/v1\/P19-1356","article-title":"\u201cWhat does BERT learn about the structure of language?,\u201d","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Jawahar","year":"2019"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00330-023-10213-1","article-title":"ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports","author":"Jeblick","year":"2023","journal-title":"Eur. Radiol"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2401.04088","author":"Jiang","year":"2024","journal-title":"Mixtral of Experts"},{"key":"B19","doi-asserted-by":"crossref","DOI":"10.1145\/3571884.3604313","article-title":"\u201cHarnessing large language models for cognitive assistants in factories,\u201d","volume-title":"Proceedings of the 5th International Conference on Conversational User Interfaces, CUI '23","author":"Kernan Freire","year":""},{"key":"B20","first-page":"1","article-title":"\u201cTacit knowledge elicitation for shop-floor workers with an intelligent assistant,\u201d","author":"Kernan Freire","year":"","journal-title":"Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems"},{"key":"B21","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1145\/3544549.3585755","article-title":"\u201cAn empirical study on pseudo-log-likelihood bias measures for masked language models using paraphrased sentences,\u201d","author":"Kwon","year":"2022","journal-title":"TrustNLP 2022"},{"key":"B22","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1108\/IJOPM-05-2022-0282","article-title":"The mediating role of knowledge management processes in the effective use of artificial intelligence in manufacturing firms","volume":"42","author":"Leoni","year":"2022","journal-title":"Int. J. Operat. Prod. Manag"},{"key":"B23","article-title":"\u201cRetrieval-augmented generation for knowledge-intensive nlp tasks,\u201d","volume-title":"Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20","author":"Lewis","year":"2020"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2209.07753","author":"Liang","year":"2022","journal-title":"Code as Policies: Language Model Programs for Embodied Control"},{"key":"B25","unstructured":"LiuJ.\n          LlamaIndex2022"},{"key":"B26","doi-asserted-by":"publisher","first-page":"100257","DOI":"10.1016\/j.jii.2021.100257","article-title":"Industry 5.0: a survey on enabling technologies and potential applications","volume":"26","author":"Maddikunta","year":"2022","journal-title":"J. Ind. Inf. Integr"},{"key":"B27","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1016\/j.procir.2014.07.112","article-title":"A new human-centric factory model","volume":"26","author":"May","year":"2015","journal-title":"Proc CIRP"},{"key":"B28","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1016\/j.procir.2021.01.046","article-title":"Digital shop floor management enhanced by natural language processing","volume":"96","author":"M\u00fcller","year":"2021","journal-title":"Procedia CIRP"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2301.10035","author":"Nov","year":"2023","journal-title":"Putting Chatgpt's Medical Advice to the (Turing) Test"},{"key":"B30","doi-asserted-by":"crossref","DOI":"10.5121\/ijnlc.2020.9403","article-title":"A semantic question answering through heterogeneous data source in the domain of smart factory","author":"Oru\u00e7","year":"2020","journal-title":"Int. J. Nat. Lang. Comput"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4018\/IJKM.2019040101","article-title":"Knowledge management in the dark: the role of shadow IT in practices in manufacturing","volume":"15","author":"Richter","year":"2019","journal-title":"IJKM"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.14292","author":"Semnani","year":"2023","journal-title":"Wikichat: A Few-Shot Llm-Based Chatbot Grounded With Wikipedia"},{"key":"B33","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1007\/978-981-10-0983-9_32","author":"Serrat","year":"2017","journal-title":"The Five Whys Technique. Knowledge Solutions: Tools, Methods, and Approaches to Drive Organizational Performance"},{"key":"B34","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780192845290.001.0001","volume-title":"Human-Centered AI","author":"Shneiderman","year":"2022"},{"key":"B35","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.04360","author":"Tang","year":"2023","journal-title":"Does Synthetic Data Generation of Llms Help Clinical Text Mining"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.09288","author":"Touvron","year":"2023","journal-title":"Llama 2: Open Foundation And fine-Tuned Chat Models"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.02199","author":"Trautmann","year":"2022","journal-title":"Legal Prompt Engineering for Multilingual Legal Judgement Prediction"},{"key":"B39","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1016\/j.procir.2023.04.001","article-title":"Chatgpt for design, manufacturing, and education","volume":"119","author":"Wang","year":"","journal-title":"Proc. CIRP"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-industry.29","article-title":"Empower large language model to perform better on industrial domain-specific question answering","author":"Wang","year":"","journal-title":"arXiv"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1561\/116.00000010","author":"Wei","year":"2023","journal-title":"An Overview on Language Models: Recent Developments and Outlook"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.05759","author":"Wei","year":"","journal-title":"Emergent Abilities of Large Language Models"},{"key":"B43","first-page":"24824","article-title":"\u201cChain-of-thought prompting elicits reasoning in large language models,\u201d","volume-title":"Advances in Neural Information Processing Systems, Vol. 35","author":"Wei","year":""},{"key":"B44","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1007\/978-3-030-85910-7_34","article-title":"\u201cAnatomy of a digital assistant,\u201d","volume-title":"Advances in Production Management Systems. Artificial Intelligence for Sustainable and Resilient Production Systems","author":"Wellsandt","year":"2021"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.15290","article-title":"Chathome: development and evaluation of a domain-specific language model for home renovation","author":"Wen","year":"2023","journal-title":"ArXiv"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1109\/ETFA54631.2023.10275362","article-title":"Towards autonomous system: flexible modular production system enhanced with large language model agents","author":"Xia","year":"2023","journal-title":"arXiv"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2306.05443","article-title":"Pixiu: A large language model, instruction data and evaluation benchmark for finance","author":"Xie","year":"","journal-title":"arXiv"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2308.13565","article-title":"Darwin series: domain specific large language models for natural science","author":"Xie","year":"","journal-title":"arXiv"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2202.13169","article-title":"A systematic evaluation of large language models of code","author":"Xu","year":"2022","journal-title":"arXiv"},{"key":"B50","doi-asserted-by":"publisher","first-page":"530","DOI":"10.1016\/j.jmsy.2021.10.006","article-title":"Industry 4.0 and industry 5.0\u2014inception, conception and perception","volume":"61","author":"Xu","year":"2021","journal-title":"J. Manuf. Syst"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4450322","article-title":"A preliminary evaluation of chatgpt in requirements information retrieval","author":"Zhang","year":"","journal-title":"arXiv"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2311.03778","article-title":"Bridging the information gap between domain-specific model and general llm for personalized recommendation","author":"Zhang","year":"","journal-title":"arXiv"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.18223","author":"Zhao","year":"2023","journal-title":"A Survey of Large Language Models"},{"key":"B54","first-page":"46","article-title":"\u201cChatgpt hallucinates when attributing answers,\u201d","volume-title":"Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, SIGIR-AP '23","author":"Zuccon","year":"2023"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1293084\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,27]],"date-time":"2024-03-27T04:50:49Z","timestamp":1711515049000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1293084\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,27]]},"references-count":54,"alternative-id":["10.3389\/frai.2024.1293084"],"URL":"https:\/\/doi.org\/10.3389\/frai.2024.1293084","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,27]]},"article-number":"1293084"}}