{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T11:03:43Z","timestamp":1790766223595,"version":"4.1.0"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T00:00:00Z","timestamp":1788220800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T00:00:00Z","timestamp":1788825600000},"content-version":"vor","delay-in-days":7,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100018693","name":"HORIZON EUROPE Framework Programme","doi-asserted-by":"publisher","award":["GA 101137074"],"award-info":[{"award-number":["GA 101137074"]}],"id":[{"id":"10.13039\/100018693","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Digit Libr"],"published-print":{"date-parts":[[2026,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Large Language Models (LLMs) are increasingly used as flexible access layers for information-rich collections, raising the question of whether they can also support structured access to cultural heritage data as if they were digital libraries. This work investigates that question through Galois, a database-inspired system that treats the LLM as a noisy storage layer. We evaluate six querying modes: direct natural language prompting, direct SQL prompting, and four Galois strategies that combine LLM-facing tuple retrieval with deterministic relational processing. The evaluation uses two cultural heritage benchmarks: Paintings Extended, based on paintings data, and a new benchmark based on Europeana metadata. For both benchmarks, we define fourteen query templates with ten variants each. The main experiments use GPT-4o-mini, Granite 3.3 8B, and Llama 3 70B, with an additional diagnostic analysis of two GPT-5 deployments. We complement the benchmark metrics with a stratified manual audit of 60 unique disagreements. Performance is strongly shaped by model capability, deployment behavior, dataset structure, and execution strategy. Direct natural language prompting is the strongest and most robust baseline, while direct SQL prompting becomes nearly equivalent on the strongest open models. Among the structured strategies, only GaloisA and GaloisF become meaningfully competitive, especially on the flatter Europeana benchmark. In the audited sample, 35 disagreements were model-originated failures, 21 reflected reference-coverage or metadata-normalization effects, and four remained indeterminate. The additional GPT-5 deployment analysis reveals frequent empty outputs and provider-level constraints. These diagnostic results characterize the specific tested deployments and must not be interpreted as a general ranking of GPT-5 model capability. Querying LLMs as digital libraries is feasible, but its effectiveness depends on model strength, deployment conditions, dataset structure, and execution strategy. Direct prompting is already strong for exploratory cultural heritage access, while Galois remains valuable when relational discipline and controlled query execution are required.<\/jats:p>","DOI":"10.1007\/s00799-026-00448-w","type":"journal-article","created":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T19:07:46Z","timestamp":1788894466000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Large language models as digital libraries: a multi-benchmark and multi-model study"],"prefix":"10.1007","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-3856-7207","authenticated-orcid":false,"given":"Mirco","family":"Cazzaro","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4970-4554","authenticated-orcid":false,"given":"Gianmaria","family":"Silvello","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,9,8]]},"reference":[{"key":"448_CR1","doi-asserted-by":"publisher","DOI":"10.1145\/3748304","author":"Y Zhu","year":"2025","unstructured":"Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Chen, H., Liu, Z., Dou, Z., Wen, J.-R.: Large language models for information retrieval: a survey. ACM Trans. Inf. Syst. (2025). https:\/\/doi.org\/10.1145\/3748304","journal-title":"ACM Trans. Inf. Syst."},{"key":"448_CR2","doi-asserted-by":"publisher","unstructured":"Petroni, F., Rockt\u00e4schel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., Miller, A.H.: Language models as knowledge bases? In: Inui, K., Jiang, J., Ng, V., Wan, X. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, pp. 2463\u20132473. Association for Computational Linguistics (2019). https:\/\/doi.org\/10.18653\/V1\/D19-1250","DOI":"10.18653\/V1\/D19-1250"},{"key":"448_CR3","doi-asserted-by":"publisher","unstructured":"Roberts, A., Raffel, C., Shazeer, N.: How much knowledge can you pack into the parameters of a language model? In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 5418\u20135426. Association for Computational Linguistics (2020). https:\/\/doi.org\/10.18653\/v1\/2020.emnlp-main.437","DOI":"10.18653\/v1\/2020.emnlp-main.437"},{"key":"448_CR4","unstructured":"Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K\u00fcttler, H., Lewis, M., Yih, W., Rockt\u00e4schel, T., Riedel, S., Kiela, D.: Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, pp. 9459\u20139474 (2020). https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/6b493230205f780e1bc26945df7481e5-Paper.pdf"},{"key":"448_CR5","doi-asserted-by":"publisher","unstructured":"Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., Hajishirzi, H.: When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 9802\u20139822. Association for Computational Linguistics (2023). https:\/\/doi.org\/10.18653\/v1\/2023.acl-long.546","DOI":"10.18653\/v1\/2023.acl-long.546"},{"key":"448_CR6","doi-asserted-by":"publisher","unstructured":"Wang, C., Liu, X., Yue, Y., Guo, Q., Hu, X., Tang, X., Zhang, T., Jiayang, C., Yao, Y., Hu, X., Qi, Z., Gao, W., Wang, Y., Yang, L., Wang, J., Xie, X., Zhang, Z., Zhang, Y.: Survey on factuality in large language models. ACM Comput. Surv. 58(1), 1\u201337 (2026). https:\/\/doi.org\/10.1145\/3742420","DOI":"10.1145\/3742420"},{"key":"448_CR7","doi-asserted-by":"publisher","unstructured":"Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., Liu, T.: A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst. 43(2), 1\u201355 (2025). https:\/\/doi.org\/10.1145\/3703155","DOI":"10.1145\/3703155"},{"key":"448_CR8","doi-asserted-by":"publisher","unstructured":"Satriani, D., Veltri, E., Santoro, D., Rosato, S., Varriale, S., Papotti, P.: Logical and physical optimizations for SQL query execution over large language models. Proc. ACM Manag. Data 3(3), 1\u201328 (2025). https:\/\/doi.org\/10.1145\/3725411","DOI":"10.1145\/3725411"},{"key":"448_CR9","doi-asserted-by":"publisher","DOI":"10.1145\/3756016","author":"I Vasic","year":"2025","unstructured":"Vasic, I., Fill, H.-G., Quattrini, R., Pierdicca, R.: Knowledge graphs vs. large language models: competitors or partners in supporting virtual museums. J. Comput. Cult. Herit. (2025). https:\/\/doi.org\/10.1145\/3756016","journal-title":"J. Comput. Cult. Herit."},{"key":"448_CR10","doi-asserted-by":"publisher","unstructured":"Hu, X.: Application of large language models for digital libraries. In: Chu, S.K., Hu, X. (eds.) Proceedings of the 24th ACM\/IEEE Joint Conference on Digital Libraries, JCDL 2024, pp. 1\u20132. ACM (2024). https:\/\/doi.org\/10.1145\/3677389.3702617","DOI":"10.1145\/3677389.3702617"},{"key":"448_CR11","doi-asserted-by":"publisher","unstructured":"Trichopoulos, G.: Large language models for cultural heritage. In: Proceedings of the 2nd International Conference of the ACM Greek SIGCHI Chapter, CHI 2023, pp. 1\u20135. ACM (2023). https:\/\/doi.org\/10.1145\/3609987.3610018","DOI":"10.1145\/3609987.3610018"},{"key":"448_CR12","unstructured":"Cazzaro, M., Silvello, G.: Querying llms as if they were digital libraries. In: Proceedings of the 22nd Conference on Information and Research Science Connecting to Digital and Library Science, Modena, Italy (2026). (IRCDL 2026)"},{"key":"448_CR13","unstructured":"Aloia, N., Concordia, C., Meghini, C.: Europeana: Towards the european digital library. In: Agosti, M., Esposito, F., Thanos, C. (eds.) Post-proceedings of the Fifth Italian Research Conference on Digital Libraries\u2014IRCDL 2009, Padova, Italy, 29-30 January 2009, pp. 154\u2013157. DELOS: an Association for Digital Libraries \/ Department of Information Engineering of the University of Padua (2009). http:\/\/www.dei.unipd.it\/%7Eagosti\/ircdl\/atti-ircdl2009-finale.pdf"},{"key":"448_CR14","doi-asserted-by":"publisher","unstructured":"Heinzerling, B., Inui, K.: Language models as knowledge bases: On entity representations, storage capacity, and paraphrased queries. In: Merlo, P., Tiedemann, J., Tsarfaty, R. (eds.) Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL 2021, Online, April 19\u201323, 2021, pp. 1772\u20131791. Association for Computational Linguistics (2021). https:\/\/doi.org\/10.18653\/V1\/2021.EACL-MAIN.153","DOI":"10.18653\/V1\/2021.EACL-MAIN.153"},{"key":"448_CR15","doi-asserted-by":"publisher","unstructured":"Shi, L., Tang, Z., Zhang, N., Zhang, X., Yang, Z.: A survey on employing large language models for text-to-sql tasks. ACM Comput. Surv. 58(2), 1\u201337 (2026). https:\/\/doi.org\/10.1145\/3737873","DOI":"10.1145\/3737873"},{"key":"448_CR16","doi-asserted-by":"publisher","unstructured":"Sun, K., Xu, Y., Zha, H., Liu, Y., Dong, X.L.: Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs? In: Duh, K., Gomez, H., Bethard, S. (eds.) Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 311\u2013325. Association for Computational Linguistics, Mexico City, Mexico (2024). https:\/\/doi.org\/10.18653\/v1\/2024.naacl-long.18 . https:\/\/aclanthology.org\/2024.naacl-long.18\/","DOI":"10.18653\/v1\/2024.naacl-long.18"},{"key":"448_CR17","doi-asserted-by":"publisher","unstructured":"Wang, C., Liu, P., Zhang, Y.: Can generative pre-trained language models serve as knowledge bases for closed-book qa? In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL\/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1\u20136, 2021, pp. 3241\u20133251. Association for Computational Linguistics (2021). https:\/\/doi.org\/10.18653\/V1\/2021.ACL-LONG.251","DOI":"10.18653\/V1\/2021.ACL-LONG.251"},{"key":"448_CR18","doi-asserted-by":"publisher","unstructured":"Youssef, P., Koras, O.A., Li, M., Schl\u00f6tterer, J., Seifert, C.: Give me the facts! A survey on factual knowledge probing in pre-trained language models. In: Bouamor, H., Pino, J., Bali, K. (eds.) Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6\u201310, 2023. Findings of ACL, pp. 15588\u201315605. Association for Computational Linguistics (2023). https:\/\/doi.org\/10.18653\/V1\/2023.FINDINGS-EMNLP.1043","DOI":"10.18653\/V1\/2023.FINDINGS-EMNLP.1043"},{"key":"448_CR19","unstructured":"Arnold, T.B., Tilton, L.: Explainable search and discovery of visual cultural heritage collections with multimodal large language models. In: Proceedings of the Computational Humanities Research Conference 2024, Aarhus, Denmark, December 4\u20136, 2024. CEUR Workshop Proceedings, pp. 559\u2013574. CEUR-WS.org (2024). https:\/\/ceur-ws.org\/Vol-3834\/paper1.pdf"},{"key":"448_CR20","unstructured":"Vanallemeersch, T., Szoc, S., Meeus, L.: Ai4culture: Towards multilingual access for cultural heritage data. In: Scarton, C., Prescott, C., Bayliss, C., Oakley, C., Wright, J., Wrigley, S., Song, X., Gow-Smith, E., Forcada, M., Moniz, H.L. (eds.) Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 2), EAMT 2024, Sheffield, UK, June 24\u201327, 2024, pp. 59\u201360. European Association for Machine Translation (EAMT) (2024). https:\/\/aclanthology.org\/2024.eamt-2.30"},{"key":"448_CR21","unstructured":"Liu, C., Russo, M., Cafarella, M.J., Cao, L., Chen, P.B., Chen, Z., Franklin, M.J., Kraska, T., Madden, S., Shahout, R., Vitagliano, G.: Palimpzest: Optimizing ai-powered analytics with declarative query processing. In: 15th Conference on Innovative Data Systems Research, CIDR 2025, Amsterdam, The Netherlands, January 19\u201322, 2025. www.cidrdb.org (2025). https:\/\/vldb.org\/cidrdb\/2025\/palimpzest-optimizing-ai-powered-analytics-with-declarative-query-processing.html"},{"key":"448_CR22","doi-asserted-by":"publisher","unstructured":"Giannakouris, V., Trummer, I.: Swelldb: Dynamic query-driven table generation with large language models. In: Companion of the 2025 International Conference on Management of Data. SIGMOD\/PODS \u201925, pp. 95\u201398. Association for Computing Machinery, New York, NY, USA (2025). https:\/\/doi.org\/10.1145\/3722212.3725136","DOI":"10.1145\/3722212.3725136"},{"key":"448_CR23","unstructured":"Zhao, F., Agrawal, D., El\u00a0Abbadi, A.: Hybrid querying over relational databases and large language models. In: 15th Conference on Innovative Data Systems Research, CIDR 2025. www.cidrdb.org (2025). https:\/\/vldb.org\/cidrdb\/2025\/hybrid-querying-over-relational-databases-and-large-language-models.html"},{"key":"448_CR24","doi-asserted-by":"publisher","unstructured":"Pi, X., Wang, B., Gao, Y., Guo, J., Li, Z., Lou, J.: Towards robustness of text-to-sql models against natural and realistic adversarial table perturbation. In: Muresan, S., Nakov, P., Villavicencio, A. (eds.) Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22\u201327, 2022, pp. 2007\u20132022. Association for Computational Linguistics (2022). https:\/\/doi.org\/10.18653\/V1\/2022.ACL-LONG.142","DOI":"10.18653\/V1\/2022.ACL-LONG.142"},{"key":"448_CR25","doi-asserted-by":"crossref","unstructured":"Saparina, I., Lapata, M.: Improving generalization in semantic parsing by increasing natural language variation. In: Graham, Y., Purver, M. (eds.) Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 - Volume 1: Long Papers, St. Julian\u2019s, Malta, March 17\u201322, 2024, pp. 1178\u20131193. Association for Computational Linguistics (2024). https:\/\/aclanthology.org\/2024.eacl-long.71","DOI":"10.18653\/v1\/2024.eacl-long.71"},{"key":"448_CR26","doi-asserted-by":"crossref","unstructured":"Safarzadeh, M., Oroojlooy, A., Roth, D.: Evaluating NL2SQL via SQL2NL. In: Christodoulopoulos, C., Chakraborty, T., Rose, C., Peng, V. (eds.) Findings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China, November 4\u20139, 2025, pp. 18954\u201318968. Association for Computational Linguistics (2025). https:\/\/aclanthology.org\/2025.findings-emnlp.1031\/","DOI":"10.18653\/v1\/2025.findings-emnlp.1031"},{"key":"448_CR27","doi-asserted-by":"publisher","unstructured":"Raasveldt, M., M\u00fchleisen, H.: Duckdb: an embeddable analytical database. In: Boncz, P., Manegold, S., Ailamaki, A., Deshpande, A., Kraska, T. (eds.) Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019, Amsterdam, The Netherlands, June 30 - July 5, 2019, pp. 1981\u20131984. ACM (2019). https:\/\/doi.org\/10.1145\/3299869.3320212","DOI":"10.1145\/3299869.3320212"}],"container-title":["International Journal on Digital Libraries"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00799-026-00448-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00799-026-00448-w","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00799-026-00448-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T10:26:50Z","timestamp":1790764010000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00799-026-00448-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9]]},"references-count":27,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,9]]}},"alternative-id":["448"],"URL":"https:\/\/doi.org\/10.1007\/s00799-026-00448-w","relation":{},"ISSN":["1432-5012","1432-1300"],"issn-type":[{"value":"1432-5012","type":"print"},{"value":"1432-1300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9]]},"assertion":[{"value":"5 May 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 August 2026","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 August 2026","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 September 2026","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"13"}}