{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T20:13:27Z","timestamp":1783541607341,"version":"3.55.0"},"reference-count":51,"publisher":"Springer Science and Business Media LLC","issue":"1-2","license":[{"start":{"date-parts":[[2023,9,29]],"date-time":"2023-09-29T00:00:00Z","timestamp":1695945600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,29]],"date-time":"2023-09-29T00:00:00Z","timestamp":1695945600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Hamburgische Investitions- und F\u00f6rderbank","award":["CCI 2014DE16RFOP006"],"award-info":[{"award-number":["CCI 2014DE16RFOP006"]}]},{"name":"Hamburgische Investitions- und F\u00f6rderbank","award":["CCI 2014DE16RFOP006"],"award-info":[{"award-number":["CCI 2014DE16RFOP006"]}]},{"name":"Hamburgische Investitions- und F\u00f6rderbank","award":["CCI 2014DE16RFOP006"],"award-info":[{"award-number":["CCI 2014DE16RFOP006"]}]},{"name":"Hamburgische Investitions- und F\u00f6rderbank","award":["CCI 2014DE16RFOP006"],"award-info":[{"award-number":["CCI 2014DE16RFOP006"]}]},{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Inf Retrieval J"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Recent machine learning advances demonstrate the effectiveness of zero-shot models trained on large amounts of data collected from the internet. Among these, CLIP (Contrastive Language-Image Pre-training) has been introduced as a multimodal model with high accuracy on a number of different tasks and domains. However, the unconstrained nature of the model begs the question whether it can be deployed in open-domain real-word applications effectively in front of non-technical users. In this paper, we evaluate whether CLIP can be used for multimodal video retrieval in a real-world environment. For this purpose, we implemented <jats:sc>impa<\/jats:sc>, an efficient shot-based retrieval system powered by CLIP. We additionally implemented advanced query functionality in a unified graphical user interface to facilitate an intuitive and efficient usage of CLIP for video retrieval tasks. Finally, we empirically evaluated our retrieval system by performing a user study with video editing professionals and journalists working in the TV news media industry. After having the participants solve open-domain video retrieval tasks, we collected data via questionnaires, interviews, and UI interaction logs. Our evaluation focused on the perceived intuitiveness of retrieval using natural language, retrieval accuracy, and how users interacted with the system\u2019s UI. We found that our advanced features yield higher task accuracy, user ratings, and more efficient queries. Overall, our results show the importance of designing intuitive and efficient user interfaces to be able to deploy large models such as CLIP effectively in real-world scenarios.<\/jats:p>","DOI":"10.1007\/s10791-023-09425-2","type":"journal-article","created":{"date-parts":[[2023,9,29]],"date-time":"2023-09-29T17:01:51Z","timestamp":1696006911000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Multimodal video retrieval with CLIP: a user study"],"prefix":"10.1007","volume":"26","author":[{"given":"Tayfun","family":"Alpay","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sven","family":"Magg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philipp","family":"Broze","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Speck","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,9,29]]},"reference":[{"key":"9425_CR1","unstructured":"Agarwal, S., Krueger, G., Clark, J. et\u00a0al. (2021). Evaluating clip: towards characterization of broader capabilities and downstream implications. arXiv preprint arXiv:2108.02818"},{"key":"9425_CR2","first-page":"24206","volume":"34","author":"H Akbari","year":"2021","unstructured":"Akbari, H., Yuan, L., Qian, R., et al. (2021). Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text. Advances in Neural Information Processing Systems, 34, 24206\u201324221.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"9425_CR3","doi-asserted-by":"crossref","unstructured":"Arnab, A., Dehghani, M., Heigold, G., et\u00a0al. (2021). Vivit: A video vision transformer. In: Proceedings of the IEEE\/CVF international conference on computer vision, pp 6836\u20136846.","DOI":"10.1109\/ICCV48922.2021.00676"},{"key":"9425_CR4","unstructured":"Awad, G., Butt, A.A., Curtis, K. et\u00a0al. (2020). Trecvid 2019: An evaluation campaign to benchmark video activity detection, video captioning and matching, and video search & retrieval. arXiv preprint arXiv:2009.09984"},{"key":"9425_CR5","unstructured":"Beaumont, R. (2022). Clip retrieval: Easily compute clip embeddings and build a clip retrieval system with them. https:\/\/github.com\/rom1504\/clip-retrieval."},{"key":"9425_CR6","unstructured":"Bradski, G. (2000). The OpenCV Library. Dr Dobb\u2019s Journal of Software Tools."},{"key":"9425_CR7","first-page":"1877","volume":"33","author":"T Brown","year":"2020","unstructured":"Brown, T., Mann, B., Ryder, N., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877\u20131901.","journal-title":"Advances in neural information processing systems"},{"key":"9425_CR8","doi-asserted-by":"publisher","first-page":"978","DOI":"10.1162\/tacl_a_00408","volume":"9","author":"E Bugliarello","year":"2021","unstructured":"Bugliarello, E., Cotterell, R., Okazaki, N., et al. (2021). Multimodal pretraining unmasked: A meta-analysis and a unified framework of vision-and-language berts. Transactions of the Association for Computational Linguistics, 9, 978\u2013994.","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"9425_CR9","unstructured":"Carlsson, F., Eisen, P., Rekathati, F. et\u00a0al. (2022). Cross-lingual and multilingual clip. In: Proceedings of the Language Resources and Evaluation Conference. European Language Resources Association, pp 6848\u20136854, https:\/\/aclanthology.org\/2022.lrec-1.739"},{"key":"9425_CR10","doi-asserted-by":"crossref","unstructured":"Dang-Nguyen, D.T., Gurrin, C., Larson, M., et\u00a0al. (eds) (2023). MultiMedia Modeling: 29th International Conference, MMM 2023, Bergen, Norway, January 9\u201312, 2023, Proceedings, Part I, vol 13833. Springer Nature, to appear.","DOI":"10.1007\/978-3-031-27818-1"},{"key":"9425_CR11","doi-asserted-by":"crossref","unstructured":"Gabeur, V., Sun, C., Alahari, K. et\u00a0al. (2020). Multi-modal transformer for video retrieval. In: Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK, August 23\u201328, 2020, Proceedings, Part IV 16, Springer, pp 214\u2013229.","DOI":"10.1007\/978-3-030-58548-8_13"},{"key":"9425_CR12","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1007\/s10791-020-09386-w","volume":"24","author":"G Gezici","year":"2021","unstructured":"Gezici, G., Lipani, A., Saygin, Y., et al. (2021). Evaluation metrics for measuring bias in search engine results. Information Retrieval Journal, 24, 85\u2013113.","journal-title":"Information Retrieval Journal"},{"key":"9425_CR13","doi-asserted-by":"crossref","unstructured":"Heller, S., Gasser, R., Illi, C., et al. (2021). Towards explainable interactive multi-modal video retrieval with vitrivr. In I. I. Part (Ed.), MultiMedia Modeling: 27th International Conference, MMM 2021, Prague, Czech Republic, June 22\u201324, 2021, Proceedings (pp. 435\u2013440). Springer.","DOI":"10.1007\/978-3-030-67835-7_41"},{"key":"9425_CR14","unstructured":"Huang, Z., Zeng, Z., Liu, B., et\u00a0al. (2020). Pixel-bert: Aligning image pixels with text by deep multi-modal transformers. arXiv preprint arXiv:2004.00849."},{"key":"9425_CR15","doi-asserted-by":"publisher","unstructured":"Ilharco, G., Wortsman, M., Wightman, R., et al. (2021). Openclip.https:\/\/doi.org\/10.5281\/zenodo.5143773","DOI":"10.5281\/zenodo.5143773"},{"issue":"3","key":"9425_CR16","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1109\/TBDATA.2019.2921572","volume":"7","author":"J Johnson","year":"2019","unstructured":"Johnson, J., Douze, M., & J\u00e9gou, H. (2019). Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3), 535\u2013547.","journal-title":"IEEE Transactions on Big Data"},{"key":"9425_CR17","doi-asserted-by":"crossref","unstructured":"Karpathy, A., Toderici, G., Shetty, S., et\u00a0al. (2014). Large-scale video classification with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1725\u20131732.","DOI":"10.1109\/CVPR.2014.223"},{"key":"9425_CR18","unstructured":"Kim, W., Son, B., & Kim, I. (2021). Vilt: Vision-and-language transformer without convolution or region supervision. In: International Conference on Machine Learning, PMLR, pp 5583\u20135594."},{"issue":"1","key":"9425_CR19","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1080\/10447319509526110","volume":"7","author":"JR Lewis","year":"1995","unstructured":"Lewis, J. R. (1995). Ibm computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57\u201378. https:\/\/doi.org\/10.1080\/10447319509526110","journal-title":"International Journal of Human-Computer Interaction"},{"key":"9425_CR20","unstructured":"Li, G., Duan, N., Fang, Y., et\u00a0al. (2020a). Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 11,336\u201311,344."},{"key":"9425_CR21","doi-asserted-by":"crossref","unstructured":"Li, X., Xu, C., Yang, G., et\u00a0al. (2019b). W2vv++ fully deep learning for ad-hoc video search. In: Proceedings of the 27th ACM international Conference on Multimedia, pp 1786\u20131794.","DOI":"10.1145\/3343031.3350906"},{"key":"9425_CR22","unstructured":"Li, L.H., Yatskar, M., Yin, D., et\u00a0al. (2019a). Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557."},{"key":"9425_CR23","doi-asserted-by":"crossref","unstructured":"Li, X., Yin, X., Li, C., et\u00a0al. (2020b). Oscar: Object-semantics aligned pre-training for vision-language tasks. In: European Conference on Computer Vision, Springer, pp 121\u2013137.","DOI":"10.1007\/978-3-030-58577-8_8"},{"key":"9425_CR24","unstructured":"LimeSurvey, (2003). LimeSurvey: An Open Source survey tool. LimeSurvey GmbH, Hamburg, Germany, http:\/\/www.limesurvey.org"},{"key":"9425_CR25","unstructured":"Lin, J., Yang, A., Zhang, Y., et\u00a0al. (2020). Interbert: Vision-and-language Interaction for Multi-modal Pretraining. arXiv preprint arXiv:2003.13198."},{"key":"9425_CR26","unstructured":"Liu, Y., Albanie, S., Nagrani, A., et\u00a0al. (2019). Use what you have: Video retrieval using representations from collaborative experts. arXiv preprint arXiv:1907.13487."},{"key":"9425_CR27","doi-asserted-by":"crossref","unstructured":"Liu, Z., Ning, J., Cao, Y., et\u00a0al. (2022). Video swin transformer. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp 3202\u20133211.","DOI":"10.1109\/CVPR52688.2022.00320"},{"key":"9425_CR28","doi-asserted-by":"crossref","unstructured":"Loko\u010d, J., Koval\u010d\u00edk, G., Sou\u010dek, T., et\u00a0al. (2019). Viret: A video retrieval tool for interactive known-item search. In: Proceedings of the 2019 on International Conference on Multimedia Retrieval, pp 177\u2013181.","DOI":"10.1145\/3323873.3325034"},{"key":"9425_CR29","doi-asserted-by":"crossref","unstructured":"Loko\u010d, J., Vesel\u1ef3, P., Mejzl\u00edk, F., et\u00a0al. (2021). Is the reign of interactive search eternal? findings from the video browser showdown 2020. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 17(3):1\u201326.","DOI":"10.1145\/3445031"},{"key":"9425_CR30","volume":"32","author":"J Lu","year":"2019","unstructured":"Lu, J., Batra, D., Parikh, D., et al. (2019). Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems, 32, 456985.","journal-title":"Advances in neural information processing systems"},{"key":"9425_CR31","unstructured":"Luo, H., Ji, L., Shi, B., et\u00a0al. (2020). Univl: A unified video and language pre-training model for multimodal understanding and generation. arXiv preprint arXiv:2002.06353."},{"key":"9425_CR32","doi-asserted-by":"crossref","unstructured":"Luo, H., Ji, L., Zhong, M., et\u00a0al. (2021) Clip4clip: An empirical study of clip for end to end video clip retrieval. arXiv preprint arXiv:2104.08860.","DOI":"10.1016\/j.neucom.2022.07.028"},{"issue":"4","key":"9425_CR33","doi-asserted-by":"publisher","first-page":"824","DOI":"10.1109\/TPAMI.2018.2889473","volume":"42","author":"YA Malkov","year":"2020","unstructured":"Malkov, Y. A., & Yashunin, D. A. (2020). Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4), 824\u2013836. https:\/\/doi.org\/10.1109\/TPAMI.2018.2889473","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"9425_CR34","doi-asserted-by":"crossref","unstructured":"Miech, A., Zhukov, D., Alayrac, J.B., et\u00a0al. (2019). Howto100m: Learning a text-video embedding by watching hundred million narrated video clips. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision, pp 2630\u20132640.","DOI":"10.1109\/ICCV.2019.00272"},{"key":"9425_CR35","doi-asserted-by":"crossref","unstructured":"Mithun, N.C., Li, J., Metze, F., et\u00a0al. (2018). Learning joint embedding with multimodal cues for cross-modal video-text retrieval. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp 19\u201327.","DOI":"10.1145\/3206025.3206064"},{"key":"9425_CR36","unstructured":"OpenAI (2022). ChatGPT. https:\/\/openai.com\/blog\/chatgpt, [Online; accessed 15-Ma-2023]."},{"key":"9425_CR37","unstructured":"Ouyang, L., Wu, J., Jiang, X., et\u00a0al. (2022). Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155."},{"key":"9425_CR38","doi-asserted-by":"crossref","unstructured":"Portillo-Quintero, J.A., Ortiz-Bayliss, J.C., & Terashima-Mar\u00edn, H. (2021). A straightforward framework for video retrieval using clip. In: Mexican Conference on Pattern Recognition, Springer, pp 3\u201312.","DOI":"10.1007\/978-3-030-77004-4_1"},{"key":"9425_CR39","unstructured":"Radford, A., Kim, J.W., Hallacy, C., et\u00a0al. (2021). Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning, PMLR, pp 8748\u20138763."},{"issue":"12","key":"9425_CR40","first-page":"46","volume":"81","author":"FF Reichheld","year":"2003","unstructured":"Reichheld, F. F. (2003). The one number you need to grow. Harvard Business Review, 81(12), 46\u201355.","journal-title":"Harvard Business Review"},{"issue":"1","key":"9425_CR41","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1080\/1937156X.2010.11949658","volume":"25","author":"NF Ribeiro","year":"2010","unstructured":"Ribeiro, N. F., & Yarnal, C. M. (2010). The perceived difficulty assessment questionnaire (pdaq): Methodology and applications for leisure educators and practitioners. Schole A Journal of Leisure Studies and Recreation Education, 25(1), 111\u2013115.","journal-title":"Schole A Journal of Leisure Studies and Recreation Education"},{"key":"9425_CR42","doi-asserted-by":"crossref","unstructured":"Rossetto, L., Schuldt, H., Awad, G., et\u00a0al. (2019). V3c\u2013a research video collection. In: International Conference on Multimedia Modeling, Springer, pp 349\u2013360.","DOI":"10.1007\/978-3-030-05710-7_29"},{"key":"9425_CR43","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1109\/TMM.2020.2980944","volume":"23","author":"L Rossetto","year":"2021","unstructured":"Rossetto, L., Gasser, R., Loko\u010d, J., et al. (2021). Interactive Video Retrieval in the Age of Deep Learning - Detailed Evaluation of VBS 2019. IEEE Transactions on Multimedia, 23, 243\u2013256. https:\/\/doi.org\/10.1109\/TMM.2020.2980944","journal-title":"IEEE Transactions on Multimedia"},{"key":"9425_CR44","unstructured":"Su, W., Zhu, X., Cao, Y., et\u00a0al. (2019). Vl-bert: Pre-training of generic visual-linguistic representations. arXiv preprint arXiv:1908.08530."},{"key":"9425_CR45","unstructured":"Sun, C., Baradel, F., Murphy, K., et\u00a0al. (2019a). Learning video representations using contrastive bidirectional transformer. arXiv preprint arXiv:1906.05743."},{"key":"9425_CR46","doi-asserted-by":"crossref","unstructured":"Sun, C., Myers, A., Vondrick, C., et\u00a0al. (2019b). Videobert: A joint model for video and language representation learning. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision, pp 7464\u20137473.","DOI":"10.1109\/ICCV.2019.00756"},{"key":"9425_CR47","doi-asserted-by":"crossref","unstructured":"Tan, H., & Bansal, M. (2019). Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490.","DOI":"10.18653\/v1\/D19-1514"},{"key":"9425_CR48","doi-asserted-by":"crossref","unstructured":"Vesel\u1ef3, P., Mejzl\u00edk, F., & Loko\u010d, J. (2021). Somhunter v2 at video browser showdown 2021. In: MultiMedia Modeling: 27th International Conference, MMM 2021, Prague, Czech Republic, June 22\u201324, 2021, Proceedings, Part II 27, Springer, pp 461\u2013466.","DOI":"10.1007\/978-3-030-67835-7_45"},{"key":"9425_CR49","doi-asserted-by":"publisher","unstructured":"Vohra, D. (2016). Apache Parquet, Apress, Berkeley, CA, pp 325\u2013335. https:\/\/doi.org\/10.1007\/978-1-4842-2199-0_8","DOI":"10.1007\/978-1-4842-2199-0_8"},{"key":"9425_CR50","unstructured":"Yu, L., Chen, Y.C., & Li, L. (2020). Self-supervised learning for vision-and-language. https:\/\/rohit497.github.io\/Recent-Advances-in-Vision-and-Language-Research\/slides\/tutorial-part5-pretraining.pdf, accessed: 2023-5-15."},{"key":"9425_CR51","doi-asserted-by":"crossref","unstructured":"Yu, F., Tang, J., Yin, W., et\u00a0al. (2021). Ernie-vil: Knowledge enhanced vision-language representations through scene graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 3208\u20133216.","DOI":"10.1609\/aaai.v35i4.16431"}],"container-title":["Information Retrieval Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10791-023-09425-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10791-023-09425-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10791-023-09425-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T10:14:21Z","timestamp":1703153661000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10791-023-09425-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,29]]},"references-count":51,"journal-issue":{"issue":"1-2","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["9425"],"URL":"https:\/\/doi.org\/10.1007\/s10791-023-09425-2","relation":{},"ISSN":["1386-4564","1573-7659"],"issn-type":[{"value":"1386-4564","type":"print"},{"value":"1573-7659","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,29]]},"assertion":[{"value":"20 May 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 September 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 September 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"P.B. is employed by nachtblau GmbH, who intend to use the results for further development of their commercial media asset management system \u201cmedialoopster\u201d.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The study involving human participants was reviewed and approved by the internal HITeC e.V. ethical review board.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"All participants of the presented study provided informed consent, including the participation and collection of pseudonomized data for the purposes of statistical evaluation and research.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"All participants of the presented study provided informed consent, including the publication of non-identifying results using anonymized and\/or aggregated data from the study.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to publication"}}],"article-number":"6"}}