{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T08:01:35Z","timestamp":1761897695104,"version":"3.41.0"},"reference-count":66,"publisher":"Association for Computing Machinery (ACM)","issue":"3s","license":[{"start":{"date-parts":[[2022,10,31]],"date-time":"2022-10-31T00:00:00Z","timestamp":1667174400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"University of Bologna with the Alma Attrezzature 2017 grant"},{"name":"AEFFE S.p.a. and the Golinelli Foundation with the funding of two Ph.D. scholarships"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2022,10,31]]},"abstract":"<jats:p>Although one of the most popular practices in photography since the end of the 19th century, an increase in scholarly interest in family photo albums dates back to the early 1980s. Such collections of photos may reveal sociological and historical insights regarding specific cultures and times. They are, however, in most cases scattered among private homes and only available on paper or photographic film, thus making their collection and analysis by historians, socio-cultural anthropologists, and cultural theorists very cumbersome. Computer-based methodologies could aid such a process in various ways, speeding up the cataloging step, for example, with the use of modern computer vision techniques. We here investigate such an approach, introducing the design and development of a multimedia application that may automatically catalog vernacular pictures drawn from family photo albums. To this aim, we introduce the IMAGO dataset, which is composed of photos belonging to family albums assembled at the University of Bologna\u2019s Rimini campus since 2004. Exploiting the proposed application, IMAGO has offered the opportunity of experimenting with photos taken between the years 1845 and 2009. In particular, it has been possible to estimate their socio-historical content, i.e., the dates and contexts of the images, without resorting to any other sources of information. Exceeding our initial expectations, such an approach has revealed its merit not only in terms of performance but also in terms of the foreseeable implications for the benefit of socio-historical research. To the best of our knowledge, this contribution is among the few that move along this path at the intersection of socio-historical studies, multimedia computing, and artificial intelligence.<\/jats:p>","DOI":"10.1145\/3507918","type":"journal-article","created":{"date-parts":[[2022,2,18]],"date-time":"2022-02-18T19:42:08Z","timestamp":1645213328000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Toward a Holistic Approach to the Socio-historical Analysis of Vernacular Photos"],"prefix":"10.1145","volume":"18","author":[{"given":"Lorenzo","family":"Stacchio","sequence":"first","affiliation":[{"name":"Department for Life Quality Studies, University of Bologna, Rimini, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alessia","family":"Angeli","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Bologna, Bologna, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giuseppe","family":"Lisanti","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Bologna, Bologna, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniela","family":"Calanca","sequence":"additional","affiliation":[{"name":"Department of the Arts, University of Bologna, Rimini, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gustavo","family":"Marfia","sequence":"additional","affiliation":[{"name":"Department of the Arts, University of Bologna, Rimini, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,11]]},"reference":[{"key":"e_1_3_2_2_2","article-title":"Amazon SageMaker Ground Truth","year":"2021","unstructured":"Amazon. 2021. Amazon SageMaker Ground Truth. https:\/\/aws.amazon.com\/it\/sagemaker\/groundtruth\/.","journal-title":"https:\/\/aws.amazon.com\/it\/sagemaker\/groundtruth\/"},{"doi-asserted-by":"crossref","unstructured":"S. Barba F. Fiorillo P. Ortiz Coder S. D\u2019auria and E. De Feo. 2011. An application for cultural heritage in Erasmus Placement. Surveys and 3D cataloguing archaeological finds in Merida (Spain). (2011).","key":"e_1_3_2_3_2","DOI":"10.5194\/isprsarchives-XXXVIII-5-W16-213-2011"},{"doi-asserted-by":"publisher","key":"e_1_3_2_4_2","DOI":"10.1109\/ICIP.2014.7025524"},{"doi-asserted-by":"publisher","key":"e_1_3_2_5_2","DOI":"10.1080\/00397679.2015.1095012"},{"key":"e_1_3_2_6_2","volume-title":"Pattern Recognition and Machine Learning","author":"Bishop C. M.","year":"2006","unstructured":"C. M. Bishop. 2006. Pattern Recognition and Machine Learning. Springer."},{"doi-asserted-by":"publisher","key":"e_1_3_2_7_2","DOI":"10.1109\/CTRQ.2010.35"},{"doi-asserted-by":"publisher","key":"e_1_3_2_8_2","DOI":"10.1093\/acprof:oso\/9780198719571.003.0006"},{"doi-asserted-by":"publisher","key":"e_1_3_2_9_2","DOI":"10.1177\/026327696013003002"},{"doi-asserted-by":"publisher","key":"e_1_3_2_10_2","DOI":"10.1111\/0018-2656.00183"},{"key":"e_1_3_2_11_2","volume-title":"Postsocial History: An Introduction","author":"Cabrera M. \u00c1.","year":"2004","unstructured":"M. \u00c1. Cabrera. 2004. Postsocial History: An Introduction. Lexington Books."},{"issue":"5","key":"e_1_3_2_12_2","first-page":"203","article-title":"Percorsi di storia della famiglia","volume":"5","author":"Calanca D.","year":"2004","unstructured":"D. Calanca. 2004. Percorsi di storia della famiglia. Rivista Di Storia E Storiografia 5, 5 (Nov. 2004), 203\u2013210.","journal-title":"Rivista Di Storia E Storiografia"},{"key":"e_1_3_2_13_2","article-title":"Album di famiglia. Autorappresentazioni tra pubblico e privato (1870-1950).","author":"Calanca D.","year":"2005","unstructured":"D. Calanca. 2005. Album di famiglia. Autorappresentazioni tra pubblico e privato (1870-1950). Storia e Futuro 8\u20139 (2005).","journal-title":"Storia e Futuro 8\u20139"},{"key":"e_1_3_2_14_2","article-title":"Fotografie amatoriali e fotografie professionali nell\u2019Italia del boom economico","author":"Calanca D.","year":"2006","unstructured":"D. Calanca. 2006. Fotografie amatoriali e fotografie professionali nell\u2019Italia del boom economico. Storia e Futuro 12 (2006), 134\u2013144.","journal-title":"Storia e Futuro 12"},{"issue":"3","key":"e_1_3_2_15_2","first-page":"1","article-title":"Italians posing between public and private. Theories and practices of Social Heritage","volume":"2","author":"Calanca D.","year":"2011","unstructured":"D. Calanca. 2011. Italians posing between public and private. Theories and practices of Social Heritage. Almatourism-Journal of Tourism, Culture and Territorial Development 2, 3 (2011), 1\u20139.","journal-title":"Almatourism-Journal of Tourism, Culture and Territorial Development"},{"doi-asserted-by":"publisher","key":"e_1_3_2_16_2","DOI":"10.1109\/TBDATA.2018.2839919"},{"key":"e_1_3_2_17_2","article-title":"Learning and fusing multiple user interest representations for micro-video and movie recommendations","author":"Chen X.","year":"2020","unstructured":"X. Chen, D. Liu, Z. Xiong, and Z-J. Zha. 2020. Learning and fusing multiple user interest representations for micro-video and movie recommendations. IEEE Transactions on Multimedia (2020).","journal-title":"IEEE Transactions on Multimedia"},{"unstructured":"E. S. Clemens and M. D. Hughes. 2002. Recovering past protest: Historical research on social movements. In Methods of Social Movement Research. Minneapolis B. Klandermans and S. Staggenborg (Eds.). University of Minnesota Press 201\u2013230.","key":"e_1_3_2_18_2"},{"doi-asserted-by":"publisher","key":"e_1_3_2_19_2","DOI":"10.1177\/0340035209359561"},{"key":"e_1_3_2_20_2","volume-title":"L\u2019Italia Del Novecento: Le Fotografie e la Storia","author":"Criscenti L.","year":"2005","unstructured":"L. Criscenti, G. D\u2019autilia, and G. De Luna. 2005. L\u2019Italia Del Novecento: Le Fotografie e la Storia. Giulio Einaudi editore."},{"key":"e_1_3_2_21_2","article-title":"Neural Network Architectures","author":"Culurciello E.","year":"2021","unstructured":"E. Culurciello. 2021. Neural Network Architectures. https:\/\/towardsdatascience.com\/neural-network-architectures-156e5bad51ba.","journal-title":"https:\/\/towardsdatascience.com\/neural-network-architectures-156e5bad51ba"},{"doi-asserted-by":"publisher","key":"e_1_3_2_22_2","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_23_2","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin J.","year":"2018","unstructured":"J. Devlin, M. Chang, K. Lee, and K. Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).","journal-title":"arXiv preprint arXiv:1810.04805"},{"unstructured":"A. Dosovitskiy L. Beyer A. Kolesnikov D. Weissenborn X. Zhai T. Unterthiner M. Dehghani M. Minderer G. Heigold S. Gelly J. Uszkoreit and N. Houlsby. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.","key":"e_1_3_2_24_2"},{"doi-asserted-by":"publisher","key":"e_1_3_2_25_2","DOI":"10.5032\/jae.2015.03069"},{"doi-asserted-by":"publisher","key":"e_1_3_2_26_2","DOI":"10.1017\/S002085900011510X"},{"doi-asserted-by":"publisher","key":"e_1_3_2_27_2","DOI":"10.1109\/ICCVW.2015.87"},{"key":"e_1_3_2_28_2","article-title":"AI Platform Data Labeling Service","year":"2021","unstructured":"Google. 2021. AI Platform Data Labeling Service. https:\/\/cloud.google.com\/ai-platform\/data-labeling\/docs.","journal-title":"https:\/\/cloud.google.com\/ai-platform\/data-labeling\/docs"},{"doi-asserted-by":"publisher","key":"e_1_3_2_29_2","DOI":"10.1109\/TPAMI.2022.3152247"},{"doi-asserted-by":"crossref","unstructured":"G. Huang Z. Liu L. Van Der Maaten and K. Q. Weinberger. 2018. Densely Connected Convolutional Networks. arxiv:1608.06993 [cs.CV]","key":"e_1_3_2_30_2","DOI":"10.1109\/CVPR.2017.243"},{"key":"e_1_3_2_31_2","article-title":"YOLO: Real Time Object Detection","author":"Redmon J.","year":"2019","unstructured":"J. Redmon. 2019. YOLO: Real Time Object Detection. Retrieved August 3, 2020, from https:\/\/github.com\/pjreddie\/darknet\/wiki\/YOLO:-Real-Time-Object-Detection.","journal-title":"https:\/\/github.com\/pjreddie\/darknet\/wiki\/YOLO:-Real-Time-Object-Detection"},{"unstructured":"H. Kaiming Z. Xiangyu R. Shaoqing and S. Jian. 2015. Deep Residual Learning for Image Recognition. arxiv:1512.03385 [cs.CV]","key":"e_1_3_2_32_2"},{"doi-asserted-by":"publisher","key":"e_1_3_2_33_2","DOI":"10.1109\/MCE.2016.2640698"},{"doi-asserted-by":"publisher","key":"e_1_3_2_34_2","DOI":"10.1109\/TMM.2019.2939711"},{"doi-asserted-by":"publisher","key":"e_1_3_2_35_2","DOI":"10.1184\/R1\/12791807.v2"},{"doi-asserted-by":"publisher","key":"e_1_3_2_36_2","DOI":"10.7208\/chicago\/9780226129259.001.0001"},{"key":"e_1_3_2_37_2","article-title":"Vernacular Photography","year":"2020","unstructured":"MoMA. 2020. Vernacular Photography. https:\/\/www.moma.org\/collection\/terms\/vernacular-photography.","journal-title":"https:\/\/www.moma.org\/collection\/terms\/vernacular-photography"},{"doi-asserted-by":"publisher","key":"e_1_3_2_38_2","DOI":"10.1007\/978-3-319-56608-5_57"},{"doi-asserted-by":"publisher","key":"e_1_3_2_39_2","DOI":"10.1007\/978-3-642-33783-3_36"},{"doi-asserted-by":"publisher","key":"e_1_3_2_40_2","DOI":"10.4324\/9780080550947"},{"unstructured":"T. H. Phan and K. Yamamoto. 2020. Resolving Class Imbalance in Object Detection with Weighted Cross Entropy Losses. arxiv:2006.01413 [cs.CV]","key":"e_1_3_2_41_2"},{"doi-asserted-by":"publisher","key":"e_1_3_2_42_2","DOI":"10.1109\/CIEL.2014.7015739"},{"key":"e_1_3_2_43_2","volume-title":"35th Conference on Neural Information Processing Systems","author":"Raghu M.","year":"2021","unstructured":"M. Raghu, T. Unterthiner, S. Kornblith, C. Zhang, and A. Dosovitskiy. 2021. Do vision transformers see like convolutional neural networks? In 35th Conference on Neural Information Processing Systems."},{"doi-asserted-by":"publisher","key":"e_1_3_2_44_2","DOI":"10.1109\/TMM.2016.2629761"},{"doi-asserted-by":"publisher","key":"e_1_3_2_45_2","DOI":"10.1145\/3411170.3411254"},{"doi-asserted-by":"publisher","key":"e_1_3_2_46_2","DOI":"10.1145\/2602695.2602701"},{"doi-asserted-by":"publisher","key":"e_1_3_2_47_2","DOI":"10.1145\/1281500.1281602"},{"doi-asserted-by":"publisher","key":"e_1_3_2_48_2","DOI":"10.1109\/WACV.2016.7477678"},{"doi-asserted-by":"publisher","key":"e_1_3_2_49_2","DOI":"10.3402\/jac.v6.25419"},{"doi-asserted-by":"publisher","key":"e_1_3_2_50_2","DOI":"10.1080\/00131881.2014.898911"},{"doi-asserted-by":"publisher","key":"e_1_3_2_51_2","DOI":"10.1093\/acref\/9780199533008.001.0001"},{"doi-asserted-by":"publisher","key":"e_1_3_2_52_2","DOI":"10.5555\/3306931"},{"doi-asserted-by":"publisher","key":"e_1_3_2_53_2","DOI":"10.1007\/s11263-019-01228-7"},{"doi-asserted-by":"publisher","key":"e_1_3_2_54_2","DOI":"10.1108\/9781787564954"},{"issue":"5","key":"e_1_3_2_55_2","first-page":"200","article-title":"Imago. Laboratorio di ricerca storica e di documentazione iconografica sulla condizione giovanile nel XX secolo","volume":"5","author":"Sorcinelli P.","year":"2004","unstructured":"P. Sorcinelli. 2004. Imago. Laboratorio di ricerca storica e di documentazione iconografica sulla condizione giovanile nel XX secolo. Rivista Di Storia E Storiografia 5, 5 (Nov. 2004), 200\u2013202.","journal-title":"Rivista Di Storia E Storiografia"},{"doi-asserted-by":"crossref","unstructured":"C. Szegedy V. Vanhoucke S. Ioffe J. Shlens and Z. Wojna. 2015. Rethinking the Inception architecture for computer vision. arxiv:1512.00567 [cs.CV]","key":"e_1_3_2_56_2","DOI":"10.1109\/CVPR.2016.308"},{"issue":"1","key":"e_1_3_2_57_2","first-page":"i3\u2013i16","article-title":"Distant viewing: Analyzing large visual corpora","volume":"34","author":"Arnold L. Tilton and T.","year":"2019","unstructured":"L. Tilton and T. Arnold. 2019. Distant viewing: Analyzing large visual corpora. Digital Scholarship in the Humanities 34, Supplement 1 (2019), i3\u2013i16.","journal-title":"Digital Scholarship in the Humanities"},{"key":"e_1_3_2_58_2","article-title":"Yolo Face Implementation","author":"Nguyen T.","year":"2018","unstructured":"T. Nguyen. 2018. Yolo Face Implementation. Retrieved August 3, 2020, from https:\/\/github.com\/sthanhng\/yoloface.","journal-title":"https:\/\/github.com\/sthanhng\/yoloface"},{"key":"e_1_3_2_59_2","first-page":"10347","volume-title":"International Conference on Machine Learning","author":"Touvron H.","year":"2021","unstructured":"H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. J\u00e9gou. 2021. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning. PMLR, 10347\u201310357."},{"doi-asserted-by":"publisher","key":"e_1_3_2_60_2","DOI":"10.1145\/3372278.3390732"},{"key":"e_1_3_2_61_2","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani A.","year":"2017","unstructured":"A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, I. Polosukhin, and \u0141. Kaiser. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998\u20136008."},{"doi-asserted-by":"crossref","unstructured":"X. Wang K. Yu S. Wu J. Gu Y. Liu C. Dong Y. Qiao and C. Change Loy. 2018. ESRGAN: Enhanced super-resolution generative adversarial networks. arxiv:1809.00219 [cs.CV]","key":"e_1_3_2_62_2","DOI":"10.1007\/978-3-030-11021-5_5"},{"issue":"1","key":"e_1_3_2_63_2","first-page":"194","article-title":"The visual digital turn: Using neural networks to study historical images","volume":"35","author":"Wevers M.","year":"2020","unstructured":"M. Wevers and T. Smits. 2020. The visual digital turn: Using neural networks to study historical images. Digital Scholarship in the Humanities 35, 1 (2020), 194\u2013207.","journal-title":"Digital Scholarship in the Humanities"},{"doi-asserted-by":"publisher","key":"e_1_3_2_64_2","DOI":"10.1109\/TMM.2013.2283468"},{"key":"e_1_3_2_65_2","article-title":"Image Restoration Toolbox","author":"Zhang K.","year":"2019","unstructured":"K. Zhang. 2019. Image Restoration Toolbox. https:\/\/github.com\/cszn\/KAIR.","journal-title":"https:\/\/github.com\/cszn\/KAIR"},{"doi-asserted-by":"publisher","key":"e_1_3_2_66_2","DOI":"10.1109\/TIP.2018.2839891"},{"doi-asserted-by":"publisher","key":"e_1_3_2_67_2","DOI":"10.1145\/3279952"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3507918","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3507918","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:10:16Z","timestamp":1750183816000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3507918"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,31]]},"references-count":66,"journal-issue":{"issue":"3s","published-print":{"date-parts":[[2022,10,31]]}},"alternative-id":["10.1145\/3507918"],"URL":"https:\/\/doi.org\/10.1145\/3507918","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2022,10,31]]},"assertion":[{"value":"2021-09-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-21","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}