{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T06:07:27Z","timestamp":1774418847784,"version":"3.50.1"},"reference-count":32,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2020,5,5]],"date-time":"2020-05-05T00:00:00Z","timestamp":1588636800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJGI"],"abstract":"<jats:p>Documenting Cultural Heritage through the extraction of 3D measures with photogrammetry is fundamental for the conservation of the memory of the past. However, when the heritage has been lost the only way to recover this information is the use of historical images from archives. The aim of this study is to experiment with new ways to search for architectural heritage in video material and to save the effort of the operator in the archive in terms of efficiency and time. A workflow is proposed to automatically detect lost heritage in film footage using Deep Learning to find suitable images to process with photogrammetry for its 3D virtual reconstruction. The performance of the network was tested on two case studies considering different architectural scenarios, the Tour Saint Jacques which still exists for the tuning of the networks, and Les Halles to test the algorithms on a real case of an architecture which has been destroyed. Despite the poor quantity and low quality of the historical images available for the training of the network, it has been demonstrated that, with few frames, it was possible to reach the same results in terms of performance of a network trained on a large dataset. Moreover, with the introduction of new metrics based on time intervals the measure of the real time saving in terms of human effort was achieved. These findings represent an important innovation in the documentation of destroyed monuments and open new ways to recover information about the past.<\/jats:p>","DOI":"10.3390\/ijgi9050297","type":"journal-article","created":{"date-parts":[[2020,5,7]],"date-time":"2020-05-07T03:10:38Z","timestamp":1588821038000},"page":"297","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["A Neural Networks Approach to Detecting Lost Heritage in Historical Video"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3786-4868","authenticated-orcid":false,"given":"Francesca","family":"Condorelli","sequence":"first","affiliation":[{"name":"DAD, Department of Architecture and Design, Politecnico di Torino, 10125 Turin, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9592-1341","authenticated-orcid":false,"given":"Fulvio","family":"Rinaudo","sequence":"additional","affiliation":[{"name":"DAD, Department of Architecture and Design, Politecnico di Torino, 10125 Turin, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Francesco","family":"Salvadore","sequence":"additional","affiliation":[{"name":"HPC Department, CINECA, 00185 Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stefano","family":"Tagliaventi","sequence":"additional","affiliation":[{"name":"HPC Department, CINECA, 00185 Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,5,5]]},"reference":[{"key":"ref_1","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"1","author":"Krizhevsky","year":"2012","journal-title":"NIPS"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, X., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015). Going deeper with convolutions. arXiv.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_3","unstructured":"Simonyan, K., and Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Radenovi\u0107, F., Tolias, G., and Chum, O. (2016). CNN image retrieval learns from BoW: Unsupervised fine-tuning with hard examples. Eur. Conf. Comput. Vis. ECCV, 1\u201317.","DOI":"10.1007\/978-3-319-46448-0_1"},{"key":"ref_5","unstructured":"Tolias, G., Sicre, R., and J\u00e9gou, H. (2016). Particular object retrieval with integral max-pooling of CNN activations. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Llamas, J., Lerones, P.M., Medina, R., Zalama, E., and G\u00f3mez-Garc\u00eda-Bermejo, J. (2017). Classification of architectural heritage images using deep learning techniques. Appl. Sci., 7.","DOI":"10.3390\/app7100992"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Saini, A., Gupta, T., Kumar, G., Kumar Gupta, A., Panwar, M., and Mittal, A. (2017). Image based indian monument recognition using convoluted neural networks. Int. Conf. Big Data IOT Data Sci., 1\u20135.","DOI":"10.1109\/BID.2017.8336587"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Belhi, A., Bouras, A., and Foufou, S. (2018). Leveraging Known data for missing label prediction in cultural heritage context. Appl. Sci., 8.","DOI":"10.3390\/app8101768"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Montoya Obeso, A., Benois-Pineau, J., Sara\u00ed Garc\u00eda V\u00e1zquez, M., and Ram\u00edrez Acosta, A.A. (2019, January 21). organizing cultural heritage with deep features. Proceedings of the 1st Workshop on Structuring and Understanding of Multimedia heritAge Contents (SUMAC \u203219), Nice, France.","DOI":"10.1145\/3347317.3357244"},{"key":"ref_10","unstructured":"Shi, X., Khademi, S., and van Gemert, J. (2019). Deep visual city recognition visualization. arXiv."},{"key":"ref_11","first-page":"95","article-title":"Challenges in content-based image indexing of cultural heritage collections","volume":"32","author":"Picard","year":"2015","journal-title":"IEEE Signal Process. Mag. Inst. Electr. Electron. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yasser, A.M., Clawson, K., and Bowerman, C. (2017, January 11\u201313). Saving cultural heritage with digital make-believe: Machine learning and digital techniques to the rescue. Proceedings of the Electronic Visualisation and the Arts, London, UK.","DOI":"10.14236\/ewic\/HCI2017.97"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Gominski, D., Poreba, M., Gouet-Brunet, V., and Chen, L. (2019, January 21). challenging deep image descriptors for retrieval in heterogeneous iconographic collections. Proceedings of the 1st Workshop on Structuring and Understanding of Multimedia heritAge Contents (SUMAC \u203219), Nice, France.","DOI":"10.1145\/3347317.3357246"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Bhargav, S., van Noord, N., and Kamps, J. (2019, January 21). Deep learning as a tool for early cinema analysis. Proceedings of the 1st Workshop on Structuring and Understanding of Multimedia heritAge Contents (SUMAC \u203219), Nice, France.","DOI":"10.1145\/3347317.3357240"},{"key":"ref_15","unstructured":"Caraceni, S., Carpen\u00e8, M., D\u2019Antonio, M., Fiameni, G., Guidazzoli, A., Imboden, S., Liguori, M.C., Montanari, M., Trotta, G., and Scipione, G. (November, January 31). I-media-cities, a searchable platform on moving images with automatic and manual annotations. Proceedings of the 23rd International Conference on Virtual System & Multimedia, Dublin, Ireland."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"343","DOI":"10.5194\/isprs-archives-XLII-2-W15-343-2019","article-title":"Architectural heritage recognition in historical film footage using neural ntworks","volume":"XLII-2\/W15","author":"Condorelli","year":"2019","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_17","first-page":"4104","article-title":"Structure-from-motion revisited","volume":"2016","author":"Frahm","year":"2016","journal-title":"IEEE Conf. Comput. Vis. Pattern Recognit."},{"key":"ref_18","unstructured":"(2020, March 09). COLMAP -Structure-From-Motion and Multi-View Stereo. Available online: https:\/\/github.com\/colmap\/colmap."},{"key":"ref_19","unstructured":"(2020, March 09). Luminoth Open Source Computer Vision Toolkit. Available online: https:\/\/github.com\/tryolabs\/luminoth."},{"key":"ref_20","unstructured":"(2020, March 09). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Available online: http:\/\/tensorflow.org."},{"key":"ref_21","first-page":"1","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_23","first-page":"21","article-title":"SSD: Single shot multibox detector","volume":"9905","author":"Liu","year":"2016","journal-title":"Comput. Vis. Eccv Lect. Notes Comput. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dutta, A., and Zisserman, A. (2019, January 21\u201325). The via annotation software for images, audio and video. Proceedings of the 27th ACM International Conference on Multimedia (MM \u203219), Nice, France.","DOI":"10.1145\/3343031.3350535"},{"key":"ref_25","unstructured":"Ruder, S. (2017). An overview of gradient descent optimization algorithms. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"443","DOI":"10.5194\/isprs-archives-XLII-2-W11-443-2019","article-title":"Benchmark of metric quality assessment in photogrammetric reconstruction for historical film footage","volume":"XLII-2\/W11","author":"Condorelli","year":"2019","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_27","unstructured":"Meurgey, J. (1926). Histoire de la paroisse Saint-Jacques de-la-Boucherie, Biblioth\u00e8que de l\u2019\u00c9cole des chartes."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"450","DOI":"10.2307\/991730","article-title":"Afterlives of the tour saint-jacques: Plotting the perceptual history of an urban fragment","volume":"60","year":"2001","journal-title":"J. Soc. Archit. Hist."},{"key":"ref_29","unstructured":"Lemoine, B. (1980). Les Halles de Paris: L\u2019Histoire d\u2019un Lieu, les P\u00e9rip\u00e9ties d\u2019une Reconstruction, la Succession Des Projets, l\u2019Architecture d\u2019un Monument, l\u2019enjeu d\u2019Une Cit\u00e9, Equerre, Les Laboratoires de L\u2019imaginaire."},{"key":"ref_30","unstructured":"Tamborrino, R. (2012). Il \u201ccaso Francia\u201d: Spazio Pubblico e Trasformazioni Urbane Sullo Scorcio Degli Anni Ottanta. La Citt\u00e0 che si Rinnova, Franco Angeli."},{"key":"ref_31","first-page":"1","article-title":"Convolutional neural network architectures for matching natural language sentences","volume":"27","author":"Hu","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst. Nips"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kalal, Z., Matas, J., and Mikolajczyk, K. (2010, January 13\u201318). P-n learning: Bootstrapping binary classifiers by structural constraints. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5540231"}],"container-title":["ISPRS International Journal of Geo-Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2220-9964\/9\/5\/297\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:32:42Z","timestamp":1760362362000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2220-9964\/9\/5\/297"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,5]]},"references-count":32,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2020,5]]}},"alternative-id":["ijgi9050297"],"URL":"https:\/\/doi.org\/10.3390\/ijgi9050297","relation":{},"ISSN":["2220-9964"],"issn-type":[{"value":"2220-9964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5,5]]}}}