{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T15:42:59Z","timestamp":1772725379350,"version":"3.50.1"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,9,16]],"date-time":"2024-09-16T00:00:00Z","timestamp":1726444800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NCAER National Data Innovation Center"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM J. Comput. Sustain. Soc."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>Precise and comprehensive measurements of socioeconomic status are crucial for both academic investigations and policy making. However, in developing countries, such measures are available at the local household level only at extremely low frequencies (e.g., via a decadal census). A number of works have attempted predicting economic status at aggregated geographical levels such as district or neighborhood using deep learning on images with varying degrees of success. However the utility of such an approach at the household level remains open. In this study, we utilize deep learning models on household images collected from four northeastern states in India to assess the feasibility and ethics of household-level income status prediction. We categorize households into classes based on income and then train a Swin Transformer model with cross-entropy loss and triplet loss to predict the socioeconomic class of the household. We then compare the prediction accuracy of our model with predictions using a simple list of household assets and predictions from a set of expert human annotators. We find that the use of deep learning on images does not lead to any substantial gains in prediction accuracy. Further, we note that human accuracy on this prediction tasks is low, raising questions on the information contained within the images. Our study raises important questions regarding the ethical implications of utilizing household images for predicting socioeconomic status. We explore these ethical implications, emphasizing the importance of a cautious and considerate approach in incorporating image-based techniques.<\/jats:p>","DOI":"10.1145\/3675160","type":"journal-article","created":{"date-parts":[[2024,6,28]],"date-time":"2024-06-28T11:05:39Z","timestamp":1719572739000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Assessing the Feasibility and Ethics of Economic Status Prediction using Deep Learning on Household Images"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-0597-286X","authenticated-orcid":false,"given":"Aatif Nisar","family":"Dar","sequence":"first","affiliation":[{"name":"Indian Institute of Technology Delhi, New Delhi, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3874-7308","authenticated-orcid":false,"given":"Nandana","family":"Sengupta","sequence":"additional","affiliation":[{"name":"Indian Institute of Technology Delhi, New Delhi, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0155-0250","authenticated-orcid":false,"given":"Chetan","family":"Arora","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, IIT Delhi, New Delhi, India"}]}],"member":"320","published-online":{"date-parts":[[2024,9,16]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISIVC.2018.8709173"},{"key":"e_1_3_3_3_2","unstructured":"Ambika Acharya Helen Fang and Shubha Raghvendra. 2017. Neighborhood Watch: Using CNNs to Predict Income Brackets from Google Street View Images. Retrieved July 2 2024 from https:\/\/cs231n.stanford.edu\/reports\/2017\/pdfs\/556.pdf"},{"key":"e_1_3_3_4_2","unstructured":"A. Atkinson. 2016. Monitoring Global Poverty: Report of the Commission on Global Poverty. World Bank Group Washington DC."},{"key":"e_1_3_3_5_2","article-title":"Controlling machine-learning algorithms and their biases","author":"Baer Tobias","year":"2017","unstructured":"Tobias Baer and Vishnu Kamalnath. 2017. Controlling machine-learning algorithms and their biases. McKinsey Insights, November 10, 2017.","journal-title":"McKinsey Insights"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3371158.3371167"},{"key":"e_1_3_3_7_2","doi-asserted-by":"crossref","unstructured":"Mark P. Bauman and Kenneth W. Shaw. 2002. Interim income tax data and earnings prediction. SSRN. Available at https:\/\/ssrn.com\/abstract=305704","DOI":"10.2139\/ssrn.305704"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aac4420"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.3390\/jrfm15040165"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.abe8628"},{"key":"e_1_3_3_11_2","unstructured":"C. Chandramouli and Registrar General. 2011. Census of India 2011. In Provisional Population Totals. Government of India New Delhi 409\u2013413."},{"key":"e_1_3_3_12_2","article-title":"Predicting poverty level from satellite imagery using deep neural networks","author":"Chitturi Varun","year":"2021","unstructured":"Varun Chitturi and Zaid Nabulsi. 2021. Predicting poverty level from satellite imagery using deep neural networks. arXiv preprint arXiv:2112.00011 (2021).","journal-title":"arXiv preprint arXiv:2112.00011"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"issue":"1","key":"e_1_3_3_14_2","first-page":"283","article-title":"Protecting privacy in the digital age","volume":"18","author":"DeVries Will Thomas","year":"2003","unstructured":"Will Thomas DeVries. 2003. Protecting privacy in the digital age. Berkeley Technology Law Journal 18, 1 (2003), 283\u2013311.","journal-title":"Berkeley Technology Law Journal"},{"key":"e_1_3_3_15_2","article-title":"An image is worth 16x16 words: Transformers for image recognition at scale","author":"Dosovitskiy Alexey","year":"2020","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).","journal-title":"arXiv preprint arXiv:2010.11929"},{"key":"e_1_3_3_16_2","volume-title":"Cyber Privacy: Who Has Your Data and Why You Should Care","author":"Doss April Falcon","year":"2020","unstructured":"April Falcon Doss. 2020. Cyber Privacy: Who Has Your Data and Why You Should Care. BenBella Books."},{"key":"e_1_3_3_17_2","article-title":"AGI safety literature review","author":"Everitt Tom","year":"2018","unstructured":"Tom Everitt, Gary Lea, and Marcus Hutter. 2018. AGI safety literature review. arXiv preprint arXiv:1805.01109 (2018).","journal-title":"arXiv preprint arXiv:1805.01109"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v14i1.7361"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.3390\/su5124988"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1257\/aer.102.2.994"},{"key":"e_1_3_3_21_2","unstructured":"Aparajita Bakshi. 2008. A Note on Household Income Surveys in India. Indian Statistical Institute."},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaf7894"},{"key":"e_1_3_3_23_2","volume-title":"Nighttime Light Predictions from Satellite Imagery","author":"Jean Neal","year":"2016","unstructured":"Neal Jean, Rachel Luo, and Jae Hyun Kim. 2016. Nighttime Light Predictions from Satellite Imagery. Stanford University."},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1093\/intqhc\/mzg031"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1093\/intqhc\/mzg031"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3065386"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.5334\/dsj-2022-006"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.120644"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1080\/0952813X.2021.1964003"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1177\/2515245917747656"},{"key":"e_1_3_3_32_2","first-page":"771","article-title":"Privacy, mass intrusion, and the modern data breach","volume":"69","author":"Mills Jon L.","year":"2017","unstructured":"Jon L. Mills and Kelsey Harclerode. 2017. Privacy, mass intrusion, and the modern data breach. Florida Law Review 69 (2017), 771.","journal-title":"Florida Law Review"},{"key":"e_1_3_3_33_2","first-page":"779","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops","author":"Naik Nikhil","year":"2014","unstructured":"Nikhil Naik, Jade Philipoom, Ramesh Raskar, and C\u00e9sar Hidalgo. 2014. Streetscore\u2014Predicting the perceived safety of one million streetscapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 779\u2013785."},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1257\/aer.p20161030"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781315172231"},{"key":"e_1_3_3_36_2","unstructured":"Luisa Natali and Chris De Neubourg. 2014. A Tale of Two Short Consumption Modules. Innocenti."},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/LGRS.2020.3006019"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-3876-2_3"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13063-021-05493-1"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.worlddev.2012.11.015"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/IGARSS39084.2020.9324399"},{"key":"e_1_3_3_42_2","first-page":"71e94","article-title":"Survey design and sample design in household budget surveys","author":"Pettersson H.","year":"2005","unstructured":"H. Pettersson, O. Ajayi, G. Kalton, K. Krotki, J. Lepkowski, H. Pettersson, and A. Turner. 2005. Survey design and sample design in household budget surveys. In Household Sample Surveys in Developing and Transition Countries: Design, Implementation and Analysis. Department of Economic and Social Affairs, United Nations, 71e94.","journal-title":"Household Sample Surveys in Developing and Transition Countries: Design, Implementation and Analysis."},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1093\/poq\/nfz017"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3129396"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_3_3_46_2","unstructured":"Vaishaal Shankar Rebecca Roelofs Horia Mania Alex Fang Benjamin Recht and Ludwig Schmidt. 2020. Evaluating machine accuracy on ImageNet. In Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research Hal Daume III and Aarti Singh (Eds.) Vol. 119. PMLR 8634\u20138644. https:\/\/proceedings.mlr.press\/v119\/shankar20c.html"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigMM.2019.00-25"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1086\/269421"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.rse.2021.112339"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-42036-w"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.2991\/icaita-16.2016.24"},{"key":"e_1_3_3_52_2","first-page":"10347","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Touvron Hugo","year":"2021","unstructured":"Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herv\u00e9 J\u00e9gou. 2021. Training data-efficient image transformers & distillation through attention. In Proceedings of the International Conference on Machine Learning. 10347\u201310357."},{"key":"e_1_3_3_53_2","article-title":"Attention is all you need","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017), 1\u201311.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2887138"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3440207"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.309"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICMLC48188.2019.8949252"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2017.8258131"}],"container-title":["ACM Journal on Computing and Sustainable Societies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3675160","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3675160","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:05:36Z","timestamp":1750291536000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3675160"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,16]]},"references-count":57,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3675160"],"URL":"https:\/\/doi.org\/10.1145\/3675160","relation":{},"ISSN":["2834-5533"],"issn-type":[{"value":"2834-5533","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,16]]},"assertion":[{"value":"2024-02-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-26","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}