{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T22:20:36Z","timestamp":1770330036989,"version":"3.49.0"},"reference-count":36,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2023,9,7]],"date-time":"2023-09-07T00:00:00Z","timestamp":1694044800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Research Foundation of Korea","award":["2021R1G1A1007097"],"award-info":[{"award-number":["2021R1G1A1007097"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>As the applications of robots expand across a wide variety of areas, high-level task planning considering human\u2013robot interactions is emerging as a critical issue. Various elements that facilitate flexible responses to humans in an ever-changing environment, such as scene understanding, natural language processing, and task planning, are thus being researched extensively. In this study, a visual question answering (VQA) task was examined in detail from among an array of technologies. By further developing conventional neuro-symbolic approaches, environmental information is stored and utilized in a symmetric graph format, which enables more flexible and complex high-level task planning. We construct a symmetric graph composed of information such as color, size, and position for the objects constituting the environmental scene. VQA, using graphs, largely consists of a part expressing a scene as a graph, a part converting a question into SPARQL, and a part reasoning the answer. The proposed method was verified using a public dataset, CLEVR, with which it successfully performed VQA. We were able to directly confirm the process of inferring answers using SPARQL queries converted from the original queries and environmental symmetric graph information, which is distinct from existing methods that make it difficult to trace the path to finding answers.<\/jats:p>","DOI":"10.3390\/sym15091713","type":"journal-article","created":{"date-parts":[[2023,9,7]],"date-time":"2023-09-07T09:51:37Z","timestamp":1694080297000},"page":"1713","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Symmetric Graph-Based Visual Question Answering Using Neuro-Symbolic Approach"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5208-7836","authenticated-orcid":false,"given":"Jiyoun","family":"Moon","sequence":"first","affiliation":[{"name":"Department of Electronics Engineering, Chosun University, Gwangju 61452, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Gonzalez-Aguirre, J.A., Osorio-Oliveros, R., Rodr\u00edguez-Hern\u00e1ndez, K.L., Liz\u00e1rraga-Iturralde, J., Morales Menendez, R., Ram\u00edrez-Mendoza, R.A., Ram\u00edrez-Moreno, M.A., and Lozoya-Santos, J.D.J. (2021). Service robots: Trends and technology. Appl. Sci., 11.","DOI":"10.3390\/app112210702"},{"key":"ref_2","first-page":"13","article-title":"A review of the applicability of robots in education","volume":"1","author":"Mubin","year":"2013","journal-title":"J. Technol. Educ. Learn."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Holland, J., Kingston, L., McCarthy, C., Armstrong, E., O\u2019Dwyer, P., Merz, F., and McConnell, M. (2021). Service robots in the healthcare sector. Robotics, 10.","DOI":"10.3390\/robotics10010047"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Echelmeyer, W., Kirchheim, A., and Wellbrock, E. (2008, January 1\u20133). Robotics-logistics: Challenges for automation of logistic processes. Proceedings of the 2008 IEEE International Conference on Automation and Logistics, Qingdao, China.","DOI":"10.1109\/ICAL.2008.4636510"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3564696","article-title":"Multiple mobile robot task and motion planning: A survey","volume":"55","author":"Antonyshyn","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1080\/0951192X.2015.1130251","article-title":"Human\u2013robot interaction review and challenges on task planning and programming","volume":"29","author":"Tsarouchi","year":"2016","journal-title":"Int. J. Comput. Integr. Manuf."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Cashmore, M., Fox, M., Long, D., Magazzeni, D., Ridder, B., Carrera, A., Palomeras, N., Hurtos, N., and Carreras, M. (2015, January 7\u201311). Rosplan: Planning in the robot operating system. Proceedings of the International Conference on Automated Planning and Scheduling, Jerusalem, Israel.","DOI":"10.1609\/icaps.v25i1.13699"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Crosby, M., Petrick, R., Rovida, F., and Krueger, V. (2017, January 18\u201323). Integrating mission and task planning in an industrial robotics framework. Proceedings of the International Conference on Automated Planning and Scheduling, Pittsburgh, PA, USA.","DOI":"10.1609\/icaps.v27i1.13857"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"955","DOI":"10.1016\/j.robot.2008.08.007","article-title":"Robot task planning using semantic maps","volume":"56","author":"Galindo","year":"2008","journal-title":"Robot. Auton. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1869397.1869404","article-title":"Human-aware task planning: An application to mobile robots","volume":"1","author":"Cirillo","year":"2010","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Alami, R., Clodic, A., Montreuil, V., Sisbot, E.A., and Chatila, R. (2005, January 12\u201314). Task planning for human-robot interaction. Proceedings of the 2005 Joint Conference on Smart Objects and Ambient Intelligence: Innovative Context-Aware Services: Usages and Technologies, Grenoble, France.","DOI":"10.1145\/1107548.1107574"},{"key":"ref_12","unstructured":"Srivastava, Y., Murali, V., Dubey, S.R., and Mukherjee, S. (2020, January 4\u20136). Visual question answering using deep learning: A survey and performance analysis. Proceedings of the Computer Vision and Image Processing: 5th International Conference (CVIP 2020), Prayagraj, India."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yang, Z., He, X., Gao, J., Deng, L., and Smola, A. (2016, January 27\u201330). Stacked attention networks for image question answering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.10"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Marino, K., Chen, X., Parikh, D., Gupta, A., and Rohrbach, M. (2021, January 20\u201325). Krisp: Integrating implicit and symbolic knowledge for open-domain knowledge-based vqa. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01389"},{"key":"ref_15","unstructured":"Mao, J., Gan, C., Kohli, P., Tenenbaum, J.B., and Wu, J. (2019). The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. arXiv."},{"key":"ref_16","unstructured":"Yi, K., Wu, J., Gan, C., Torralba, A., Kohli, P., and Tenenbaum, J. (2018). Neural-symbolic vqa: Disentangling reasoning from vision and language understanding. Adv. Neural Inf. Process. Syst., 31."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Johnson, J., Hariharan, B., Van Der Maaten, L., Fei-Fei, L., Lawrence Zitnick, C., and Girshick, R. (2017, January 21\u201326). Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.215"},{"key":"ref_18","unstructured":"Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015, January 6\u201311). Show, attend and tell: Neural image caption generation with visual attention. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Fukui, A., Park, D.H., Yang, D., Rohrbach, A., Darrell, T., and Rohrbach, M. (2016). Multimodal compact bilinear pooling for visual question answering and visual grounding. arXiv.","DOI":"10.18653\/v1\/D16-1044"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wu, Q., Wang, P., Shen, C., Dick, A., and Van Den Hengel, A. (2016, January 27\u201330). Ask me anything: Free-form visual question answering based on knowledge from external sources. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.500"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, M., Xu, R., Wang, S., Zhou, L., Lin, X., Zhu, C., Zeng, M., Ji, H., and Chang, S.F. (2022, January 18\u201324). Clip-event: Connecting text and images with event structures. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01593"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yang, X., Gao, C., Zhang, H., and Cai, J. (2021, January 11\u201317). Auto-parsing network for image captioning and visual question answering. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00220"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Nam, H., Ha, J.W., and Kim, J. (2017, January 21\u201326). Dual attention networks for multimodal reasoning and matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.232"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Noh, H., Seo, P.H., and Han, B. (2016, January 27\u201330). Image question answering using convolutional neural network with dynamic parameter prediction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.11"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Lei, S.W., Gao, D., Wu, J.Z., Wang, Y., Liu, W., Zhang, M., and Shou, M.Z. (2022). Symbolic replay: Scene graph as prompt for continual learning on vqa task. arXiv.","DOI":"10.1609\/aaai.v37i1.25208"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"104425","DOI":"10.1016\/j.engappai.2021.104425","article-title":"Graph matching based reasoner: A symbolic approach to question answering","volume":"105","author":"Han","year":"2021","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_27","first-page":"1682","article-title":"A multi-world approach to question answering about real-world scenes based on uncertain input","volume":"27","author":"Malinowski","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_28","unstructured":"Amizadeh, S., Palangi, H., Polozov, A., Huang, Y., and Koishida, K. (2020, January 3\u201318). Neuro-symbolic visual reasoning: Disentangling. Proceedings of the International Conference on Machine Learning, Virtual Event."},{"key":"ref_29","unstructured":"Vedantam, R., Desai, K., Lee, S., Rohrbach, M., Batra, D., and Parikh, D. (2019, January 9\u201315). Probabilistic neural symbolic models for interpretable visual question answering. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Bosselut, A., Le Bras, R., and Choi, Y. (2021, January 2\u20139). Dynamic neuro-symbolic knowledge graph construction for zero-shot commonsense question answering. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual Event. No. 6.","DOI":"10.1609\/aaai.v35i6.16625"},{"key":"ref_31","unstructured":"Banu, A. (2017). Semantic Web Technologies, CRC Press."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"789","DOI":"10.4236\/jsea.2012.510091","article-title":"A Facilitated Interface to Generate a Combined Textual and Graphical Database System Using Widely Available Software","volume":"5","author":"Lawson","year":"2012","journal-title":"J. Softw. Eng. Appl."},{"key":"ref_33","first-page":"49","article-title":"Knowledge graph-based knowledge map for efficient expression and inference of associated knowledge","volume":"27","author":"Yoo","year":"2021","journal-title":"J. Intell. Inf. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"149787","DOI":"10.1109\/ACCESS.2020.3016676","article-title":"Diagnosis method of thyroid disease combining knowledge graph and deep learning","volume":"8","author":"Chai","year":"2020","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Tsai, H., Riesa, J., Johnson, M., Arivazhagan, N., Li, X., and Archer, A. (2019). Small and practical BERT models for sequence labeling. arXiv.","DOI":"10.18653\/v1\/D19-1374"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Szymanski, B.K. (1988, January 4\u20138). A simple solution to Lamport\u2019s concurrent programming problem with linear wait. Proceedings of the 2nd International Conference on Supercomputing, Saint Malo, France.","DOI":"10.1145\/55364.55425"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/15\/9\/1713\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:46:34Z","timestamp":1760129194000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/15\/9\/1713"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,7]]},"references-count":36,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["sym15091713"],"URL":"https:\/\/doi.org\/10.3390\/sym15091713","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,7]]}}}