{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T10:44:53Z","timestamp":1756896293399,"version":"3.37.3"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2024,1,27]],"date-time":"2024-01-27T00:00:00Z","timestamp":1706313600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2024,1,27]],"date-time":"2024-01-27T00:00:00Z","timestamp":1706313600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/501100001348","name":"Agency for Science, Technology and Research","doi-asserted-by":"publisher","award":["AME Programmatic Funding Scheme (Project# A18A2b0046)."],"award-info":[{"award-number":["AME Programmatic Funding Scheme (Project# A18A2b0046)."]}],"id":[{"id":"10.13039\/501100001348","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Vis Comput"],"published-print":{"date-parts":[[2024,11]]},"DOI":"10.1007\/s00371-023-03232-y","type":"journal-article","created":{"date-parts":[[2024,1,27]],"date-time":"2024-01-27T20:02:11Z","timestamp":1706385731000},"page":"8189-8203","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Manufacturing domain instruction comprehension using synthetic data"],"prefix":"10.1007","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6647-0246","authenticated-orcid":false,"given":"Kritika","family":"Johari","sequence":"first","affiliation":[]},{"given":"Christopher Tay Zi","family":"Tong","sequence":"additional","affiliation":[]},{"given":"Rishabh","family":"Bhardwaj","sequence":"additional","affiliation":[]},{"given":"Vigneshwaran","family":"Subbaraju","sequence":"additional","affiliation":[]},{"given":"Jung-Jae","family":"Kim","sequence":"additional","affiliation":[]},{"given":"U.-Xuan","family":"Tan","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,1,27]]},"reference":[{"key":"3232_CR1","first-page":"171","volume":"14","author":"KL Du","year":"1999","unstructured":"Du, K.L., Huang, X., Wang, M., Hu, J.: Assembly robotics research: A survey. Int. J. Robot. Autom. 14, 171\u2013183 (1999)","journal-title":"Int. J. Robot. Autom."},{"key":"3232_CR2","doi-asserted-by":"crossref","unstructured":"Hatori, J., Kikuchi, Y., Kobayashi, S., Takahashi, K., Tsuboi, Y., Unno, Y., Ko, W., Tan, J.: Interactively picking real-world objects with unconstrained spoken language instructions. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 3774\u20133781 IEEE (2018)","DOI":"10.1109\/ICRA.2018.8460699"},{"key":"3232_CR3","doi-asserted-by":"publisher","first-page":"10826","DOI":"10.1109\/LRA.2022.3195198","volume":"7","author":"P Pramanick","year":"2022","unstructured":"Pramanick, P., Sarkar, C., Paul, S., dev Roychoudhury, R., Bhowmick, B.: Doro: Disambiguation of referred object for embodied agents. IEEE Robot. Autom. Lett. 7, 10826\u201310833 (2022)","journal-title":"IEEE Robot. Autom. Lett."},{"key":"3232_CR4","doi-asserted-by":"crossref","unstructured":"Thomason, J., Padmakumar, A., Sinapov, J., Walker, N., Jiang, Y., Yedidsion, H., Hart, J., Stone, P., Mooney, R.J.: Improving grounded natural language understanding through human-robot dialog. In: 2019 International Conference on Robotics and Automation (ICRA), IEEE, pp. 6934\u20136941 (2019)","DOI":"10.1109\/ICRA.2019.8794287"},{"key":"3232_CR5","doi-asserted-by":"publisher","first-page":"4426","DOI":"10.1109\/TMM.2020.3042066","volume":"23","author":"Y Qiao","year":"2020","unstructured":"Qiao, Y., Deng, C., Wu, Q.: Referring expression comprehension: A survey of methods and datasets. IEEE Trans. Multimed. 23, 4426\u20134440 (2020)","journal-title":"IEEE Trans. Multimed."},{"key":"3232_CR6","doi-asserted-by":"crossref","unstructured":"Kazemzadeh, S., Ordonez, V., Matten, M., Berg, T.: Referitgame: Referring to objects in photographs of natural scenes. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 787\u2013798 (2014)","DOI":"10.3115\/v1\/D14-1086"},{"key":"3232_CR7","doi-asserted-by":"crossref","unstructured":"Yu, L., Poirson, P., Yang, S., Berg, A.C., Berg, T.L.: Modeling context in referring expressions. In: European Conference on Computer Vision, Springer, pp. 69\u201385 (2016)","DOI":"10.1007\/978-3-319-46475-6_5"},{"key":"3232_CR8","doi-asserted-by":"crossref","unstructured":"Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A.L., Murphy, K.: Generation and comprehension of unambiguous object descriptions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 11\u201320 (2016)","DOI":"10.1109\/CVPR.2016.9"},{"key":"3232_CR9","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European Conference on Computer Vision, Springer pp. 740\u2013755 (2014)","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"3232_CR10","unstructured":"Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497 (2015)"},{"key":"3232_CR11","doi-asserted-by":"crossref","unstructured":"Deng, C., Wu, Q., Wu, Q., Hu, F., Lyu, F., Tan, M.: Visual grounding via accumulated attention. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7746\u20137755 (2018)","DOI":"10.1109\/CVPR.2018.00808"},{"key":"3232_CR12","doi-asserted-by":"crossref","unstructured":"Yu, L., Lin, Z., Shen, X., Yang, J., Lu, X., Bansal, M., Berg, T.L.: Mattnet: Modular attention network for referring expression comprehension. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1307\u20131315 (2018)","DOI":"10.1109\/CVPR.2018.00142"},{"key":"3232_CR13","doi-asserted-by":"crossref","unstructured":"Sadhu, A., Chen, K., Nevatia, R.: Zero-shot grounding of objects from natural language queries. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 4694\u20134703 (2019)","DOI":"10.1109\/ICCV.2019.00479"},{"key":"3232_CR14","unstructured":"Zhou, Y., Ji, R., Luo, G., Sun, X., Su, J., Ding, X., Lin, C.w., Tian, Q.: A real-time global inference network for one-stage referring expression comprehension. arXiv preprint arXiv:1912.03478 (2019)"},{"key":"3232_CR15","unstructured":"Chen, X., Ma, L., Chen, J., Jie, Z., Liu, W., Luo, J.: Real-time referring expression comprehension by single-stage grounding network. arXiv preprint arXiv:1812.03426 (2018)"},{"key":"3232_CR16","doi-asserted-by":"crossref","unstructured":"Liao, Y., Liu, S., Li, G., Wang, F., Chen, Y., Qian, C., Li, B.: A real-time cross-modality correlation filtering method for referring expression comprehension. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. pp. 10880\u201310889 (2020)","DOI":"10.1109\/CVPR42600.2020.01089"},{"key":"3232_CR17","doi-asserted-by":"publisher","first-page":"558","DOI":"10.1177\/0278364918760992","volume":"37","author":"R Scalise","year":"2018","unstructured":"Scalise, R., Li, S., Admoni, H., Rosenthal, S., Srinivasa, S.S.: Natural language instructions for human\u2013robot collaborative manipulation. Int. J. Robot. Res. 37, 558\u2013565 (2018)","journal-title":"Int. J. Robot. Res."},{"key":"3232_CR18","doi-asserted-by":"crossref","unstructured":"Shridhar, M., Hsu, D.: Interactive visual grounding of referring expressions for human\u2013robot interaction. arXiv preprint arXiv:1806.03831 (2018)","DOI":"10.15607\/RSS.2018.XIV.028"},{"key":"3232_CR19","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1177\/0278364919897133","volume":"39","author":"M Shridhar","year":"2020","unstructured":"Shridhar, M., Mittal, D., Hsu, D.: Ingress: Interactive visual grounding of referring expressions. Int. J. Robot. Res. 39, 217\u2013232 (2020)","journal-title":"Int. J. Robot. Res."},{"key":"3232_CR20","unstructured":"Borkman, S., Crespi, A., Dhakad, S., Ganguly, S., Hogins, J., Jhang, Y.C., Kamalzadeh, M., Li, B., Leal, S., Parisi, P., et\u00a0al.: Unity perception: Generate synthetic data for computer vision. arXiv preprint arXiv:2107.04259 (2021)"},{"key":"3232_CR21","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1613\/jair.1327","volume":"21","author":"P Gorniak","year":"2004","unstructured":"Gorniak, P., Roy, D.: Grounded semantic composition for visual scenes. J. Artif. Intell. Res. 21, 429\u2013470 (2004)","journal-title":"J. Artif. Intell. Res."},{"key":"3232_CR22","doi-asserted-by":"crossref","unstructured":"Liu, R., Liu, C., Bai, Y., Yuille, A.L.: Clevr-ref+: Diagnosing visual reasoning with referring expressions. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. (2019) 4185\u20134194","DOI":"10.1109\/CVPR.2019.00431"},{"key":"3232_CR23","unstructured":"Kazakos, I., Ventura, C., Bellver, M., Silberer, C., Gir\u00f3-i Nieto, X.: Synthref: Generation of synthetic referring expressions for object segmentation. arXiv preprint arXiv:2106.04403 (2021)"},{"key":"3232_CR24","doi-asserted-by":"crossref","unstructured":"Johnson, J., Hariharan, B., Van Der\u00a0Maaten, L., Fei-Fei, L., Lawrence\u00a0Zitnick, C., Girshick, R.: Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2901\u20132910 (2017)","DOI":"10.1109\/CVPR.2017.215"},{"key":"3232_CR25","doi-asserted-by":"crossref","unstructured":"Yang, L., Fan, Y., Xu, N.: Video instance segmentation. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision. pp. 5188\u20135197 (2019)","DOI":"10.1109\/ICCV.2019.00529"},{"key":"3232_CR26","first-page":"1","volume":"8","author":"X Kong","year":"2023","unstructured":"Kong, X., Xia, S., Liu, N., Wei, M.: Gada-segnet: Gated attentive domain adaptation network for semantic segmentation of lidar point clouds. Vis. Comput. 8, 1\u201311 (2023)","journal-title":"Vis. Comput."},{"key":"3232_CR27","unstructured":"Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)"},{"key":"3232_CR28","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1016\/j.cviu.2009.03.008","volume":"114","author":"HJ Escalante","year":"2010","unstructured":"Escalante, H.J., Hern\u00e1ndez, C.A., Gonzalez, J.A., L\u00f3pez-L\u00f3pez, A., Montes, M., Morales, E.F., Sucar, L.E., Villasenor, L., Grubinger, M.: The segmented and annotated iapr tc-12 benchmark. Comput. Vis. Image Underst. 114, 419\u2013428 (2010)","journal-title":"Comput. Vis. Image Underst."},{"key":"3232_CR29","doi-asserted-by":"crossref","unstructured":"Tang, P., Guo, Y., Zheng, G., Zheng, L., Pu, J., Wang, J., Chen, Z.: Two-stage filtering method to improve the performance of object detection trained by synthetic dataset in heavily cluttered industry scenes. Vis. Comput. (2023) 1\u201320","DOI":"10.1007\/s00371-023-02899-7"},{"key":"3232_CR30","doi-asserted-by":"crossref","unstructured":"Wood, E., Baltrusaitis, T., Zhang, X., Sugano, Y., Robinson, P., Bulling, A.: Rendering of eyes for eye-shape registration and gaze estimation. In: Proceedings of the IEEE International Conference on Computer Vision. (2015) 3756\u20133764","DOI":"10.1109\/ICCV.2015.428"},{"key":"3232_CR31","doi-asserted-by":"crossref","unstructured":"Wood, E., Baltru\u0161aitis, T., Hewitt, C., Dziadzio, S., Cashman, T.J., Shotton, J.: Fake it till you make it: face analysis in the wild using synthetic data alone. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision. pp. 3681\u20133691 (2021)","DOI":"10.1109\/ICCV48922.2021.00366"},{"key":"3232_CR32","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., Doll\u00e1r, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2980\u20132988 (2017)","DOI":"10.1109\/ICCV.2017.324"},{"key":"3232_CR33","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770\u2013778 (2016)","DOI":"10.1109\/CVPR.2016.90"},{"key":"3232_CR34","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE 2009, 248\u2013255 (2009)","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"3232_CR35","doi-asserted-by":"crossref","unstructured":"Bhardwaj, R., Saha, A., Hoi, S.C.: Vector-quantized input-contextualized soft prompts for natural language understanding. arXiv preprint arXiv:2205.11024 (2022)","DOI":"10.18653\/v1\/2022.emnlp-main.455"},{"key":"3232_CR36","doi-asserted-by":"crossref","unstructured":"Nagaraja, V.K., Morariu, V.I., Davis, L.S.: Modeling context between objects for referring expression understanding. In: European Conference on Computer Vision, Springer pp. 792\u2013807 (2016)","DOI":"10.1007\/978-3-319-46493-0_48"},{"key":"3232_CR37","unstructured":"Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)"},{"key":"3232_CR38","doi-asserted-by":"publisher","first-page":"2751","DOI":"10.1007\/s00371-021-02153-y","volume":"38","author":"N Ahmad","year":"2022","unstructured":"Ahmad, N., Asghar, S., Gillani, S.A.: Transfer learning-assisted multi-resolution breast cancer histopathological images classification. Vis. Comput. 38, 2751\u20132770 (2022)","journal-title":"Vis. Comput."},{"key":"3232_CR39","doi-asserted-by":"publisher","first-page":"1765","DOI":"10.1007\/s00371-022-02443-z","volume":"39","author":"AJ Prakash","year":"2023","unstructured":"Prakash, A.J., Prakasam, P.: An intelligent fruits classification in precision agriculture using bilinear pooling convolutional neural networks. Vis. Comput. 39, 1765\u20131781 (2023)","journal-title":"Vis. Comput."},{"key":"3232_CR40","doi-asserted-by":"publisher","first-page":"987","DOI":"10.1007\/s00371-008-0271-7","volume":"24","author":"R Reif","year":"2008","unstructured":"Reif, R., Walch, D.: Augmented & virtual reality applications in the field of logistics. Vis. Comput. 24, 987\u2013994 (2008)","journal-title":"Vis. Comput."},{"key":"3232_CR41","doi-asserted-by":"publisher","first-page":"2051","DOI":"10.1007\/s00371-020-01911-8","volume":"36","author":"K Yu","year":"2020","unstructured":"Yu, K., Ahn, J., Lee, J., Kim, M., Han, J.: Collaborative slam and ar-guided navigation for floor layout inspection. Vis. Comput. 36, 2051\u20132063 (2020)","journal-title":"Vis. Comput."},{"key":"3232_CR42","doi-asserted-by":"publisher","first-page":"1491","DOI":"10.1007\/s00371-019-01745-z","volume":"36","author":"UK Latif","year":"2020","unstructured":"Latif, U.K., Shin, S.Y.: Op-mr: the implementation of order picking based on mixed reality in a smart warehouse. Vis. Comput. 36, 1491\u20131500 (2020)","journal-title":"Vis. Comput."},{"key":"3232_CR43","doi-asserted-by":"crossref","unstructured":"Qin, Y., Chi, X., Sheng, B., Lau, R.W.: Guiderender: large-scale scene navigation based on multi-modal view frustum movement prediction. Vis. Comput. pp. 1\u201311 (2023)","DOI":"10.1007\/s00371-023-02922-x"},{"key":"3232_CR44","doi-asserted-by":"crossref","unstructured":"Xiang, N., Liang, H.N., Yu, L., Yang, X., Zhang, J.J.: A mixed reality framework for microsurgery simulation with visual-tactile perception. Vis. Comput. pp. 1\u201313 (2023)","DOI":"10.1007\/s00371-023-02964-1"},{"key":"3232_CR45","doi-asserted-by":"publisher","first-page":"789","DOI":"10.1007\/s00371-020-01830-8","volume":"37","author":"M Ayadi","year":"2021","unstructured":"Ayadi, M., Scuturici, M., Ben Amar, C., Miguet, S.: A skyline-based approach for mobile augmented reality. Vis. Comput. 37, 789\u2013804 (2021)","journal-title":"Vis. Comput."},{"key":"3232_CR46","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.3390\/s21041123","volume":"21","author":"D Jurado","year":"2021","unstructured":"Jurado, D., Jurado, J.M., Ortega, L., Feito, F.R.: Geuinf: Real-time visualization of indoor facilities using mixed reality. Sensors 21, 1123 (2021)","journal-title":"Sensors"},{"key":"3232_CR47","doi-asserted-by":"publisher","first-page":"2149","DOI":"10.1007\/s00371-022-02470-w","volume":"39","author":"P Bhagat","year":"2023","unstructured":"Bhagat, P., Choudhary, P., Singh, K.M.: A study on zero-shot learning from semantic viewpoint. Vis. Comput. 39, 2149\u20132163 (2023)","journal-title":"Vis. Comput."},{"key":"3232_CR48","doi-asserted-by":"crossref","unstructured":"Yang, Y., Lou, X., Choi, C.: Interactive robotic grasping with attribute-guided disambiguation. In: 2022 IEEE International Conference on Robotics and Automation (ICRA), IEEE (2022)","DOI":"10.1109\/ICRA46639.2022.9812360"},{"key":"3232_CR49","doi-asserted-by":"crossref","unstructured":"Liang, H., Yuan, J., Thalmann, D., Thalmann, N.M.: Ar in hand: Egocentric palm pose tracking and gesture recognition for augmented reality applications. In: Proceedings of the 23rd ACM International Conference on Multimedia. pp. 743\u2013744 (2015)","DOI":"10.1145\/2733373.2807972"},{"key":"3232_CR50","doi-asserted-by":"crossref","unstructured":"Johari, K., Tong, C.T.Z., Subbaraju, V., Kim, J.J., Tan, U., et\u00a0al.: Gaze assisted visual grounding. In: International Conference on Social Robotics, Springer pp. 191\u2013202 (2021)","DOI":"10.1007\/978-3-030-90525-5_17"}],"container-title":["The Visual Computer"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00371-023-03232-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00371-023-03232-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00371-023-03232-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T16:15:17Z","timestamp":1730909717000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00371-023-03232-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,27]]},"references-count":50,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["3232"],"URL":"https:\/\/doi.org\/10.1007\/s00371-023-03232-y","relation":{},"ISSN":["0178-2789","1432-2315"],"issn-type":[{"type":"print","value":"0178-2789"},{"type":"electronic","value":"1432-2315"}],"subject":[],"published":{"date-parts":[[2024,1,27]]},"assertion":[{"value":"14 December 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 January 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This research is supported by the Agency for Science, Technology and Research (A*STAR) under its AME Programmatic Funding Scheme (Project # A18A2b0046). All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"In this paper, we have developed an approach whereby we generate synthetic data, thus skipping the process of data collection for machine learning. The objective of performing the experiment with humans is to demonstrate the feasibility. As the tasks are simple (minimal risks to participants) and no personal identifier is collected, we have obtained IRB exemption approval.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}