{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T16:41:07Z","timestamp":1783010467207,"version":"3.54.6"},"reference-count":46,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2019,1,29]],"date-time":"2019-01-29T00:00:00Z","timestamp":1548720000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010418","name":"Institute for Information and communications Technology Promotion","doi-asserted-by":"publisher","award":["B0101-15-0525"],"award-info":[{"award-number":["B0101-15-0525"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002582","name":"Gwangju Institute of Science and Technology","doi-asserted-by":"publisher","award":["Autonomous Vehicle project"],"award-info":[{"award-number":["Autonomous Vehicle project"]}],"id":[{"id":"10.13039\/501100002582","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With recent advances in object detection, the tracking-by-detection method has become mainstream for multi-object tracking in computer vision. The tracking-by-detection scheme necessarily has to resolve a problem of data association between existing tracks and newly received detections at each frame. In this paper, we propose a new deep neural network (DNN) architecture that can solve the data association problem with a variable number of both tracks and detections including false positives. The proposed network consists of two parts: encoder and decoder. The encoder is the fully connected network with several layers that take bounding boxes of both detection and track-history as inputs. The outputs of the encoder are sequentially fed into the decoder which is composed of the bi-directional Long Short-Term Memory (LSTM) networks with a projection layer. The final output of the proposed network is an association matrix that reflects matching scores between tracks and detections. To train the network, we generate training samples using the annotation of Stanford Drone Dataset (SDD). The experiment results show that the proposed network achieves considerably high recall and precision rate as the binary classifier for the assignment tasks. We apply our network to track multiple objects on real-world datasets and evaluate the tracking performance. The performance of our tracker outperforms previous works based on DNN and comparable to other state-of-the-art methods.<\/jats:p>","DOI":"10.3390\/s19030559","type":"journal-article","created":{"date-parts":[[2019,1,29]],"date-time":"2019-01-29T11:27:52Z","timestamp":1548761272000},"page":"559","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":55,"title":["Data Association for Multi-Object Tracking via Deep Neural Networks"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7843-631X","authenticated-orcid":false,"given":"Kwangjin","family":"Yoon","sequence":"first","affiliation":[{"name":"School of Electrical Engineering and Computer Science, Gwanju Institute of Science and Technology, Gwangju 61005, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6882-2324","authenticated-orcid":false,"given":"Du Yong","family":"Kim","sequence":"additional","affiliation":[{"name":"School of Engineering, RMIT University, Melbourne, VIC 3000, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5586-862X","authenticated-orcid":false,"given":"Young-Chul","family":"Yoon","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Computer Science, Gwanju Institute of Science and Technology, Gwangju 61005, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2775-7789","authenticated-orcid":false,"given":"Moongu","family":"Jeon","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Computer Science, Gwanju Institute of Science and Technology, Gwangju 61005, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,1,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Robicquet, A., Sadeghian, A., Alahi, A., and Savarese, S. (2016). Learning social etiquette: Human trajectory understanding in crowded scenes. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46484-8_33"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sadeghian, A., Alahi, A., and Savarese, S. (arXiv, 2017). Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies, arXiv.","DOI":"10.1109\/ICCV.2017.41"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"728","DOI":"10.1049\/iet-cvi.2017.0600","article-title":"Visual Multiple-Object Tracking for Unknown Clutter Rate","volume":"12","author":"Kim","year":"2018","journal-title":"IET Comput. Vis."},{"key":"ref_4","unstructured":"Viola, P., and Jones, M. (2001, January 8\u201314). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), Kauai, HI, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1627","DOI":"10.1109\/TPAMI.2009.167","article-title":"Object detection with discriminatively trained part-based models","volume":"32","author":"Felzenszwalb","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Girshick, R. (arXiv, 2015). Fast r-cnn, arXiv.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). Ssd: Single shot multibox detector. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_9","unstructured":"Ristani, E., and Tomasi, C. (2014). Tracking multiple people online and in real time. Asian Conference on Computer Vision, Springer."},{"key":"ref_10","unstructured":"Kumar, R., Charpiat, G., and Thonnat, M. (2014). Multiple object tracking by efficient graph partitioning. Asian Conference on Computer Vision, Springer."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zamir, A.R., Dehghan, A., and Shah, M. (2012). Gmcp-tracker: Global multi-object tracking using generalized minimum clique graphs. Computer Vision\u2014ECCV 2012, Springer.","DOI":"10.1007\/978-3-642-33709-3_25"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1806","DOI":"10.1109\/TPAMI.2011.21","article-title":"Multiple object tracking using k-shortest paths optimization","volume":"33","author":"Berclaz","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_13","unstructured":"Zhang, L., Li, Y., and Nevatia, R. (2008, January 23\u201328). Global data association for multi-object tracking using network flows. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, AK, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Pirsiavash, H., Ramanan, D., and Fowlkes, C.C. (2011, January 20\u201325). Globally-optimal greedy algorithms for tracking a variable number of objects. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995604"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yoon, Y.C., Boragule, A., Yoon, K., and Jeon, M. (2018, January 27\u201330). Online Multi-Object Tracking with Historical Appearance Matching and Scene Adaptive Detection Filtering. Proceedings of the IEEE International Conference on Advanced Video and Signal-Based Surveillance(AVSS), Auckland, New Zealand.","DOI":"10.1109\/AVSS.2018.8639078"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"6731","DOI":"10.1007\/s11042-015-3219-8","article-title":"Multi-human tracking using part-based appearance modelling and grouping-based tracklet association for visual surveillance applications","volume":"76","author":"Yang","year":"2017","journal-title":"Multimedia Tools Appl."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1109\/TIP.2012.2218824","article-title":"Spatio-Temporal Auxiliary Particle Filtering With l1-Norm-Based Appearance Model Learning for Robust Visual Tracking","volume":"22","author":"Kim","year":"2013","journal-title":"IEEE Trans. Image Process."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hamid Rezatofighi, S., Milan, A., Zhang, Z., Shi, Q., Dick, A., and Reid, I. (2016, January 27\u201330). Joint probabilistic matching using m-best solutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.22"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1109\/JOE.1983.1145560","article-title":"Sonar tracking of multiple targets using joint probabilistic data association","volume":"8","author":"Fortmann","year":"1983","journal-title":"IEEE J. Ocean. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1049\/iet-ipr.2017.1244","article-title":"Multiple hypothesis tracking algorithm for multi-target multi-camera tracking with disjoint views","volume":"12","author":"Yoon","year":"2018","journal-title":"IET Image Process."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kim, C., Li, F., Ciptadi, A., and Rehg, J.M. (2015, January 7\u201313). Multiple hypothesis tracking revisited. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.533"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1109\/TAC.1979.1102177","article-title":"An algorithm for tracking multiple targets","volume":"24","author":"Reid","year":"1979","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Song, Y.M., and Jeon, M. (2016, January 26\u201328). Online multiple object tracking with the hierarchically adopted gm-phd filter using motion and appearance. Proceedings of the 2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), Seoul, Korea.","DOI":"10.1109\/ICCE-Asia.2016.7804800"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Milan, A., Rezatofighi, S.H., Garg, R., Dick, A.R., and Reid, I.D. (2017, January 4\u20139). Data-Driven Approximations to NP-Hard Problems. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10750"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Rezatofighi, S.H., Milan, A., Zhang, Z., Shi, Q., Dick, A.R., and Reid, I.D. (2015, January 7\u201313). Joint Probabilistic Data Association Revisited. Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.349"},{"key":"ref_26","unstructured":"Leal-Taix\u00e9, L., Milan, A., Reid, I., Roth, S., and Schindler, K. (arXiv, 2015). Motchallenge 2015: Towards a benchmark for multi-target tracking, arXiv."},{"key":"ref_27","unstructured":"Leal-Taix\u00e9, L., Canton-Ferrer, C., and Schindler, K. (July, January 26). Learning by tracking: Siamese CNN for robust target association. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) Deep Vision: Deep Learning for Computer Vision, Las Vegas, NV, USA."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Milan, A., Leal-Taix\u00e9, L., Schindler, K., and Reid, I. (2015, January 7\u201312). Joint Tracking and Segmentation of Multiple Targets. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299178"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hosang, J., Benenson, R., and Schiele, B. (arXiv, 2017). Learning non-maximum suppression, arXiv.","DOI":"10.1109\/CVPR.2017.685"},{"key":"ref_30","unstructured":"Vinyals, O., Fortunato, M., and Jaitly, N. (2015, January 7\u201312). Pointer networks. Proceedings of the 29th Annual Conference on Neural Information Processing Systems (NIPS 2015), Montreal, QC, Canada."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Milan, A., Rezatofighi, S.H., Dick, A., Reid, I., and Schindler, K. (2017, January 4\u20139). Online Multi-Target Tracking using Recurrent Neural Networks. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11194"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ma, X., and Hovy, E. (arXiv, 2016). End-to-end sequence labeling via bi-directional lstm-cnns-crf, arXiv.","DOI":"10.18653\/v1\/P16-1101"},{"key":"ref_33","unstructured":"Blackman, S., and Popoli, R. (1999). Design and Analysis of Modern Tracking Systems, Artech House."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_35","unstructured":"Sutskever, I., Vinyals, O., and Le, Q.V. (2014, January 8\u201313). Sequence to sequence learning with neural networks. Proceedings of the 28th Annual Conference on Neural Information Processing Systems (NIPS 2014), Montreal, QC, Canada."},{"key":"ref_36","unstructured":"Bahdanau, D., Cho, K., and Bengio, Y. (arXiv, 2014). Neural machine translation by jointly learning to align and translate, arXiv."},{"key":"ref_37","first-page":"246309","article-title":"Evaluating multiple object tracking performance: the CLEAR MOT metrics","volume":"2008","author":"Bernardin","year":"2008","journal-title":"J. Image Video Process."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Ristani, E., Solera, F., Zou, R., Cucchiara, R., and Tomasi, C. (2016). Performance measures and a data set for multi-target, multi-camera tracking. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-48881-3_2"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Davis, J., and Goadrich, M. (2006, January 25\u201329). The relationship between Precision-Recall and ROC curves. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA.","DOI":"10.1145\/1143844.1143874"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A new approach to linear filtering and prediction problems","volume":"82","author":"Kalman","year":"1960","journal-title":"J. Basic Eng."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Alahi, A., and Savarese, S. (2015, January 7\u201313). Learning to track: Online multi-object tracking by decision making. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.534"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Chen, L., Ai, H., Shang, C., Zhuang, Z., and Bai, B. (2017, January 17\u201320). Online multi-object tracking with convolutional neural networks. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.","DOI":"10.1109\/ICIP.2017.8296360"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Sanchez-Matilla, R., Poiesi, F., and Cavallaro, A. (2016). Online multi-target tracking with strong and weak detections. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-48881-3_7"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Yoon, J.H., Yang, M.H., Lim, J., and Yoon, K.J. (2015, January 5\u20139). Bayesian multi-object tracking using motion context from multiple objects. Proceedings of the 2015 IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV.2015.12"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Dicle, C., Camps, O.I., and Sznaier, M. (2013, January 1\u20138). The way they move: Tracking multiple targets with similar appearance. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.286"},{"key":"ref_46","unstructured":"Milan, A., Leal-Taix\u00e9, L., Reid, I.D., Roth, S., and Schindler, K. (arXiv, 2016). MOT16: A Benchmark for Multi-Object Tracking, arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/3\/559\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:29:30Z","timestamp":1760185770000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/3\/559"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,29]]},"references-count":46,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2019,2]]}},"alternative-id":["s19030559"],"URL":"https:\/\/doi.org\/10.3390\/s19030559","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,29]]}}}