{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,25]],"date-time":"2026-07-25T01:40:43Z","timestamp":1784943643561,"version":"3.55.0"},"reference-count":90,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2020,10,8]],"date-time":"2020-10-08T00:00:00Z","timestamp":1602115200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,10,8]],"date-time":"2020-10-08T00:00:00Z","timestamp":1602115200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["ERC-2017-COG-773161ERC-2017-COG-773161"],"award-info":[{"award-number":["ERC-2017-COG-773161ERC-2017-COG-773161"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006785","name":"Google","doi-asserted-by":"publisher","award":["Faculty Research Award"],"award-info":[{"award-number":["Faculty Research Award"]}],"id":[{"id":"10.13039\/100006785","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100005156http:\/\/dx.doi.org\/10.13039\/100005156","name":"Alexander von Humboldt-Stiftung","doi-asserted-by":"publisher","award":["Sofja Kovalevskaja AwardSofja Kovalevskaja Award"],"award-info":[{"award-number":["Sofja Kovalevskaja AwardSofja Kovalevskaja Award"]}],"id":[{"id":"10.13039\/100005156http:\/\/dx.doi.org\/10.13039\/100005156","id-type":"DOI","asserted-by":"publisher"}]},{"name":"CCAV","award":["Streetwise"],"award-info":[{"award-number":["Streetwise"]}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/N019474\/1EP\/N019474\/1"],"award-info":[{"award-number":["EP\/N019474\/1EP\/N019474\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2021,2]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Multi-object tracking (MOT) has been notoriously difficult to evaluate. Previous metrics overemphasize the importance of either detection or association. To address this, we present a novel MOT evaluation metric, higher order tracking accuracy (HOTA), which explicitly balances the effect of performing accurate detection, association and localization into a single unified metric for comparing trackers. HOTA decomposes into a family of sub-metrics which are able to evaluate each of five basic error types separately, which enables clear analysis of tracking performance. We evaluate the effectiveness of HOTA on the MOTChallenge benchmark, and show that it is able to capture important aspects of MOT performance not previously taken into account by established metrics. Furthermore, we show HOTA scores better align with human visual evaluation of tracking performance.<\/jats:p>","DOI":"10.1007\/s11263-020-01375-2","type":"journal-article","created":{"date-parts":[[2020,10,8]],"date-time":"2020-10-08T05:02:33Z","timestamp":1602133353000},"page":"548-578","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1108,"title":["HOTA: A Higher Order Metric for Evaluating Multi-object Tracking"],"prefix":"10.1007","volume":"129","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4133-7764","authenticated-orcid":false,"given":"Jonathon","family":"Luiten","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aljos\u0306a","family":"Os\u0306ep","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Patrick","family":"Dendorfer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philip","family":"Torr","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas","family":"Geiger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laura","family":"Leal-Taix\u00e9","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bastian","family":"Leibe","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,10,8]]},"reference":[{"key":"1375_CR1","doi-asserted-by":"crossref","unstructured":"Andriluka, M., Iqbal, U., Insafutdinov, E., Pishchulin, L., Milan, A., Gall, J., & Schiele, B. (2018). Posetrack: A benchmark for human pose estimation and tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5167\u20135176).","DOI":"10.1109\/CVPR.2018.00542"},{"key":"1375_CR2","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1016\/j.neucom.2019.08.008","volume":"368","author":"M Babaee","year":"2019","unstructured":"Babaee, M., Li, Z., & Rigoll, G. (2019). A dual cnn-rnn for multiple people tracking. Neurocomputing, 368, 69\u201383.","journal-title":"Neurocomputing"},{"key":"1375_CR3","doi-asserted-by":"crossref","unstructured":"Baisa, N.L. (2018). Online multi-target visual tracking using a hisp filter. In Proceedings of the 13th international joint conference on computer vision, imaging and computer graphics theory and applications (Vol. 5: VISAPP, pp. 429\u2013438).","DOI":"10.5220\/0006564504290438"},{"key":"1375_CR4","doi-asserted-by":"crossref","unstructured":"Baisa, N.L. (2019). Online multi-object visual tracking using a gm-phd filter with deep appearance learning. In 2019 22th international conference on information fusion (FUSION) (pp. 1\u20138).","DOI":"10.23919\/FUSION43075.2019.9011441"},{"key":"1375_CR5","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1016\/j.jvcir.2019.01.026","volume":"59","author":"NL Baisa","year":"2019","unstructured":"Baisa, N. L., & Wallace, A. (2019). Development of a n-type gm-phd filter for multiple target, multiple type visual tracking. Journal of Visual Communication and Image Representation, 59, 257\u2013271.","journal-title":"Journal of Visual Communication and Image Representation"},{"key":"1375_CR6","unstructured":"Bento, J., & Zhu, J.J. (2016). A metric for sets of trajectories that is practical and mathematically consistent. arXiv preprint arXiv:1601.03094"},{"key":"1375_CR7","doi-asserted-by":"crossref","unstructured":"Bergmann, P., Meinhardt, T., & Leal-Taix\u00e9, L. (2019). Tracking without bells and whistles. In ICCV.","DOI":"10.1109\/ICCV.2019.00103"},{"key":"1375_CR8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2008\/246309","volume":"2008","author":"K Bernardin","year":"2008","unstructured":"Bernardin, K., & Stiefelhagen, R. (2008). Evaluating multiple object tracking performance: The clear mot metrics. EURASIP Journal on Image and Video Processing, 2008, 1\u201310.","journal-title":"EURASIP Journal on Image and Video Processing"},{"key":"1375_CR9","doi-asserted-by":"crossref","unstructured":"Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. (2016). Simple online and realtime tracking. In 2016 IEEE international conference on image processing (ICIP) (pp. 3464\u20133468).","DOI":"10.1109\/ICIP.2016.7533003"},{"key":"1375_CR10","doi-asserted-by":"crossref","unstructured":"Bochinski, E., Eiselein, V., & Sikora, T. (2017). High-speed tracking-by-detection without using image information. In AVSS.","DOI":"10.1109\/AVSS.2017.8078516"},{"key":"1375_CR11","doi-asserted-by":"crossref","unstructured":"Bras\u00f3, G., & Leal-Taix\u00e9, L. (2020). Learning a neural solver for multiple object tracking. In CVPR.","DOI":"10.1109\/CVPR42600.2020.00628"},{"key":"1375_CR12","doi-asserted-by":"crossref","unstructured":"Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020). nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 11621\u201311631).","DOI":"10.1109\/CVPR42600.2020.01164"},{"key":"1375_CR13","doi-asserted-by":"crossref","unstructured":"Carletta, J., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., Kadlec, J., Karaiskos, V., Kraaij, W., & Kronenthal, M., et\u00a0al. (2005). The ami meeting corpus: A pre-announcement. In International workshop on machine learning for multimodal interaction (pp. 28\u201339). Springer.","DOI":"10.1007\/11677482_3"},{"key":"1375_CR14","doi-asserted-by":"crossref","unstructured":"Chang, M.F., Lambert, J., Sangkloy, P., Singh, J., Bak, S., Hartnett, A., Wang, D., Carr, P., Lucey, S., Ramanan, D., & Hays, J. (2019). Argoverse: 3d tracking and forecasting with rich maps. In The IEEE conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2019.00895"},{"key":"1375_CR15","doi-asserted-by":"crossref","unstructured":"Chen, J., Sheng, H., Zhang, Y., & Xiong, Z. (2017). Enhancing detection model for multiple hypothesis tracking. In BMTT workshop.","DOI":"10.1109\/CVPRW.2017.266"},{"issue":"11","key":"1375_CR16","doi-asserted-by":"publisher","first-page":"1613","DOI":"10.1109\/LSP.2019.2940922","volume":"26","author":"L Chen","year":"2019","unstructured":"Chen, L., Ai, H., Chen, R., & Zhuang, Z. (2019). Aggregate tracklet appearance features for multi-object tracking. IEEE Signal Processing Letters, 26(11), 1613\u20131617.","journal-title":"IEEE Signal Processing Letters"},{"key":"1375_CR17","doi-asserted-by":"crossref","unstructured":"Chu, P., & Ling, H. (2019). Famnet: Joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking. In ICCV.","DOI":"10.1109\/ICCV.2019.00627"},{"key":"1375_CR18","doi-asserted-by":"crossref","unstructured":"Dave, A., Khurana, T., Tokmakov, P., Schmid, C., & Ramanan, D. (2020). Tao: A large-scale benchmark for tracking any object. In ECCV.","DOI":"10.1007\/978-3-030-58558-7_26"},{"key":"1375_CR19","unstructured":"Dendorfer, P., Rezatofighi, H., Milan, A., Shi, J., Cremers, D., Reid, I., Roth, S., Schindler, K., & Leal-Taix\u00e9, L. (2019). CVPR19 tracking and detection challenge: How crowded can it get? arXiv:1906.04567 [cs]"},{"key":"1375_CR20","unstructured":"Dendorfer, P., Rezatofighi, H., Milan, A., Shi, J., Cremers, D., Reid, I., Roth, S., Schindler, K., & Leal-Taix\u00e9, L. (2020). Mot20: A benchmark for multi object tracking in crowded scenes. arXiv:2003.09003 [cs]. arXiv:1906.04567. ArXiv: 2003.09003."},{"key":"1375_CR21","doi-asserted-by":"crossref","unstructured":"Edward, K.K., Matthew, P.D., & Michael, B.H. (2009). An information theoretic approach for tracker performance evaluation. In 2009 IEEE 12th international conference on computer vision (pp. 1523\u20131529). IEEE","DOI":"10.1109\/ICCV.2009.5459275"},{"key":"1375_CR22","doi-asserted-by":"crossref","unstructured":"Eiselein, V., Arp, D., P\u00e4tzold, M., & Sikora, T. (2012). Real-time multi-human tracking using a probability hypothesis density filter and multiple detectors. In 2012 IEEE Ninth international conference on advanced video and signal-based surveillance.","DOI":"10.1109\/AVSS.2012.59"},{"key":"1375_CR23","doi-asserted-by":"crossref","unstructured":"Ellis, A., & Ferryman, J. (2010). Pets2010 and pets2009 evaluation of results using individual ground truthed single views. In 2010 7th IEEE international conference on advanced video and signal based surveillance (pp. 135\u2013142). IEEE.","DOI":"10.1109\/AVSS.2010.89"},{"issue":"2","key":"1375_CR24","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","volume":"88","author":"M Everingham","year":"2010","unstructured":"Everingham, M., Van Gool, L., Williams, C., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (VOC) challenge. IJCV, 88(2), 303\u2013338.","journal-title":"IJCV"},{"issue":"9","key":"1375_CR25","doi-asserted-by":"publisher","first-page":"2277","DOI":"10.1109\/TMM.2019.2902480","volume":"21","author":"Z Fu","year":"2019","unstructured":"Fu, Z., Angelini, F., Chambers, J., & Naqvi, S. M. (2019). Multi-level cooperative fusion of gm-phd filters for online multiple human tracking. IEEE Transactions on Multimedia, 21(9), 2277\u20132291.","journal-title":"IEEE Transactions on Multimedia"},{"key":"1375_CR26","doi-asserted-by":"crossref","unstructured":"Fu, Z., Feng, P., Angelini, F., Chambers, J., & Naqvi, S. (2018). Particle phd filter based multiple human tracking using online group-structured dictionary learning. In IEEE access.","DOI":"10.1109\/ICASSP.2017.7952983"},{"issue":"1","key":"1375_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s13160-012-0089-6","volume":"30","author":"O Fujita","year":"2013","unstructured":"Fujita, O. (2013). Metrics based on average distance between sets. Japan Journal of Industrial and Applied Mathematics, 30(1), 1\u201319.","journal-title":"Japan Journal of Industrial and Applied Mathematics"},{"key":"1375_CR28","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? the kitti vision benchmark suite. In Conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"1375_CR29","doi-asserted-by":"crossref","unstructured":"Henschel, R., Leal-Taix\u00e9, L., Cremers, D., & Rosenhahn, B. (2018). Fusion of head and full-body detectors for multi-object tracking. In Trajnet CVPRW.","DOI":"10.1109\/CVPRW.2018.00192"},{"key":"1375_CR30","doi-asserted-by":"crossref","unstructured":"Henschel, R., Zou, Y., & Rosenhahn, B. (2019). Multiple people tracking using body and joint detections. In CVPRW.","DOI":"10.1109\/CVPRW.2019.00105"},{"key":"1375_CR31","doi-asserted-by":"publisher","first-page":"104423","DOI":"10.1109\/ACCESS.2019.2932301","volume":"7","author":"H Karunasekera","year":"2019","unstructured":"Karunasekera, H., Wang, H., & Zhang, H. (2019). Multiple object tracking with attention to appearance, structure, motion and size. IEEE Access, 7, 104423\u2013104434.","journal-title":"IEEE Access"},{"key":"1375_CR32","volume-title":"Performance evaluation protocol for face, person and vehicle detection & tracking in video analysis and content extraction (vace-ii)","author":"R Kasturi","year":"2006","unstructured":"Kasturi, R., Goldgof, D., Soundararajan, P., Manohar, V., Boonstra, M., & Korzhova, V. (2006). Performance evaluation protocol for face, person and vehicle detection & tracking in video analysis and content extraction (vace-ii). Tampa: Computer Science & Engineering University of South Florida."},{"key":"1375_CR33","unstructured":"Keuper, M., Tang, S., Andres, B., Brox, T., & Schiele, B. (2018). Motion segmentation amp; multiple object tracking by correlation co-clustering. IEEE transactions on pattern analysis and machine intelligence (pp. 1\u20131)."},{"key":"1375_CR34","doi-asserted-by":"crossref","unstructured":"Kim, C., Li, F., Ciptadi, A., & Rehg, J.M. (2015). Multiple hypothesis tracking revisited. In ICCV.","DOI":"10.1109\/ICCV.2015.533"},{"key":"1375_CR35","doi-asserted-by":"crossref","unstructured":"Kim, C., Li, F., & Rehg, J.M. (2018). Multi-object tracking with neural gating using bilinear lstm. In ECCV.","DOI":"10.1007\/978-3-030-01237-3_13"},{"key":"1375_CR36","doi-asserted-by":"crossref","unstructured":"Kirillov, A., He, K., Girshick, R., Rother, C., & Doll\u00e1r, P. (2019). Panoptic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 9404\u20139413).","DOI":"10.1109\/CVPR.2019.00963"},{"key":"1375_CR37","doi-asserted-by":"crossref","unstructured":"Kutschbach, T., Bochinski, E., Eiselein, V., & Sikora, T. (2017). Sequential sensor fusion combining probability hypothesis density and kernelized correlation filters for multi-object tracking in video data. In AVSS.","DOI":"10.1109\/AVSS.2017.8078517"},{"key":"1375_CR38","doi-asserted-by":"crossref","unstructured":"Layne, R., Hannuna, S., Camplani, M., Hall, J., Hospedales, T.M., Xiang, T., Mirmehdi, M., & Damen, D. (2017). A dataset for persistent multi-target multi-camera tracking in rgb-d. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 47\u201355).","DOI":"10.1109\/CVPRW.2017.189"},{"key":"1375_CR39","unstructured":"Leal-Taix\u00e9, L., Milan, A., Reid, I., Roth, S., & Schindler, K. (2015). MOTChallenge 2015: Towards a benchmark for multi-target tracking. arXiv:1504.01942 [cs]."},{"key":"1375_CR40","unstructured":"Leal-Taix\u00e9, L., Milan, A., Schindler, K., Cremers, D., Reid, I., & Roth, S. (2017). Tracking the trackers: an analysis of the state of the art in multiple object tracking. arXiv preprint arXiv:1704.02781."},{"key":"1375_CR41","doi-asserted-by":"publisher","first-page":"8181","DOI":"10.1109\/ACCESS.2018.2889442","volume":"7","author":"S Lee","year":"2018","unstructured":"Lee, S., & Kim, E. (2018). Multiple object tracking via feature pyramid siamese networks. IEEE Access, 7, 8181.","journal-title":"IEEE Access"},{"key":"1375_CR42","doi-asserted-by":"publisher","first-page":"67316","DOI":"10.1109\/ACCESS.2018.2879535","volume":"6","author":"SH Lee","year":"2018","unstructured":"Lee, S. H., Kim, M. Y., & Bae, S. H. (2018). Learning discriminative appearance models for online multi-object tracking with appearance discriminability measures. IEEE Access, 6, 67316\u201367328.","journal-title":"IEEE Access"},{"issue":"10","key":"1375_CR43","doi-asserted-by":"publisher","first-page":"2553","DOI":"10.1109\/TPAMI.2013.70","volume":"35","author":"I Leichter","year":"2013","unstructured":"Leichter, I., & Krupka, E. (2013). Monotonicity and error type differentiability in performance measures for target detection and tracking in video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(10), 2553\u20132560.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"1375_CR44","doi-asserted-by":"crossref","unstructured":"Li, Y., Huang, C., & Nevatia, R. (2009). Learning to associate: Hybridboosted multi-target tracker for crowded scene. In 2009 IEEE conference on computer vision and pattern recognition (pp. 2953\u20132960). IEEE.","DOI":"10.1109\/CVPR.2009.5206735"},{"key":"1375_CR45","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., & Zitnick, C.L. (2014). Microsoft coco: Common objects in context. In European conference on computer vision (pp. 740\u2013755). Springer.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"1375_CR46","doi-asserted-by":"publisher","first-page":"76489","DOI":"10.1109\/ACCESS.2019.2921975","volume":"7","author":"Q Liu","year":"2019","unstructured":"Liu, Q., Liu, B., Wu, Y., Li, W., & Yu, N. (2019). Real-time online multi-object tracking in compressed domain. IEEE Access, 7, 76489\u201376499.","journal-title":"IEEE Access"},{"key":"1375_CR47","unstructured":"Long, C., Haizhou, A., Zijie, Z., & Chong, S. (2018). Real-time multiple people tracking with deeply learned candidate selection and person re-identification. In ICME."},{"issue":"2","key":"1375_CR48","doi-asserted-by":"publisher","first-page":"1803","DOI":"10.1109\/LRA.2020.2969183","volume":"5","author":"J Luiten","year":"2020","unstructured":"Luiten, J., Fischer, T., & Leibe, B. (2020). Track to reconstruct and reconstruct to track. IEEE Robotics and Automation Letters, 5(2), 1803\u20131810.","journal-title":"IEEE Robotics and Automation Letters"},{"key":"1375_CR49","doi-asserted-by":"crossref","unstructured":"Luiten, J., Torr, P., & Leibe, B. (2019). Video instance segmentation 2019: A winning approach for combined detection, segmentation, classification and tracking. In Proceedings of the IEEE international conference on computer vision workshops (pp. 0\u20130).","DOI":"10.1109\/ICCVW.2019.00088"},{"key":"1375_CR50","unstructured":"Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Zhao, X., & Kim, T.K. (2014). Multiple object tracking: A literature review. arXiv preprint arXiv:1409.7618."},{"key":"1375_CR51","doi-asserted-by":"crossref","unstructured":"Maksai, A., & Fua, P. (2019). Eliminating exposure bias and metric mismatch in multiple object tracking. In CVPR.","DOI":"10.1109\/CVPR.2019.00477"},{"key":"1375_CR52","doi-asserted-by":"crossref","unstructured":"Maksai, A., Wang, X., Fleuret, F., & Fua, P. (2017). Non-markovian globally consistent multi-object tracking. In Proceedings of the IEEE international conference on computer vision (pp. 2544\u20132554).","DOI":"10.1109\/ICCV.2017.278"},{"key":"1375_CR53","unstructured":"Manohar, V., Boonstra, M., Korzhova, V., Soundararajan, P., Goldgof, D., Kasturi, R., Prasad, S., Raju, H., Bowers, R., & Garofolo, J. (2006). Pets vs. vace evaluation programs: A comparative study. In Proceedings of IEEE international workshop on performance evaluation of tracking and surveillance (pp. 1\u20136)."},{"key":"1375_CR54","unstructured":"Milan, A., Leal-Taix\u00e9, L., Reid, I., Roth, S., & Schindler, K. (2016). MOT16: A benchmark for multi-object tracking. arXiv:1603.00831."},{"key":"1375_CR55","doi-asserted-by":"crossref","unstructured":"Milan, A., Schindler, K., & Roth, S. (2013). Challenges of ground truth evaluation of multi-target tracking. InProceedings of the IEEE conference on computer vision and pattern recognition workshops.","DOI":"10.1109\/CVPRW.2013.111"},{"key":"1375_CR56","doi-asserted-by":"crossref","unstructured":"Naphade, M., Anastasiu, D.C., Sharma, A., Jagrlamudi, V., Jeon, H., Liu, K., Chang, M.C., Lyu, S., & Gao, Z. (2017). The nvidia ai city challenge. In 2017 IEEE SmartWorld, ubiquitous intelligence & computing, advanced & trusted computed, scalable computing & communications, cloud & big data computing, internet of people and smart city innovation (SmartWorld\/SCALCOM\/UIC\/ATC\/CBDCom\/IOP\/SCI) (pp. 1\u20136). IEEE.","DOI":"10.1109\/UIC-ATC.2017.8397673"},{"key":"1375_CR57","doi-asserted-by":"crossref","unstructured":"Nghiem, A.T., Bremond, F., Thonnat, M., & Valentin, V. (2007). Etiseo, performance evaluation for video surveillance systems. In 2007 IEEE conference on advanced video and signal based surveillance (pp. 476\u2013481). IEEE.","DOI":"10.1109\/AVSS.2007.4425357"},{"key":"1375_CR58","unstructured":"Rahmathullah, A.S., Garc\u00eda-Fern\u00e1ndez, \u00c1.F., & Svensson, L. (2016). A metric on the space of finite sets of trajectories for evaluation of multi-target tracking algorithms. arXiv preprint arXiv:1605.01177."},{"issue":"6","key":"1375_CR59","doi-asserted-by":"publisher","first-page":"843","DOI":"10.1109\/TAC.1979.1102177","volume":"24","author":"D Reid","year":"1979","unstructured":"Reid, D. (1979). An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control, 24(6), 843\u2013854.","journal-title":"IEEE Transactions on Automatic Control"},{"key":"1375_CR60","doi-asserted-by":"crossref","unstructured":"Ristani, E., Solera, F., Zou, R., Cucchiara, R., & Tomasi, C. (2016). Performance measures and a data set for multi-target, multi-camera tracking. In European conference on computer vision (pp. 17\u201335). Springer.","DOI":"10.1007\/978-3-319-48881-3_2"},{"key":"1375_CR61","doi-asserted-by":"crossref","unstructured":"Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., & Fei-Fei, L. (2015). ImageNet large scale visual recognition challenge. In International Journal of Computer Vision (IJCV).","DOI":"10.1007\/s11263-015-0816-y"},{"key":"1375_CR62","doi-asserted-by":"crossref","unstructured":"Sanchez-Matilla, R., & Cavallaro, A. (2019). A predictor of moving objects for first-person vision. In IEEE international conference on image processing (ICIP).","DOI":"10.1109\/ICIP.2019.8803140"},{"key":"1375_CR63","doi-asserted-by":"crossref","unstructured":"Sanchez-Matilla, R., Poiesi, F., & Cavallaro, A. (2016). Online multi-target tracking with strong and weak detections. In Computer vision\u2013ECCV 2016 workshops (pp. 84\u201399).","DOI":"10.1007\/978-3-319-48881-3_7"},{"key":"1375_CR64","doi-asserted-by":"crossref","unstructured":"Sheng, H., Zhang, Y., Chen, J., Xiong, Z., & Zhang, J. (2018). Heterogeneous association graph fusion for target association in multiple object tracking. In IEEE Transactions on Circuits and Systems for Video Technology.","DOI":"10.1109\/TCSVT.2018.2882192"},{"key":"1375_CR65","doi-asserted-by":"crossref","unstructured":"Shitrit, H.B., Berclaz, J., Fleuret, F., & Fua, P. (2011). Tracking multiple people under global appearance constraints. In 2011 International conference on computer vision (pp. 137\u2013144). IEEE.","DOI":"10.1109\/ICCV.2011.6126235"},{"issue":"4","key":"1375_CR66","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1109\/TIT.1974.1055256","volume":"20","author":"R Singer","year":"1974","unstructured":"Singer, R., Sea, R., & Housewright, K. (1974). Derivation and evaluation of improved tracking filter for use in dense multitarget environments. IEEE Transactions on Information Theory, 20(4), 423\u2013432.","journal-title":"IEEE Transactions on Information Theory"},{"key":"1375_CR67","doi-asserted-by":"crossref","unstructured":"Smith, K., Gatica-Perez, D., Odobez, J.M., & Ba, S. (2005). Evaluating multi-object tracking. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR\u201905)-workshops (pp. 36\u201336). IEEE.","DOI":"10.1109\/CVPR.2005.453"},{"issue":"1","key":"1375_CR68","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1109\/TAC.1975.1100851","volume":"20","author":"P Smith","year":"1975","unstructured":"Smith, P., & Buechler, G. (1975). A branching algorithm for discriminating and tracking multiple objects. IEEE Transactions on Automatic Control, 20(1), 101\u2013104.","journal-title":"IEEE Transactions on Automatic Control"},{"key":"1375_CR69","doi-asserted-by":"crossref","unstructured":"Song, Y., & Jeon, M. (2016). Online multiple object tracking with the hierarchically adopted gm-phd filter using motion and appearance. In IEEE\/IEIE the international conference on consumer electronics (ICCE) Asia.","DOI":"10.1109\/ICCE-Asia.2016.7804800"},{"key":"1375_CR70","doi-asserted-by":"crossref","unstructured":"Song, Y., Yoon, K., Yoon, Y., Yow, K., & Jeon, M. (2019). Online multi-object tracking with gmphd filter and occlusion group management. IEEE access.","DOI":"10.1109\/ACCESS.2019.2953276"},{"issue":"6","key":"1375_CR71","doi-asserted-by":"publisher","first-page":"1207","DOI":"10.1109\/TAES.1975.308178","volume":"11","author":"J Stein","year":"1975","unstructured":"Stein, J., & Blackman, S. (1975). Generalized correlation of multi-target track data. IEEE Transactions on Aerospace and Electronic Systems, 11(6), 1207\u20131217.","journal-title":"IEEE Transactions on Aerospace and Electronic Systems"},{"key":"1375_CR72","doi-asserted-by":"crossref","unstructured":"Stiefelhagen, R., Bernardin, K., Bowers, R., Garofolo, J., Mostefa, D., & Soundararajan, P. (2006). The clear 2006 evaluation. In International evaluation workshop on classification of events, activities and relationships (pp. 1\u201344). Springer.","DOI":"10.1007\/978-3-540-69568-4_1"},{"key":"1375_CR73","doi-asserted-by":"crossref","unstructured":"Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., & Caine, B., et\u00a0al. (2019). Scalability in perception for autonomous driving: Waymo open dataset. arXiv pp. arXiv\u20131912.","DOI":"10.1109\/CVPR42600.2020.00252"},{"key":"1375_CR74","doi-asserted-by":"crossref","unstructured":"Voigtlaender, P., Krause, M., Osep, A., Luiten, J., Sekar, B.B.G., Geiger, A., & Leibe, B. (2019). Mots: Multi-object tracking and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition.","DOI":"10.1109\/CVPR.2019.00813"},{"key":"1375_CR75","doi-asserted-by":"crossref","unstructured":"Waibel, A., Steusloff, H., Stiefelhagen, R., & Watson, K. (2009). Computers in the human interaction loop. In Computers in the human interaction loop (pp. 3\u20136). Springer.","DOI":"10.1007\/978-1-84882-054-8_1"},{"key":"1375_CR76","doi-asserted-by":"crossref","unstructured":"Wang, G., Wang, Y., Zhang, H., Gu, R., & Hwang, J.N. (2019). Exploit the connectivity: Multi-object tracking with trackletnet. In ACM international conference on multimedia (pp. 482\u2013490).","DOI":"10.1145\/3343031.3350853"},{"key":"1375_CR77","doi-asserted-by":"crossref","unstructured":"Wang, X., Zhang, X., Zhu, Y., Guo, Y., Yuan, X., Xiang, L., Wang, Z., Ding, G., Brady, D., & Dai, Q., et\u00a0al. (2020). Panda: A gigapixel-level human-centric video dataset. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 3268\u20133278).","DOI":"10.1109\/CVPR42600.2020.00333"},{"key":"1375_CR78","doi-asserted-by":"publisher","first-page":"102907","DOI":"10.1016\/j.cviu.2020.102907","volume":"193","author":"L Wen","year":"2020","unstructured":"Wen, L., Du, D., Cai, Z., Lei, Z., Chang, M. C., Qi, H., et al. (2020). Ua-detrac: A new benchmark and protocol for multi-object detection and tracking. Computer Vision and Image Understanding, 193, 102907.","journal-title":"Computer Vision and Image Understanding"},{"key":"1375_CR79","doi-asserted-by":"crossref","unstructured":"Weng, X., Wang, J., Held, D., & Kitani, K. (2020). 3d multi-object tracking: A baseline and new evaluation metrics. In IROS.","DOI":"10.1109\/IROS45743.2020.9341164"},{"key":"1375_CR80","unstructured":"Wu, B., & Nevatia, R. (2006). Tracking of multiple, partially occluded humans based on static body part detection. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR\u201906) (vol.\u00a01, pp. 951\u2013958). IEEE."},{"key":"1375_CR81","doi-asserted-by":"crossref","unstructured":"Wu, C.W., Zhong, M.T., Tsao, Y., Yang, S.W., Chen, Y.K., & Chien, S.Y. (2017). Track-clustering error evaluation for track-based multi-camera tracking system employing human re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 1\u20139).","DOI":"10.1109\/CVPRW.2017.184"},{"key":"1375_CR82","doi-asserted-by":"crossref","unstructured":"Xu, J., Cao, Y., Zhang, Z., & Hu, H. (2019). Spatial-temporal relation networks for multi-object tracking. In ICCV.","DOI":"10.1109\/ICCV.2019.00409"},{"key":"1375_CR83","doi-asserted-by":"crossref","unstructured":"Yang, L., Fan, Y., & Xu, N. (2019). Video instance segmentation. In Proceedings of the IEEE international conference on computer vision (pp. 5188\u20135197).","DOI":"10.1109\/ICCV.2019.00529"},{"key":"1375_CR84","doi-asserted-by":"publisher","first-page":"38060","DOI":"10.1109\/ACCESS.2020.2975912","volume":"8","author":"K Yoon","year":"2020","unstructured":"Yoon, K., Gwak, J., Song, Y. M., Yoon, Y. C., & Jeon, M. (2020). Oneshotda: Online multi-object tracker with one-shot-learning-based data association. IEEE Access, 8, 38060\u201338072.","journal-title":"IEEE Access"},{"key":"1375_CR85","doi-asserted-by":"crossref","unstructured":"Yoon, Y., Boragule, A., Song, Y., Yoon, K., & Jeon, M. (2018). Online multi-object tracking with historical appearance matching and scene adaptive detection filtering. In IEEE AVSS.","DOI":"10.1109\/AVSS.2018.8639078"},{"key":"1375_CR86","doi-asserted-by":"crossref","unstructured":"Young, D.P., & Ferryman, J.M. (2005). Pets metrics: On-line performance evaluation service. In 2005 IEEE international workshop on visual surveillance and performance evaluation of tracking and surveillance (pp. 317\u2013324). IEEE.","DOI":"10.1109\/VSPETS.2005.1570931"},{"key":"1375_CR87","doi-asserted-by":"crossref","unstructured":"Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F., Madhavan, V., & Darrell, T. (2020). Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 2636\u20132645).","DOI":"10.1109\/CVPR42600.2020.00271"},{"key":"1375_CR88","doi-asserted-by":"crossref","unstructured":"Yu, S.I., Meng, D., Zuo, W., Hauptmann, A.(2016). The solution path algorithm for identity-aware multi-object tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3871\u20133879).","DOI":"10.1109\/CVPR.2016.420"},{"key":"1375_CR89","doi-asserted-by":"crossref","unstructured":"Zhu, J., Yang, H., Liu, N., Kim, M., Zhang, W., & Yang, M.H. (2018). Online multi-object tracking with dual matching attention networks. In ECCV.","DOI":"10.1007\/978-3-030-01228-1_23"},{"key":"1375_CR90","unstructured":"Zhu, P., Wen, L., Du, D., Bian, X., Hu, Q., & Ling, H. (2020). Vision meets drones: Past, present and future. arXiv preprint arXiv:2001.06303."}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-020-01375-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11263-020-01375-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-020-01375-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,22]],"date-time":"2022-11-22T14:31:26Z","timestamp":1669127486000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11263-020-01375-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,8]]},"references-count":90,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,2]]}},"alternative-id":["1375"],"URL":"https:\/\/doi.org\/10.1007\/s11263-020-01375-2","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"value":"0920-5691","type":"print"},{"value":"1573-1405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,8]]},"assertion":[{"value":"2 May 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 August 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 October 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}