{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T02:46:49Z","timestamp":1784342809726,"version":"3.55.0"},"reference-count":64,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,10,29]],"date-time":"2021-10-29T00:00:00Z","timestamp":1635465600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,10,29]],"date-time":"2021-10-29T00:00:00Z","timestamp":1635465600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100009591","name":"Ministerium f\u00fcr Innovation, Wissenschaft und Forschung des Landes Nordrhein-Westfalen","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100009591","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["952647"],"award-info":[{"award-number":["952647"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["830929"],"award-info":[{"award-number":["830929"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100008007","name":"Universit\u00e4t Paderborn","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100008007","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Due to the lack of established real-world benchmark suites for static taint analyses of Android applications, evaluations of these analyses are often restricted and hard to compare. Even in evaluations that do use real-world apps, details about the ground truth in those apps are rarely documented, which makes it difficult to compare and reproduce the results. To push Android taint analysis research forward, this paper thus recommends criteria for constructing real-world benchmark suites for this specific domain, and presents <jats:sc>TaintBench<\/jats:sc>, the first real-world <jats:italic>malware<\/jats:italic> benchmark suite with documented taint flows. <jats:sc>TaintBench<\/jats:sc> benchmark apps include taint flows with complex structures, and addresses static challenges that are commonly agreed on by the community. Together with the <jats:sc>TaintBench<\/jats:sc> suite, we introduce the <jats:sc>TaintBench<\/jats:sc> framework, whose goal is to simplify real-world benchmarking of Android taint analyses. First, a usability test shows that the framework improves experts\u2019 performance and perceived usability when documenting and inspecting taint flows. Second, experiments using <jats:sc>TaintBench<\/jats:sc> reveal new insights for the taint analysis tools <jats:sc>Amandroid<\/jats:sc> and <jats:sc>FlowDroid<\/jats:sc>: (i) They are less effective on real-world malware apps than on synthetic benchmark apps. (ii) Predefined lists of sources and sinks heavily impact the tools\u2019 accuracy. (iii) Surprisingly, up-to-date versions of both tools are less accurate than their predecessors.<\/jats:p>","DOI":"10.1007\/s10664-021-10013-5","type":"journal-article","created":{"date-parts":[[2021,10,29]],"date-time":"2021-10-29T02:02:32Z","timestamp":1635472952000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["TaintBench: Automatic real-world malware benchmarking of Android taint analyses"],"prefix":"10.1007","volume":"27","author":[{"given":"Linghui","family":"Luo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felix","family":"Pauck","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Goran","family":"Piskachev","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manuel","family":"Benz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ivan","family":"Pashchenko","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Mory","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric","family":"Bodden","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ben","family":"Hermann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabio","family":"Massacci","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,10,29]]},"reference":[{"key":"10013_CR1","unstructured":"Amandroid (2017) https:\/\/bintray.com\/arguslab\/maven\/argus-saf\/3.1.2, Accessed 02\/16\/2020"},{"key":"10013_CR2","unstructured":"Amandroid* (2018) https:\/\/bintray.com\/arguslab\/maven\/argus-saf\/3.2.0, Accessed 02\/16\/2020"},{"key":"10013_CR3","unstructured":"Androguard (2011) https:\/\/github.com\/androguard\/androguard, Accessed 02\/16\/2020"},{"key":"10013_CR4","unstructured":"AQL (2020) Android app analysis query language (aql). https:\/\/foellix.github.io\/AQL-System, Accessed 02\/16\/2020"},{"key":"10013_CR5","doi-asserted-by":"crossref","unstructured":"Arp D, Spreitzenbarth M, Hubner M, Gascon H, Rieck K (2014) DREBIN: effective and explainable detection of android malware in your pocket. In: Proceedings of the 21st NDSS, The Internet Society","DOI":"10.14722\/ndss.2014.23247"},{"key":"10013_CR6","doi-asserted-by":"crossref","unstructured":"Arzt S, Rasthofer S, Fritz C, Bodden E, Bartel A, Klein J, Traon Y L, Octeau D, McDaniel P D (2014) Flowdroid: precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps. In: Proceedings of PLDI. ACM","DOI":"10.1145\/2594291.2594299"},{"key":"10013_CR7","doi-asserted-by":"publisher","unstructured":"Avdiienko V, Kuznetsov K, Gorla A, Zeller A, Arzt S, Rasthofer S, Bodden E (2015) Mining apps for abnormal usage of sensitive data. In: 2015 IEEE\/ACM 37th IEEE International conference on software engineering, vol 1, pp 426\u2013436. https:\/\/doi.org\/10.1109\/ICSE.2015.61","DOI":"10.1109\/ICSE.2015.61"},{"key":"10013_CR8","doi-asserted-by":"crossref","unstructured":"Benz M, Krogh Kristensen E, Luo L, Borges PN Jr, Bodden E, Zeller A (2020) Heaps\u2019n leaks: How heap snapshots improve android taint analysis. In: Proceedings of the 42nd International conference on software engineering. https:\/\/2020.icse-conferences.org\/details\/icse-2020-papers\/114\/Heaps-n-Leaks-How-Heap-Snapshots-Improve-Android-Taint-Analysis","DOI":"10.1145\/3377811.3380438"},{"key":"10013_CR9","doi-asserted-by":"crossref","unstructured":"Blackburn SM, Garner R, Hoffmann C, Khan AM, McKinley KS, Bentzur R, Diwan A, Feinberg D, Frampton D, Guyer SZ, Hirzel M, Hosking AL, Jump M, Lee HB, Moss JEB, Phansalkar A, Stefanovic D, VanDrunen T, von Dincklage D, Wiedermann B (2006) The dacapo benchmarks: java benchmarking development and analysis. In: Proceedings of the 21th OOPSLA. ACM","DOI":"10.1145\/1167473.1167488"},{"key":"10013_CR10","doi-asserted-by":"crossref","unstructured":"Bohluli Z, Shahriari HR (2018) Detecting privacy leaks in android apps using inter-component information flow control analysis. In: Proceedings of the 15th ISCISC. IEEE","DOI":"10.1109\/ISCISC.2018.8546876"},{"key":"10013_CR11","doi-asserted-by":"crossref","unstructured":"Bosu A, Liu F, Yao D D, Wang G (2017) Collusive data leak and more: Large-scale threat analysis of inter-app communications. In: Proceedings of AsiaCCS. ACM","DOI":"10.1145\/3052973.3053004"},{"key":"10013_CR12","unstructured":"Brooke J (1996) Sus: a \u201dquick and dirty\u2019usability. Usability evaluation in industry, pp 189"},{"key":"10013_CR13","doi-asserted-by":"publisher","unstructured":"Cam N, Hau P, Nguyen T (2016) Android security analysis based on inter-application relationships, pp 689\u2013700. https:\/\/doi.org\/10.1007\/978-981-10-0557-2_68","DOI":"10.1007\/978-981-10-0557-2_68"},{"key":"10013_CR14","doi-asserted-by":"crossref","unstructured":"Chidamber SR, Kemerer CF (1994) A metrics suite for object oriented design. Trans Softw Eng IEEE","DOI":"10.1109\/32.295895"},{"key":"10013_CR15","unstructured":"Contagio Mobile (2012) http:\/\/contagiominidump.blogspot.com, Accessed 02\/16\/2020"},{"key":"10013_CR16","unstructured":"Contagio Mobile Malware (2018) http:\/\/contagiomobile.deependresearch.org\/index.html, Accessed 02\/18\/2021"},{"key":"10013_CR17","doi-asserted-by":"crossref","unstructured":"Do LNQ, Eichberg M, Bodden E (2016) Toward an automated benchmark management system. In: Proceedings of the 5th SOAP@PLDI. ACM","DOI":"10.1145\/2931021.2931023"},{"key":"10013_CR18","unstructured":"DroidBench 3-0 (2016) https:\/\/github.com\/secure-software-engineering\/DroidBench\/tree\/develop, Accessed 02\/16\/2020"},{"key":"10013_CR19","doi-asserted-by":"crossref","unstructured":"Eichberg M, Hermann B, Mezini M, Glanz L (2015) Hidden truths in dead software paths. In: Proceedings of the 10th ESEC\/FSE. ACM","DOI":"10.1145\/2786805.2786865"},{"key":"10013_CR20","doi-asserted-by":"crossref","unstructured":"Enck W, Gilbert P, Chun B, Cox LP, Jung J, McDaniel PD, Sheth A (2014) Taintdroid: an information flow tracking system for real-time privacy monitoring on smartphones. Commun ACM","DOI":"10.1145\/2494522"},{"key":"10013_CR21","unstructured":"F-Droid (2020) F-droid. https:\/\/F-Droid.org, Accessed 02\/16\/2020"},{"key":"10013_CR22","unstructured":"FlowDroid (2017) https:\/\/github.com\/secure-software-engineering\/soot-infoflow-android\/wiki, Accessed 02\/16\/2020"},{"key":"10013_CR23","unstructured":"FlowDroid* (2019) https:\/\/github.com\/secure-software-engineering\/FlowDroid\/releases\/tag\/v2.7.1, Accessed 02\/16\/2020"},{"key":"10013_CR24","doi-asserted-by":"crossref","unstructured":"Gordon MI, Kim D, Perkins JH, Gilham L, Nguyen N, Rinard MC (2015) Information flow analysis of android applications in droidsafe. In: Proceedings of the 22nd NDSS. The Internet Society","DOI":"10.14722\/ndss.2015.23089"},{"key":"10013_CR25","unstructured":"GitHub (2020) Github actions. https:\/\/docs.github.com\/en\/actions, Accessed 07\/05\/2020"},{"key":"10013_CR26","unstructured":"Gitpod (2019) Online ide for github. https:\/\/www.gitpod.io, Accessed 02\/16\/2020"},{"key":"10013_CR27","doi-asserted-by":"publisher","unstructured":"Grech N, Smaragdakis Y (2017) P\/taint: Unified points-to and taint analysis. Proc ACM Program Lang 1(OOPSLA). https:\/\/doi.org\/10.1145\/3133926","DOI":"10.1145\/3133926"},{"key":"10013_CR28","doi-asserted-by":"crossref","unstructured":"Huang J, Zhang X, Tan L, Wang P, Liang B (2014) Asdroid: detecting stealthy behaviors in android applications by user interface and program behavior contradiction. In: Proceedings of the 36th ICSE. ACM","DOI":"10.1145\/2568225.2568301"},{"key":"10013_CR29","doi-asserted-by":"crossref","unstructured":"Huang W, Dong Y, Milanova A, Dolby J (2015) Scalable and precise taint analysis for android, Proceedings of ISSTA. ACM","DOI":"10.1145\/2771783.2771803"},{"key":"10013_CR30","unstructured":"JADX (2020) Dex to Java decompiler. https:\/\/github.com\/skylot\/jadx, Accessed 02\/16\/2020"},{"key":"10013_CR31","unstructured":"Lam P, Bodden E, Lhot\u00e1k O, Hendren L (2011) The Soot framework for Java program analysis: a retrospective. In: Proceedings of CETUS. http:\/\/www.bodden.de\/pubs\/lblh11soot.pdf,"},{"key":"10013_CR32","doi-asserted-by":"crossref","unstructured":"Li L, Bartel A, Bissyand\u00e9 TF, Klein J, Traon YL, Arzt S, Rasthofer S, Bodden E, Octeau D, McDaniel PD (2015) Iccta: Detecting inter-component privacy leaks in android apps. In: Proceedings of the 37th ICSE. IEEE Computer Society","DOI":"10.1109\/ICSE.2015.48"},{"key":"10013_CR33","unstructured":"Livshits VB, Lam MS (2005) Finding security vulnerabilities in java applications with static analysis. In: Proceedings of the 14th USENIX security symposium. USENIX Association"},{"key":"10013_CR34","doi-asserted-by":"crossref","unstructured":"Luo L, Bodden E, Sp\u00e4th J (2019a) A qualitative analysis of android taint-analysis results. In: Proceedings of the 34th ASE. IEEE","DOI":"10.1109\/ASE.2019.00020"},{"key":"10013_CR35","unstructured":"Luo L, Dolby J, Bodden E (2019b) Magpiebridge: A general approach to integrating static analyses into ides and editors (tool insights paper). In: Proceedings of the 33rd ECOOP, Schloss Dagstuhl - Leibniz-Zentrum fu\u0307r Informatik, LIPIcs, vol 134"},{"key":"10013_CR36","unstructured":"Maur\u00edcio Aniche (2015) Java code metrics calculator (CK). https:\/\/github.com\/mauricioaniche\/ck, Accessed 02\/16\/2020"},{"key":"10013_CR37","doi-asserted-by":"crossref","unstructured":"Mitra J, Ranganath V (2017) Ghera: A repository of android app vulnerability benchmarks. In: Proceedings of the 13th PROMISE. ACM","DOI":"10.1145\/3127005.3127010"},{"key":"10013_CR38","unstructured":"Micro T (2020) New tekya ad fraud found on google play. https:\/\/blog.trendmicro.com\/trendlabs-security-intelligence\/new-tekya-ad-fraud-found-on-google-play\/, Accessed 29.06.2020"},{"key":"10013_CR39","unstructured":"Microsoft (2020) VSC - Visual Studio Code. https:\/\/code.visualstudio.com, Accessed 02\/16\/2020"},{"key":"10013_CR40","unstructured":"OASIS (2019) Static analysis results interchange format (sarif) version 2.0. https:\/\/docs.oasis-open.org\/sarif\/sarif\/v2.0\/csprd02\/sarif-v2.0-csprd02.html, Accessed 02\/16\/2020"},{"key":"10013_CR41","unstructured":"OWASP (2021) Owasp benchmark. https:\/\/owasp.org\/www-project-benchmark\/, Accessed 07.05.2021"},{"key":"10013_CR42","doi-asserted-by":"crossref","unstructured":"Pauck F, Wehrheim H (2019) Together strong: cooperative android app analysis. In: Proceedings of ESEC\/FSE. ACM","DOI":"10.1145\/3338906.3338915"},{"key":"10013_CR43","doi-asserted-by":"crossref","unstructured":"Pauck F, Zhang S (2019) Android app merging for benchmark speed-up and analysis lift-up. In: Proceedings of the 2nd A-Mobile@ASE. IEEE","DOI":"10.1109\/ASEW.2019.00019"},{"key":"10013_CR44","doi-asserted-by":"crossref","unstructured":"Pauck F, Bodden E, Wehrheim H (2018) Do android taint analysis tools keep their promises?. In: Proceedings of ESEC\/FSE. ACM","DOI":"10.1145\/3236024.3236029"},{"key":"10013_CR45","doi-asserted-by":"crossref","unstructured":"Piskachev G, Do LNQ, Bodden E (2019) Codebase-adaptive detection of security-relevant methods. In: Proceedings of the 28th ISSTA. ACM","DOI":"10.1145\/3293882.3330556"},{"key":"10013_CR46","doi-asserted-by":"crossref","unstructured":"Prokopec A, Ros\u00e0 A, Leopoldseder D, Duboscq G, Tuma P, Studener M, Bulej L, Zheng Y, Villaz\u00f3n A, Simon D, W\u00fcrthinger T, Binder W (2019) Renaissance: benchmarking suite for parallel applications on the JVM. In: Proceedings of the 40th PLDI. ACM","DOI":"10.1145\/3314221.3314637"},{"key":"10013_CR47","doi-asserted-by":"crossref","unstructured":"Qiu L, Wang Y, Rubin J (2018) Analyzing the analyzers: Flowdroid\/iccta, amandroid, and droidsafe. In: Proceedings of the 27th ISSTA. ACM","DOI":"10.1145\/3213846.3213873"},{"issue":"1","key":"10013_CR48","doi-asserted-by":"publisher","first-page":"178","DOI":"10.1007\/s10664-019-09749-y","volume":"25","author":"V Ranganath","year":"2020","unstructured":"Ranganath V, Mitra J (2020) Are free android app security analysis tools effective in detecting known vulnerabilities? Empir. Softw. Eng. 25(1):178\u2013219. https:\/\/doi.org\/10.1007\/s10664-019-09749-y","journal-title":"Empir. Softw. Eng."},{"key":"10013_CR49","unstructured":"Rasthofer S (2013) The android logging service \u2013 a dangerous feature for user privacy? https:\/\/blogs.uni-paderborn.de\/sse\/2013\/05\/17\/privacy-threatened-by-logging, Accessed 02\/16\/2020"},{"key":"10013_CR50","doi-asserted-by":"crossref","unstructured":"Rasthofer S, Arzt S, Bodden E (2014) A machine-learning approach for classifying and categorizing android sources and sinks. In: Proceedings of the 21st NDSS. The Internet Society","DOI":"10.14722\/ndss.2014.23039"},{"key":"10013_CR51","doi-asserted-by":"crossref","unstructured":"Rastogi V, Chen Y, Jiang X (2013) Droidchameleon: evaluating android anti-malware against transformation attacks. In: 8th ACM symposium on information, computer and communications security, ASIA CCS \u201913, Hangzhou, China - May 08 - 10, 2013. ACM","DOI":"10.1145\/2484313.2484355"},{"key":"10013_CR52","doi-asserted-by":"crossref","unstructured":"Reif M, Eichberg M, Hermann B, Mezini M (2017) Hermes: assessment and creation of effective test corpora. In: Proceedings of the 6th SOAP@PLDI. ACM","DOI":"10.1145\/3088515.3088523"},{"key":"10013_CR53","doi-asserted-by":"crossref","unstructured":"Rountev A, Kagan S, Gibas M (2004) Static and dynamic analysis of call chains in java. In: Proceedings of ISSTA. ACM","DOI":"10.1145\/1007512.1007514"},{"key":"10013_CR54","unstructured":"Soni J (2020) This dangerous malware got around google play store security. https:\/\/www.techradar.com\/news\/phantomlance-malware-breaches-google-play-store-security, Accessed 29.06.2020"},{"key":"10013_CR55","unstructured":"statcounter (2019) Operating system market share worldwide jan - dec 2019. https:\/\/gs.statcounter.com\/os-market-share#monthly-201901-201912-bar, Accessed 02\/16\/2020"},{"key":"10013_CR56","unstructured":"Vall\u0117e-Rai R, Co P, Gagnon E, Hendren LJ, Lam P, Sundaresan V (1999) Soot - a java bytecode optimization framework. In: Proceedings of CASCON. IBM"},{"key":"10013_CR57","doi-asserted-by":"crossref","unstructured":"Wei F, Roy S, Ou X, Robby (2014) Amandroid: A precise and general inter-component data flow analysis framework for security vetting of android apps. In: Proceedings of CCS. ACM","DOI":"10.1145\/2660267.2660357"},{"key":"10013_CR58","unstructured":"VirusShare (2014) https:\/\/virusshare.com, Accessed 02\/16\/2020"},{"key":"10013_CR59","doi-asserted-by":"crossref","unstructured":"Wei F, Li Y, Roy S, Ou X, Zhou W (2017) Deep ground truth analysis of current android malware. In: Proceedings of the 14th DIMVA, Lecture Notes in Computer Science, vol 10327. Springer","DOI":"10.1007\/978-3-319-60876-1_12"},{"key":"10013_CR60","doi-asserted-by":"crossref","unstructured":"Wong MY, Lie D (2016) Intellidroid: A targeted input generator for the dynamic analysis of android malware. In: Proceedings of the 23rd NDSS. The Internet Society","DOI":"10.14722\/ndss.2016.23118"},{"key":"10013_CR61","doi-asserted-by":"crossref","unstructured":"Yang T, Qian K, Li L, Lo D C, Tao L (2016) Static mining and dynamic taint for mobile security threats analysis In. In: Proceedings of SmartCloud. IEEE Computer Society","DOI":"10.1109\/SmartCloud.2016.43"},{"key":"10013_CR62","doi-asserted-by":"publisher","unstructured":"Youssef A, Shosha AF (2017) Quantitave dynamic taint analysis of privacy leakage in android arabic apps. In: Proceedings of the 12th international conference on availability, reliability and security, Association for Computing Machinery, New York, NY, USA, ARES \u201917. https:\/\/doi.org\/10.1145\/3098954.3105827","DOI":"10.1145\/3098954.3105827"},{"key":"10013_CR63","doi-asserted-by":"crossref","unstructured":"Zhang J, Tian C, Duan Z (2019) Fastdroid: efficient taint analysis for android applications. In: Proceedings of the 41st ICSE. IEEE \/ ACM","DOI":"10.1109\/ICSE-Companion.2019.00092"},{"key":"10013_CR64","doi-asserted-by":"crossref","unstructured":"Zheng C, Zhu S, Dai S, Gu G, Gong X, Han X, Zou W (2012) Smartdroid: an automatic system for revealing ui-based trigger conditions in android applications. In: Proceedings of SPSM@CCS. ACM","DOI":"10.1145\/2381934.2381950"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-10013-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-021-10013-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-10013-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,1,26]],"date-time":"2022-01-26T08:09:58Z","timestamp":1643184598000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-021-10013-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,29]]},"references-count":64,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1]]}},"alternative-id":["10013"],"URL":"https:\/\/doi.org\/10.1007\/s10664-021-10013-5","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,29]]},"assertion":[{"value":"1 July 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 October 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Ethical approval was not sought for the presented usability test described in Section 5, because it is not reporting the main research findings and the test involved only 5 participants. The purpose of the usability test was just to give some information about how usable our tools are for experts.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"Not applicable","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Competing interests"}},{"value":"Not applicable","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of Interests"}}],"article-number":"16"}}