{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,20]],"date-time":"2025-06-20T11:45:21Z","timestamp":1750419921439,"version":"3.41.0"},"reference-count":120,"publisher":"Association for Computing Machinery (ACM)","issue":"EICS","license":[{"start":{"date-parts":[[2020,6,18]],"date-time":"2020-06-18T00:00:00Z","timestamp":1592438400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100010785","name":"Canada First Research Excellence Fund","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100010785","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2020,6,18]]},"abstract":"<jats:p>Scientific workflow management systems such as Galaxy, Taverna and Workspace, have been developed to automate scientific workflow management and are increasingly being used to accelerate the specification, execution, visualization, and monitoring of data-intensive tasks. For example, the popular bioinformatics platform Galaxy is installed on over 168 servers around the world and the social networking space myExperiment shares almost 4,000 Galaxy scientific workflows among its 10,665 members. Most of these systems offer graphical interfaces for composing workflows. However, while graphical languages are considered easier to use, graphical workflow models are more difficult to comprehend and maintain as they become larger and more complex. Text-based languages are considered harder to use but have the potential to provide a clean and concise expression of workflow even for large and complex workflows. A recent study showed that some scientists prefer script\/text-based environments to perform complex scientific analysis with workflows. Unfortunately, such environments are unable to meet the needs of scientists who prefer graphical workflows. In order to address the needs of both types of scientists and at the same time to have script-based workflow models because of their underlying benefits, we propose a visually guided workflow modeling framework that combines interactive graphical user interface elements in an integrated development environment with the power of a domain-specific language to compose independently developed and loosely coupled services into workflows. Our domain-specific language provides scientists with a clean, concise, and abstract view of workflow to better support workflow modeling. As a proof of concept, we developed VizSciFlow, a generalized scientific workflow management system that can be customized for use in a variety of scientific domains. As a first use case, we configured and customized VizSciFlow for the bioinformatics domain. We conducted three user studies to assess its usability, expressiveness, efficiency, and flexibility. Results are promising, and in particular, our user studies show that VizSciFlow is more desirable for users to use than either Python or Galaxy for solving complex scientific problems.<\/jats:p>","DOI":"10.1145\/3394976","type":"journal-article","created":{"date-parts":[[2020,6,18]],"date-time":"2020-06-18T21:54:38Z","timestamp":1592517278000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["VizSciFlow: A Visually Guided Scripting Framework for Supporting Complex Scientific Data Analysis"],"prefix":"10.1145","volume":"4","author":[{"given":"Muhammad M.","family":"Hossain","sequence":"first","affiliation":[{"name":"University of Saskatchewan, Saskatoon, SK, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Banani","family":"Roy","sequence":"additional","affiliation":[{"name":"University of Saskatchewan, Saskatoon, SK, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chanchal K.","family":"Roy","sequence":"additional","affiliation":[{"name":"University of Saskatchewan, Saskatoon, SK, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kevin A.","family":"Schneider","sequence":"additional","affiliation":[{"name":"University of Saskatchewan, Saskatoon, SK, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,6,18]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Daniel Blankenberg, Dave Bouvier, Martin vC ech, John Chilton, Dave Clements, Nate Coraor, Carl Eberhard, et al.","author":"Afgan Enis","year":"2016","unstructured":"Enis Afgan , Dannon Baker , Marius Van den Beek , Daniel Blankenberg, Dave Bouvier, Martin vC ech, John Chilton, Dave Clements, Nate Coraor, Carl Eberhard, et al. 2016 . The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2016 update. Nucleic acids research, Vol. 44, W1 (2016), W3--W10. Enis Afgan, Dannon Baker, Marius Van den Beek, Daniel Blankenberg, Dave Bouvier, Martin vC ech, John Chilton, Dave Clements, Nate Coraor, Carl Eberhard, et al. 2016. The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2016 update. Nucleic acids research, Vol. 44, W1 (2016), W3--W10."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2110205.2110215"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/SSDM.2004.1311241"},{"key":"e_1_2_1_4_1","volume-title":"OASIS Standard","volume":"11","author":"Alves A.","year":"2007","unstructured":"A. Alves , A. Arkin , S. Askary , C. Barreto , B. Bloch , Francisco Curbera , M. Ford , Y. Goland , A. Guzar , Neelakantan Kartha , C. Liu , and Rania Khalaf . 2007 . Web Services Business Process Execution Language Version 2.0 (OASIS Standard) . OASIS Standard , Vol. 11 (01 2007). A. Alves, A. Arkin, S. Askary, C. Barreto, B. Bloch, Francisco Curbera, M. Ford, Y. Goland, A. Guzar, Neelakantan Kartha, C. Liu, and Rania Khalaf. 2007. Web Services Business Process Execution Language Version 2.0 (OASIS Standard). OASIS Standard, Vol. 11 (01 2007)."},{"key":"e_1_2_1_5_1","volume-title":"Crusoe","author":"Amstutz Peter","year":"2015","unstructured":"Peter Amstutz , Nebojsa Tijanic , Stian Soiland-Reyes , John Kern , Luka Stojanovic , Tim Pierce , John Chilton , Maxim Mikheev , Samuel Lampa , Herv\u00e9 M\u00e9nager , Scott Frazer , Venkat S. Malladi , and Michael R . Crusoe . 2015 . Beyond Galaxy: portable workflows and tool definitions with the CWL. https:\/\/cesgo.genouest.org\/resources\/129 Peter Amstutz, Nebojsa Tijanic, Stian Soiland-Reyes, John Kern, Luka Stojanovic, Tim Pierce, John Chilton, Maxim Mikheev, Samuel Lampa, Herv\u00e9 M\u00e9nager, Scott Frazer, Venkat S. Malladi, and Michael R. Crusoe. 2015. Beyond Galaxy: portable workflows and tool definitions with the CWL. https:\/\/cesgo.genouest.org\/resources\/129"},{"key":"e_1_2_1_6_1","unstructured":"Simon Andrews et al. 2010. FastQC: a quality control tool for high throughput sequence data.  Simon Andrews et al. 2010. FastQC: a quality control tool for high throughput sequence data."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/2835587.2835589"},{"key":"e_1_2_1_8_1","volume-title":"Miguel Goul ao, and Bruno Barroca","author":"Ankica Barivs","year":"2014","unstructured":"Ankica Barivs ic, Vasco Amaral , Miguel Goul ao, and Bruno Barroca . 2014 . Evaluating the usability of domain-specific languages. In Software Design and Development: Concepts, Methodologies, Tools, and Applications. IGI Global , 2120--2141. Ankica Barivs ic, Vasco Amaral, Miguel Goul ao, and Bruno Barroca. 2014. Evaluating the usability of domain-specific languages. In Software Design and Development: Concepts, Methodologies, Tools, and Applications. IGI Global, 2120--2141."},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the 19th Conference on Pattern Languages of Programs","author":"Barivsi\u0107 Ankica","year":"2012","unstructured":"Ankica Barivsi\u0107 , Pedro Monteiro , Vasco Amaral , Miguel Goul ao, and Miguel Monteiro . 2012 . Patterns for Evaluating Usability of Domain-Specific Languages . In Proceedings of the 19th Conference on Pattern Languages of Programs ( Tucson, Arizona) (PLoP '12). The Hillside Group, USA, Article 14, 34 pages. Ankica Barivsi\u0107, Pedro Monteiro, Vasco Amaral, Miguel Goul ao, and Miguel Monteiro. 2012. Patterns for Evaluating Usability of Domain-Specific Languages. In Proceedings of the 19th Conference on Pattern Languages of Programs (Tucson, Arizona) (PLoP '12). The Hillside Group, USA, Article 14, 34 pages."},{"key":"e_1_2_1_10_1","volume-title":"International Conference on Parallel Processing and Applied Mathematics","author":"Barker Adam","year":"2007","unstructured":"Adam Barker and Jano Van Hemert . 2007 . Scientific workflow: a survey and research directions . In International Conference on Parallel Processing and Applied Mathematics . Springer, New York, NY, 746--753. Adam Barker and Jano Van Hemert. 2007. Scientific workflow: a survey and research directions. In International Conference on Parallel Processing and Applied Mathematics. Springer, New York, NY, 746--753."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/6424.315691"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/765891.765936"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/VL.1996.545293"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/11568322_24"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/2817912.2817913"},{"key":"e_1_2_1_16_1","unstructured":"John Brooke et al. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry Vol. 189 194 (1996) 4--7.  John Brooke et al. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry Vol. 189 194 (1996) 4--7."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1142473.1142574"},{"key":"e_1_2_1_18_1","volume-title":"Julia K Goodrich, Jeffrey I Gordon, et al.","author":"Caporaso J Gregory","year":"2010","unstructured":"J Gregory Caporaso , Justin Kuczynski , Jesse Stombaugh , Kyle Bittinger , Frederic D Bushman , Elizabeth K Costello , Noah Fierer , Antonio Gonzalez Pena , Julia K Goodrich, Jeffrey I Gordon, et al. 2010 . QIIME allows analysis of high-throughput community sequencing data. Nature methods, Vol. 7 , 5 (2010), 335. J Gregory Caporaso, Justin Kuczynski, Jesse Stombaugh, Kyle Bittinger, Frederic D Bushman, Elizabeth K Costello, Noah Fierer, Antonio Gonzalez Pena, Julia K Goodrich, Jeffrey I Gordon, et al. 2010. QIIME allows analysis of high-throughput community sequencing data. Nature methods, Vol. 7, 5 (2010), 335."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2018.8622351"},{"key":"e_1_2_1_20_1","first-page":"1","article-title":"A Data Management Scheme for Micro-Level Modular Computation-intensive Programs in Big Data Platforms. In In: Alhajj R., Moshirpour M.,and Far B. (eds), Data Management and Analysis: Case Studies in Education, Healthcare and Beyond","volume":"65","author":"Chakroborti Debasish","year":"2020","unstructured":"Debasish Chakroborti , Banani Roy , Amit Kumar Mondal , Golam Mostaeen , Ralph Deters , Chanchal K. Roy , and Schneider Kevin A. 2020 . A Data Management Scheme for Micro-Level Modular Computation-intensive Programs in Big Data Platforms. In In: Alhajj R., Moshirpour M.,and Far B. (eds), Data Management and Analysis: Case Studies in Education, Healthcare and Beyond , Studies in Big Data , Vol. 65. 1 -- 20 . Debasish Chakroborti, Banani Roy, Amit Kumar Mondal, Golam Mostaeen, Ralph Deters, Chanchal K. Roy, and Schneider Kevin A. 2020. A Data Management Scheme for Micro-Level Modular Computation-intensive Programs in Big Data Platforms. In In: Alhajj R., Moshirpour M.,and Far B. (eds), Data Management and Analysis: Case Studies in Education, Healthcare and Beyond, Studies in Big Data, Vol. 65. 1--20.","journal-title":"Studies in Big Data"},{"key":"e_1_2_1_21_1","volume-title":"On scientific workflows","author":"Chen Jinjun","year":"2007","unstructured":"Jinjun Chen and Wil van der Aalst . 2007. On scientific workflows . IEEE Computer Society's Technical Committee for Scalable Computing , Vol . 9 ( 2007 ). Jinjun Chen and Wil van der Aalst. 2007. On scientific workflows. IEEE Computer Society's Technical Committee for Scalable Computing, Vol. 9 (2007)."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.csi.2011.06.002"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.992"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings eResearch","author":"Cleary Paul","year":"2014","unstructured":"Paul Cleary , Matt Bolger , Lachlan Hetherton , Chris Rucinski , David Thomas , and Damien Watkins . 2014 . Workspace: A Platform for Delivering Scientific Applications . Proceedings eResearch (2014), 4. Paul Cleary, Matt Bolger, Lachlan Hetherton, Chris Rucinski, David Thomas, and Damien Watkins. 2014. Workspace: A Platform for Delivering Scientific Applications. Proceedings eResearch (2014), 4."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btp163"},{"key":"e_1_2_1_26_1","unstructured":"Steve Cook Gareth Jones Stuart Kent and Alan Cameron Wills. 2007. Domain-specific development with visual studio dsl tools .Pearson Education.  Steve Cook Gareth Jones Stuart Kent and Alan Cameron Wills. 2007. Domain-specific development with visual studio dsl tools .Pearson Education."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2607023.2607036"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2014.10.008"},{"key":"e_1_2_1_30_1","volume-title":"Emilio Palumbo, and Cedric Notredame.","author":"Tommaso Paolo Di","year":"2017","unstructured":"Paolo Di Tommaso , Maria Chatzou , Evan W Floden , Pablo Prieto Barja , Emilio Palumbo, and Cedric Notredame. 2017 . Nextflow enables reproducible computational workflows. Nature biotechnology, Vol. 35 , 4 (2017), 316--319. Paolo Di Tommaso, Maria Chatzou, Evan W Floden, Pablo Prieto Barja, Emilio Palumbo, and Cedric Notredame. 2017. Nextflow enables reproducible computational workflows. Nature biotechnology, Vol. 35, 4 (2017), 316--319."},{"volume-title":"UML Activity Diagrams as a Workflow Specification Language","author":"Dumas Marlon","key":"e_1_2_1_31_1","unstructured":"Marlon Dumas and Arthur ter Hofstede . 2001. UML Activity Diagrams as a Workflow Specification Language . Springer Berlin Heidelberg , Berlin, Heidelberg , 76--90. https:\/\/doi.org\/10.1007\/3--540--45441--1_7 10.1007\/3--540--45441--1_7 Marlon Dumas and Arthur ter Hofstede. 2001. UML Activity Diagrams as a Workflow Specification Language .Springer Berlin Heidelberg, Berlin, Heidelberg, 76--90. https:\/\/doi.org\/10.1007\/3--540--45441--1_7"},{"key":"e_1_2_1_32_1","unstructured":"elephantlaboratories. 2019. Does anyone use CWL? Does it actually help you get work done? https:\/\/www.reddit.com\/r\/bioinformatics\/comments\/7gxsk0\/does_anyone_use_cwl_does_it_actually_help_you_get\/ visited 2019-07-08.  elephantlaboratories. 2019. Does anyone use CWL? Does it actually help you get work done? https:\/\/www.reddit.com\/r\/bioinformatics\/comments\/7gxsk0\/does_anyone_use_cwl_does_it_actually_help_you_get\/ visited 2019-07-08."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1869542.1869625"},{"key":"e_1_2_1_34_1","first-page":"1","article-title":"Workflow Provenance for Big Data: From Modelling to Reporting. In Alhajj R., Moshirpour M.,and Far B. (eds), Data Management and Analysis: Case Studies in Education, Healthcare and Beyond","volume":"65","author":"Ferdous Rayhan","year":"2020","unstructured":"Rayhan Ferdous , Banani Roy , Chanchal Roy , and Kevin Schneider . 2020 . Workflow Provenance for Big Data: From Modelling to Reporting. In Alhajj R., Moshirpour M.,and Far B. (eds), Data Management and Analysis: Case Studies in Education, Healthcare and Beyond , Studies in Big Data , Vol. 65. 1 -- 18 . Rayhan Ferdous, Banani Roy, Chanchal Roy, and Kevin Schneider. 2020. Workflow Provenance for Big Data: From Modelling to Reporting. In Alhajj R., Moshirpour M.,and Far B. (eds), Data Management and Analysis: Case Studies in Education, Healthcare and Beyond, Studies in Big Data, Vol. 65. 1--18.","journal-title":"Studies in Big Data"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3220134.3225217"},{"key":"e_1_2_1_36_1","unstructured":"Martin Fowler. 2010a. Domain-specific languages .Pearson Education.  Martin Fowler. 2010a. Domain-specific languages .Pearson Education."},{"key":"e_1_2_1_37_1","volume-title":"Domain Specific Languages","author":"Fowler Martin","unstructured":"Martin Fowler . 2010b. Domain Specific Languages 1 st ed.). Addison-Wesley Professional , Reading, MA . Martin Fowler. 2010b. Domain Specific Languages 1st ed.). Addison-Wesley Professional, Reading, MA.","edition":"1"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1965724.1965740"},{"key":"e_1_2_1_39_1","doi-asserted-by":"crossref","unstructured":"Belinda Giardine Cathy Riemer Ross C Hardison Richard Burhans Laura Elnitski Prachi Shah Yi Zhang Daniel Blankenberg Istvan Albert James Taylor etal 2005. Galaxy: a platform for interactive large-scale genome analysis. Genome research Vol. 15 10 (2005) 1451--1455.  Belinda Giardine Cathy Riemer Ross C Hardison Richard Burhans Laura Elnitski Prachi Shah Yi Zhang Daniel Blankenberg Istvan Albert James Taylor et al. 2005. Galaxy: a platform for interactive large-scale genome analysis. Genome research Vol. 15 10 (2005) 1451--1455.","DOI":"10.1101\/gr.4086505"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2007.421"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(84)80037-1"},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Carole A Goble etal 2010. myExperiment: a repository and social network for the sharing of bioinformatics workflows. Nucleic acids research Vol. 38 suppl_2 (2010) W677--W682.  Carole A Goble et al. 2010. myExperiment: a repository and social network for the sharing of bioinformatics workflows. Nucleic acids research Vol. 38 suppl_2 (2010) W677--W682.","DOI":"10.1093\/nar\/gkq429"},{"key":"e_1_2_1_43_1","doi-asserted-by":"crossref","unstructured":"Jeremy Goecks Anton Nekrutenko etal 2010. Galaxy: a comprehensive approach for supporting accessible reproducible and transparent computational research in the life sciences. Genome biology Vol. 11 8 (2010) R86.  Jeremy Goecks Anton Nekrutenko et al. 2010. Galaxy: a comprehensive approach for supporting accessible reproducible and transparent computational research in the life sciences. Genome biology Vol. 11 8 (2010) R86.","DOI":"10.1186\/gb-2010-11-8-r86"},{"key":"e_1_2_1_44_1","volume-title":"Conventional workflow technology for scientific simulation. In Guide to e-Science","author":"G\u00f6rlach Katharina","year":"2011","unstructured":"Katharina G\u00f6rlach , Mirko Sonntag , Dimka Karastoyanova , Frank Leymann , and Michael Reiter . 2011 . Conventional workflow technology for scientific simulation. In Guide to e-Science . Springer, New York, NY, 323--352. Katharina G\u00f6rlach, Mirko Sonntag, Dimka Karastoyanova, Frank Leymann, and Michael Reiter. 2011. Conventional workflow technology for scientific simulation. In Guide to e-Science. Springer, New York, NY, 323--352."},{"key":"e_1_2_1_45_1","volume-title":"Proceedings of ECCE-6 (6th European Conference on Cognitive Ergonomics). GC van der Veer, MJ Tauber, S. Bagnarola and M. Antavolits","author":"Green Thomas RG","year":"1992","unstructured":"Thomas RG Green and Marian Petre . 1992 . When visual programs are harder to read than textual programs. In Human-Computer Interaction: Tasks and Organisation , Proceedings of ECCE-6 (6th European Conference on Cognitive Ergonomics). GC van der Veer, MJ Tauber, S. Bagnarola and M. Antavolits . Rome, CUD. 167--180. Thomas RG Green and Marian Petre. 1992. When visual programs are harder to read than textual programs. In Human-Computer Interaction: Tasks and Organisation, Proceedings of ECCE-6 (6th European Conference on Cognitive Ergonomics). GC van der Veer, MJ Tauber, S. Bagnarola and M. Antavolits. Rome, CUD. 167--180."},{"key":"e_1_2_1_46_1","first-page":"121","article-title":"Comprehensibility of visual and textual programs: A test of superlativism against the'match-mismatch'conjecture","volume":"91","author":"Green Thomas RG","year":"1991","unstructured":"Thomas RG Green , Marian Petre , and RKE Bellamy . 1991 . Comprehensibility of visual and textual programs: A test of superlativism against the'match-mismatch'conjecture . ESP , Vol. 91 , 743 (1991), 121 -- 146 . Thomas RG Green, Marian Petre, and RKE Bellamy. 1991. Comprehensibility of visual and textual programs: A test of superlativism against the'match-mismatch'conjecture. ESP, Vol. 91, 743 (1991), 121--146.","journal-title":"ESP"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1177\/154193120605000909"},{"volume-title":"Advances in psychology.","author":"Hart Sandra G","key":"e_1_2_1_48_1","unstructured":"Sandra G Hart and Lowell E Staveland . 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research . In Advances in psychology. Vol. 52 . Elsevier , 139--183. Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139--183."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/295665.295675"},{"key":"e_1_2_1_50_1","volume-title":"Workflow Management Coalition","volume":"68","author":"Hollingsworth D","year":"1995","unstructured":"D Hollingsworth . 1995 . Workflow Management Coalition: The Workflow Reference Model . Workflow Management Coalition , Vol. 68 (01 1995). D Hollingsworth. 1995. Workflow Management Coalition: The Workflow Reference Model. Workflow Management Coalition, Vol. 68 (01 1995)."},{"key":"e_1_2_1_51_1","unstructured":"David Hollingsworth et al. 2004. The workflow reference model: 10 years on. In Fujitsu Services UK; Technical Committee Chair of WfMC. Citeseer 295--312.  David Hollingsworth et al. 2004. The workflow reference model: 10 years on. In Fujitsu Services UK; Technical Committee Chair of WfMC. Citeseer 295--312."},{"key":"e_1_2_1_52_1","doi-asserted-by":"crossref","first-page":"1904","DOI":"10.1101\/gr.1363103","article-title":"Biopipe: a flexible framework for protocol-based bioinformatics analysis","volume":"13","author":"Hoon Shawn","year":"2003","unstructured":"Shawn Hoon , Kiran Kumar Ratnapu , Jer-ming Chia, Balamurugan Kumarasamy , Xiao Juguang , Michele Clamp , Arne Stabenau , Simon Potter , Laura Clarke , and Elia Stupka . 2003 . Biopipe: a flexible framework for protocol-based bioinformatics analysis . Genome Research , Vol. 13 , 8 (2003), 1904 -- 1915 . Shawn Hoon, Kiran Kumar Ratnapu, Jer-ming Chia, Balamurugan Kumarasamy, Xiao Juguang, Michele Clamp, Arne Stabenau, Simon Potter, Laura Clarke, and Elia Stupka. 2003. Biopipe: a flexible framework for protocol-based bioinformatics analysis. Genome Research, Vol. 13, 8 (2003), 1904--1915.","journal-title":"Genome Research"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSR.1998.685738"},{"key":"e_1_2_1_54_1","volume-title":"Apache Flink: Distributed stream data processing. Technical Report.","author":"Jacobs Kevin","year":"2016","unstructured":"Kevin Jacobs and Kacper Surdy . 2016 . Apache Flink: Distributed stream data processing. Technical Report. Kevin Jacobs and Kacper Surdy. 2016. Apache Flink: Distributed stream data processing. Technical Report."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1173706.1173744"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2017.8258410"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.5555\/227726.227842"},{"key":"e_1_2_1_58_1","volume-title":"Eclipse Modeling Symposium","volume":"20062","author":"Kolovos Dimitrios S","year":"2006","unstructured":"Dimitrios S Kolovos , Richard F Paige , and Fiona AC Polack . 2006 . Eclipse development tools for epsilon. In Eclipse Summit Europe , Eclipse Modeling Symposium , Vol. 20062 . 200. Dimitrios S Kolovos, Richard F Paige, and Fiona AC Polack. 2006. Eclipse development tools for epsilon. In Eclipse Summit Europe, Eclipse Modeling Symposium, Vol. 20062. 200."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2015.11.001"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/bts480"},{"key":"e_1_2_1_61_1","volume-title":"CEUR Workshop Proceedings","volume":"1255","author":"Lajmi A.","year":"2014","unstructured":"A. Lajmi , J. Martinez , and T. Ziadi . 2014. DSLFORGE: Textual modeling on the web . CEUR Workshop Proceedings , Vol. 1255 (01 2014 ), 25--29. A. Lajmi, J. Martinez, and T. Ziadi. 2014. DSLFORGE: Textual modeling on the web. CEUR Workshop Proceedings, Vol. 1255 (01 2014), 25--29."},{"key":"e_1_2_1_62_1","unstructured":"Peter Lawrence (Ed.). 1997. Workflow Handbook 1997 .John Wiley & Sons Inc. New York NY USA.  Peter Lawrence (Ed.). 1997. Workflow Handbook 1997 .John Wiley & Sons Inc. New York NY USA."},{"key":"e_1_2_1_63_1","volume-title":"A review of bioinformatic pipeline frameworks. Briefings in bioinformatics","author":"Leipzig Jeremy","year":"2017","unstructured":"Jeremy Leipzig . 2017. A review of bioinformatic pipeline frameworks. Briefings in bioinformatics , Vol. 18 , 3 ( 2017 ), 530--536. Jeremy Leipzig. 2017. A review of bioinformatic pipeline frameworks. Briefings in bioinformatics, Vol. 18, 3 (2017), 530--536."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.5555\/3190867.3190870"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02806-9_12"},{"key":"e_1_2_1_66_1","volume-title":"ArXiv","volume":"1303","author":"Li Heng","year":"2013","unstructured":"Heng Li . 2013 . Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM . ArXiv , Vol. 1303 (2013). Heng Li. 2013. Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM. ArXiv, Vol. 1303 (2013)."},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btp352"},{"key":"e_1_2_1_68_1","unstructured":"William Lidwell et al. 2010. Universal principles of design revised and updated: 125 ways to enhance usability influence perception increase appeal make better design decisions and teach through design .Rockport Pub.  William Lidwell et al. 2010. Universal principles of design revised and updated: 125 ways to enhance usability influence perception increase appeal make better design decisions and teach through design .Rockport Pub."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSC.2009.4"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10723-015-9329-8"},{"key":"e_1_2_1_71_1","unstructured":"Bertram Lud\"ascher et al. 2009a. Scientific process automation and workflow management. In Scientific Data Management: Challenges Technology and Deployment. CRC press.  Bertram Lud\"ascher et al. 2009a. Scientific process automation and workflow management. In Scientific Data Management: Challenges Technology and Deployment. CRC press."},{"key":"e_1_2_1_72_1","volume-title":"International Conference on Business Process Management","author":"Bertram","year":"2009","unstructured":"Bertram Lud\"ascher, Mathias Weske , Timothy McPhillips , and Shawn Bowers . 2009 b. Scientific workflows: Business as usual? . In International Conference on Business Process Management . Springer, New York, NY, 31--47. Bertram Lud\"ascher, Mathias Weske, Timothy McPhillips, and Shawn Bowers. 2009b. Scientific workflows: Business as usual?. In International Conference on Business Process Management. Springer, New York, NY, 31--47."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1109\/SSDM.2003.1214990"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btr507"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/SERVICES.2010.14"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2015.07.392"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICWS.2004.1314777"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1186\/2041-1480-5-25"},{"key":"e_1_2_1_79_1","unstructured":"Daniel D McCracken and Edwin D Reilly. 2003. Backus-naur form (bnf). (2003).  Daniel D McCracken and Edwin D Reilly. 2003. Backus-naur form (bnf). (2003)."},{"volume-title":"Getting started with pyparsing .\" O'Reilly Media","author":"McGuire Paul","key":"e_1_2_1_80_1","unstructured":"Paul McGuire . 2007. Getting started with pyparsing .\" O'Reilly Media , Inc.\", 1005 Gravenstein Highway North, Sebastopol, CA 95472, USA. Paul McGuire. 2007. Getting started with pyparsing .\" O'Reilly Media, Inc.\", 1005 Gravenstein Highway North, Sebastopol, CA 95472, USA."},{"key":"e_1_2_1_81_1","volume-title":"When and how to develop domain-specific languages. ACM computing surveys (CSUR)","author":"Mernik Marjan","year":"2005","unstructured":"Marjan Mernik , Jan Heering , and Anthony M Sloane . 2005. When and how to develop domain-specific languages. ACM computing surveys (CSUR) , Vol. 37 , 4 ( 2005 ), 316--344. Marjan Mernik, Jan Heering, and Anthony M Sloane. 2005. When and how to develop domain-specific languages. ACM computing surveys (CSUR), Vol. 37, 4 (2005), 316--344."},{"key":"e_1_2_1_82_1","volume-title":"Fine-Grained Attribute Level Locking Scheme for Collaborative Scientific Workflow Development. In 2018 IEEE International Conference on Services Computing (SCC). 273--277","author":"Mostaeen Golam","year":"2018","unstructured":"Golam Mostaeen , Banani Roy , Chanchal Roy , and Kevin Schneider . 2018 . Fine-Grained Attribute Level Locking Scheme for Collaborative Scientific Workflow Development. In 2018 IEEE International Conference on Services Computing (SCC). 273--277 . Golam Mostaeen, Banani Roy, Chanchal Roy, and Kevin Schneider. 2018. Fine-Grained Attribute Level Locking Scheme for Collaborative Scientific Workflow Development. In 2018 IEEE International Conference on Services Computing (SCC). 273--277."},{"key":"e_1_2_1_83_1","article-title":"Designing for Real-Time Groupware Systems to Support Complex Scientific Data Analysis","volume":"3","author":"Mostaeen Golam","year":"2019","unstructured":"Golam Mostaeen , Banani Roy , Chanchal Roy , and Kevin Schneider . 2019 . Designing for Real-Time Groupware Systems to Support Complex Scientific Data Analysis . Journal Proceedings of the ACM on Human-Computer Interaction , Vol. 3 , EICS, Article 9 (June 2019), 28 pages. Golam Mostaeen, Banani Roy, Chanchal Roy, and Kevin Schneider. 2019. Designing for Real-Time Groupware Systems to Support Complex Scientific Data Analysis. Journal Proceedings of the ACM on Human-Computer Interaction, Vol. 3, EICS, Article 9 (June 2019), 28 pages.","journal-title":"Journal Proceedings of the ACM on Human-Computer Interaction"},{"volume-title":"Usability engineering","author":"Nielsen Jakob","key":"e_1_2_1_84_1","unstructured":"Jakob Nielsen . 1994. Usability engineering . Elsevier . Jakob Nielsen. 1994. Usability engineering. Elsevier."},{"key":"e_1_2_1_85_1","unstructured":"Jakob Nielsen. 2000. Why You Only Need to Test with 5 Users Jakob Nielsen's Alertbox. \"https:\/\/www.nngroup.com\/articles\/why-you-only-need-to-test-with-5-users\/\"  Jakob Nielsen. 2000. Why You Only Need to Test with 5 Users Jakob Nielsen's Alertbox. \"https:\/\/www.nngroup.com\/articles\/why-you-only-need-to-test-with-5-users\/\""},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137770"},{"volume-title":"Model and Data Engineering","author":"Ojamaa Andres","key":"e_1_2_1_87_1","unstructured":"Andres Ojamaa , Hele-Mai Haav , and Jaan Penjam . 2015. Semi-automated generation of DSL meta models from formal domain ontologies . In Model and Data Engineering . Springer , New York, NY , 3--15. Andres Ojamaa, Hele-Mai Haav, and Jaan Penjam. 2015. Semi-automated generation of DSL meta models from formal domain ontologies. In Model and Data Engineering. Springer, New York, NY, 3--15."},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.86"},{"key":"e_1_2_1_89_1","unstructured":"Terence Parr. 2013. The definitive ANTLR 4 reference .Pragmatic Bookshelf.  Terence Parr. 2013. The definitive ANTLR 4 reference .Pragmatic Bookshelf."},{"volume-title":"Modern Business Process Automation","author":"Pesic Maja","key":"e_1_2_1_90_1","unstructured":"Maja Pesic , Helen Schonenberg , and Wil van der Aalst . 2010. Declarative workflow . In Modern Business Process Automation . Springer , 175--201. Maja Pesic, Helen Schonenberg, and Wil van der Aalst. 2010. Declarative workflow. In Modern Business Process Automation. Springer, 175--201."},{"key":"e_1_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.1145\/203241.203251"},{"volume-title":"Proceedings of the 4th International Workshop on Rapid Continuous Software Engineering. ACM","author":"Akond","key":"e_1_2_1_93_1","unstructured":"Akond Rahman et al. 2018. What questions do programmers ask about configuration as code? . In Proceedings of the 4th International Workshop on Rapid Continuous Software Engineering. ACM , New York, NY, USA, 16--22. Akond Rahman et al. 2018. What questions do programmers ask about configuration as code?. In Proceedings of the 4th International Workshop on Rapid Continuous Software Engineering. ACM, New York, NY, USA, 16--22."},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btg1031"},{"key":"e_1_2_1_95_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSA.2017.42"},{"key":"e_1_2_1_96_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/bts167"},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.1093\/pcp\/pcs183"},{"key":"e_1_2_1_98_1","unstructured":"Jeff Sauro. 2011. Measuring usability with the system usability scale (SUS).  Jeff Sauro. 2011. Measuring usability with the system usability scale (SUS)."},{"key":"e_1_2_1_99_1","doi-asserted-by":"publisher","DOI":"10.1128\/AEM.01541-09"},{"volume-title":"Advances in enterprise engineering","author":"Schonenberg Helen","key":"e_1_2_1_100_1","unstructured":"Helen Schonenberg , Ronny Mans , Nick Russell , Nataliya Mulyar , and Wil van der Aalst . 2008. Process flexibility: A survey of contemporary approaches . In Advances in enterprise engineering I. Springer , New York, NY , 16--30. Helen Schonenberg, Ronny Mans, Nick Russell, Nataliya Mulyar, and Wil van der Aalst. 2008. Process flexibility: A survey of contemporary approaches. In Advances in enterprise engineering I. Springer, New York, NY, 16--30."},{"key":"e_1_2_1_101_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btt199"},{"key":"e_1_2_1_102_1","doi-asserted-by":"crossref","unstructured":"Barry Smith Michael Ashburner Cornelius Rosse Jonathan Bard William Bug Werner Ceusters Louis J Goldberg Karen Eilbeck Amelia Ireland Christopher J Mungall etal 2007. The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nature biotechnology Vol. 25 11 (2007) 1251.  Barry Smith Michael Ashburner Cornelius Rosse Jonathan Bard William Bug Werner Ceusters Louis J Goldberg Karen Eilbeck Amelia Ireland Christopher J Mungall et al. 2007. The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nature biotechnology Vol. 25 11 (2007) 1251.","DOI":"10.1038\/nbt1346"},{"key":"e_1_2_1_103_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvlc.2012.04.001"},{"key":"e_1_2_1_104_1","doi-asserted-by":"publisher","DOI":"10.1145\/634067.634236"},{"key":"e_1_2_1_105_1","volume-title":"Guest editors","author":"Sprinkle Jonathan","year":"2009","unstructured":"Jonathan Sprinkle , Marjan Mernik , Juha-Pekka Tolvanen , and Diomidis Spinellis . 2009. Guest editors ' introduction: What kinds of nails need a domain-specific hammer? IEEE software, Vol. 26 , 4 ( 2009 ), 15--18. Jonathan Sprinkle, Marjan Mernik, Juha-Pekka Tolvanen, and Diomidis Spinellis. 2009. Guest editors' introduction: What kinds of nails need a domain-specific hammer? IEEE software, Vol. 26, 4 (2009), 15--18."},{"key":"e_1_2_1_106_1","volume-title":"International Conference on Model Driven Engineering Languages and Systems","author":"Tairas Robert","year":"2008","unstructured":"Robert Tairas , Marjan Mernik , 2008 . Using ontologies in the domain analysis of domain-specific languages . In International Conference on Model Driven Engineering Languages and Systems . Springer, New York, NY, 332--342. Robert Tairas, Marjan Mernik, et al. 2008. Using ontologies in the domain analysis of domain-specific languages. In International Conference on Model Driven Engineering Languages and Systems. Springer, New York, NY, 332--342."},{"key":"e_1_2_1_107_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2014.02.016"},{"key":"#cr-split#-e_1_2_1_108_1.1","doi-asserted-by":"crossref","unstructured":"Tom Tullis and Bill Albert. 2013. Chapter 6 - Self-Reported Metrics. In Measuring the User Experience (Second Edition) second edition ed.) Tom Tullis and Bill Albert (Eds.). Morgan Kaufmann Boston 121 -- 161. https:\/\/doi.org\/10.1016\/B978-0--12--415781--1.00006--6 10.1016\/B978-0--12--415781--1.00006--6","DOI":"10.1016\/B978-0-12-415781-1.00006-6"},{"key":"#cr-split#-e_1_2_1_108_1.2","doi-asserted-by":"crossref","unstructured":"Tom Tullis and Bill Albert. 2013. Chapter 6 - Self-Reported Metrics. In Measuring the User Experience (Second Edition) second edition ed.) Tom Tullis and Bill Albert (Eds.). Morgan Kaufmann Boston 121 -- 161. https:\/\/doi.org\/10.1016\/B978-0--12--415781--1.00006--6","DOI":"10.1016\/B978-0-12-415781-1.00006-6"},{"key":"e_1_2_1_109_1","doi-asserted-by":"publisher","DOI":"10.1145\/3102113.3102147"},{"key":"e_1_2_1_110_1","unstructured":"Wil van der Aalst and Arthur ter Hofstede. 1999. Workflow Patterns Initiative. http:\/\/www.workflowpatterns.com Retrieved December 12 2019 from  Wil van der Aalst and Arthur ter Hofstede. 1999. Workflow Patterns Initiative. http:\/\/www.workflowpatterns.com Retrieved December 12 2019 from"},{"key":"e_1_2_1_111_1","volume-title":"YAWL: yet another workflow language. Information systems","author":"van der Aalst Wil","year":"2005","unstructured":"Wil van der Aalst and Arthur ter Hofstede . 2005. YAWL: yet another workflow language. Information systems , Vol. 30 , 4 ( 2005 ), 245--275. Wil van der Aalst and Arthur ter Hofstede. 2005. YAWL: yet another workflow language. Information systems, Vol. 30, 4 (2005), 245--275."},{"key":"e_1_2_1_112_1","doi-asserted-by":"publisher","DOI":"10.1145\/2523616.2523633"},{"key":"e_1_2_1_113_1","unstructured":"Markus V\u00f6lter Sebastian Benz Christian Dietrich etal 2013. DSL Engineering - Designing Implementing and Using Domain-Specific Languages .dslbook.org. http:\/\/www.dslbook.org  Markus V\u00f6lter Sebastian Benz Christian Dietrich et al. 2013. DSL Engineering - Designing Implementing and Using Domain-Specific Languages .dslbook.org. http:\/\/www.dslbook.org"},{"key":"e_1_2_1_114_1","volume-title":"Hadoop: The Definitive Guide","author":"White Tom","year":"2009","unstructured":"Tom White . 2009 . Hadoop: The Definitive Guide 1 st ed.). O'Reilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472, USA. Tom White. 2009. Hadoop: The Definitive Guide 1st ed.). O'Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472, USA.","edition":"1"},{"key":"e_1_2_1_115_1","doi-asserted-by":"publisher","DOI":"10.1007\/10722620_3"},{"key":"e_1_2_1_116_1","doi-asserted-by":"crossref","unstructured":"Katherine Wolstencroft Robert Haines Fellows etal 2013. The Taverna workflow suite: designing and executing workflows of Web Services on the desktop web or in the cloud. Nucleic acids research Vol. 41 W1 (2013) W557--W561.  Katherine Wolstencroft Robert Haines Fellows et al. 2013. The Taverna workflow suite: designing and executing workflows of Web Services on the desktop web or in the cloud. Nucleic acids research Vol. 41 W1 (2013) W557--W561.","DOI":"10.1093\/nar\/gkt328"},{"key":"e_1_2_1_117_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICWS.2007.107"},{"key":"e_1_2_1_118_1","first-page":"10","article-title":"Spark: Cluster computing with working sets","volume":"10","author":"Zaharia Matei","year":"2010","unstructured":"Matei Zaharia , Mosharaf Chowdhury , Michael J Franklin , Scott Shenker , and Ion Stoica . 2010 . Spark: Cluster computing with working sets . HotCloud , Vol. 10 , 10 -- 10 (2010), 95. Matei Zaharia, Mosharaf Chowdhury, Michael J Franklin, Scott Shenker, and Ion Stoica. 2010. Spark: Cluster computing with working sets. HotCloud, Vol. 10, 10--10 (2010), 95.","journal-title":"HotCloud"},{"key":"e_1_2_1_119_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934664"},{"key":"e_1_2_1_120_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btt593"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394976","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3394976","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:48:00Z","timestamp":1750193280000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394976"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,18]]},"references-count":120,"journal-issue":{"issue":"EICS","published-print":{"date-parts":[[2020,6,18]]}},"alternative-id":["10.1145\/3394976"],"URL":"https:\/\/doi.org\/10.1145\/3394976","relation":{},"ISSN":["2573-0142"],"issn-type":[{"type":"electronic","value":"2573-0142"}],"subject":[],"published":{"date-parts":[[2020,6,18]]},"assertion":[{"value":"2020-06-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}