{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T00:25:55Z","timestamp":1774398355696,"version":"3.50.1"},"reference-count":57,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2018,1,17]],"date-time":"2018-01-17T00:00:00Z","timestamp":1516147200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Protein structure prediction servers use various computational methods to predict the three-dimensional structure of proteins from their amino acid sequence. Predicted models are used to infer protein function and guide experimental efforts. This can contribute to solving the problem of predicting tertiary protein structures, one of the main unsolved problems in bioinformatics. The challenge is to understand the relationship between the amino acid sequence of a protein and its three-dimensional structure, which is related to the function of these macromolecules. This article is an extended version of the article wCReF: The Web Server for the Central Residue Fragment-based Method (CReF) Protein Structure Predictor, published in the 14th International Conference on Information Technology: New Generations. In the first version, we presented the wCReF, a protein structure prediction server for the central residue fragment-based method. The wCReF interface was developed with a focus on usability and user interaction. With this tool, users can enter the amino acid sequence of their target protein and obtain its approximate 3D structure without the need to install all the multitude of necessary tools. In this extended version, we present the design process of the prediction server in detail, which includes: (A) identification of user needs: aiming at understanding the features of a protein structure prediction server, the end user profiles and the commonly-performed tasks; (B) server usability inspection: in order to define wCReF\u2019s requirements and features, we have used heuristic evaluation guided by experts in both the human-computer interaction and bioinformatics domain areas, applied to the protein structure prediction servers I-TASSER, QUARK and Robetta; as a result, changes were found in all heuristics resulting in 89 usability problems; (C) software requirements document and prototype: assessment results guiding the key features that wCReF must have compiled in a software requirements document; from this step, prototyping was carried out; (D) wCReF usability analysis: a glimpse at the detection of new usability problems with end users by adapting the Ssemugabi satisfaction questionnaire; users\u2019 evaluation had 80% positive feedback; (E) finally, some specific guidelines for interface design are presented, which may contribute to the design of interactive computational resources for the field of bioinformatics. In addition to the results of the original article, we present the methodology used in wCReF\u2019s design and evaluation process (sample, procedures, evaluation tools) and the results obtained.<\/jats:p>","DOI":"10.3390\/info9010020","type":"journal-article","created":{"date-parts":[[2018,1,17]],"date-time":"2018-01-17T12:17:11Z","timestamp":1516191431000},"page":"20","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Usability as the Key Factor to the Design of a Web Server for the CReF Protein Structure Predictor: The wCReF"],"prefix":"10.3390","volume":"9","author":[{"given":"Vanessa","family":"Machado Paix\u00e3o-Cortes","sequence":"first","affiliation":[{"name":"Graduate Program in Computer Science\u2014PPGCC, Pontifical Catholic University of Rio Grande do Sul (PUCRS), Ipiranga Avenue, 6681 Partenon, Porto Alegre 90619-900, Rio Grande do Sul, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michele","family":"Dos Santos da Silva Tanus","sequence":"additional","affiliation":[{"name":"Graduate Program in Computer Science\u2014PPGCC, Pontifical Catholic University of Rio Grande do Sul (PUCRS), Ipiranga Avenue, 6681 Partenon, Porto Alegre 90619-900, Rio Grande do Sul, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Walter","family":"Paix\u00e3o-Cortes","sequence":"additional","affiliation":[{"name":"Graduate Program in Computer Science\u2014PPGCC, Pontifical Catholic University of Rio Grande do Sul (PUCRS), Ipiranga Avenue, 6681 Partenon, Porto Alegre 90619-900, Rio Grande do Sul, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4602-1208","authenticated-orcid":false,"given":"Osmar","family":"De Souza","sequence":"additional","affiliation":[{"name":"Graduate Program in Computer Science\u2014PPGCC, Pontifical Catholic University of Rio Grande do Sul (PUCRS), Ipiranga Avenue, 6681 Partenon, Porto Alegre 90619-900, Rio Grande do Sul, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marcia","family":"De Borba Campos","sequence":"additional","affiliation":[{"name":"Graduate Program in Computer Science\u2014PPGCC, Pontifical Catholic University of Rio Grande do Sul (PUCRS), Ipiranga Avenue, 6681 Partenon, Porto Alegre 90619-900, Rio Grande do Sul, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2159-551X","authenticated-orcid":false,"given":"Milene","family":"Silveira","sequence":"additional","affiliation":[{"name":"Graduate Program in Computer Science\u2014PPGCC, Pontifical Catholic University of Rio Grande do Sul (PUCRS), Ipiranga Avenue, 6681 Partenon, Porto Alegre 90619-900, Rio Grande do Sul, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,1,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"348","DOI":"10.1145\/1227504.1227434","article-title":"A multidisciplinary bioinformatics minor","volume":"39","author":"Bruhn","year":"2007","journal-title":"ACM SIGCSE Bull."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ranganathan, S. (2005). Bioinformatics education\u2014Perspectives and challenges. PLoS Comput. Biol., 1.","DOI":"10.1371\/journal.pcbi.0010052"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"W695","DOI":"10.1093\/nar\/gkq313","article-title":"A new bioinformatics analysis tools framework at EMBL-EBI","volume":"38","author":"Goujon","year":"2010","journal-title":"Nucleic Acids Res."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Fulekar, M. (2009). Bioinformatics: Applications in Life and Environmental Sciences, Springer Science & Business Media.","DOI":"10.1007\/978-1-4020-8880-3"},{"key":"ref_5","first-page":"2","article-title":"What is bioinformatics? An introduction and overview","volume":"1","author":"Luscombe","year":"2001","journal-title":"Yearb. Med. Inform."},{"key":"ref_6","unstructured":"Gibas, C., and Jambeck, P. (2001). Developing Bioinformatics Computer Skills, O\u2019Reilly Media, Inc."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1187\/cbe.13-10-0193","article-title":"A survey of scholarly literature describing the field of bioinformatics education and bioinformatics educational research","volume":"13","author":"Magana","year":"2014","journal-title":"CBE-Life Sci. Educ."},{"key":"ref_8","unstructured":"Lesk, A. (2013). Introduction to Bioinformatics, Oxford University Press."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2003","DOI":"10.1016\/j.promfg.2015.07.247","article-title":"Human factors in the design and evaluation of bioinformatics tools","volume":"3","author":"Badr","year":"2015","journal-title":"Procedia Manuf."},{"key":"ref_10","first-page":"D13","article-title":"Database resources of the national center for biotechnology information","volume":"42","author":"Acland","year":"2014","journal-title":"Nucleic Acids Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1179\/030801811X12941390545726","article-title":"Socio-Cultural characteristics of usability of bioinformatics databases and tools","volume":"36","author":"Douglas","year":"2011","journal-title":"Interdiscip. Sci. Rev."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1093\/bioinformatics\/btn633","article-title":"Better bioinformatics through usability analysis","volume":"25","author":"Bolchini","year":"2008","journal-title":"Bioinformatics"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Bolchini, D., Finkestein, A., and Paolini, P. (2009). Designing usable bio-information architectures. Human-Computer Interaction. Interacting in Various Application Domains, Springer.","DOI":"10.1007\/978-3-642-02583-9_71"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mirel, B. (2007, January 1\u20133). Usability and usefulness in bioinformatics: Evaluating a tool for querying and analyzing protein interactions based on scientists\u2019 actual research questions. Proceedings of the IEEE International Professional Communication Conference (IPCC 2007), Seattle, WA, USA.","DOI":"10.1109\/IPCC.2007.4464064"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mirel, B., and Wright, Z. (2009). Heuristic evaluations of bioinformatics tools: A development case. Human-Computer Interaction. New Trends, Springer.","DOI":"10.1007\/978-3-642-02574-7_37"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Shaer, O., Kol, G., Strait, M., Fan, C., Grevet, C., and Elfenbein, S. (2010, January 10\u201315). G-nome surfer: A tabletop interface for collaborative exploration of genomic data. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Atlanta, GA, USA.","DOI":"10.1145\/1753326.1753539"},{"key":"ref_17","unstructured":"Machado, V.S., Paix\u00e3o-Cortes, W.R., de Souza, O.N., and de Borba Campos, M. (2017, January 9\u201314). Decision-Making for Interactive Systems: A Case Study for Teaching and Learning in Bioinformatics. Proceedings of the International Conference on Learning and Collaboration Technologies, Vancouver, BC, Canada."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Veretnik, S., Fink, J.L., and Bourne, P.E. (2008). Computational biology resources lack persistence and usability. PLoS Comput. Biol., 4.","DOI":"10.1371\/journal.pcbi.1000136"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/1029496.1029527","article-title":"Beyond power: Making bioinformatics tools user-centered","volume":"47","author":"Javahery","year":"2004","journal-title":"Commun. ACM"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/2047-217X-2-15","article-title":"Ten recommendations for creating usable bioinformatics command line software","volume":"2","author":"Seemann","year":"2013","journal-title":"GigaScience"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Pavelin, K., Cham, J.A., de Matos, P., Brooksbank, C., Cameron, G., and Steinbeck, C. (2012). Bioinformatics meets user-centred design: A perspective. PLoS Comput. Biol., 8.","DOI":"10.1371\/journal.pcbi.1002554"},{"key":"ref_22","first-page":"33","article-title":"Usability of navigation tools for browsing genetic sequences","volume":"Volume 106","author":"Rutherford","year":"2010","journal-title":"Proceedings of the Eleventh Australasian Conference on User Interface"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Dorn, M., and Norberto de Souza, O. (2008, January 16\u201320). CReF: A central-residue-fragment-based method for predicting approximate 3-D polypeptides structures. Proceedings of the 2008 ACM Symposium on Applied Computing, Fortaleza, Brazil.","DOI":"10.1145\/1363686.1363979"},{"key":"ref_24","unstructured":"Dorn, M., Breda, A., and Norberto de Souza, O. (2008, January 28\u201330). A hybrid method for the protein structure prediction problem. Proceedings of the Brazilian Symposium on Bioinformatics, Santo Andr\u00e9, Brazil."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1504\/IJDMB.2010.033521","article-title":"Mining the Protein Data Bank with CReF to predict approximate 3-D structures of polypeptides","volume":"4","author":"Dorn","year":"2010","journal-title":"Int. J. Data Min. Bioinform."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Machado, V.S., da Silva Tanus, M.D.S., Paix\u00e3o-Cortes, W.R., Norberto de Souza, O., de Borba Campos, M., and Silveira, M.S. (2018). wCReF\u2013A Web Server for the CReF Protein Structure Predictor. Information Technology-New Generations, Springer.","DOI":"10.1007\/978-3-319-54978-1_103"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wu, S., Skolnick, J., and Zhang, Y. (2007). Ab initio modeling of small proteins by iterative TASSER simulations. BMC Biol., 5.","DOI":"10.1186\/1741-7007-5-17"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, Y. (2008). I-TASSER server for protein 3D structure prediction. BMC Bioinform., 9.","DOI":"10.1186\/1471-2105-9-40"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1002\/prot.24341","article-title":"Interplay of I-TASSER and QUARK for template-based and ab initio protein structure prediction in CASP10","volume":"82","author":"Zhang","year":"2014","journal-title":"Proteins Struct. Funct. Bioinform."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"W526","DOI":"10.1093\/nar\/gkh468","article-title":"Protein structure prediction and analysis using the Robetta server","volume":"32","author":"Kim","year":"2004","journal-title":"Nucleic Acids Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1126\/science.181.4096.223","article-title":"Principles that govern the folding of protein chains","volume":"181","author":"Anfinsen","year":"1973","journal-title":"Science"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1186\/s13062-015-0082-5","article-title":"TMFoldWeb: A web server for predicting transmembrane protein fold class","volume":"10","author":"Kozma","year":"2015","journal-title":"Biol. Direct"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1072","DOI":"10.1038\/nbt.2419","article-title":"Protein structure prediction from sequence variation","volume":"30","author":"Marks","year":"2012","journal-title":"Nat. Biotechnol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1590\/S0100-40422003000200019","article-title":"Modelagem de prote\u00ednas por homologia","volume":"26","author":"Alencastro","year":"2003","journal-title":"Qu\u00edmica Nova"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1042","DOI":"10.1126\/science.1219021","article-title":"The protein-folding problem, 50 years on","volume":"338","author":"Dill","year":"2012","journal-title":"Science"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"W171","DOI":"10.1093\/nar\/gkr184","article-title":"The IntFOLD server: An integrated web resource for protein fold recognition, 3D model quality assessment, intrinsic disorder prediction, domain prediction and ligand binding site prediction","volume":"39","author":"Roche","year":"2011","journal-title":"Nucleic Acids Res."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lee, J., Wu, S., and Zhang, Y. (2009). Ab Initio Protein Structure Prediction. From Protein Structure to Function with Bioinformatics, Springer.","DOI":"10.1007\/978-1-4020-9058-5_1"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Cao, R., Freitas, C., Chan, L., Sun, M., Jiang, H., and Chen, Z. (2017). ProLanGO: Protein Function Prediction Using Neural Machine Translation Based on a Recurrent Neural Network. Molecules, 22.","DOI":"10.3390\/molecules22101732"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2568","DOI":"10.1016\/j.eswa.2012.10.062","article-title":"An expert protein loop refinement protocol by molecular dynamics simulations with restraints","volume":"40","year":"2013","journal-title":"Expert Syst. Appl."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Moult, J., Fidelis, K., Kryshtafovych, A., Schwede, T., and Tramontano, A. (2017). Critical Assessment of Methods of Protein Structure Prediction (CASP)\u2013Round XII. Proteins Struct. Funct. Bioinform.","DOI":"10.1002\/prot.25415"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1186\/s13059-016-1037-6","article-title":"An expanded evaluation of protein function prediction methods shows an improvement in accuracy","volume":"17","author":"Jiang","year":"2016","journal-title":"Genome Biol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1021\/acs.jchemed.5b00404","article-title":"RCSB Protein Data Bank: A Resource for Chemical, Biochemical, and Structural Explorations of Large and Small Biomolecules","volume":"93","author":"Zardecki","year":"2016","journal-title":"Chem. Educ."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1093\/nar\/28.1.235","article-title":"The Protein Data Bank","volume":"28","author":"Berman","year":"2000","journal-title":"Nucleic Acids Res."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1016\/S0022-2836(05)80360-2","article-title":"Basic local alignment search tool","volume":"215","author":"Altschul","year":"1990","journal-title":"J. Mol. Biol."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/S0164-1212(98)10026-2","article-title":"Human\u2013computer interaction: Interdisciplinary roots and trends","volume":"43","author":"Hartson","year":"1998","journal-title":"J. Syst. Softw."},{"key":"ref_46","unstructured":"Sharp, H., Rogers, Y., and Preece, J. (2007). Interaction Design: Beyond Human-Computer Interaction, Wiley."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Nielsen, J. (1994, January 24\u201328). Usability inspection methods. Proceedings of the Conference Companion on Human Factors in Computing Systems, Boston, MA, USA.","DOI":"10.1145\/259963.260531"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Nielsen, J. (1994). Usability Engineering, Elsevier.","DOI":"10.1016\/B978-0-08-052029-2.50007-3"},{"key":"ref_49","unstructured":"Winckler, M.A.A., Farenc, C., Palanque, P., and Pimenta, M.S. (2001, January 15\u201317). Avalia\u00e7\u00e3o da Navega\u00e7\u00e3o de Interfaces Web a partir de Modelos. Proceedings of the Workshop Sobre Fatores Humanos Em Sistemas Computacionais, Florian\u00f3polis, Brazil."},{"key":"ref_50","first-page":"1","article-title":"Realce das normas e padr\u00f5es: A usabilidade como fator primordial para a boa interatividade do usu\u00e1rio","volume":"1","author":"Oliveira","year":"2013","journal-title":"Cad. Ci\u00eanc. Hum. Soc. Apl."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"10080","DOI":"10.1073\/pnas.94.19.10080","article-title":"Structural mimicry of a native protein by a minimized binding domain","volume":"94","author":"Starovasnik","year":"1997","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Li, H., Leung, K.S., Nakane, T., and Wong, M.H. (2014). iview: An interactive WebGL visualizer for protein-ligand complex. BMC Bioinform., 15.","DOI":"10.1186\/1471-2105-15-56"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"2460","DOI":"10.1093\/bioinformatics\/bth248","article-title":"POLYVIEW: A flexible visualization tool for structural and functional annotations of proteins","volume":"20","author":"Porollo","year":"2004","journal-title":"Bioinformatics"},{"key":"ref_54","unstructured":"Ssemugabi, S. (2009). Usability Evaluation of a Web-Based E-Learning Application: A Study of Two Evaluation Methods. [Ph.D. Thesis, University of South Africa (UNISA)]."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Ssemugabi, S., and De Villiers, R. (2007, January 2\u20133). A comparative study of two usability evaluation methods using a web-based e-learning application. Proceedings of the 2007 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists on IT Research in Developing Countries, Port Elizabeth, South Africa.","DOI":"10.1145\/1292491.1292507"},{"key":"ref_56","unstructured":"Sahakian, W.S. (1972). Likert technique for attitude measurement. Social Psychology: Experimentation, Theory, Research, Intext Educational Publishers."},{"key":"ref_57","first-page":"189","article-title":"Avalia\u00e7\u00e3o de usabilidade de sites web","volume":"3","author":"Campos","year":"2012","journal-title":"Rev. Caminhos"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/9\/1\/20\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:51:38Z","timestamp":1760194298000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/9\/1\/20"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,1,17]]},"references-count":57,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2018,1]]}},"alternative-id":["info9010020"],"URL":"https:\/\/doi.org\/10.3390\/info9010020","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,1,17]]}}}