{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,4]],"date-time":"2025-07-04T05:17:43Z","timestamp":1751606263120,"version":"3.41.0"},"reference-count":77,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2021,4,13]],"date-time":"2021-04-13T00:00:00Z","timestamp":1618272000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100007059","name":"Northwestern University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007059","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS 153083"],"award-info":[{"award-number":["IIS 153083"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100026024","name":"Adobe Research","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100026024","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,4,13]]},"abstract":"<jats:p>Freelancers increasingly earn their livelihood through online marketplaces. To attract new clients, freelancers continuously curate their online portfolios to convey their unique skills and style. However, many lack access to rapid, regular, and inexpensive feedback needed to improve their portfolios. Existing crowd feedback systems, which collect feedback on individual creative projects (i.e., decomposed approach), could fill this need, but it is unclear how they might support feedback on multiple projects (i.e., holistic approach). In a between-subjects study with 30 freelancers, we compared decomposed and holistic feedback collection approaches using CrowdFolio, a crowd feedback system for portfolios. The holistic approach helped freelancers discover new ways to describe their work, while the decomposed approach provided detailed insight about the visual attractiveness of projects. This study contributes evidence that portfolio feedback systems, regardless of collection approach, can positively support professional development by impacting how freelancers portray themselves online and reflect on their identity.<\/jats:p>","DOI":"10.1145\/3449096","type":"journal-article","created":{"date-parts":[[2021,4,22]],"date-time":"2021-04-22T17:47:23Z","timestamp":1619113643000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["CrowdFolio"],"prefix":"10.1145","volume":"5","author":[{"given":"Eureka","family":"Foong","sequence":"first","affiliation":[{"name":"University of Tokyo, Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joy O.","family":"Kim","sequence":"additional","affiliation":[{"name":"Adobe Research, San Francisco, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mira","family":"Dontcheva","sequence":"additional","affiliation":[{"name":"Adobe Research, Seattle, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elizabeth M.","family":"Gerber","sequence":"additional","affiliation":[{"name":"Northwestern University, Evanston, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,4,22]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"[n.d.]. Amazon Mechanical Turk. https:\/\/www.mturk.com\/"},{"key":"e_1_2_2_2_1","unstructured":"[n.d.]. Grammarly. https:\/\/www.grammarly.com\/ Library Catalog: www.grammarly.com."},{"key":"e_1_2_2_3_1","unstructured":"[n.d.]. stringr package | R Documentation. https:\/\/www.rdocumentation.org\/packages\/stringr\/versions\/1.4.0"},{"key":"e_1_2_2_4_1","unstructured":"2018. Behance:: Best of Behance. https:\/\/www.behance.net\/"},{"key":"e_1_2_2_5_1","unstructured":"2018. Dribbble. https:\/\/dribbble.com\/"},{"key":"e_1_2_2_6_1","unstructured":"2018. Facebook. https:\/\/www.facebook.com\/"},{"key":"e_1_2_2_7_1","unstructured":"2018. reddit. https:\/\/www.reddit.com\/"},{"key":"e_1_2_2_8_1","unstructured":"2019. Upwork. https:\/\/www.upwork.com\/"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2017.23"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(02)00056-X"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819943"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531653"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/0030-5073(75)90028-8"},{"volume-title":"A Guide to Creating Your Design Portfolio","author":"Barrett Carlie","key":"e_1_2_2_15_1","unstructured":"Carlie Barrett, Alex Broerman, Emmanuel Carillo, Morgen Depenthal, Kevin Doyle, Allison Dunphy, Matt Gill, Natalie Miklosic, Samantha Gustafson, Mei Hsieh, Aaron Kurosu, Rachel Lang, Andrew Leemhuis, Jon Panichella, Casey Schneider, James Sloss, Steve Speaker, Aaron Watkins, and Jacklyn Woniger. 2011. Hire Me?! The Portfolio Handbook: A Guide to Creating Your Design Portfolio. University of Cincinnati, Cincinnati, OH. http:\/\/www.blurb.com\/b\/2516994-hire-me-the-portfolio-handbook"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173868"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858411"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2466265"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2005.02.004"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24770-0_19"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0026256"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2684822.2685286"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.20429\/ijsotl.2008.020212"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025659"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196709.3196770"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1108\/13620430510620629"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470686"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1879831.1879836"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097286.3097291"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.14742\/ajet.2464"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1083-6101.2006.00020.x"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858535"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025791"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134680"},{"key":"e_1_2_2_35_1","unstructured":"The Python Software Foundation. 2020. difflib - Helpers for computing deltas - Python 3.7.7 documentation. https:\/\/docs.python.org\/3.7\/library\/difflib.html"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1177\/1470593115607942"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2757226.2757249"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2007.06.039"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0013899"},{"volume-title":"Develop Innovative Ideas, and Design Effective Solutions","author":"Hanington Bruce","key":"e_1_2_2_40_1","unstructured":"Bruce Hanington and Bella Martin. 2019. Universal Methods of Design Expanded and Revised: 125 Ways to Research Complex Problems, Develop Innovative Ideas, and Design Effective Solutions. Rockport Publishers. Google-Books-ID: SFnBDwAAQBAJ."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174023"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1177\/0265407510361614"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.poetic.2009.10.001"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.4324\/9780203855881"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1108\/TG-09-2013-0035"},{"key":"e_1_2_2_46_1","unstructured":"Behance Inc. 2012a. 6 Steps To Creating A Knockout Online Portfolio. https:\/\/99u.adobe.com\/articles\/7127\/6-steps-to-creating-a-knockout-online-portfolio"},{"key":"e_1_2_2_47_1","unstructured":"Behance Inc. 2012b. A Portfolio for Clients: A Beginner's Guide to Getting More Gigs. https:\/\/99u.adobe.com\/articles\/52662\/a-portfolio-for-clients-a-beginners-guide-to-getting-more-gigs Library Catalog: 99u.adobe.com."},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-019-09368-7"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174180"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998196"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441923"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047202"},{"key":"e_1_2_2_53_1","unstructured":"Gary Klein Jennifer K Phillips Erica L Rall and Deborah Peluso. 2007. A data-frame theory of sensemaking. In Expertise out of context: Proceedings of the Sixth International Conference on Naturalistic Decision Making. http:\/\/books.google.com\/books?hl=en&lr=&id=GQK8mccWlVoC&oi=fnd&pg=PA113&dq=A+data+frame+theoryofsensemaking&ots=cXazVlI4PI&sig=Amx9bbxRp9g_J6eXwJzcY6LyKUU"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134695"},{"key":"e_1_2_2_55_1","article-title":"5 Revision of the Self-Monitoring Scale","volume":"5","author":"Lennox Richard D","year":"1984","unstructured":"Richard D Lennox and Raymond N Wolfe. 1984. 5 Revision of the Self-Monitoring Scale. Journal of Personality and Social Psychology 5, Vol. 46, 6 (1984), 1349--1364. 55http:\/\/www.communicationcache.com\/uploads\/1\/0\/8\/8\/10887248\/revision_of_the_self-monitoring_scale..pdf","journal-title":"Journal of Personality and Social Psychology"},{"volume-title":"Portfolio Design","author":"Linton Harold","key":"e_1_2_2_56_1","unstructured":"Harold Linton. 2012. Portfolio Design. W.W. Norton & Company. Google-Books-ID: ZEQppwAACAAJ."},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675283"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1016\/0030-5073(82)90240-9"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531651"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0285(77)90012-3"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858169"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173629"},{"volume-title":"Oh sh*t what now?: honest advice for new graphic designers","author":"Oldham Craig","key":"e_1_2_2_63_1","unstructured":"Craig Oldham. 2017. Oh sh*t what now?: honest advice for new graphic designers. Laurence King Publishing Ltd, London. OCLC: 1029212644."},{"key":"e_1_2_2_64_1","unstructured":"O*NET OnLine. 2020. Summary Report for: 27--1024.00 - Graphic Designers. https:\/\/www.onetonline.org\/link\/summary\/27--1024.00"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4020-8905-3_4"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3177882"},{"volume-title":"The reflective practitioner. Temple Smith","author":"Sch\u00f6n Donald","key":"e_1_2_2_67_1","unstructured":"Donald Sch\u00f6n. 1983. The reflective practitioner. Temple Smith, London. http:\/\/scholar.google.com\/scholar?q=related:im1nTiraZZAJ:scholar.google.com\/&hl=en&num=20&as_sdt=0,5"},{"key":"e_1_2_2_68_1","volume-title":"Corbin","author":"Strauss Anselm","year":"1997","unstructured":"Anselm Strauss and Juliet M. Corbin. 1997. Grounded Theory in Practice. SAGE. Google-Books-ID: TtRMolAapBYC."},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1177\/0950017019886511"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/1124772.1124960"},{"volume-title":"The Think Aloud Method","author":"van Someren Maarten W","key":"e_1_2_2_71_1","unstructured":"Maarten W van Someren, Yvonne F Barnard, and Jacobijn A C Sandberg. 1994. The Think Aloud Method. Academic Press, London. http:\/\/books.google.com\/books?id=lnp9AAAAMAAJ&q=intitle: THE+THINK+ALOUD+METHOD+A+practical+guide+to+modelling+cognitive+processes&dq=intitle:THE+ THINK+ALOUD+METHOD+A+practical+guide+to+modelling+cognitive+processes&hl=&cd=1 source=gbs_api"},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531604"},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675140"},{"key":"e_1_2_2_74_1","volume-title":"Proceedings of the 2016 ACM Conference on Designing Interactive Systems. ACM Press, 773--784","author":"Grace Yen Yu-Chun","year":"2016","unstructured":"Yu-Chun Grace Yen, Steven Dow, Elizabeth M Gerber, and Brian P Bailey. 2016. Social network, web forum, or task market? Comparing different crowd genres for design feedback exchange. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. ACM Press, 773--784. \/\/www.google.com\/intl\/en_us\/mail\/help\/about.html"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1037\/xge0000564"},{"key":"e_1_2_2_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819953"},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2207708"},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240704"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449096","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3449096","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3449096","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:46Z","timestamp":1750191466000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449096"}},"subtitle":["Understanding How Holistic and Decomposed Workflows Influence Feedback on Online Portfolios"],"short-title":[],"issued":{"date-parts":[[2021,4,13]]},"references-count":77,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2021,4,13]]}},"alternative-id":["10.1145\/3449096"],"URL":"https:\/\/doi.org\/10.1145\/3449096","relation":{},"ISSN":["2573-0142"],"issn-type":[{"type":"electronic","value":"2573-0142"}],"subject":[],"published":{"date-parts":[[2021,4,13]]},"assertion":[{"value":"2021-04-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}