{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T09:40:54Z","timestamp":1763458854991,"version":"3.45.0"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2016,5,24]],"date-time":"2016-05-24T00:00:00Z","timestamp":1464048000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation","award":["1330596"],"award-info":[{"award-number":["1330596"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2016,8,22]]},"abstract":"<jats:p>Privacy policies describe high-level goals for corporate data practices; regulators require industries to make available conspicuous, accurate privacy policies to their customers. Consequently, software requirements must conform to those privacy policies. To help stakeholders extract privacy goals from policies, we introduce a semiautomated framework that combines crowdworker annotations, natural language typed dependency parses, and a reusable lexicon to improve goal-extraction coverage, precision, and recall. The framework evaluation consists of a five-policy corpus governing web and mobile information systems, yielding an average precision of 0.73 and recall of 0.83. The results show that no single framework element alone is sufficient to extract goals; however, the overall framework compensates for elemental limitations. Human annotators are highly adaptive at discovering annotations in new texts, but those annotations can be inconsistent and incomplete; dependency parsers lack sophisticated, tacit knowledge, but they can perform exhaustive text search for prospective requirements indicators; and while the lexicon may never completely saturate, the lexicon terms can be reliably used to improve recall. Lexical reuse reduces false negatives by 41%, increasing the average recall to 0.85. Last, crowd workers were able to identify and remove false positives by around 80%, which improves average precision to 0.93.<\/jats:p>","DOI":"10.1145\/2907942","type":"journal-article","created":{"date-parts":[[2016,5,25]],"date-time":"2016-05-25T14:07:06Z","timestamp":1464185226000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":38,"title":["Mining Privacy Goals from Privacy Policies Using Hybridized Task Recomposition"],"prefix":"10.1145","volume":"25","author":[{"given":"Jaspreet","family":"Bhatia","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, Pennsylvania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Travis D.","family":"Breaux","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, Pennsylvania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Florian","family":"Schaub","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, Pennsylvania"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2016,5,24]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/182.358441"},{"key":"e_1_2_1_2_1","unstructured":"Annie I. Ant\u00f3n. 1997. Goal Identification and Refinement in the Specification of Information Systems. Ph.D. Thesis. Georgia Institute of Technology Atlanta GA."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-003-0183-z"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2013.20"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531653"},{"key":"e_1_2_1_6_1","volume-title":"Krieger","author":"Berry Daniel M.","year":"2003","unstructured":"Daniel M. Berry, Erik Kamsties, and Michael M. Krieger. 2003. From Contract Drafting to Software Specification: Linguistic Sources of Ambiguity. Technical Report. University of Waterloo, Waterloo, ON."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-28714-5_18"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/RELAW.2015.7330207"},{"volume-title":"21st IEEE International Requirements Engineering Conference (RE'13)","author":"Travis","key":"e_1_2_1_9_1","unstructured":"Travis D. Breaux and Ashwini Rao. 2013. Formal analysis of privacy requirements specifications for multi-tier applications. In 21st IEEE International Requirements Engineering Conference (RE'13). Rio de Janeiro, Brazil, 14--23."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2014.6912258"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1416563.1416565"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-013-0190-7"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2015.7320419"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2339811"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15516709cog0502_2"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/0167-6423(93)90021-G"},{"volume-title":"Proceedings of International Conference on Language Resources and Evaluation, 449--454","author":"de Marneffe Marie-Catherine","key":"e_1_2_1_17_1","unstructured":"Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of International Conference on Language Resources and Evaluation, 449--454."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-41154-0_1"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-12206-9_22"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324900000048"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-28244-0_11"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047202"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the 19th International Conference on Automated Software Engineering Workshops, 91--102","author":"Kof Leonid","year":"2004","unstructured":"Leonid Kof. 2004. Natural language processing for requirements engineering: Applicability to large requirements documents. In Proceedings of the 19th International Conference on Automated Software Engineering Workshops, 91--102."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2011.36"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2370216.2370290"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2014.27"},{"key":"e_1_2_1_27_1","first-page":"540","article-title":"The cost of reading privacy policies","volume":"4","author":"McDonald Aleecia M.","year":"2008","unstructured":"Aleecia M. McDonald and Lorrie F. Cranor. 2008. The cost of reading privacy policies. I\/S: A Journal of Law and Policy for the Information Society 4, 3, 540--565.","journal-title":"I\/S: A Journal of Law and Policy for the Information Society"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2007.126"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-89778-1_10"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.datak.2015.07.007"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979148"},{"key":"e_1_2_1_32_1","first-page":"39","article-title":"Disagreeable privacy policies: Mismatches between meaning and users\u2019 understanding","volume":"30","author":"Reidenberg J. R.","year":"2015","unstructured":"J. R. Reidenberg, T. D. Breaux, L. F. Cranor, B. French, A. Grannis, J. T. Graves, F. Liu, A. M. McDonald, T. B. Norton, R. Ramanath, N. C. Russell, N. Sadeh, and F. Schaub. 2015. Disagreeable privacy policies: Mismatches between meaning and users\u2019 understanding. Berkeley Technology Law Journal 30, 1, 39--88.","journal-title":"Berkeley Technology Law Journal"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0285(76)90013-X"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2362456.2362479"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1037\/0278-7393.29.4.641"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.5555\/1613715.1613751"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0285(91)90016-H"},{"volume-title":"2nd AAAI Conference on Human Computation and Crowdsourcing, 210--219","author":"Verroios Vasilis","key":"e_1_2_1_38_1","unstructured":"Vasilis Verroios and Michael S. Bernstein. 2014. Context trees: Crowdsourcing global understanding from local views. In 2nd AAAI Conference on Human Computation and Crowdsourcing, 210--219."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2207709"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1858996.1859007"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-011-0119-y"},{"key":"e_1_2_1_42_1","volume-title":"Case study research","author":"Yin K.","unstructured":"Robert. K. Yin. 2009. Case study research (4th ed.). In Applied Social Research Methods Series, Vol. 5. Sage Publications.","edition":"4"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2492517.2492640"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2907942","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2907942","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2907942","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T09:36:57Z","timestamp":1763458617000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2907942"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,5,24]]},"references-count":43,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2016,8,22]]}},"alternative-id":["10.1145\/2907942"],"URL":"https:\/\/doi.org\/10.1145\/2907942","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"type":"print","value":"1049-331X"},{"type":"electronic","value":"1557-7392"}],"subject":[],"published":{"date-parts":[[2016,5,24]]},"assertion":[{"value":"2015-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-03-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-05-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}