{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:08:37Z","timestamp":1750306117326,"version":"3.41.0"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2017,4,26]],"date-time":"2017-04-26T00:00:00Z","timestamp":1493164800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CAREER CNS-1350941,CNS-1320071"],"award-info":[{"award-number":["CAREER CNS-1350941,CNS-1320071"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006228","name":"Oak Ridge National Laboratory","doi-asserted-by":"publisher","award":["DE-AC05-00OR22725"],"award-info":[{"award-number":["DE-AC05-00OR22725"]}],"id":[{"id":"10.13039\/100006228","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Model. Perform. Eval. Comput. Syst."],"published-print":{"date-parts":[[2017,6,30]]},"abstract":"<jats:p>Online data-intensive (OLDI) services use anytime algorithms to compute over large amounts of data and respond quickly. Interactive response times are a priority, so OLDI services parallelize query execution across distributed software components and return best effort answers based on the data so far processed. Omitted data from slow components could lead to better answers, but tracing online how much better the answers could be is difficult. We propose Ubora, a design approach to measure the effect of slow-running components on the quality of answers. Ubora randomly samples online queries and executes them a second time. The first online execution omits data from slow components and provides interactive answers. The second execution uses mature results from intermediate components completed after the online execution finishes. Ubora uses memoization to speed up mature executions by replaying network messages exchanged between components. Our systems-level implementation works for a wide range of services, including Hadoop\/Yarn, Apache Lucene, the EasyRec Recommendation Engine, and the OpenEphyra question-answering system. Ubora computes answer quality with more mature executions per second than competing approaches that do not use memoization. With Ubora, we show that answer quality is effective at guiding online admission control. While achieving the same answer quality on high-priority queries, our adaptive controller had 55% higher peak throughput on low-priority queries than a competing controller guided by the rate of timeouts.<\/jats:p>","DOI":"10.1145\/3055280","type":"journal-article","created":{"date-parts":[[2017,4,28]],"date-time":"2017-04-28T12:38:23Z","timestamp":1493383103000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Obtaining and Managing Answer Quality for Online Data-Intensive Services"],"prefix":"10.1145","volume":"2","author":[{"given":"Jaimie","family":"Kelley","sequence":"first","affiliation":[{"name":"Ohio State University, Columbus, OH"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christopher","family":"Stewart","sequence":"additional","affiliation":[{"name":"Ohio State University, Columbus, OH"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nathaniel","family":"Morris","sequence":"additional","affiliation":[{"name":"Ohio State University, Columbus, OH"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Devesh","family":"Tiwari","sequence":"additional","affiliation":[{"name":"Northeastern University, Boston,MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuxiong","family":"He","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond,WA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sameh","family":"Elnikety","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond,WA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,4,26]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458109"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICAC.2015.47"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1088149.1088185"},{"volume-title":"Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201912)","year":"2012","author":"Attariyan Mona","key":"e_1_2_1_4_1"},{"volume-title":"Proceedings of the International Conference on Very Large Data Bases (VLDB\u201913)","year":"2013","author":"Barykin Oleksandr","key":"e_1_2_1_5_1"},{"volume-title":"Retrieved","year":"2017","author":"Falsett R.","key":"e_1_2_1_6_1"},{"key":"e_1_2_1_7_1","article-title":"The AI behind Watson-the technical article","author":"Ferrucci D.","year":"2010","journal-title":"AI Magazine. Retrieved"},{"volume-title":"Proceedings of the 4th USENIX Conference on Networked Systems Design and Implementation (NSDI\u201907)","year":"2007","author":"Fonseca Rodrigo","key":"e_1_2_1_8_1"},{"volume-title":"Retrieved","year":"2009","author":"Forrest B.","key":"e_1_2_1_9_1"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/98.709365"},{"volume-title":"Proceedings of the 2014 International Conference on Autonomic Computing.","year":"2014","author":"Gandhi Anshul","key":"e_1_2_1_11_1"},{"volume-title":"Retrieved","year":"2011","author":"Gelfond A.","key":"e_1_2_1_12_1"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463712"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2694344.2694351"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2013.12"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2391229.2391241"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2011.91"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2371536.2371557"},{"volume-title":"Retrieved","year":"2016","author":"Intel Corporation","key":"e_1_2_1_19_1"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2486001.2486028"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2465351.2465367"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2014.6816674"},{"volume-title":"Proceedings of the Workshop on Large-Scale Distributed Systems and Middleware.","author":"Kelley J.","key":"e_1_2_1_23_1"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICAC.2015.33"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICAC.2015.16"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213840"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2371536.2371547"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2005.304"},{"volume-title":"Retrieved","year":"2011","author":"Lenchner Jon","key":"e_1_2_1_29_1"},{"key":"e_1_2_1_30_1","unstructured":"Lucid Imagination. 2010. The Case for Lucene\/Solr: Real World Search Applications. White Paper. Lucid Imagination.  Lucid Imagination. 2010. The Case for Lucene\/Solr: Real World Search Applications. White Paper. Lucid Imagination."},{"volume-title":"Introduction to Information Retrieval","author":"Manning Christopher","key":"e_1_2_1_31_1"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1508244.1508269"},{"volume-title":"Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XIV). 205--216","author":"Meisner D.","key":"e_1_2_1_33_1"},{"volume-title":"Proceedings of the 2016 Spring Undergraduate Research Expo.","year":"2016","author":"Muhammad Stephanie","key":"e_1_2_1_34_1"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2517349.2522738"},{"volume-title":"Retrieved","year":"2009","key":"e_1_2_1_36_1"},{"volume-title":"Proceedings of the 2013 International Conference on Autonomic Computing.","year":"2013","author":"Paiva Joao","key":"e_1_2_1_37_1"},{"volume-title":"Retrieved","year":"2016","key":"e_1_2_1_38_1"},{"volume-title":"Proceedings of the 2013 International Conference on Autonomic Computing.","year":"2013","author":"Ren Shaolei","key":"e_1_2_1_39_1"},{"volume-title":"Retrieved","year":"2014","author":"Austria Research Studios","key":"e_1_2_1_40_1"},{"volume-title":"Retrieved","year":"2013","author":"Schlaefer Nico","key":"e_1_2_1_41_1"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2451116.2451124"},{"volume-title":"Google Technical Report. Retrieved","year":"2017","author":"Sigelman B.","key":"e_1_2_1_43_1"},{"volume-title":"Proceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201909)","year":"2009","author":"Song Yee Jiun","key":"e_1_2_1_44_1"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568088.2576093"},{"volume-title":"Zoolander: Efficiently meeting very strict, low-latency SLOs. In Proceedings of the 2013 International Conference on Autonomic Computing.","year":"2013","author":"Stewart Christopher","key":"e_1_2_1_46_1"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1272996.1273002"},{"volume-title":"Text Retrieval Conference Data. Retrieved","year":"2014","author":"Technology Laboratory\u2019s (ITL) Retrieval Group","key":"e_1_2_1_48_1"},{"volume-title":"Retrieved","year":"2015","author":"Software Foundation The Apache","key":"e_1_2_1_49_1"},{"volume-title":"Retrieved","year":"2016","author":"Software Foundation The Apache","key":"e_1_2_1_50_1"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1735970.1736051"},{"volume-title":"Retrieved","year":"2014","author":"Foundation Wikimedia","key":"e_1_2_1_52_1"},{"volume-title":"Retrieved","year":"2014","author":"Wikipedia","key":"e_1_2_1_53_1"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/CLOUD.2015.47"},{"key":"e_1_2_1_55_1","first-page":"73","article-title":"Using anytime algorithms in intelligent systems","volume":"17","author":"Zilberstein Shlomo","year":"1996","journal-title":"AI Magazine"}],"container-title":["ACM Transactions on Modeling and Performance Evaluation of Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3055280","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3055280","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3055280","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:36:37Z","timestamp":1750217797000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3055280"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,4,26]]},"references-count":55,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2017,6,30]]}},"alternative-id":["10.1145\/3055280"],"URL":"https:\/\/doi.org\/10.1145\/3055280","relation":{},"ISSN":["2376-3639","2376-3647"],"issn-type":[{"type":"print","value":"2376-3639"},{"type":"electronic","value":"2376-3647"}],"subject":[],"published":{"date-parts":[[2017,4,26]]},"assertion":[{"value":"2016-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-02-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-04-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}