{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T02:56:24Z","timestamp":1776394584434,"version":"3.51.2"},"reference-count":23,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2015,8]]},"abstract":"<jats:p>While there has been significant focus on collecting and managing data feeds, it is only now that attention is turning to their quality. In this paper, we propose a principled approach to online data quality monitoring in a dynamic feed environment. Our goal is to alert quickly when feed behavior deviates from expectations.<\/jats:p><jats:p>We make contributions in two distinct directions. First, we propose novel enhancements to permit a publish-subscribe approach to incorporate data quality modules into the DFMS architecture. Second, we propose novel temporal extensions to standard statistical techniques to adapt them to online feed monitoring for outlier detection and alert generation at<jats:italic>multiple scales<\/jats:italic>along three dimensions: aggregation at multiple time intervals to detect at varying levels of sensitivity; multiple lengths of data history for varying the speed at which models adapt to change; and multiple levels of monitoring delay to address lagged data arrival.<\/jats:p><jats:p>FIT, or Feed Inspection Tool, is the result of a successful implementation of our approach. We present several case studies outlining the effective deployment of FIT in real applications along with user testimonials.<\/jats:p>","DOI":"10.14778\/2824032.2824070","type":"journal-article","created":{"date-parts":[[2015,9,16]],"date-time":"2015-09-16T12:18:17Z","timestamp":1442405897000},"page":"1728-1739","source":"Crossref","is-referenced-by-count":3,"title":["FIT to monitor feed quality"],"prefix":"10.14778","volume":"8","author":[{"given":"Tamraparni","family":"Dasu","sequence":"first","affiliation":[{"name":"AT&amp;T Labs, Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vladislav","family":"Shkapenyuk","sequence":"additional","affiliation":[{"name":"AT&amp;T Labs, Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Divesh","family":"Srivastava","sequence":"additional","affiliation":[{"name":"AT&amp;T Labs, Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Deborah F.","family":"Swayne","sequence":"additional","affiliation":[{"name":"AT&amp;T Labs, Research"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2015,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"IBM InfoSphere DataStage. http:\/\/www.ibm.com\/software\/products\/en\/ibminfodata. IBM InfoSphere DataStage. http:\/\/www.ibm.com\/software\/products\/en\/ibminfodata."},{"key":"e_1_2_1_2_1","unstructured":"Informatica PowerCenter. http:\/\/www.informatica.com\/in\/etl\/. Informatica PowerCenter. http:\/\/www.informatica.com\/in\/etl\/."},{"key":"e_1_2_1_3_1","unstructured":"Oracle Data Integrator. http:\/\/www.oracle.com\/us\/products\/middleware\/data-integration\/enterprise-edition\/overview\/index.html. Oracle Data Integrator. http:\/\/www.oracle.com\/us\/products\/middleware\/data-integration\/enterprise-edition\/overview\/index.html."},{"key":"e_1_2_1_4_1","unstructured":"SAP BusinessObjects Data Services. http:\/\/www.sap.com\/pc\/tech\/enterprise-information-management\/software\/data-integrator\/index.html. SAP BusinessObjects Data Services. http:\/\/www.sap.com\/pc\/tech\/enterprise-information-management\/software\/data-integrator\/index.html."},{"key":"e_1_2_1_5_1","volume-title":"Methodologies and Techniques. Data-Centric Systems and Applications. Springer","author":"Batini C.","year":"2006"},{"key":"e_1_2_1_6_1","first-page":"21","volume-title":"ICDM","author":"Berti-Equille L.","year":"2013"},{"issue":"1","key":"e_1_2_1_7_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/qre.1759","article-title":"Editorial to the special issue: Nonparametric statistical process control charts","volume":"31","author":"Chakraborti S.","year":"2015","journal-title":"Quality and Reliability Eng. Int."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2011.31"},{"key":"e_1_2_1_9_1","unstructured":"W. G. Cochran. Sampling Techniques. John Wiley & Sons 1977. W. G. Cochran. Sampling Techniques. John Wiley & Sons 1977."},{"key":"e_1_2_1_10_1","volume-title":"John Wiley","author":"Dasu T.","year":"2003"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350279"},{"key":"e_1_2_1_12_1","first-page":"572","volume-title":"KDD","author":"Dasu T.","year":"2014"},{"key":"e_1_2_1_13_1","volume-title":"Synthesis Lectures on Data Management. Morgan & Claypool Publishers","author":"Dong X. L.","year":"2015"},{"key":"e_1_2_1_14_1","volume-title":"Synthesis Lectures on Data Management. Morgan & Claypool Publishers","author":"Fan W.","year":"2012"},{"key":"e_1_2_1_15_1","first-page":"8","volume-title":"BNCOD","author":"Fan W.","year":"2009"},{"issue":"3","key":"e_1_2_1_16_1","first-page":"26","article-title":"Efficient and effective analysis of data quality using pattern tableaux","volume":"34","author":"Golab L.","year":"2011","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_1_17_1","first-page":"605","volume-title":"EDBT","author":"Grover R.","year":"2015"},{"key":"e_1_2_1_18_1","unstructured":"J. D. Hamilton. Time series analysis volume 2. Princeton university press Princeton 1994. J. D. Hamilton. Time series analysis volume 2. Princeton university press Princeton 1994."},{"key":"e_1_2_1_19_1","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1080\/10618600.1996.10474713","article-title":"R: A language for data analysis and graphics","volume":"5","author":"Ihaka R.","year":"1996","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/269012.269025"},{"key":"e_1_2_1_21_1","first-page":"1059","volume-title":"SIGMOD","author":"Shkapenyuk V.","year":"2011"},{"issue":"6","key":"e_1_2_1_22_1","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1080\/10485250215317","article-title":"A nonparametric confidence interval for the trimmed mean","volume":"14","author":"Tsao M.","year":"2002","journal-title":"Nonparametric Statistics"},{"key":"e_1_2_1_23_1","first-page":"244","volume-title":"ICDE","author":"Volkovs M.","year":"2014"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/2824032.2824070","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,11]],"date-time":"2024-06-11T05:06:20Z","timestamp":1718082380000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/2824032.2824070"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,8]]},"references-count":23,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2015,8]]}},"alternative-id":["10.14778\/2824032.2824070"],"URL":"https:\/\/doi.org\/10.14778\/2824032.2824070","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2015,8]]}}}