{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T04:31:58Z","timestamp":1777696318694,"version":"3.51.4"},"reference-count":22,"publisher":"SAGE Publications","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IDA"],"published-print":{"date-parts":[[2023,1,30]]},"abstract":"<jats:p>We propose a new method of constructing a variable bin width histogram that can accommodate the unbalanced distribution of the samples yet retaining, as a whole, the good aspect of both equal width (EW) and equal-area (EA) histograms that are being used popularly for data visualization and analysis. We formulate this as an optimal change point detection problem in which the bin boundaries are determined by minimizing the sum of the absolute error or the squared error in each bin. The former is based on Distance Minimization (DM) and new, and the latter is based on Variance Minimization (VM) and is considered the state-of-the-art. The constructed histograms can effectively be used to detect and visualize hidden outliers\/anomalies by applying the interquartile range method in each bin. The final histograms are obtained by adjusting bin boundaries and heights accordingly after removing the detected outliers\/anomalies. We further propose a method to annotate the constructed bins if the data for annotation is given for each sample as a set of nominal variables, using z-score with respect to their distribution within each bin. We applied our method to both real vinyl greenhouse datasets and two different sets of three synthetic datasets, and confirmed that both DM and VM methods work as intended, both can represent the sample distribution with a smaller number of bins than those by EW and EA methods, The use of interquartile range method can detect anomalies as well as outliers, and the terms selected for annotation are interpretable and reasonable. EW and EA methods have contrasting properties. DM and VM methods lie in between, but the former is closer to EA method and the latter to EW method. DM method runs substantially faster than VM method and performs slightly better than VM method in outlier detection and annotation tasks.<\/jats:p>","DOI":"10.3233\/ida-216316","type":"journal-article","created":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T11:18:06Z","timestamp":1675163886000},"page":"5-29","source":"Crossref","is-referenced-by-count":1,"title":["Constructing outlier-free histograms with variable bin-width based on distance minimization"],"prefix":"10.1177","volume":"27","author":[{"given":"Takayasu","family":"Fushimi","sequence":"first","affiliation":[{"name":"School of Computer Science, Tokyo University of Technology, Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kazumi","family":"Saito","sequence":"additional","affiliation":[{"name":"Faculty of Science, Kanagawa University, Kanagawa, Japan"},{"name":"Center for Advanced Intelligence Project, RIKEN, Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hiroshi","family":"Motoda","sequence":"additional","affiliation":[{"name":"Institute of Scientific and Industrial Research, Osaka University, Osaka, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"key":"10.3233\/IDA-216316_ref1","doi-asserted-by":"crossref","unstructured":"L. Araujo, J.A. Cuesta and J.J. Merelo, Genetic algorithm for burst detection and activity tracking in event streams, In Proceedings of the 9th International Conference on Parallel Problem Solving from Nature (PPSN\u201906), 2006, pp.\u00a0302\u2013311.","DOI":"10.1007\/11844297_31"},{"key":"10.3233\/IDA-216316_ref2","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1198\/jcgs.2009.0002","article-title":"Variations on the histogram","volume":"18","author":"Denby","year":"2009","journal-title":"Journal of Computational and Graphical Statistics \u2013 J COMPUT GRAPH STAT"},{"key":"10.3233\/IDA-216316_ref3","doi-asserted-by":"crossref","unstructured":"R. Ebina, K. Nakamura and S. Oyanagi, A real-time burst detection method, In Proceedings of the 23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI), 2011, pp.\u00a01040\u20131046.","DOI":"10.1109\/ICTAI.2011.177"},{"issue":"284","key":"10.3233\/IDA-216316_ref4","doi-asserted-by":"crossref","first-page":"7890","DOI":"10.1080\/01621459.1958.10501479","article-title":"On grouping for maximum homogeneity","volume":"53","author":"Fisher","year":"1958","journal-title":"Journal of the American Statistical Association"},{"issue":"4","key":"10.3233\/IDA-216316_ref5","first-page":"453","article-title":"On the histogram as a density estimator: l2 theory","volume":"57","author":"Freedman","year":"1981","journal-title":"Probability Theory and Related Fields"},{"key":"10.3233\/IDA-216316_ref6","first-page":"40","article-title":"Construction of histogram with variable bin-width based on change point detection","author":"Fushimi","journal-title":"Discovery Science"},{"key":"10.3233\/IDA-216316_ref7","first-page":"59","article-title":"Histogram-based outlier score (hbos): A fast unsupervised anomaly detection algorithm","author":"Goldstein","year":"2012","journal-title":"Poster and Demo Track of the 35th German Conference on Artificial Intelligence (KI-2012)"},{"key":"10.3233\/IDA-216316_ref8","unstructured":"Y.E. Ioannidis, Universality of serial histograms, In Proceedings of the 19th VLDB Conference, VLDB1993, San Francisco, CA, USA, 1993. Morgan Kaufmann Publishers Inc, pp.\u00a0256\u2013267."},{"key":"10.3233\/IDA-216316_ref9","doi-asserted-by":"crossref","unstructured":"Y.E. Ioannidis and V. Poosala, Balancing histogram optimality and practicality for query result size estimation, In Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, SIGMOD \u201995, New York, NY, USA, 1995. Association for Computing Machinery, pp.\u00a0233\u2013244.","DOI":"10.1145\/223784.223841"},{"key":"10.3233\/IDA-216316_ref10","first-page":"99","article-title":"Optimal histogram representation of large data sets: Fisher vs piecewise linear approximation","author":"Irpino","year":"2007","journal-title":"Extraction et gestion des connaissances (EGC\u20192007), Actes des cinqui\u00e8mes journ\u00e9es Extraction et Gestion des Connaissances, Namur, Belgique, 23-26 janvier 2007, 2 Volumes"},{"key":"10.3233\/IDA-216316_ref11","doi-asserted-by":"crossref","unstructured":"J. Kleinberg, Bursty and hierarchical structure in streams, In Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2002), 2002, pp.\u00a091\u2013101.","DOI":"10.1145\/775047.775061"},{"issue":"4","key":"10.3233\/IDA-216316_ref12","first-page":"5","article-title":"Approximate query answering using histograms","volume":"22","author":"Poosala","year":"1999","journal-title":"IEEE Data Eng Bull"},{"key":"10.3233\/IDA-216316_ref13","first-page":"294","article-title":"Improved histograms for selectivity estimation of range predicates","author":"Poosala","year":"1996","journal-title":"Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data"},{"issue":"2","key":"10.3233\/IDA-216316_ref14","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s10844-013-0283-2","article-title":"Change point detection for burst analysis from an observed information diffusion sequence of tweets","volume":"44","author":"Saito","year":"2015","journal-title":"Journal of Intelligent Information Systems"},{"issue":"3","key":"10.3233\/IDA-216316_ref15","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1093\/biomet\/66.3.605","article-title":"On optimal and data-based histograms","volume":"66","author":"Scott","year":"1979","journal-title":"Biometrika"},{"key":"10.3233\/IDA-216316_ref16","doi-asserted-by":"crossref","unstructured":"D.W. Scott, Multivariate Density Estimation: Theory, Practice, and Visualization, Wiley, 1 edition, 8 1992.","DOI":"10.1002\/9780470316849"},{"key":"10.3233\/IDA-216316_ref17","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1007\/978-3-540-77002-2_10","article-title":"Change detection in learning histograms from data streams","author":"Sebasti\u00e3o","year":"2007","journal-title":"Progress in Artificial Intelligence"},{"issue":"153","key":"10.3233\/IDA-216316_ref18","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1080\/01621459.1926.10502161","article-title":"The choice of a class interval","volume":"21","author":"Sturges","year":"1926","journal-title":"Journal of the American Statistical Association"},{"key":"10.3233\/IDA-216316_ref19","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1109\/TSMCC.2009.2037311","article-title":"Burst detection from multiple data streams: A network-based approach","volume":"40","author":"Sun","year":"2010","journal-title":"IEEE Transactions on Systems, Man, & Cybernetics Society, Part C"},{"key":"10.3233\/IDA-216316_ref20","doi-asserted-by":"crossref","unstructured":"R. Swan and J. Allan, Automatic generation of overview timelines, In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2000), 2000, pp.\u00a049\u201356.","DOI":"10.1145\/345508.345546"},{"key":"10.3233\/IDA-216316_ref21","unstructured":"X. Zhang, Fast Algorithms for Burst Detection, PhD dissertation (New York University), 2006. http:\/\/pdf.aminer.org\/000\/301\/507\/better_burst_detection.pdf."},{"key":"10.3233\/IDA-216316_ref22","doi-asserted-by":"crossref","unstructured":"Y. Zhu and D. Shasha, Efficient elastic burst detection in data streams, In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2003), 2003, pp.\u00a0336\u2013345.","DOI":"10.1145\/956750.956789"}],"container-title":["Intelligent Data Analysis"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/IDA-216316","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:19:49Z","timestamp":1777454389000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/IDA-216316"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,30]]},"references-count":22,"journal-issue":{"issue":"1"},"URL":"https:\/\/doi.org\/10.3233\/ida-216316","relation":{},"ISSN":["1088-467X","1571-4128"],"issn-type":[{"value":"1088-467X","type":"print"},{"value":"1571-4128","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,30]]}}}