{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:14:43Z","timestamp":1750306483063,"version":"3.41.0"},"reference-count":18,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2016,5,12]],"date-time":"2016-05-12T00:00:00Z","timestamp":1463011200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Storage"],"published-print":{"date-parts":[[2016,6,27]]},"abstract":"<jats:p>System log files contains messages emitted from several modules within a system and carries valuable information about the system state such as device status and error conditions and also about the various tasks within the system such as program names, execution path, including function names and parameters, and the task completion status. For customers with remote support, the system collects and transmits these logs to a central enterprise repository, where these are monitored for alerts, problem forecasting, and troubleshooting.<\/jats:p>\n          <jats:p>Very large log files limit the interpretability for the support engineers. For an expert, a large volume of log messages may not pose any problem; however, an inexperienced person may get flummoxed due to the presence of a large number of log messages. Often it is desired to present the log messages in a comprehensive manner where a person can view the important messages first and then go into details if required.<\/jats:p>\n          <jats:p>In this article, we present a user-friendly log viewer where we first hide the unimportant or inconsequential messages from the log file. A user can then click a particular hidden view and get the details of the hided messages. Messages with low utility are considered inconsequential as their removal does not impact the end user for the aforesaid purpose such as problem forecasting or troubleshooting. We relate the utility of a message to the probability of its appearance in the due context. We present machine-learning-based techniques that computes the usefulness of individual messages in a log file. We demonstrate identification and discarding of inconsequential messages to shrink the log size to acceptable limits. We have tested this over real-world logs and observed that eliminating such low value data can reduce the log files significantly (30% to 55%), with minimal error rates (7% to 20%). When limited user feedback is available, we show modifications to the technique to learn the user intent and accordingly further reduce the error.<\/jats:p>","DOI":"10.1145\/2846101","type":"journal-article","created":{"date-parts":[[2016,5,13]],"date-time":"2016-05-13T14:30:58Z","timestamp":1463149858000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["A User-Friendly Log Viewer for Storage Systems"],"prefix":"10.1145","volume":"12","author":[{"given":"Jayanta","family":"Basak","sequence":"first","affiliation":[{"name":"NetApp, Inc., Bangalore, India"}]},{"given":"P. C.","family":"Nagesh","sequence":"additional","affiliation":[{"name":"NetApp, Inc."}]}],"member":"320","published-online":{"date-parts":[[2016,5,12]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/170035.170072"},{"key":"e_1_2_1_2_1","volume-title":"28th Large Installation System Administration Conference (LISA14)","author":"Alspaugh S.","year":"2014","unstructured":"S. Alspaugh , Beidi Chen , Jessica Lin , Archana Ganapathi , Marti Hearst , and Randy Katz . 2014 . Analyzing log analysis: An empirical study of user log mining . In 28th Large Installation System Administration Conference (LISA14) . USENIX Association, Seattle, WA, 62--77. S. Alspaugh, Beidi Chen, Jessica Lin, Archana Ganapathi, Marti Hearst, and Randy Katz. 2014. Analyzing log analysis: An empirical study of user log mining. In 28th Large Installation System Administration Conference (LISA14). USENIX Association, Seattle, WA, 62--77."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1009715923555"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2005.10.010"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.485891"},{"key":"e_1_2_1_6_1","first-page":"43","article-title":"Understanding customer problem troubleshooting from storage system logs","volume":"9","author":"Jiang Weihang","year":"2009","unstructured":"Weihang Jiang , Chongfeng Hu , Shankar Pasupathy , Arkady Kanevsky , Zhenmin Li , and Yuanyuan Zhou . 2009 . Understanding customer problem troubleshooting from storage system logs . In FAST , Vol. 9. 43 -- 56 . Weihang Jiang, Chongfeng Hu, Shankar Pasupathy, Arkady Kanevsky, Zhenmin Li, and Yuanyuan Zhou. 2009. Understanding customer problem troubleshooting from storage system logs. In FAST, Vol. 9. 43--56.","journal-title":"FAST"},{"key":"e_1_2_1_7_1","unstructured":"J. Koshy. 2007. PMC based Performance Measurement in FreeBSD. Retrieved from http:\/\/people.freebsd.org\/&sim;jkoshy\/projects\/perf-measurement.  J. Koshy. 2007. PMC based Performance Measurement in FreeBSD. Retrieved from http:\/\/people.freebsd.org\/&sim;jkoshy\/projects\/perf-measurement."},{"key":"e_1_2_1_8_1","unstructured":"Time Kramer. 2003. Effective Log Reduction and Analysis Using Linux and Open Source Tools. Retrieved from http:\/\/www.giac.org\/paper\/gsec\/3144\/effective-log-reduction-analysis-linux-open-source-tools\/105234.  Time Kramer. 2003. Effective Log Reduction and Analysis Using Linux and Open Source Tools. Retrieved from http:\/\/www.giac.org\/paper\/gsec\/3144\/effective-log-reduction-analysis-linux-open-source-tools\/105234."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2007.370635"},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the Workshop on Deep Learning and Unsupervised Feature Learning, NIPS","volume":"10","author":"Andrew","unstructured":"Andrew L. Maas and Andrew Y. Ng. 2010. A probabilistic model for semantic word vectors . In Proceedings of the Workshop on Deep Learning and Unsupervised Feature Learning, NIPS , Vol. 10 . Andrew L. Maas and Andrew Y. Ng. 2010. A probabilistic model for semantic word vectors. In Proceedings of the Workshop on Deep Learning and Unsupervised Feature Learning, NIPS, Vol. 10."},{"key":"e_1_2_1_11_1","unstructured":"Network Appliance. 2007. Proactive health management with autosupport. http:\/\/www.netapp.com\/us\/media\/wp-7027.pdf.  Network Appliance. 2007. Proactive health management with autosupport. http:\/\/www.netapp.com\/us\/media\/wp-7027.pdf."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1089815.1089822"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1656395.1656397"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/347319.346322"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/SECURWARE.2010.37"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00175354"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the SLAML","author":"Xu Wei","year":"2010","unstructured":"Wei Xu , Ling Huang , Armando Fox , David Patterson , and Michael Jordan . 2010 . Experience mining google.s production console logs . In Proceedings of the SLAML (2010). Wei Xu, Ling Huang, Armando Fox, David Patterson, and Michael Jordan. 2010. Experience mining google.s production console logs. In Proceedings of the SLAML (2010)."},{"volume-title":"Proceedings of the IEEE\/IFIP International Conference on Dependable Systems & Networks (DSN&rsquo;\u201909)","author":"Ziming Z.","key":"e_1_2_1_18_1","unstructured":"Z. Ziming , L. Zhiling , B. H. Park , and A. Geist . 2009. System log pre-processing to improve failure prediction . In Proceedings of the IEEE\/IFIP International Conference on Dependable Systems & Networks (DSN&rsquo;\u201909) . 572--577. Z. Ziming, L. Zhiling, B. H. Park, and A. Geist. 2009. System log pre-processing to improve failure prediction. In Proceedings of the IEEE\/IFIP International Conference on Dependable Systems & Networks (DSN&rsquo;\u201909). 572--577."}],"container-title":["ACM Transactions on Storage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2846101","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2846101","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T05:48:25Z","timestamp":1750225705000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2846101"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,5,12]]},"references-count":18,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2016,6,27]]}},"alternative-id":["10.1145\/2846101"],"URL":"https:\/\/doi.org\/10.1145\/2846101","relation":{},"ISSN":["1553-3077","1553-3093"],"issn-type":[{"type":"print","value":"1553-3077"},{"type":"electronic","value":"1553-3093"}],"subject":[],"published":{"date-parts":[[2016,5,12]]},"assertion":[{"value":"2015-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-05-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}