{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T05:31:34Z","timestamp":1780637494655,"version":"3.54.1"},"reference-count":58,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,2,14]],"date-time":"2024-02-14T00:00:00Z","timestamp":1707868800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Stiftelsen Bergteknisk Forskning (BeFo), Rock Engineering Research Foundation of Sweden","award":["448"],"award-info":[{"award-number":["448"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In the context of geo-infrastructures and specifically tunneling projects, analyzing the large-scale sensor-based measurement-while-drilling (MWD) data plays a pivotal role in assessing rock engineering conditions. However, handling the big MWD data due to multiform stacking is a time-consuming and challenging task. Extracting valuable insights and improving the accuracy of geoengineering interpretations from MWD data necessitates a combination of domain expertise and data science skills in an iterative process. To address these challenges and efficiently normalize and filter out noisy data, an automated processing approach integrating the stepwise technique, mode, and percentile gate bands for both single and peer group-based holes was developed. Subsequently, the mathematical concept of a novel normalizing index for classifying such big datasets was also presented. The visualized results from different geo-infrastructure datasets in Sweden indicated that outliers and noisy data can more efficiently be eliminated using single hole-based normalizing. Additionally, a relational unified PostgreSQL database was created to store and automatically transfer the processed and raw MWD as well as real time grouting data that offers a cost effective and efficient data extraction tool. The generated database is expected to facilitate in-depth investigations and enable application of the artificial intelligence (AI) techniques to predict rock quality conditions and design appropriate support systems based on MWD data.<\/jats:p>","DOI":"10.3390\/s24041209","type":"journal-article","created":{"date-parts":[[2024,2,14]],"date-time":"2024-02-14T06:59:26Z","timestamp":1707893966000},"page":"1209","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":71,"title":["Normalizing Large Scale Sensor-Based MWD Data: An Automated Method toward A Unified Database"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3832-572X","authenticated-orcid":false,"given":"Abbas","family":"Abbaszadeh Shahri","sequence":"first","affiliation":[{"name":"Johan Lundberg AB, 754 50 Uppsala, Sweden"},{"name":"Division of Rock Engineering, Tyrens, 118 86 Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chunling","family":"Shan","sequence":"additional","affiliation":[{"name":"Division of Rock Engineering, Tyrens, 118 86 Stockholm, Sweden"},{"name":"Division of Soil and Rock Mechanics, Royal Institute of Technology, KTH, 114 28 Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefan","family":"Larsson","sequence":"additional","affiliation":[{"name":"Division of Soil and Rock Mechanics, Royal Institute of Technology, KTH, 114 28 Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fredrik","family":"Johansson","sequence":"additional","affiliation":[{"name":"Division of Soil and Rock Mechanics, Royal Institute of Technology, KTH, 114 28 Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Gearhart, M., Moseley, L.M., and Foste, M. (1986, January 17). Current state of the art of MWD and its application in exploration and development drilling. Proceedings of the International Meeting on Petroleum Engineering, Beijing, China. SPE-14071-MS.","DOI":"10.2523\/14071-MS"},{"key":"ref_2","first-page":"310","article-title":"Improvements in blast fragmentation using measurement while drilling parameters","volume":"6","author":"Smith","year":"2002","journal-title":"Int. J. Blasting Fragm."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1016\/S0148-9062(97)00332-X","article-title":"Rock characterization using percussive drilling","volume":"35","author":"Schunnesson","year":"1998","journal-title":"Int. J. Rock Mech. Min. Sc."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"103467","DOI":"10.1016\/j.tust.2020.103467","article-title":"Improved filtering and normalizing of Measurement-While-Drilling (MWD) data in tunnel excavation","volume":"103","author":"Schunnesson","year":"2020","journal-title":"Tunn. Undergr. Space Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1076\/frag.6.3.287.14052","article-title":"Blast design using measurement while drilling parameters","volume":"6","author":"Segui","year":"2002","journal-title":"Fragblast"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.ijrmms.2018.06.015","article-title":"Development of a geological model for chargeability assessment of borehole using drill monitoring technique","volume":"109","author":"Ghosh","year":"2018","journal-title":"Int. J. Rock Mech. Min. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/j.jrmge.2015.01.006","article-title":"Rock characterization while drilling and application of roof bolter drilling data for evaluation of ground conditions","volume":"7","author":"Rostami","year":"2015","journal-title":"J. Rock Mech. Geo. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"563","DOI":"10.9711\/KTAJ.2015.17.5.563","article-title":"Main challenges for deep subsea tunnels based on Norwegian experience","volume":"17","author":"Nilsen","year":"2015","journal-title":"J. Korean Tunn. Undergr. Space Assoc."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1002\/geot.202100070","article-title":"Improving face decisions in tunnelling by machine learning-based MWD analysis","volume":"15","author":"Hansen","year":"2022","journal-title":"Geomech. Tunneling"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.tust.2017.10.011","article-title":"On the mutual relations of drill monitoring variables and the drill control system in tunneling operations","volume":"72","author":"Navarro","year":"2018","journal-title":"Tunn. Undergr. Space Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"989","DOI":"10.1016\/j.ijmst.2016.09.005","article-title":"Processing of measurement while drilling data for rock mass characterization","volume":"26","author":"Khorzougi","year":"2016","journal-title":"Int. J. Min. Sci. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1016\/j.ijmst.2018.01.001","article-title":"Rock fracture density characterization using measurement while drilling (MWD) techniques","volume":"28","author":"Khorzoughi","year":"2018","journal-title":"Int. J. Min. Sci. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Isheyskiy, V., Martinyskin, E., Smirnov, S., Vasilyev, A., Knyazev, K., and Fatyanov, T. (2021). Specifics of MWD data collection and verification during formation of training datasets. Minerals, 11.","DOI":"10.3390\/min11080798"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Isheyskiy, V., and Sanchidrian, J.A. (2020). Prospects of applying MWD technology for quality management of drilling and blasting operations at mining enterprises. Minerals, 10.","DOI":"10.3390\/min10100925"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1144\/GSL.SP.1995.097.01.06","article-title":"Improving the value of geological data: A standardized data model for industry","volume":"97","author":"Saunders","year":"1996","journal-title":"Geol. Soc."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Krogstie, J., Opdahl, A.L., and Brinkkemper, S. (2007). Conceptual Modelling in Information Systems Engineering, Springer.","DOI":"10.1007\/978-3-540-72677-7"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ress.2012.12.021","article-title":"A review on coarse warranty data and analysis","volume":"114","author":"Wu","year":"2013","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_18","first-page":"309","article-title":"Repository metadata: Approaches and challenges","volume":"47","author":"Chapman","year":"2009","journal-title":"Cat. Classif. Quaterly"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00766-016-0254-6","article-title":"Requirements for cloud-based BIM governance solutions to facilitate team collaboration in construction projects","volume":"23","author":"Alreshidi","year":"2018","journal-title":"Requir. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1023\/A:1009841028985","article-title":"DMajor- Application programming interface for database mining","volume":"3","author":"Virmani","year":"1999","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.proeng.2015.10.001","article-title":"Analysis of normalization methods influencing results: A review to honour professor Friedel Peldschus on the occasion of his 75th birthday","volume":"122","author":"Kaplinski","year":"2015","journal-title":"Procedia Eng."},{"key":"ref_22","first-page":"22","article-title":"Development of data normalization methods for multi-criteria decision making: Applying for MARCOS method","volume":"9","author":"Trung","year":"2022","journal-title":"Manuf. Rev."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Mukhametzyanov, I.Z. (2023). Normalization of Multidimensional Data for Multi-Criteria Decision Making Problems, Springer.","DOI":"10.1007\/978-3-031-33837-3"},{"key":"ref_24","unstructured":"J\u00fcttler, H. (1996). Untersuchungen zu Fragen der Operationsforschung und ihrer Anwendungsm\u00f6glichkeiten auf \u00f6konomische Problemstellungen unter besonderer Ber\u00fccksichtigung der Spieltheorie. [Ph.D. Thesis, Wirtschftswissenschaftliche Fakult\u00e4t der Humbold-Universit\u00e4t Berlin]."},{"key":"ref_25","unstructured":"Weitendorf, D. (1976). Beitrag zur Optimierung der R\u00e4umlichen Struktur Eines Geb\u00e4udes. [Ph.D. Thesis, Hochschule f\u00fcr Architektur und Bauwesen Weimar]."},{"key":"ref_26","first-page":"173","article-title":"Technologische entscheidungen bei der ber\u00fccksichtigung mehrerer ziehle","volume":"37","author":"Peldschus","year":"1983","journal-title":"Bauplan. Bautech."},{"key":"ref_27","unstructured":"Peldschus, F. (1986). Zur Anwendung der Theorie der Spiele f\u00fcr Aufgaben der Bautechnologie. [Ph.D. Thesis, Technischen Hochschule Leipzig]."},{"key":"ref_28","first-page":"531","article-title":"Experience of the game theory application in construction management","volume":"14","author":"Peldschus","year":"2008","journal-title":"Ukio Technol. Ir Ekon. Vystym."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"303","DOI":"10.15388\/Informatica.2008.215","article-title":"A new normalization method in games theory","volume":"19","author":"Zavadskas","year":"2008","journal-title":"Informatica"},{"key":"ref_30","unstructured":"B\u00f6rner, I. (1980). Untersuchungen zur Optimierung Nach Mehreren Zielen f\u00fcr Aufgaben der Bautechnologie. [Ph.D. Thesis, Sektion Technologie der Bauproduktion; Diplomarbeit]."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1007\/3-540-58435-8_183","article-title":"A unified framework for indexing in database systems","volume":"Volume 856","author":"Karagiannis","year":"1994","journal-title":"Database and Expert Systems Applications"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhussupbekov, A., Alibekova, N., Akhazhanov, S., and Sarsembayeva, A. (2021). Development of a unified geotechnical database and data processing on the example of Nur-Sultan City. Appl. Sci., 11.","DOI":"10.3390\/app11010306"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2681","DOI":"10.1007\/s10639-022-11293-0","article-title":"Advances in database systems education: Methods; tools; curricula; and way forward","volume":"28","author":"Ishaq","year":"2023","journal-title":"Educ. Inf. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.sesci.2018.06.002","article-title":"Progress and challenges of big data research on petrology and geochemistry","volume":"3","author":"Jiao","year":"2018","journal-title":"Solid Earth Sci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"112082","DOI":"10.1016\/j.measurement.2022.112082","article-title":"Measurement while core drilling based on a small-scale drilling platfrom: Mechanical and energy analysis","volume":"204","author":"Deng","year":"2022","journal-title":"Measurement"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1566","DOI":"10.1016\/j.cpc.2009.04.011","article-title":"Novel two-step filtering scheme for a logging while-drilling system","volume":"180","author":"Zhao","year":"2019","journal-title":"Comput. Phys. Commun."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"819","DOI":"10.1007\/s13202-020-01024-x","article-title":"Drilling data quality improvement and information extraction with case studies","volume":"11","author":"Geekiyanage","year":"2021","journal-title":"J. Pet. Explor. Prod. Technol."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Yang, Y., Li, F., Gao, Y., and Mao, Y. (2020). Multi-sensor combined measurement while drilling based on the improved adaptive fading square root unscented Kalman filter. Sensors, 20.","DOI":"10.3390\/s20071897"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1080\/10916460903419206","article-title":"Exact lithologic boundary detection based on wavelet transform analysis and real-time investigation of facies discontinuities using drilling data","volume":"29","author":"Arabjamaloei","year":"2011","journal-title":"Pet. Sci. Technol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"04023028","DOI":"10.1061\/IJGNAI.GMENG-8080","article-title":"Deep learning for intelligent prediction of rock strength by adopting measurement while drilling data","volume":"23","author":"Zhao","year":"2023","journal-title":"Int. J. Geomech."},{"key":"ref_41","unstructured":"Eren, T., and Ozbayoglu, M.E. (2010). SPE Oil and Gas India Conference and Exhibition, SPE. SPE-1291126-MS."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.ijrmms.2014.10.012","article-title":"Automated coal seam detection using modulated specific energy measure in a monitor-while-drilling context","volume":"75","author":"Leung","year":"2015","journal-title":"Int. J. Rock Mech. Min. Sci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"11318","DOI":"10.1038\/s41598-022-15493-z","article-title":"Real-time prediction of formation pressure gradient while drilling","volume":"12","author":"Abdelaal","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"76833","DOI":"10.1109\/ACCESS.2021.3082557","article-title":"Deep learning and time-series analysis for the early detection of lost circulation incidents during drilling operations","volume":"9","author":"Aljubran","year":"2021","journal-title":"IEEE Access"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1363","DOI":"10.1016\/S0890-6955(00)00112-7","article-title":"Tool wear condition monitoring in drilling operations using hidden Markov models (HMMs)","volume":"41","author":"Ertunc","year":"2001","journal-title":"Int. J. Mach. Tools Manuf."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1733","DOI":"10.1139\/cgj-2019-0094","article-title":"Measuring while drilling in Florida limestone for geotechnical site investigation","volume":"57","author":"Rodgers","year":"2020","journal-title":"Can. Geotech. J."},{"key":"ref_47","unstructured":"Purkayastha, A.D., and Nair, P.V. (2017). SPE Oil and Gas India Conference and Exhibition, SPE. SPE-185394-MS."},{"key":"ref_48","unstructured":"Wesseloo, J. (2017). Deep Mining 2017: Proceedings of the Eighth International Conference on Deep and High Stress Mining, Australian Centre for Geomechanics."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Drebenstedt, C., and Singhal, R. (2014). Mine Planning and Equipment Selection, Springer.","DOI":"10.1007\/978-3-319-02678-7"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Martin, C.A., Philo, R.M., Decker, D.P., and Burgess, T.M. (1994, January 15\u201318). Innovative advances in MWD. In Proceeding of the IADC\/SPE Drilling Conference, Dallas, Dallas, TX, USA. SPE-27516-MS.","DOI":"10.2523\/27516-MS"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1016\/j.ijmst.2023.02.004","article-title":"Rock mass structural recognition from drill monitoring technology in underground mining using discontinuity index and machine learning techniques","volume":"33","author":"Segarra","year":"2023","journal-title":"Int. J. Min. Sci. Technol."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Reckmann, H., Jogi, P., Kpetehoto, F., Chandrasekaran, S., and Macpherson, J. (2010, January 2\u20134). MWD failure rates due to drilling dynamics. Proceedings of the ADC\/SPE Drilling Conference and Exhibition, New Orleans, LA, USA. Paper Number: SPE-127413-MS.","DOI":"10.2118\/127413-MS"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"085114","DOI":"10.1063\/5.0090505","article-title":"Full waveform vibration and shock measurement tool for measurement-while-drilling","volume":"12","author":"Song","year":"2022","journal-title":"AIP Adv."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"269","DOI":"10.7569\/JSEE.2014.629515","article-title":"Strategies in high-data-rate MWD mud pulse telemetry","volume":"2","author":"Su","year":"2014","journal-title":"J. Sustain. Energy Eng."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Abbaszadeh Shahri, A., Shan, C., and Larsson, S. (2023). A hybrid ensemble-based automated deep learning approach to generate 3D geo-models and uncertainty analysis. Eng. Comput.","DOI":"10.1007\/s00366-023-01852-5"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Duking, M.F., Kraaikamp, C., Lopuhaa, P., and Meester, L.E. (2005). A Modern Introduction to Probability and Statistics, Springer.","DOI":"10.1007\/1-84628-168-7"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"615","DOI":"10.1109\/TFUZZ.2015.2466080","article-title":"Law of large numbers for uncertain random variables","volume":"24","author":"Yao","year":"2016","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1111\/j.1467-9574.1980.tb00681.x","article-title":"Mean; median and mode in binomial distribution","volume":"34","author":"Kaas","year":"1980","journal-title":"Stat. Neerl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/4\/1209\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:59:31Z","timestamp":1760104771000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/4\/1209"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,14]]},"references-count":58,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,2]]}},"alternative-id":["s24041209"],"URL":"https:\/\/doi.org\/10.3390\/s24041209","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,14]]}}}