{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T05:09:05Z","timestamp":1778476145005,"version":"3.51.4"},"reference-count":20,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T00:00:00Z","timestamp":1778457600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:p>This paper proposes a preprocessing framework for optimizing large-scale graph database ingestion through intelligent edge filtering based on value ranking. We combine adapted PageRank algorithms with business-specific metrics and edge type importance to evaluate and rank edges, enabling selective retention of high-value relationships. The framework introduces three PageRank variants (maximum weight normalization, weighted average, and log-based normalization) with type-specific business value normalization to handle heterogeneous graphs. Current graph database ingestion approaches struggle with scale: loading 6.2TB of data (38 billion objects) requires over 3 weeks, forcing organizations to limit historical data retention. Our approach addresses this through preprocessing-stage filtering before database ingestion. While requiring experimental validation, preliminary analysis suggests potential for 40%\u201380% data volume reduction depending on graph characteristics, with corresponding improvements in loading efficiency and storage costs. The paper details the theoretical framework, computational complexity analysis, formal property preservation guarantees, and comprehensive validation methodology. This work represents a novel direction in graph database optimization: value-based preprocessing rather than runtime query optimization.<\/jats:p>","DOI":"10.3389\/fdata.2026.1768571","type":"journal-article","created":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T04:23:44Z","timestamp":1778473424000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Optimizing large-scale graph database ingestion through edge value ranking: a proposed framework"],"prefix":"10.3389","volume":"9","author":[{"given":"Phanindra Reddy","family":"Madduru","sequence":"first","affiliation":[{"name":"Amazon.com","place":["Seattle, WA, United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bijo","family":"Thomas","sequence":"additional","affiliation":[{"name":"Amazon.com","place":["Seattle, WA, United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2026,5,11]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1145\/3183713.3190654","article-title":"\u201cG-CORE: a core for future graph query languages,\u201d","volume-title":"Proceedings of the 2018 International Conference on Management of Data","author":"Angles","year":"2018"},{"key":"B2","unstructured":"Survey and taxonomy of lossless graph compression and space-efficient graph representations\n          \n          \n            \n              Besta\n              M.\n            \n            \n              Hoefler\n              T.\n            \n          \n          arXiv preprint\n          \n          2019"},{"key":"B3","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1145\/3335409.3335411","article-title":"Graph queries: from theory to practice","volume":"47","author":"Bonifati","year":"2018","journal-title":"SIGMOD Rec"},{"key":"B4","first-page":"49","article-title":"\u201cTAO: Facebook's distributed data store for the social graph,\u201d","author":"Bronson","year":"2013","journal-title":"2013 USENIX Annual Technical Conference (held in San Jose, CA, June 26-28, 2013)"},{"key":"B5","author":"Cormen","year":"2009","journal-title":"Introduction to Algorithms"},{"key":"B6","unstructured":"TigerGraph: a native MPP graph database\n          \n          \n            \n              Deutsch\n              A.\n            \n            \n              Xu\n              Y.\n            \n            \n              Wu\n              M.\n            \n            \n              Lee\n              V.\n            \n          \n          arXiv preprint\n          \n          2019"},{"key":"B7","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1145\/2213836.2213855","article-title":"\u201cQuery preserving graph compression,\u201d","volume-title":"Proceedings of the 2012 ACM SIGMOD international conference on management of data","author":"Fan","year":"2012"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3556974","article-title":"Distributed graph database management systems: a systematic literature review","volume":"55","author":"Han","year":"2023","journal-title":"ACM Comput. Surv"},{"key":"B9","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1145\/3627703.3629571","article-title":"\u201cCore graph: exploiting edge centrality to speedup the evaluation of iterative graph queries,\u201d","volume-title":"Proceedings of the nineteenth European conference on computer systems, EuroSys-24","author":"Jiang","year":"2024"},{"key":"B10","unstructured":"The TrieJax Architecture: Accelerating Graph Operations Through Relational Joins\n          \n          \n            \n              Kalinsky\n              O.\n            \n            \n              Kimelfeld\n              B.\n            \n            \n              Etsion\n              Y.\n            \n          \n          arXiv preprint\n          \n          2019"},{"key":"B11","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1090\/S0002-9939-1956-0078686-7","article-title":"On the shortest spanning subtree of a graph and the traveling salesman problem","volume":"7","author":"Kruskal","year":"1956","journal-title":"Proc. Am. Math. Soc"},{"key":"B12","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1109\/ICDM.2016.0033","article-title":"\u201cEdge weight prediction in weighted signed networks,\u201d","volume-title":"2016 IEEE 16th International Conference on Data Mining (ICDM)","author":"Kumar","year":"2016"},{"key":"B13","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1145\/2907294.2907312","article-title":"\u201cEfficient processing of large graphs via input reduction,\u201d","volume-title":"Proceedings of the 25th ACM international symposium on high-performance parallel and distributed computing, HPDC '16","author":"Kusum","year":"2016"},{"key":"B14","article-title":"\u201cA heterogeneous graph-based framework for scalable fraud detection.\u201d","volume-title":"Proceedings of the 19th international workshop on mining and learning with graphs (MLG'23)","author":"Madduru","year":"2023"},{"key":"B15","first-page":"351","article-title":"\u201cWormhole: reliable pub-sub to support geo-replicated internet services,\u201d","volume-title":"12th USENIX symposium on networked systems design and implementation (NSDI 15)","author":"Sharma","year":"2016"},{"key":"B16","first-page":"2830","article-title":"\u201cPreprocessing techniques for streaming graphs.\u201d","volume-title":"2019 IEEE International Conference on Big Data (Big Data)","author":"Wang","year":"2019"},{"key":"B17","first-page":"1666","article-title":"\u201cEnterprise graph search for Neo4j,\u201d","volume-title":"2020 IEEE 36th international conference on data engineering (ICDE)","author":"Weber","year":"2020"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3677132","article-title":"Efficient and accurate PageRank approximation on large graphs","volume":"2","author":"Wu","year":"2024","journal-title":"Proc. ACM Manag. Data"},{"key":"B19","first-page":"608","article-title":"\u201cWonderland: a novel abstraction-based out-of-core graph processing system,\u201d","volume-title":"Proceedings of the twenty-third international conference on architectural support for programming languages and operating systems, ASPLOS '18","author":"Zhang","year":"2018"},{"key":"B20","first-page":"5833","article-title":"Efficient streaming graph embeddings","volume":"34","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Knowl. Data Eng"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2026.1768571\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T04:23:51Z","timestamp":1778473431000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2026.1768571\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,11]]},"references-count":20,"alternative-id":["10.3389\/fdata.2026.1768571"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2026.1768571","relation":{},"ISSN":["2624-909X"],"issn-type":[{"value":"2624-909X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,11]]},"article-number":"1768571"}}