{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T03:56:09Z","timestamp":1783655769471,"version":"3.55.0"},"reference-count":38,"publisher":"National Academy of Sciences","issue":"38","license":[{"start":{"date-parts":[[2021,3,4]],"date-time":"2021-03-04T00:00:00Z","timestamp":1614816000000},"content-version":"vor","delay-in-days":181,"URL":"https:\/\/www.pnas.org\/site\/aboutpnas\/licenses.xhtml"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS-1452718"],"award-info":[{"award-number":["IIS-1452718"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS-1452718"],"award-info":[{"award-number":["IIS-1452718"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"crossref","award":["W911NF-15-1-0259"],"award-info":[{"award-number":["W911NF-15-1-0259"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.pnas.org"],"crossmark-restriction":true},"short-container-title":["Proc. Natl. Acad. Sci. U.S.A."],"published-print":{"date-parts":[[2020,9,22]]},"abstract":"<jats:title>Significance<\/jats:title>\n                  <jats:p>Networks are a powerful tool for modeling complex biological and social systems. However, most networks are incomplete, and missing connections can negatively affect scientific analyses. Today, many algorithms can predict missing connections, but it is unknown how accuracy varies across algorithms and networks and whether link predictability varies across scientific domains. Analyzing 203 link prediction algorithms applied to 550 diverse real-world networks, we show that no predictor is best or worst overall. We then combine these many predictors into a single state-of-the-art algorithm that achieves nearly optimal performance on both synthetic networks with known optimality and real-world networks. Not all networks are equally predictable, however, and we find that social networks are easiest, while biological and technological networks are hardest.<\/jats:p>","DOI":"10.1073\/pnas.1914950117","type":"journal-article","created":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T19:56:23Z","timestamp":1599249383000},"page":"23393-23400","update-policy":"https:\/\/doi.org\/10.1073\/pnas.cm10313","source":"Crossref","is-referenced-by-count":129,"title":["Stacking models for nearly optimal link prediction in complex networks"],"prefix":"10.1073","volume":"117","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3515-3504","authenticated-orcid":false,"given":"Amir","family":"Ghasemian","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Colorado, Boulder, CO 80309;"},{"name":"Information Sciences Institute, University of Southern California, Marina del Rey, CA 90292;"},{"name":"Department of Statistics, Harvard University, Cambridge, MA 02138;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Homa","family":"Hosseinmardi","sequence":"additional","affiliation":[{"name":"Information Sciences Institute, University of Southern California, Marina del Rey, CA 90292;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aram","family":"Galstyan","sequence":"additional","affiliation":[{"name":"Information Sciences Institute, University of Southern California, Marina del Rey, CA 90292;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3512-0542","authenticated-orcid":false,"given":"Edoardo M.","family":"Airoldi","sequence":"additional","affiliation":[{"name":"Department of Statistics, Harvard University, Cambridge, MA 02138;"},{"name":"Department of Statistical Science, Fox School of Business, Temple University, Philadelphia, PA 19122;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3529-8746","authenticated-orcid":false,"given":"Aaron","family":"Clauset","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Colorado, Boulder, CO 80309;"},{"name":"BioFrontiers Institute, University of Colorado, Boulder, CO 80303;"},{"name":"Santa Fe Institute, Santa Fe, NM 87501"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"341","published-online":{"date-parts":[[2020,9,4]]},"reference":[{"key":"e_1_3_4_1_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.socnet.2005.07.002"},{"key":"e_1_3_4_2_2","first-page":"10","article-title":"Computationally efficient link prediction in a variety of social networks","volume":"5","author":"Fire M.","year":"2013","unstructured":"M. Fire , Computationally efficient link prediction in a variety of social networks. ACM Trans. Intell. Syst. Technol. (TIST) 5, 10 (2013).","journal-title":"ACM Trans. Intell. Syst. Technol. (TIST)"},{"key":"e_1_3_4_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.physa.2010.11.027"},{"key":"e_1_3_4_4_2","first-page":"2019","volume-title":"Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Nagarajan M.","year":"2015","unstructured":"M. Nagarajan , \u201cPredicting future scientific discoveries based on a networked analysis of the past literature\u201d in Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Association for Computing Machinery, New York, NY, 2015), pp. 2019\u20132028."},{"key":"e_1_3_4_5_2","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2014\/38.1.13"},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0153384"},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0033721"},{"key":"e_1_3_4_8_2","doi-asserted-by":"crossref","first-page":"1722","DOI":"10.1109\/TKDE.2019.2911585","article-title":"Evaluating overfit and underfit in models of network community structure","volume":"32","author":"Ghasemian A.","year":"2019","unstructured":"A. Ghasemian, H. Hosseinmardi, A. Clauset, Evaluating overfit and underfit in models of network community structure. IEEE Trans. Knowl. Data Eng. 32, 1722\u20131735 (2019).","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.97.062316"},{"key":"e_1_3_4_10_2","doi-asserted-by":"publisher","DOI":"10.1214\/09-SS054"},{"key":"e_1_3_4_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7"},{"key":"e_1_3_4_12_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature06830"},{"key":"e_1_3_4_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3012704"},{"key":"e_1_3_4_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-8462-3_9"},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.20591"},{"key":"e_1_3_4_16_2","doi-asserted-by":"publisher","DOI":"10.1140\/epjb\/e2009-00335-8"},{"key":"e_1_3_4_17_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0908366106"},{"key":"e_1_3_4_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939754"},{"key":"e_1_3_4_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2807452"},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/4235.585893"},{"key":"e_1_3_4_21_2","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.1602548"},{"key":"e_1_3_4_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00116037"},{"key":"e_1_3_4_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00058655"},{"key":"e_1_3_4_24_2","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava N.","year":"2014","unstructured":"N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929\u20131958 (2014).","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80023-1"},{"key":"e_1_3_4_26_2","first-page":"1401","volume-title":"Proceedings of the 16th International Joint Conference on Artificial intelligence","author":"Schapire R. E.","year":"1999","unstructured":"R. E. Schapire, \u201cA brief introduction to boosting\u201d in Proceedings of the 16th International Joint Conference on Artificial intelligence, T. Dean, Ed. (Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999), vol. 2, pp. 1401\u20131406."},{"key":"e_1_3_4_27_2","doi-asserted-by":"publisher","DOI":"10.1214\/16-BA1023"},{"key":"e_1_3_4_28_2","doi-asserted-by":"publisher","DOI":"10.1214\/17-BA1091"},{"key":"e_1_3_4_29_2","first-page":"382","article-title":"Bayesian model averaging: A tutorial","volume":"14","author":"Hoeting J. A.","year":"1999","unstructured":"J. A. Hoeting, D. Madigan, A. E. Raftery, C. T. Volinsky, Bayesian model averaging: A tutorial. Stat. Sci. 14, 382\u2013401 (1999).","journal-title":"Stat. Sci."},{"key":"e_1_3_4_30_2","first-page":"155","volume-title":"Proceedings of the Third International Conference on Knowledge Discovery and Data Mining","author":"Domingos P.","year":"1997","unstructured":"P. Domingos, \u201cWhy does bagging work? A Bayesian account and its implications\u201d in Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, D. Heckerman, R. Uthurusamy, H. Mannila, D. Pregibon, Eds. (AAAI Press, 1997), pp. 155\u2013158."},{"key":"e_1_3_4_31_2","unstructured":"T. P. Minka Bayesian model averaging is not model combination MIT Media Lab note (2000). https:\/\/tminka.github.io\/papers\/bma.html. Accessed 17 August 2020."},{"key":"e_1_3_4_32_2","first-page":"619","volume-title":"Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS","author":"Kim H. C.","year":"2012","unstructured":"H. C. Kim, Z. Ghahramani, \u201cBayesian classifier combination\u201d in Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS, N. D. Lawrence, M. Girolami, Eds. (Proceedings of Machine Learning Research, 2012), pp. 619\u2013627."},{"key":"e_1_3_4_33_2","unstructured":"Y. Koren The BellKor solution to the Netflix Grand Prize. Netflix Prize Documentation 81 1\u201310 (2009). https:\/\/netflixprize.com\/assets\/GrandPrize2009_BPC_BellKor.pdf. Accessed 17 August 2020."},{"key":"e_1_3_4_34_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btm158"},{"key":"e_1_3_4_35_2","doi-asserted-by":"publisher","DOI":"10.1006\/jcss.1997.1504"},{"key":"e_1_3_4_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313660"},{"key":"e_1_3_4_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijforecast.2018.06.001"},{"key":"e_1_3_4_38_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0194889"}],"container-title":["Proceedings of the National Academy of Sciences"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.pnas.org\/syndication\/doi\/10.1073\/pnas.1914950117","content-type":"unspecified","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/pnas.org\/doi\/pdf\/10.1073\/pnas.1914950117","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,13]],"date-time":"2024-08-13T02:28:57Z","timestamp":1723516137000},"score":1,"resource":{"primary":{"URL":"https:\/\/pnas.org\/doi\/full\/10.1073\/pnas.1914950117"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,4]]},"references-count":38,"journal-issue":{"issue":"38","published-print":{"date-parts":[[2020,9,22]]}},"alternative-id":["10.1073\/pnas.1914950117"],"URL":"https:\/\/doi.org\/10.1073\/pnas.1914950117","relation":{},"ISSN":["0027-8424","1091-6490"],"issn-type":[{"value":"0027-8424","type":"print"},{"value":"1091-6490","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,4]]},"assertion":[{"value":"2020-09-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}