{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T14:41:51Z","timestamp":1774363311386,"version":"3.50.1"},"reference-count":30,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2019,9,30]],"date-time":"2019-09-30T00:00:00Z","timestamp":1569801600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"crossref","award":["R01LM012095"],"award-info":[{"award-number":["R01LM012095"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2019,9,30]]},"abstract":"<jats:p>Many causal discovery algorithms infer graphical structure from observational data. The PC algorithm in particular estimates a completed partially directed acyclic graph (CPDAG), or an acyclic graph containing directed edges identifiable with conditional independence testing. However, few groups have investigated strategies for estimating and controlling the false discovery rate (FDR) of the edges in the CPDAG. In this article, we introduce PC with p-values (PC-p), a fast algorithm that robustly computes edge-specific p-values and then estimates and controls the FDR across the edges. PC-p specifically uses the p-values returned by many conditional independence (CI) tests to upper bound the p-values of more complex edge-specific hypothesis tests. The algorithm then estimates and controls the FDR using the bounded p-values and the Benjamini-Yekutieli FDR procedure. Modifications to the original PC algorithm also help PC-p accurately compute the upper bounds despite non-zero Type II error rates. Experiments show that PC-p yields more accurate FDR estimation and control across the edges in a variety of CPDAGs compared to alternative methods.<\/jats:p>","DOI":"10.1145\/3351342","type":"journal-article","created":{"date-parts":[[2019,10,10]],"date-time":"2019-10-10T13:13:05Z","timestamp":1570713185000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Estimating and Controlling the False Discovery Rate of the PC Algorithm Using Edge-specific P-Values"],"prefix":"10.1145","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0077-6314","authenticated-orcid":false,"given":"Eric V.","family":"Strobl","sequence":"first","affiliation":[{"name":"Vanderbilt University Medical Center, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter L.","family":"Spirtes","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shyam","family":"Visweswaran","sequence":"additional","affiliation":[{"name":"University of Pittsburgh, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,10,10]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Learning large-scale Bayesian networks with the sparsebn package. Retrieved from: arXiv preprint arXiv:1703.04025","author":"Aragam Bryon","year":"2017","unstructured":"Bryon Aragam , Jiaying Gu , and Qing Zhou . 2017. Learning large-scale Bayesian networks with the sparsebn package. Retrieved from: arXiv preprint arXiv:1703.04025 ( 2017 ). Bryon Aragam, Jiaying Gu, and Qing Zhou. 2017. Learning large-scale Bayesian networks with the sparsebn package. Retrieved from: arXiv preprint arXiv:1703.04025 (2017)."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/2789272.2886822"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN\u201911)","author":"Armen A. P.","unstructured":"A. P. Armen and I. Tsamardinos . 2011. A unified approach to estimation and control of the false discovery rate in Bayesian network skeleton identification . In Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN\u201911) . A. P. Armen and I. Tsamardinos. 2011. A unified approach to estimation and control of the false discovery rate in Bayesian network skeleton identification. In Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN\u201911)."},{"key":"e_1_2_1_4_1","volume-title":"Technical Report TR-441. University of Crete. 1--79.","author":"Armen A. P.","year":"2014","unstructured":"A. P. Armen and I. Tsamardinos . 2014 . Estimation and Control of the False Discovery Rate of Bayesian Network Skeleton Identification . Technical Report TR-441. University of Crete. 1--79. A. P. Armen and I. Tsamardinos. 2014. Estimation and Control of the False Discovery Rate of Bayesian Network Skeleton Identification. Technical Report TR-441. University of Crete. 1--79."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1995.tb02031.x"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1013699998"},{"key":"e_1_2_1_7_1","first-page":"507","article-title":"Optimal structure identification with greedy search","author":"Chickering David Maxwell","year":"2002","unstructured":"David Maxwell Chickering . 2002 . Optimal structure identification with greedy search . J. Mach. Learn. Res. 3 , Nov. (2002), 507 -- 554 . David Maxwell Chickering. 2002. Optimal structure identification with greedy search. J. Mach. Learn. Res. 3, Nov. (2002), 507--554.","journal-title":"J. Mach. Learn. Res. 3"},{"key":"e_1_2_1_8_1","volume-title":"Optimal structure identification with greedy search. J. Mach. Learn. Res. 3 (Mar","author":"Chickering David Maxwell","year":"2003","unstructured":"David Maxwell Chickering . 2003. Optimal structure identification with greedy search. J. Mach. Learn. Res. 3 (Mar . 2003 ), 507--554. DOI:https:\/\/doi.org\/10.1162\/153244303321897717 10.1162\/153244303321897717 David Maxwell Chickering. 2003. Optimal structure identification with greedy search. J. Mach. Learn. Res. 3 (Mar. 2003), 507--554. DOI:https:\/\/doi.org\/10.1162\/153244303321897717"},{"key":"e_1_2_1_9_1","first-page":"1","article-title":"Order-independent constraint-based causal structure learning","volume":"15","author":"Colombo D.","year":"2014","unstructured":"D. Colombo and M. H. Maathuis . 2014 . Order-independent constraint-based causal structure learning . J. Mach. Learn. Res. 15 , 1 (Jan. 2014), 3741--3782. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals;2627435.2750365. D. Colombo and M. H. Maathuis. 2014. Order-independent constraint-based causal structure learning. J. Mach. Learn. Res. 15, 1 (Jan. 2014), 3741--3782. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals;2627435.2750365.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the UAI Causality Workshop. 7 pages.","author":"Cook John W.","unstructured":"John W. Cook , David Danks , and Sergey M. Plis . 2017. Learning dynamic structure from undersampled data . In Proceedings of the UAI Causality Workshop. 7 pages. John W. Cook, David Danks, and Sergey M. Plis. 2017. Learning dynamic structure from undersampled data. In Proceedings of the UAI Causality Workshop. 7 pages."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.4064\/fm-25-1-379-387"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI\u201999)","author":"Friedman N.","year":"2073","unstructured":"N. Friedman , M. Goldszmidt , and A. Wyner . 1999. Data analysis with Bayesian networks: A bootstrap approach . In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI\u201999) . Morgan Kaufmann Publishers Inc., San Francisco, CA, 196--205. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals; 2073 796.2073819. N. Friedman, M. Goldszmidt, and A. Wyner. 1999. Data analysis with Bayesian networks: A bootstrap approach. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI\u201999). Morgan Kaufmann Publishers Inc., San Francisco, CA, 196--205. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals;2073796.2073819."},{"key":"e_1_2_1_13_1","volume-title":"A survey of learning causality with data: Problems and methods. Retrieved from: arXiv preprint arXiv:1809.09337","author":"Guo Ruocheng","year":"2018","unstructured":"Ruocheng Guo , Lu Cheng , Jundong Li , P. Richard Hahn , and Huan Liu . 2018. A survey of learning causality with data: Problems and methods. Retrieved from: arXiv preprint arXiv:1809.09337 ( 2018 ). Ruocheng Guo, Lu Cheng, Jundong Li, P. Richard Hahn, and Huan Liu. 2018. A survey of learning causality with data: Problems and methods. Retrieved from: arXiv preprint arXiv:1809.09337 (2018)."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1111\/biom.12415"},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems. 689--696","author":"Hoyer Patrik O.","year":"2009","unstructured":"Patrik O. Hoyer , Dominik Janzing , Joris M. Mooij , Jonas Peters , and Bernhard Sch\u00f6lkopf . 2009 . Nonlinear causal discovery with additive noise models . In Proceedings of the Conference on Advances in Neural Information Processing Systems. 689--696 . Patrik O. Hoyer, Dominik Janzing, Joris M. Mooij, Jonas Peters, and Bernhard Sch\u00f6lkopf. 2009. Nonlinear causal discovery with additive noise models. In Proceedings of the Conference on Advances in Neural Information Processing Systems. 689--696."},{"key":"e_1_2_1_16_1","volume-title":"Estimating high-dimensional directed acyclic graphs with the PC-algorithm. J. Mach. Learn. Res. 8 (May","author":"Kalisch Markus","year":"2007","unstructured":"Markus Kalisch and Peter B\u00fchlmann . 2007. Estimating high-dimensional directed acyclic graphs with the PC-algorithm. J. Mach. Learn. Res. 8 (May 2007 ), 613--636. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals;1248659.1248681. Markus Kalisch and Peter B\u00fchlmann. 2007. Estimating high-dimensional directed acyclic graphs with the PC-algorithm. J. Mach. Learn. Res. 8 (May 2007), 613--636. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals;1248659.1248681."},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the International Conference on IEEE Engineering in Medicine and Biology Society. 4617--4620","author":"Li J.","unstructured":"J. Li , Z. Wang , and M. J. McKeown . 2008. Learning brain connectivity with the false-discovery-rate-controlled PC-algorithm . In Proceedings of the International Conference on IEEE Engineering in Medicine and Biology Society. 4617--4620 . J. Li, Z. Wang, and M. J. McKeown. 2008. Learning brain connectivity with the false-discovery-rate-controlled PC-algorithm. In Proceedings of the International Conference on IEEE Engineering in Medicine and Biology Society. 4617--4620."},{"key":"e_1_2_1_18_1","unstructured":"J. Li and Z. J. Wang. 2009. Controlling the false discovery rate of the association\/causality structure learned with the PC algorithm. J. Mach. Learn. Res. 10 (June 2009) 475--514. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals;1577069.1577086.  J. Li and Z. J. Wang. 2009. Controlling the false discovery rate of the association\/causality structure learned with the PC algorithm. J. Mach. Learn. Res. 10 (June 2009) 475--514. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals;1577069.1577086."},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the Conference on Uncertainty in Artificial Intelligence, Ronald Parr and Linda C. van der Gaag (Eds.). AUAI Press, 251--258","author":"Listgarten J.","year":"2007","unstructured":"J. Listgarten and D. Heckerman . 2007. Determining the number of non-spurious arcs in a learned DAG model: Investigation of a Bayesian and a frequentist approach . In Proceedings of the Conference on Uncertainty in Artificial Intelligence, Ronald Parr and Linda C. van der Gaag (Eds.). AUAI Press, 251--258 . Retrieved from: http:\/\/dblp.uni-trier.de\/db\/conf\/uai\/uai 2007 .html#ListgartenH07. J. Listgarten and D. Heckerman. 2007. Determining the number of non-spurious arcs in a learned DAG model: Investigation of a Bayesian and a frequentist approach. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, Ronald Parr and Linda C. van der Gaag (Eds.). AUAI Press, 251--258. Retrieved from: http:\/\/dblp.uni-trier.de\/db\/conf\/uai\/uai2007.html#ListgartenH07."},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence (UAI\u201995)","author":"Meek C.","year":"1995","unstructured":"C. Meek . 1995 . Strong completeness and faithfulness in Bayesian networks . In Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence (UAI\u201995) . Morgan Kaufmann Publishers Inc., San Francisco, CA, 411--418. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals; 2074158.2074205. C. Meek. 1995. Strong completeness and faithfulness in Bayesian networks. In Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence (UAI\u201995). Morgan Kaufmann Publishers Inc., San Francisco, CA, 411--418. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals;2074158.2074205."},{"key":"e_1_2_1_21_1","volume-title":"Varshney","author":"Pegueroles Bernat Guillen","year":"2018","unstructured":"Bernat Guillen Pegueroles , Bhanukiran Vinzamuri , Karthikeyan Shanmugam , Steve Hedden , Jonathan D. Moyer , and Kush R . Varshney . 2018 . Structure learning from time series with false discovery control. Retrieved from: arXiv preprint arXiv:1805.09909 (2018). Bernat Guillen Pegueroles, Bhanukiran Vinzamuri, Karthikeyan Shanmugam, Steve Hedden, Jonathan D. Moyer, and Kush R. Varshney. 2018. Structure learning from time series with false discovery control. Retrieved from: arXiv preprint arXiv:1805.09909 (2018)."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1063\/1.5025050"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1105809"},{"key":"e_1_2_1_24_1","first-page":"2003","article-title":"A linear non-Gaussian acyclic model for causal discovery","author":"Shimizu Shohei","year":"2006","unstructured":"Shohei Shimizu , Patrik O. Hoyer , Aapo Hyv\u00e4rinen , and Antti Kerminen . 2006 . A linear non-Gaussian acyclic model for causal discovery . J. Mach. Learn. Res. 7 , Oct. (2006), 2003 -- 2030 . Shohei Shimizu, Patrik O. Hoyer, Aapo Hyv\u00e4rinen, and Antti Kerminen. 2006. A linear non-Gaussian acyclic model for causal discovery. J. Mach. Learn. Res. 7, Oct. (2006), 2003--2030.","journal-title":"J. Mach. Learn. Res. 7"},{"key":"e_1_2_1_25_1","volume-title":"DirectLiNGAM: A direct method for learning a linear non-Gaussian structural equation model. J. Mach. Learn. Res. 12 (July","author":"Shimizu Shohei","year":"2011","unstructured":"Shohei Shimizu , Takanori Inazumi , Yasuhiro Sogawa , Aapo Hyv\u00e4rinen , Yoshinobu Kawahara , Takashi Washio , Patrik O. Hoyer , and Kenneth Bollen . 2011. DirectLiNGAM: A direct method for learning a linear non-Gaussian structural equation model. J. Mach. Learn. Res. 12 (July 2011 ), 1225--1248. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals;1953048.2021040. Shohei Shimizu, Takanori Inazumi, Yasuhiro Sogawa, Aapo Hyv\u00e4rinen, Yoshinobu Kawahara, Takashi Washio, Patrik O. Hoyer, and Kenneth Bollen. 2011. DirectLiNGAM: A direct method for learning a linear non-Gaussian structural equation model. J. Mach. Learn. Res. 12 (July 2011), 1225--1248. Retrieved from: http:\/\/dl.acm.org\/citation.cfm?id&equals;1953048.2021040."},{"key":"e_1_2_1_26_1","doi-asserted-by":"crossref","unstructured":"P. Spirtes C. Glymour and R. Scheines. 2000. Causation Prediction and Search (2nd ed.). The MIT Press.  P. Spirtes C. Glymour and R. Scheines. 2000. Causation Prediction and Search (2nd ed.). The MIT Press.","DOI":"10.7551\/mitpress\/1754.001.0001"},{"key":"e_1_2_1_27_1","volume-title":"Handbook of Graphical Models","author":"Spirtes Peter","unstructured":"Peter Spirtes and Kun Zhang . 2018. Search for causal models . In Handbook of Graphical Models . CRC Press , 457--488. Peter Spirtes and Kun Zhang. 2018. Search for causal models. In Handbook of Graphical Models. CRC Press, 457--488."},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI\u201908)","author":"Tsamardinos I.","year":"2008","unstructured":"I. Tsamardinos and L. E. Brown . 2008. Bounding the false discovery rate in local Bayesian network learning . In Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI\u201908) . 1100--1105. Retrieved from: http:\/\/www.aaai.org\/Library\/AAAI\/ 2008 \/aaai08-174.php. I. Tsamardinos and L. E. Brown. 2008. Bounding the false discovery rate in local Bayesian network learning. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI\u201908). 1100--1105. Retrieved from: http:\/\/www.aaai.org\/Library\/AAAI\/2008\/aaai08-174.php."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-006-6889-7"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems. 9472--9483","author":"Zheng Xun","unstructured":"Xun Zheng , Bryon Aragam , Pradeep K. Ravikumar , and Eric P. Xing . 2018. DAGs with NO TEARS: Continuous optimization for structure learning . In Proceedings of the Conference on Advances in Neural Information Processing Systems. 9472--9483 . Xun Zheng, Bryon Aragam, Pradeep K. Ravikumar, and Eric P. Xing. 2018. DAGs with NO TEARS: Continuous optimization for structure learning. In Proceedings of the Conference on Advances in Neural Information Processing Systems. 9472--9483."}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3351342","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3351342","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:25:51Z","timestamp":1750206351000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3351342"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,30]]},"references-count":30,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2019,9,30]]}},"alternative-id":["10.1145\/3351342"],"URL":"https:\/\/doi.org\/10.1145\/3351342","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,9,30]]},"assertion":[{"value":"2018-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-10-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}