{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T19:31:32Z","timestamp":1772911892097,"version":"3.50.1"},"reference-count":57,"publisher":"Maximum Academic Press","license":[{"start":{"date-parts":[[2018,4,18]],"date-time":"2018-04-18T00:00:00Z","timestamp":1524009600000},"content-version":"unspecified","delay-in-days":107,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["The Knowledge Engineering Review"],"published-print":{"date-parts":[[2018]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>The International Planning Competition (IPC) is a prominent event of the artificial intelligence planning community that has been organized since 1998; it aims at fostering the development and comparison of planning approaches, assessing the state-of-the-art in planning and identifying new challenging benchmarks. IPC has a strong impact also outside the planning community, by providing a large number of ready-to-use planning engines and testing pioneering applications of planning techniques.<\/jats:p>\n                  <jats:p>This paper focusses on the deterministic part of IPC 2014, and describes format, participants, benchmarks as well as a thorough analysis of the results. Generally, results of the competition indicates some significant progress, but they also highlight issues and challenges that the planning community will have to face in the future.<\/jats:p>","DOI":"10.1017\/s0269888918000012","type":"journal-article","created":{"date-parts":[[2018,4,19]],"date-time":"2018-04-19T01:15:43Z","timestamp":1524100543000},"source":"Crossref","is-referenced-by-count":17,"title":["What you always wanted to know about the deterministic part of the International Planning Competition (IPC) 2014 (but were too afraid to ask)"],"prefix":"10.48130","volume":"33","author":[{"given":"Mauro","family":"Vallati","sequence":"first","affiliation":[]},{"given":"Luk\u00e1\u0161","family":"Chrpa","sequence":"additional","affiliation":[]},{"given":"Thomas L.","family":"Mccluskey","sequence":"additional","affiliation":[]}],"member":"27968","published-online":{"date-parts":[[2018,4,18]]},"reference":[{"key":"S0269888918000012_ref35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1613\/jair.1240","article-title":"The 3rd international planning competition: results and analysis","volume":"20","author":"Long","year":"2003","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S0269888918000012_ref21","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1613\/jair.1705","article-title":"The fast downward planning system","volume":"26","author":"Helmert","year":"2006","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S0269888918000012_ref4","unstructured":"Belov A. , Diepold D. , Heule M. J. & J\u00e4rvisalo M. 2014. SAT competition 2014. http:\/\/www.satcompetition.org\/"},{"key":"S0269888918000012_ref18","doi-asserted-by":"crossref","first-page":"383","DOI":"10.7307\/ptt.v28i4.1952","article-title":"Using automated planning for traffic signals control","volume":"28","author":"Guli\u0107","year":"2016","journal-title":"PROMET-TrafficandTransportation"},{"key":"S0269888918000012_ref51","doi-asserted-by":"crossref","unstructured":"Valenzano R. A. , Nakhost H. , M\u00fcller M. , Schaeffer J. & Sturtevant N. R. 2012. Arvandherd: parallel planning with a portfolio. In ECAI, 786\u2013791.","DOI":"10.3233\/978-1-61499-098-7-786"},{"key":"S0269888918000012_ref2","unstructured":"Balint A. , Belov A. , Diepold D. , Gerber S. , J\u00e4rvisalo M. & Sinz C. 2012. SAT challenge 2012. http:\/\/www.satcompetition.org\/"},{"key":"S0269888918000012_ref5","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(95)00055-0"},{"key":"S0269888918000012_ref53","first-page":"717","volume-title":"AI Communications","volume":"28","author":"Vallati","year":"2015"},{"key":"S0269888918000012_ref40","unstructured":"N\u00fa\u00f1ez S. , Borrajo D. & L\u00f3pez C. L. 2015. Automatic construction of optimal static sequential portfolios for AI planning and beyond. Artificial Intelligence 226, 75\u2013101."},{"key":"S0269888918000012_ref45","unstructured":"Rintanen J. 2004. Phase transitions in classical planning: an experimental study. In ICAPS 2004, 101\u2013110."},{"key":"S0269888918000012_ref42","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1613\/jair.2972","article-title":"The LAMA planner: guiding cost-based anytime planning with landmarks","volume":"39","author":"Richter","year":"2010","journal-title":"Journal Artificial Intelligence Research"},{"key":"S0269888918000012_ref39","unstructured":"Nissim R. , Hoffmann J. & Helmert M. 2011. The merge-and-shrink planner: bisimulation-based abstraction for optimal planning. In IPC 2011 Planner Abstracts, 106\u2013107."},{"key":"S0269888918000012_ref55","unstructured":"Vallati M. , Hutter F. , Chrpa L. & McCluskey T. L. 2015c. On the effective configuration of planning domain models. In International Joint Conference on Artificial Intelligence (IJCAI). AAAI Press."},{"key":"S0269888918000012_ref9","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v38i1.2719"},{"key":"S0269888918000012_ref56","unstructured":"Vallati M. & Vaquero T. 2015. Towards a protocol for benchmark selection in IPC. In The 4th Workshop of the International Planning Competition."},{"key":"S0269888918000012_ref27","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1613\/jair.1982","article-title":"Engineering benchmarks for planning: the domains used in the deterministic part of IPC-4","volume":"26","author":"Hoffmann","year":"2006","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S0269888918000012_ref33","unstructured":"Kanefsky B. & Taylor W. 1991. Where the really hard problems are. In Proceedings of IJCAI 91, 163\u2013169."},{"key":"S0269888918000012_ref12","doi-asserted-by":"crossref","unstructured":"Edelkamp S. , Kissmann P. & Torralba \u00c1. 2015. Bdds strike back (in AI planning). In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence.","DOI":"10.1609\/aaai.v29i1.9349"},{"key":"S0269888918000012_ref23","unstructured":"Helmert M. & R\u00f6ger G. 2008. How good is almost perfect? In Proceedings of AAAI, 944\u2013949."},{"key":"S0269888918000012_ref6","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2015.09.008"},{"key":"S0269888918000012_ref28","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1613\/jair.855","article-title":"The FF planning system: fast plan generation through heuristic search","volume":"14","author":"Hoffmann","year":"2001","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S0269888918000012_ref8","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1613\/jair.1918","article-title":"Temporal planning using subgoal partitioning and resolution in sgplan","volume":"26","author":"Chen","year":"2006","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S0269888918000012_ref43","unstructured":"Richter S. , Westphal M. & Helmert M. 2011. Lama 2008 and 2011. In International Planning Competition, 117\u2013124."},{"key":"S0269888918000012_ref15","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1613\/jair.1183","article-title":"Planning through stochastic local search and temporal action graphs","volume":"20","author":"Gerevini","year":"2003","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S0269888918000012_ref19","unstructured":"Haslum P. 2011. Computing genome edit distances using domain-independent planning. In ICAPS'11 Scheduling and Planning Applications Workshop (SPARK)."},{"key":"S0269888918000012_ref14","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1613\/jair.1129","article-title":"PDDL2.1: an extension to PDDL for expressing temporal planning domains","volume":"20","author":"Fox","year":"2003","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S0269888918000012_ref34","doi-asserted-by":"crossref","unstructured":"Lipovetzky N. & Geffner H. 2011. Searching for plans with carefully designed probes. In Proceedings of ICAPS.","DOI":"10.1609\/icaps.v21i1.13470"},{"key":"S0269888918000012_ref25","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1613\/jair.1144","article-title":"The Metric-FF planning system: translating \u2018ignoring delete lists\u2019 to numeric state variables","volume":"20","author":"Hoffmann","year":"2003","journal-title":"Journal Artificial Intelligence Research"},{"key":"S0269888918000012_ref7","unstructured":"Cenamor I. , De La Rosa T. & Fern\u00e1ndez F. 2012. Mining IPC-2011 results. In WS-IPC 2012."},{"key":"S0269888918000012_ref17","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2008.10.012"},{"key":"S0269888918000012_ref32","unstructured":"Baier J. A. & McIlraith S. A. 2008. Planning with preferences. AI Magazine 29(4), 25\u201336."},{"key":"S0269888918000012_ref49","unstructured":"Torralba A. & Alc\u00e1zar V. 2013. Constrained symbolic search: on mutexes, BDD minimization and more. In Sixth Annual Symposium on Combinatorial Search (SoCS)."},{"key":"S0269888918000012_ref36","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2015.01.004"},{"key":"S0269888918000012_ref10","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2014.12.008"},{"key":"S0269888918000012_ref13","doi-asserted-by":"crossref","unstructured":"Fawcett C. , Vallati M. , Hutter F. , Hoffmann J. , Hoos H. H. & Leyton-Brown K. 2014. Improved features for runtime prediction of domain-independent planners. In Proceedings of the Twenty-Fourth International Conference on Automated Planning and Scheduling, ICAPS.","DOI":"10.1609\/icaps.v24i1.13680"},{"key":"S0269888918000012_ref48","unstructured":"Thimm M. , Villata S. , Cerutti F. , Oren N. , Strass H. & Vallati M. 2016. Summary report of the first international competition on computational models of argumentation. AI Magazine 37, 102."},{"key":"S0269888918000012_ref3","unstructured":"Balint A. , Belov A. , Heule M. J. & J\u00e4rvisalo M. 2013. SAT competition 2013. http:\/\/www.satcompetition.org\/"},{"key":"S0269888918000012_ref46","doi-asserted-by":"crossref","unstructured":"Rizzini M. , Fawcett C. , Vallati M. , Gerevini A. E. & Hoos H. 2015. Portfolio methods for optimal planning: an empirical analysis. In Proceedings of the IEEE International Conference on Tools with Artificial Intelligence (ICTAI-15). IEEE.","DOI":"10.1109\/ICTAI.2015.79"},{"key":"S0269888918000012_ref22","unstructured":"Helmert M. & Domshlak C. 2011. LM-cut: optimal planning with the landmark-cut heuristic. In IPC 2011 Planner Abstracts."},{"key":"S0269888918000012_ref26","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1613\/jair.3276","article-title":"Analyzing search topology without running any search: on the connection between causal graphs and h+","volume":"41","author":"Hoffmann","year":"2011","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S0269888918000012_ref37","unstructured":"Matloob R. & Soutchanski M. 2016. Exploring organic synthesis with state-of-the-art planning techniques. In Proceedings of ICAPS \u201916 Scheduling and Planning Applications Workshop (SPARK)."},{"key":"S0269888918000012_ref31","unstructured":"Hurley B. & O'Sullivan B. 2015. Statistical regimes and runtime prediction. In International Joint Conference on Artificial Intelligence (IJCAI). AAAI Press."},{"key":"S0269888918000012_ref50","doi-asserted-by":"crossref","unstructured":"Torralba \u00c1. , Edelkamp S. & Kissmann P. 2013. Transition trees for cost-optimal symbolic planning. In Twenty-Third International Conference on Automated Planning and Scheduling (ICAPS).","DOI":"10.1609\/icaps.v23i1.13558"},{"key":"S0269888918000012_ref38","unstructured":"Nakhost H. & M\u00fcller M. 2010. Action elimination and plan neighborhood graph search: two algorithms for plan improvement. In The Twentieth International Conference on Automated Planning and Scheduling (ICAPS), 121\u2013128."},{"key":"S0269888918000012_ref24","unstructured":"Helmert M. , R\u00f6ger G. & Karpas E. 2011. Fast downward stone soup: a baseline for building planner portfolios. In ICAPS 2011 Workshop on Planning and Learning, 28\u201335."},{"key":"S0269888918000012_ref16","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1613\/jair.4359","article-title":"Planning through automatic portfolio configuration: the pbp approach","volume":"50","author":"Gerevini","year":"2014","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S0269888918000012_ref41","unstructured":"Ramirez M. , Lipovetzky N. & Muise C. 2014. Lightweight automated planning toolkit. Technical report, http:\/\/lapkt. org."},{"key":"S0269888918000012_ref44","unstructured":"Rintanen J. 2012. Engineering efficient planners with SAT. In Proceedings of ECAI, 684\u2013689."},{"key":"S0269888918000012_ref57","volume-title":"Some Rapid Approximate Statistical Procedures","author":"Wilcoxon","year":"1964"},{"key":"S0269888918000012_ref1","unstructured":"Alc\u00e1zar V. , Veloso M. M. & Borrajo D. 2011. Adapting a rapidly-exploring random tree for automated planning. In Proceedings of the Fourth Annual Symposium on Combinatorial Search, SOCS."},{"key":"S0269888918000012_ref30","doi-asserted-by":"crossref","unstructured":"Howey R. , Long D. & Fox M. 2004. Val: automatic plan validation, continuous effects and mixed initiative planning using PDDL. In Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI-04), 294\u2013301. IEEE.","DOI":"10.1109\/ICTAI.2004.120"},{"key":"S0269888918000012_ref20","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(02)00364-8"},{"key":"S0269888918000012_ref52","unstructured":"Vallati M. , Chrpa L. , Grzes M. , McCluskey T. , Roberts M. & Sanner S. 2015a. The 2014 international planning competition: progress and trends. AI Magazine 36, 90\u201398."},{"key":"S0269888918000012_ref47","volume-title":"Handbook of constraint programming","author":"Rossi","year":"2006"},{"key":"S0269888918000012_ref11","unstructured":"Dr\u00e9o J. , Sav\u00e9ant P. , Schoenauer M. & Vidal V. 2011. Divide-and-evolve: the marriage of Descartes and Darwin. In Proceedings of the 7th International Planning Competition (IPC)."},{"key":"S0269888918000012_ref29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1613\/jair.935","article-title":"A critical assessment of benchmark comparison in planning","volume":"17","author":"Howe","year":"2002","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S0269888918000012_ref54","unstructured":"Vallati M. , Chrpa L. & McCluskey T. L. 2014. The 2014 IPC: description of participating planners of the deterministic track. https:\/\/helios.hud.ac.uk\/scommv\/IPC-14\/planners_actual.html"}],"container-title":["The Knowledge Engineering Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0269888918000012","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T14:42:11Z","timestamp":1767624131000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0269888918000012\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018]]},"references-count":57,"alternative-id":["S0269888918000012"],"URL":"https:\/\/doi.org\/10.1017\/s0269888918000012","relation":{},"ISSN":["0269-8889","1469-8005"],"issn-type":[{"value":"0269-8889","type":"print"},{"value":"1469-8005","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018]]},"article-number":"e3"}}