Datasets


We created essentially two sets of data:

Log Files

For both tracks of the IPC, you can download a full dataset and a reduced dataset. Note that all experiments were executed 10 times with different seeds. Per run, we defined a base seed. For every run, the actually used seed was the base seed plus the instance number. The full dataset contains the log files for all planners on every instance. Since this is a lot of data we also created a reduced data set.

Problem Instance Renaming

In the logs provided below, every instance is referred to with a numerical code. For technical reasons, the instance numbers are separated between the partial-order and the total-order tracks and partially overlap! After the experiments were run, the structure of the domain repository was changed to allow for the usage of LAB. As such, the file names of the instances in the log files may not be correct anymore. The following two files contain a mapping of instance numbers to the instance files. The paths refer to the github repository.

Full Data Set

Every log contains the full output of the planner and the output of the subsequent verification of the found plan. In some cases, the verification took longer than the maximum time-limit of the job on our compute cluster (primaliry very long plans in the Towers domain). All such plans were valid. The planners are identified by a number which is shown in the table of planner above. The file names of the full logs comprise several information. The most notable one is the last number before the long sequence of dashs: it is the instance number (see above).

Reduced Data Set

In addition, we offer a reduced dataset. This dataset contains one line per run of a planner in which it found a solution (not necessarily a correct one). Each line contains 6 entries:

The reduced data sets can be found here:

Plans

In addition to the logs, we also offer the plans that were generated by the planners in the IPC as a resource. This can for example be used for research in plan verification. We are still in the process of gathering these plans and translating them into a uniform format (casing, handling of special characters). We will also include plans generated by other planners on the competition domains. So far, we have gathered all plans that were generated by the IPC planners during the evaluation runs and plans generated by the planner PANDA. The current version of the plans is available on github. All duplicate plans have already been removed. We plan to add further plans to the repository in the future.