Datasets

---

We created essentially two sets of data:

• All log files of the planners for the competition (with all detailed results) and
• all produced plans (including invalid ones)

Log Files

For both tracks of the IPC, you can download a full dataset and a reduced dataset. Note that all experiments were executed 10 times with different seeds. Per run, we defined a base seed. For every run, the actually used seed was the base seed plus the instance number. The full dataset contains the log files for all planners on every instance. Since this is a lot of data we also created a reduced data set.

Problem Instance Renaming

In the logs provided below, every instance is referred to with a numerical code. For technical reasons, the instance numbers are separated between the partial-order and the total-order tracks and partially overlap! After the experiments were run, the structure of the domain repository was changed to allow for the usage of LAB. As such, the file names of the instances in the log files may not be correct anymore. The following two files contain a mapping of instance numbers to the instance files. The paths refer to the github repository.

• Instance name mapping of partial order track
• Instance name mapping of total order track

Full Data Set

Every log contains the full output of the planner and the output of the subsequent verification of the found plan. In some cases, the verification took longer than the maximum time-limit of the job on our compute cluster (primaliry very long plans in the Towers domain). All such plans were valid. The planners are identified by a number which is shown in the table of planner above. The file names of the full logs comprise several information. The most notable one is the last number before the long sequence of dashs: it is the instance number (see above).

• partial order track (148 MB)
• total order track (6.7 GB)

Reduced Data Set

In addition, we offer a reduced dataset. This dataset contains one line per run of a planner in which it found a solution (not necessarily a correct one). Each line contains 6 entries:

• A string describing the planner. The number contained in this string corresponds to the number of the planner.
• The instance number (see above).
• The domain number. This number has no deeper meaning, it is just a unique number given to all instances of one domain.
• Runtime in seconds. This already includes the reduction of 0.12 seconds given to all planners due to singularity startup time.
• Peak memory usage in kilobytes. This already includes the reduction by 20000 kb given to all planners due to the memory used by singularity.
• Length of the found plans in number of actions.

The reduced data sets can be found here:

• partial order track
• total order track

Plans

In addition to the logs, we also offer the plans that were generated by the planners in the IPC as a resource. This can for example be used for research in plan verification. We are still in the process of gathering these plans and translating them into a uniform format (casing, handling of special characters). We will also include plans generated by other planners on the competition domains. So far, we have gathered all plans that were generated by the IPC planners during the evaluation runs and plans generated by the planner PANDA. The current version of the plans is available on github. All duplicate plans have already been removed. We plan to add further plans to the repository in the future.