Instructions for Building and Running the PPDDL MDP Simulator
-------------------------------------------------------------

This package has been prepared with the GNU build tools.  See
`INSTALL' for generic instruction on how to build the MDP simulator.

On most systems you should be able to build the server and a simple
client in two easy steps:

1. Type `./configure' to configure the package for your system.

2. Type `make' to compile the package.

If all goes well, this should produce two executables: the server
`mdpsim' and a simple client `mdpclient'.

Run `./mdpsim --help' for brief information on how to run the MDP
server.

To test, first start a server (we specify a time limit of 10 seconds
instead of using the default time limit of 15 minutes):

  ./mdpsim --port=2323 --time-limit=10000 examples/john.pddl &

Next, start a simple client:

  ./mdpclient --host=localhost --port=2323 examples/john.pddl

The client should print information on the simulation runs and exit.
To stop the server, kill the associated process.


Troubleshooting
---------------

The package has been successfully compiled on the following platforms
using various versions of GCC:

  RedHat Linux 6.2 [GCC 2.95.3]
  RedHat Linux 7.1 [GCC 3.2.3]
  RedHat Linux 9.0 [GCC 3.2.2, 4.0.1]
  SunOS 5.7 [GCC 3.2.3]
  SunOS 5.9 [GCC 4.0.0]

If you see something like

  ...
  checking whether the C++ compiler works... configure: error: cannot run C++ compiled programs.
  If you meant to cross compile, use `--host'.
  See `config.log' for more details.

when you run `./configure', then make sure that libstdc++.so.* is in
the search path for shared libraries.  You may have to set the
environment variable LD_LIBRARY_PATH before running the configure
script.

If you encounter any other problems when configuring or compiling the
package, send the output produced by the configure script, including
the file `config.log', or the output generated by make to
<[email protected]>.

Bug reports and feature requests should be filed online:

  http://code.google.com/p/mdpsim/issues/list


Interpreting the Test Output
----------------------------

The server tests the sample planner (mdpclient, which executes random
legal actions) on the domain defined in the file examples/john.pddl on
the problem `john-problem' defined there.  It executes the domain
twice---this parameter is set in the configuration file comp.cfg (the
second number on the first line).

Here is an example output (newlines added for clarity):

<end-round>
 <state>
  <atom><predicate>has-soap</predicate></atom>
  <atom><predicate>has-soccer-ball</predicate></atom>
  <atom><predicate>shower-is-on</predicate></atom>
  <fluent><function>reward</function><value>14</value></fluent>
 </state>
 <time-spent>3</time-spent>
 <turns-used>5662</turns-used>
</end-round>

<end-round>
 <state>
  <atom><predicate>has-soap</predicate></atom>
  <atom><predicate>has-soccer-ball</predicate></atom>
  <fluent><function>reward</function><value>-30</value></fluent>
 </state>
 <time-spent>3</time-spent>
 <turns-used>7861</turns-used>
</end-round>

<end-session>
 <sessionID>62</sessionID>
 <rounds>2</rounds>
 <goals>
  <failed>2</failed>
  <reached><successes>0</successes></reached>
 </goals>
 <metric-average>-8</metric-average>
</end-session>

It shows the final state of two rounds for this domain (positive
literals are listed).  It also reports the reward value for each
round, the time spent, and the number of steps taken.  A final report
gives the average value of the reward score ((+14-30)/2 = -8) as well
the number of successful vs. unsuccessful rounds.


Communication Protocol
----------------------

Upon connecting to the server, the client issues a message of the form:

  <session-request>
    <name> -some arbitrary identifier- </name>
    <problem> -the name of the problem to work on- </problem>
  </session-request>

The server will respond with:

  <session-init>
    <sessionID> -some number- </sessionID>
    <setting>
      <rounds> -the times you can try the problem- </rounds>
      <allowed-time> -time limit- </allowed-time>
      <allowed-turns> -turn limit- </allowed-turns>
    </setting>
  </session-init>


The client will then send:

  <round-request/>


The server responds with:

  <round-init>
    <sessionID> -that same  number- </sessionID>
    <round> -the count of how many rounds you've done- </round>
    <time-left> -time remaining until time limit is reached- </time-left>
    <rounds-left> -rounds remaining- </rounds-left>
  </round-init>

At this point the client may do any calculations, as the clock is running.

The server will send a state or end-round message:

  <end-round>
    <sessionID> -that same number- </sessionID>
    <round> -the count of how many rounds you've done- </round>
    -state-
    <goal-reached/> (this will only appear if the goal has been met)
    <time-spent> -time spend in this round- </time-spent>
    <turns-used> -turns used in this round- </turns-used>
  </end-round>

  <state>
    <is-goal/> (this will only appear if the goal has been met)
    -atoms-
    -fluents-
  </state>

An atom:

  <atom>
    <predicate> -the name of the predicate- </predicate>
    <term> -a term- </term>  (0 or more of these)
    <term> -another term- </term>
  </atom>

A fluent:

  <fluent>
    <function> -the name of the function- </function>
    <term> -a term- </term>  (0 or more of these)
    <term> -another term- </term>
    <value> -fluent value- </value>
  </fluent>

The client responds to state messages with actions:

  <act>
    <action>
      <name> -the name of the action- </name>
      <term> -a term- </term>  (0 or more of these)
      <term> -another term- </term>
    </action>
  </act>

After the last round, the server sends a final message summarizing the
session:

  <end-session>
    <sessionID> -that same number- </sessionID>
    <rounds> -number of rounds- </rounds>
    <goals>
      <failed> -number of failed rounds- </failed>
      <reached>
        <successes> -number of successful rounds- </successes>
        <time-average> -time average per successful round- </time-average>
        <turn-average> -turn average per successful round- </turn-average>
      </reached>
    </goal>
    <metric-average> -metric average over all rounds- </metric-average>
  </end-session>

Time and turn average is present only if there is at least one successful
round.

For more details on the communication protocol, see:

  Håkan L. S. Younes, Michael L. Littman, David Weissman, and John
    Asmuth. 2005. "The first probabilistic track of the international planning
    competition." Journal of Artificial Intelligence Research 24: 851-887.


Conversion to ADD/MTBDD Representation
--------------------------------------

The file `mtbdd.cc' contains code for constructing ADD/MTBDD
representations of transition probability matrices and reward vectors
for PPDDL actions.  The code uses the CUDD package for decision
diagram manipulation, which can be obtained at the following location:

  http://vlsi.colorado.edu/~fabio/

You can build a simple client program that uses the converter with
`make mtbddclient', but before doing so you need to install the CUDD
link libraries and headers in a single directory and configure the
package with the following command:

  ./configure CUDDDIR=<location of cudd files>

The default value of CUDDDIR is `./cudd'.

To set up CUDD, try the following:

  wget ftp://vlsi.colorado.edu/pub/cudd-2.4.1.tar.gz
  gunzip cudd-2.4.1.tar.gz
  tar xvf cudd-2.4.1.tar
  ln -s cudd-2.4.1/cudd
  cd cudd-2.4.1
  [hand-edit the makefile]
  make

To test the MTBDD client, first start a server:

  ./mdpsim --port=2323 --config=comp.cfg examples/coffee-domain.pddl examples/coffee-problem.pddl &

Next, start the client:

  ./mtbddclient -v1 --host=localhost --port=2323 examples/coffee-domain.pddl examples/coffee-problem.pddl

You can increase the verbosity level by using -v2 or -v3 instead of
-v1.  This will provide more detailed output from the conversion to
MTBDDs and the progress of value iteration.