jmarkov.jmdp

## Class MDP<S extends State,A extends Action>

- java.lang.Object
- - jmarkov.jmdp.MDP<S,A>

- Type Parameters:
    `S` - Current state.
    `A` - Action taken.
  Direct Known Subclasses:
    FiniteMDP, InfiniteMDP

      public abstract class MDP<S extends State,A extends Action>
      extends java.lang.Object

  This class is the main framework to build a Dynamic Programming Problem. It was initially created to work over Markov Decision Problems which imply random probabilities but can easily be worked out for deterministic problems if the probabilities are set to one. This class should not me extended directly on problems. The default package has FiniteMDP and InfiniteMDP classes that are intended to be extended on problems. See the examples for a clearer reference.
  Author:
    Andres Sarmiento, Germán Riaño - Universidad de Los Andes
  See Also:
    `FiniteMDP`, `DTMDP`

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `MDP()`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `debug(int level, java.lang.String message)` Prints a message in the reporter.
    `void` | `debug(int level, java.lang.String s, boolean newline)` Prints debug information in the reporter.
    `void` | `debug(int level, java.lang.String s, boolean newline, boolean indent)` Prints debug information in the reporter.
    `int` | `getDebugLevel()` Gets the current debug level.
    `Policy<S,A>` | `getOptimalPolicy()` Returns the optimal policy.
    `ValueFunction<S>` | `getOptimalValueFunction()` Returns the optimal ValueFunction.
    `DebugReporter` | `getReporter()`
    `Solver<S,A>` | `getSolver()`
    `boolean` | `isFinite()`
    `boolean` | `isSolved()` Indicates if the problems has been solved
    `double` | `operation(double present, double future)` The Operator between present and future costs.
    `void` | `printSolution()` Prints the solution to Standard output.
    `void` | `printSolution(java.io.PrintWriter pw)` Prints the solution to the given PrintWriter
    `void` | `setDebugLevel(int level)` Sets teh current level
    `void` | `setReporter(DebugReporter reporter)`
    `void` | `setSolver(Solver<S,A> solver)`
    `void` | `solve()` Solves the problem.

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### MDP

          public MDP()

  - ### Method Detail

    - #### getOptimalPolicy

          public final Policy<S,A> getOptimalPolicy()
                                                                          throws SolverException

      Returns the optimal policy. This function should be called only AFTER solve has been called.
      Returns:
        The optimal policy.
      Throws:
        `SolverException` - Exception thrown if a solution cannot be found

    - #### isSolved

          public final boolean isSolved()

      Indicates if the problems has been solved
      Returns:
        true if solved

    - #### isFinite

          public final boolean isFinite()

      Returns:
        Returns true if the problem's horizon is finite.

    - #### solve

          public final void solve()
                           throws SolverException

      Solves the problem. If no solver has been defined, this used the default solver.
      Throws:
        `SolverException` - This Exception is thrown if a solution cannot be found.
      See Also:
        `getDefaultSolver()`

    - #### getSolver

          public Solver<S,A> getSolver()

      Returns:
        Returns the solver.

    - #### setSolver

          public void setSolver(Solver<S,A> solver)

      Parameters:
        `solver` - The solver to set.

    - #### getOptimalValueFunction

          public ValueFunction<S> getOptimalValueFunction()
                                                                 throws SolverException

      Returns the optimal ValueFunction. This causes the problem to be solved if it has not been solved.
      Returns:
        Returns the valueFunction.
      Throws:
        `SolverException` - This exception is thrown if a solution cannot be found.

    - #### operation

          public double operation(double present,
                         double future)

      The Operator between present and future costs. By default is sum, but can be changed by the user, by overriding this method.
      Parameters:
        `present` - Cost of current transition
        `future` - Cost of future transitions.
      Returns:
        By default it returns present + future.

    - #### getReporter

          public DebugReporter getReporter()

      Returns:
        Returns the reporter.

    - #### setReporter

          public void setReporter(DebugReporter reporter)

      Parameters:
        `reporter` - The reporter to set.

    - #### debug

          public void debug(int level,
                   java.lang.String message)

      Prints a message in the reporter.
      Parameters:
        `level` - maximum debug level at which to show message
        `message` - message
      See Also:
        `DebugReporter`

    - #### debug

          public void debug(int level,
                   java.lang.String s,
                   boolean newline,
                   boolean indent)

      Prints debug information in the reporter.
      Parameters:
        `level` - the level for the info
        `s` - Message
        `newline` - true if a new line is to be inserted
        `indent` - true if the info is indented according to level
      See Also:
        `DebugReporter`, `DebugReporter.debug(int, java.lang.String, boolean, boolean)`

    - #### debug

          public void debug(int level,
                   java.lang.String s,
                   boolean newline)

      Prints debug information in the reporter.
      Parameters:
        `level` - the level for the info
        `s` - Message
        `newline` - true if a new line is to be inserted
      See Also:
        `DebugReporter`, `DebugReporter.debug(int, java.lang.String, boolean)`

    - #### getDebugLevel

          public int getDebugLevel()

      Gets the current debug level.
      Returns:
        The current debug level
      See Also:
        `DebugReporter`, `DebugReporter.getDebugLevel()`

    - #### setDebugLevel

          public void setDebugLevel(int level)

      Sets teh current level
      Parameters:
        `level` - The new level to level
      See Also:
        `DebugReporter`, `DebugReporter.setDebugLevel(int)`

    - #### printSolution

          public void printSolution()

      Prints the solution to Standard output.

    - #### printSolution

          public void printSolution(java.io.PrintWriter pw)

      Prints the solution to the given PrintWriter
      Parameters:
        `pw` - The PrintWriter where the solution will be printed. It must have been initialized.