jmarkov.jmdp.solvers

## Class PolicyIterationSolverAvg<S extends State,A extends Action>

- java.lang.Object
- - jmarkov.jmdp.solvers.Solver<S,A>
  - - jmarkov.jmdp.solvers.AbstractInfiniteSolver<S,A>
    - - jmarkov.jmdp.solvers.AbstractAverageSolver<S,A>
      - - jmarkov.jmdp.solvers.PolicyIterationSolverAvg<S,A>

- Type Parameters:
    `S` - States class.
    `A` - Actions class.
  All Implemented Interfaces:
    JMarkovElement

      public class PolicyIterationSolverAvg<S extends State,A extends Action>
      extends AbstractAverageSolver<S,A>

  This class solves infinite horizon non-discounted problems using the policy iteration algorithm. It extends Solver and should only be used on infinite horizon problems. The objective function the solver uses is the long-run-Average cost. The result is a deterministic optimal policy for the given structure. Policy Iteration is a solver method this is always convergent in a finite number of iterations, under the right conditions. The algorithm has to solve a linear system of equations as big as the amount of states. When there are too many states, it is recommendable to use other solvers, or using the modified policy iteration (by using the second constructor). The advantage of using Policy Iteration is that the result is the true optimal solution and not an aproximation as in other common methods. The method starts with a policy. It solves the system of linear equations for the value functions for that policy. With this values it looks for a better policy. It then solves the value functions again and looks for a better policy. If this policy is equal to the last policy tried, it stops, in any other case it keeps improving the policy and updating the value functions.
  Author:
    Daniel F. Silva

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `PolicyIterationSolverAvg(DTMDP<S,A> problem)` The constructor method exclusively receives a problem of the type InfiniteMDP because this solver is only designed to work on infinite horizon problems.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `java.lang.String` | `description()` This method return a complete verbal describtion of this element.
    `ValueFunction<S>` | `getBias()`
    `double` | `getGain()`
    `long` | `getIterations()`
    `long` | `getProcessTime()`
    `java.lang.String` | `label()` The sub classes must return the Solver name.
    `void` | `printSolution()` Prints the solution in the default PrintWriter (System.out)
    `void` | `printSolution(java.io.PrintWriter pw)` Prints the solution on a given PrintWriter.
    `void` | `setPrintBias(boolean val)`
    `void` | `setPrintGain(boolean val)`
    `Solution<S,A>` | `solve()` Called to solve the problem.

    - ### Methods inherited from class jmarkov.jmdp.solvers.AbstractInfiniteSolver

      `getProblem`

    - ### Methods inherited from class jmarkov.jmdp.solvers.Solver

      `getOptimalPolicy, getOptimalValueFunction, getValueFunction, isSolved, setPrintProcessTime, setPrintValueFunction, toString`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

    - ### Methods inherited from interface jmarkov.basic.JMarkovElement

      `equals`

- - ### Constructor Detail

    - #### PolicyIterationSolverAvg

          public PolicyIterationSolverAvg(DTMDP<S,A> problem)

      The constructor method exclusively receives a problem of the type InfiniteMDP because this solver is only designed to work on infinite horizon problems. This solver solves the average objective function problem. There is no discount factor.
      Parameters:
        `problem` - the structure of the problem of type InfiniteMDP

  - ### Method Detail

    - #### solve

          public Solution<S,A> solve()
                                                           throws SolverException

      Description copied from class: `Solver`
      Called to solve the problem. This method MUST write the local variable policy and valueFunction.
      Specified by:
        `solve` in class `Solver<S extends State,A extends Action>`
      Returns:
        The solution Object taht contains the plicy and value fuenction.
      Throws:
        `SolverException` - This exception is thrown if the solver cannot find a solution for some reason.

    - #### description

          public java.lang.String description()

      Description copied from interface: `JMarkovElement`
      This method return a complete verbal describtion of this element. This description may contain multiple text rows.
      Specified by:
        `description` in interface `JMarkovElement`
      Overrides:
        `description` in class `Solver<S extends State,A extends Action>`
      Returns:
        A String describing this element.
      See Also:
        `JMarkovElement.label()`

    - #### label

          public java.lang.String label()

      Description copied from class: `Solver`
      The sub classes must return the Solver name.
      Specified by:
        `label` in interface `JMarkovElement`
      Specified by:
        `label` in class `Solver<S extends State,A extends Action>`
      Returns:
        A String label.
      See Also:
        `Solver.toString()`

    - #### getProcessTime

          public final long getProcessTime()

      Specified by:
        `getProcessTime` in class `Solver<S extends State,A extends Action>`
      Returns:
        Returns the processTime.

    - #### getIterations

          public final long getIterations()

      Specified by:
        `getIterations` in class `AbstractInfiniteSolver<S extends State,A extends Action>`
      Returns:
        Returns the iterations.

    - #### getGain

          public final double getGain()

    - #### getBias

          public final ValueFunction<S> getBias()

    - #### setPrintBias

          public void setPrintBias(boolean val)

    - #### setPrintGain

          public void setPrintGain(boolean val)

    - #### printSolution

          public void printSolution(java.io.PrintWriter pw)

      Prints the solution on a given PrintWriter.
      Overrides:
        `printSolution` in class `AbstractInfiniteSolver<S extends State,A extends Action>`
      Parameters:
        `pw` -
      See Also:
        `PrintWriter`

    - #### printSolution

          public void printSolution()
                             throws java.lang.Exception

      Prints the solution in the default PrintWriter (System.out)
      Overrides:
        `printSolution` in class `Solver<S extends State,A extends Action>`
      Throws:
        `java.lang.Exception`