jmarkov.jmdp

## Class DTMDP<S extends State,A extends Action>

- java.lang.Object
- - jmarkov.jmdp.MDP<S,A>
  - - jmarkov.jmdp.InfiniteMDP<S,A>
    - - jmarkov.jmdp.DTMDP<S,A>

- Type Parameters:
    `S` - State class
    `A` - Action class
  Direct Known Subclasses:
    ControlProdNonEvents, CT2DTConverter, DTMDPEv, DTMDPEvA, InfStochasticDemand, LeadTimeStochasticDemand, OrderProcessing, ProbsExample, StochasticShortestPath

      public abstract class DTMDP<S extends State,A extends Action>
      extends InfiniteMDP<S,A>

  This class represents a discrete time infnite horizon MDP problem. It must be extended in order to represent the appropriate structure for each problem. The user must implement at least the functions that have been declared abstract.
  Author:
    Andres Sarmiento, Germán Riaño - Universidad de Los Andes
  See Also:
    `PolicyIterationSolver`, `ValueIterationSolver`

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `DTMDP(S initial)` Creates a new infinite horizon discrete time (MDP) Problem.
    `DTMDP(States<S> initial)` Creates a new infinite horizon discrete time (MDP) Problem.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `ValueFunction<S>` | `getSteadyStateProbabilities()`
    `abstract double` | `immediateCost(S i, A a)` Cost incurred when taking action a from state i
    `abstract double` | `prob(S i, S j, A a)` Probability of going from state i to state j by taking the action a
    `abstract States<S>` | `reachable(S i, A a)` Set of states that can be reached from this state i, after taking the action a.
    `void` | `setProbabilitySolver(ProbabilitySolver<S,A> solv)`
    `Solution<S,A>` | `solve(double interestRate)` Solves the problem with the given interest rate

    - ### Methods inherited from class jmarkov.jmdp.InfiniteMDP

      `feasibleActions, getAllStates, getNumStates, getSolver`

    - ### Methods inherited from class jmarkov.jmdp.MDP

      `debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### DTMDP

          public DTMDP(States<S> initial)

      Creates a new infinite horizon discrete time (MDP) Problem.
      Parameters:
        `initial` - set of initial states for the exploration algorithm

    - #### DTMDP

          public DTMDP(S initial)

      Creates a new infinite horizon discrete time (MDP) Problem.
      Parameters:
        `initial` - An initial state for the exploration algorithm

  - ### Method Detail

    - #### immediateCost

          public abstract double immediateCost(S i,
                             A a)

      Cost incurred when taking action a from state i
      Parameters:
        `i` - Current State
        `a` - Current Action
      Returns:
        The cost incurred per transition

    - #### prob

          public abstract double prob(S i,
                    S j,
                    A a)

      Probability of going from state i to state j by taking the action a
      Parameters:
        `i` - Current state.
        `j` - Destination State
        `a` - Action
      Returns:
        The probability.

    - #### reachable

          public abstract States<S> reachable(S i,
                            A a)

      Set of states that can be reached from this state i, after taking the action a.
      Parameters:
        `i` - Current State
        `a` - Action taken
      Returns:
        The reachable states.

    - #### getSteadyStateProbabilities

          public ValueFunction<S> getSteadyStateProbabilities()
                                                                     throws SolverException

      Returns:
        a map with the probability for each state.
      Throws:
        `SolverException`

    - #### setProbabilitySolver

          public void setProbabilitySolver(ProbabilitySolver<S,A> solv)

      Parameters:
        `solv` - Sets the solver that solves the steady state probabilities.

    - #### solve

          public final Solution<S,A> solve(double interestRate)
                                                                 throws SolverException

      Solves the problem with the given interest rate
      Parameters:
        `interestRate` - the interest rate parameter to solve the problem.
      Returns:
        The soultion to the problem.
      Throws:
        `SolverException`