jmarkov.jmdp

## Class CTMDP<S extends State,A extends Action>

- java.lang.Object
- - jmarkov.jmdp.MDP<S,A>
  - - jmarkov.jmdp.InfiniteMDP<S,A>
    - - jmarkov.jmdp.CTMDP<S,A>

- Type Parameters:
    `S` - The state class
    `A` - The Action class
  Direct Known Subclasses:
    Bank2Queues, CTInventory, CTMDPEv, CTMDPEvA

      public abstract class CTMDP<S extends State,A extends Action>
      extends InfiniteMDP<S,A>

  This class represents a continuous time MDP. It should ONLY be used in INFINITE horizon Problems. It must be extended in order to represent the appropriate structure for each INFINITE horizon MDP problem. The user must implement at least the functions that have been declared abstract. It is also necessary to create one of the extensions of the class Solver. By default, the program includes PolicyIterationSolver and ValueIterationSolver classes to solve infinite horizon problems. The FiniteSolver class is only for finite horizon problems. To solve the problem follow the instructions in each of the solvers's instructions.
  Author:
    Andres Sarmiento, Germán Riaño - Universidad de Los Andes
  See Also:
    `PolicyIterationSolver`, `ValueIterationSolver`

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `CTMDP(States<S> initial)` Creates a new continuous time infinite horizon MDP Problem.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `abstract double` | `continuousCost(S i, A a)` Cost incurred continuously in time until the next transition from state i given that action a is taken.
    `StatesSet<S>` | `getAllStates()` Complete set of states explored
    `double` | `getMaxRate()`
    `ValueFunction<S>` | `getSteadyStateProbabilities()`
    `abstract double` | `lumpCost(S i, A a)` Cost incurred instantaneously in the moment when action a is taken from state i.
    `abstract double` | `rate(S i, S j, A a)` Rate of going from state i to state j by taking the action a
    `abstract States<S>` | `reachable(S i, A a)` Set of States that can be reached from this state i, after taking the action a.
    `void` | `setConverter(CT2DTConverter<S,A> converter)` Sets the class in charge of making a DTMDP equivalent to the CTMDP
    `Solution<S,A>` | `solve(double interestRate)` Solves the problem with the given interest rate

    - ### Methods inherited from class jmarkov.jmdp.InfiniteMDP

      `feasibleActions, getNumStates, getSolver`

    - ### Methods inherited from class jmarkov.jmdp.MDP

      `debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### CTMDP

          public CTMDP(States<S> initial)

      Creates a new continuous time infinite horizon MDP Problem.
      Parameters:
        `initial` - set of initial states for the exploration algorithm

  - ### Method Detail

    - #### getAllStates

          public StatesSet<S> getAllStates()

      Complete set of states explored
      Overrides:
        `getAllStates` in class `InfiniteMDP<S extends State,A extends Action>`
      Returns:
        set of states explored

    - #### setConverter

          public void setConverter(CT2DTConverter<S,A> converter)

      Sets the class in charge of making a DTMDP equivalent to the CTMDP
      Parameters:
        `converter` - class that makes a DTMDP equivalent to the CTMDP

    - #### getMaxRate

          public double getMaxRate()

      Returns:
        maximum exit rate for all states and all actions

    - #### getSteadyStateProbabilities

          public ValueFunction<S> getSteadyStateProbabilities()
                                                                     throws SolverException

      Returns:
        The steady state probability for each state
      Throws:
        `SolverException`

    - #### solve

          public Solution<S,A> solve(double interestRate)
                                                           throws SolverException

      Solves the problem with the given interest rate
      Parameters:
        `interestRate` - the interest rate parameter to solve the problem.
      Returns:
        The Optimal solution.
      Throws:
        `SolverException`

    - #### lumpCost

          public abstract double lumpCost(S i,
                        A a)

      Cost incurred instantaneously in the moment when action a is taken from state i.
      Parameters:
        `i` - State
        `a` - Action
      Returns:
        Lump cost received.

    - #### continuousCost

          public abstract double continuousCost(S i,
                              A a)

      Cost incurred continuously in time until the next transition from state i given that action a is taken.
      Parameters:
        `i` - State
        `a` - Action
      Returns:
        Rate at which cost is incurred when action a is taken.

    - #### reachable

          public abstract States<S> reachable(S i,
                            A a)

      Set of States that can be reached from this state i, after taking the action a.
      Parameters:
        `i` - current State
        `a` - action taken
      Returns:
        the reachable states.

    - #### rate

          public abstract double rate(S i,
                    S j,
                    A a)

      Rate of going from state i to state j by taking the action a
      Parameters:
        `i` - current state
        `j` - Destination state.
        `a` - Action taken
      Returns:
        The rate