jmarkov.jmdp

## Class FiniteMDP<S extends State,A extends Action>

- java.lang.Object
- - jmarkov.jmdp.MDP<S,A>
  - - jmarkov.jmdp.FiniteMDP<S,A>

- Type Parameters:
    `S` - States class
    `A` - Actions class
  Direct Known Subclasses:
    CattleGrowth, FiniteDP, FiniteMDPEv, Inventory, StochasticDemand

      public abstract class FiniteMDP<S extends State,A extends Action>
      extends MDP<S,A>

  This class should ONLY be used in FINITE horizon problems. It must be extended in order to represent the appropriate structure for each FINITE Dynamic Programming problem. The user must implement at least the functions that have been declared abstract. It´s also necessary to create one of the extensions of the class Solver. By default, the program includes the FiniteSolver class to solve finite horizon problems. PolicyIterationSolver and ValueIterationSolver are only for infinite horizon problems. To solve the problem follow the instructions in each of the solvers´ instructions.
  Author:
    Andres Sarmiento, Germán Riaño - Universidad de Los Andes
  See Also:
    `FiniteSolver`

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `FiniteMDP(S initial, int horizon)` Creates a finite horizon MDP.
    `FiniteMDP(States<S> initial, int horizon)` Creates a new FINITE horizon (MDP) Problem.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `double` | `defaultFinalCost(S i)` This method returns the cost incurred if the last stage ends with the system at state i choosing the best immediate cost.
    `abstract Actions<A>` | `feasibleActions(S i, int t)` Returns the actions available at this state i and at this stage t .
    `abstract double` | `finalCost(S i)` This method returns the cost incurred if the last stage ends with the system at state i.
    `int` | `getHorizon()` Returns the time lastStage
    `States<S>` | `getStates(int t)` All the states that are available at stage t.
    `abstract double` | `immediateCost(S i, A a, int t)` This function must return the Immediate cost incurred when taking action a from state i
    `abstract double` | `prob(S i, S j, A a, int t)` This is the probability of going from state i to state j by taking the action a at stage t.
    `abstract States<S>` | `reachable(S i, A a, int t)` Set of States that can be reached from this state i, at this stage t, after taking the acton a.

    - ### Methods inherited from class jmarkov.jmdp.MDP

      `debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, getSolver, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### FiniteMDP

          public FiniteMDP(States<S> initial,
                   int horizon)

      Creates a new FINITE horizon (MDP) Problem.
      Parameters:
        `initial` - set of initial states
        `horizon` - last stage at which actions can be taken

    - #### FiniteMDP

          public FiniteMDP(S initial,
                   int horizon)

      Creates a finite horizon MDP.
      Parameters:
        `initial` - a initial state
        `horizon` - horizon.

  - ### Method Detail

    - #### immediateCost

          public abstract double immediateCost(S i,
                             A a,
                             int t)

      This function must return the Immediate cost incurred when taking action a from state i
      Parameters:
        `i` - Current state
        `a` - Action
        `t` - Current time stage
      Returns:
        Cost value

    - #### prob

          public abstract double prob(S i,
                    S j,
                    A a,
                    int t)

      This is the probability of going from state i to state j by taking the action a at stage t.
      Parameters:
        `i` - Current state
        `j` - Destination state
        `a` - Action taken
        `t` - Current time stage
      Returns:
        Probability

    - #### reachable

          public abstract States<S> reachable(S i,
                            A a,
                            int t)

      Set of States that can be reached from this state i, at this stage t, after taking the acton a. The user must implement this method.
      Parameters:
        `i` - Current state
        `a` - Action taken
        `t` - Time stage
      Returns:
        Set of reachable states.

    - #### getStates

          public States<S> getStates(int t)

      All the states that are available at stage t.
      Parameters:
        `t` - time stage
      Returns:
        States available at stage t.

    - #### finalCost

          public abstract double finalCost(S i)

      This method returns the cost incurred if the last stage ends with the system at state i. The user must extend this method.
      Parameters:
        `i` - Ending state
      Returns:
        Cost.

    - #### defaultFinalCost

          public final double defaultFinalCost(S i)

      This method returns the cost incurred if the last stage ends with the system at state i choosing the best immediate cost.
      Parameters:
        `i` - Ending State
      Returns:
        Cost.

    - #### feasibleActions

          public abstract Actions<A> feasibleActions(S i,
                                   int t)

      Returns the actions available at this state i and at this stage t . The user must implement this method.
      Parameters:
        `i` - Current State
        `t` - Time stage
      Returns:
        Set of feasible actions.

    - #### getHorizon

          public int getHorizon()

      Returns the time lastStage
      Returns:
        Time horizon