jmarkov.jmdp

## Class CTMDPEvA<S extends State,A extends Action,E extends Event>

- java.lang.Object
- - jmarkov.jmdp.MDP<S,A>
  - - jmarkov.jmdp.InfiniteMDP<S,A>
    - - jmarkov.jmdp.CTMDP<StateEvent<S,E>,A>
      - - jmarkov.jmdp.CTMDPEvA<S,A,E>

- Type Parameters:
    `S` - The staes class
    `A` - Actions class
    `E` - Events class

      public abstract class CTMDPEvA<S extends State,A extends Action,E extends Event>
      extends CTMDP<StateEvent<S,E>,A>

  This class represents an Infinite horizon, continuous time Markov Decision Process with events where actions depend on events. It allows the definition of events that can occur in a given state and this makes the cost and probability definition easier than in the cases where no events are defined.
  Author:
    Andres Sarmiento and Germán Riaño. Universidad de los Andes.

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `CTMDPEvA(States<S> initial)` Creates a new continuous time infinite horizon MDP Problem with events

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `abstract Events<E>` | `activeEvents(S i)` Set of events that are active from state i given that action a is taken.
    `abstract double` | `continuousCost(S i, A a, E e)` Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.
    `double` | `continuousCost(StateEvent<S,E> i, A a)` Cost incurred continuously in time until the next transition from state i given that action a is taken.
    `abstract Actions<A>` | `feasibleAct(S i)` Returns the set of actions available at this state.
    `Actions<A>` | `feasibleActions(StateEvent<S,E> s)` Returns the set of actions available at this state.
    `abstract double` | `lumpCost(S i, A a, E e)` Reward instantaneously gained in the moment when action a is taken from state i.
    `double` | `lumpCost(StateEvent<S,E> i, A a)` Cost incurred instantaneously in the moment when action a is taken from state i.
    `abstract double` | `rate(S i, S j, A a, E e)` Rate.
    `double` | `rate(StateEvent<S,E> i, StateEvent<S,E> j, A a)` Rate of going from state i to state j by taking the action a
    `States<StateEvent<S,E>>` | `reachable(StateEvent<S,E> i, A a)` Set of States that can be reached from this state i, after taking the action a.
    `abstract States<S>` | `reached(S i, A a, E e)` Set of reachable states from state i given that action a is taken and event e occurs.

    - ### Methods inherited from class jmarkov.jmdp.CTMDP

      `getAllStates, getMaxRate, getSteadyStateProbabilities, setConverter, solve`

    - ### Methods inherited from class jmarkov.jmdp.InfiniteMDP

      `getNumStates, getSolver`

    - ### Methods inherited from class jmarkov.jmdp.MDP

      `debug, debug, debug, getDebugLevel, getOptimalPolicy, getOptimalValueFunction, getReporter, isFinite, isSolved, operation, printSolution, printSolution, setDebugLevel, setReporter, setSolver, solve`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### CTMDPEvA

          public CTMDPEvA(States<S> initial)

      Creates a new continuous time infinite horizon MDP Problem with events
      Parameters:
        `initial` - set of initial states for the exploration algorithm

  - ### Method Detail

    - #### lumpCost

          public double lumpCost(StateEvent<S,E> i,
                        A a)

      Description copied from class: `CTMDP`
      Cost incurred instantaneously in the moment when action a is taken from state i.
      Specified by:
        `lumpCost` in class `CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>`
      Parameters:
        `i` - State
        `a` - Action
      Returns:
        Lump cost received.

    - #### continuousCost

          public double continuousCost(StateEvent<S,E> i,
                              A a)

      Description copied from class: `CTMDP`
      Cost incurred continuously in time until the next transition from state i given that action a is taken.
      Specified by:
        `continuousCost` in class `CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>`
      Parameters:
        `i` - State
        `a` - Action
      Returns:
        Rate at which cost is incurred when action a is taken.

    - #### rate

          public double rate(StateEvent<S,E> i,
                    StateEvent<S,E> j,
                    A a)

      Description copied from class: `CTMDP`
      Rate of going from state i to state j by taking the action a
      Specified by:
        `rate` in class `CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>`
      Parameters:
        `i` - current state
        `j` - Destination state.
        `a` - Action taken
      Returns:
        The rate

    - #### reachable

          public States<StateEvent<S,E>> reachable(StateEvent<S,E> i,
                                          A a)

      Description copied from class: `CTMDP`
      Set of States that can be reached from this state i, after taking the action a.
      Specified by:
        `reachable` in class `CTMDP<StateEvent<S extends State,E extends Event>,A extends Action>`
      Parameters:
        `i` - current State
        `a` - action taken
      Returns:
        the reachable states.

    - #### feasibleActions

          public Actions<A> feasibleActions(StateEvent<S,E> s)

      Description copied from class: `InfiniteMDP`
      Returns the set of actions available at this state.
      Specified by:
        `feasibleActions` in class `InfiniteMDP<StateEvent<S extends State,E extends Event>,A extends Action>`
      Parameters:
        `s` - Current State
      Returns:
        set of Actions that can be taken at this state.

    - #### rate

          public abstract double rate(S i,
                    S j,
                    A a,
                    E e)

      Rate. Rate of going of reaching state j given that the current state is i, the action taken is a and the event that occurs is e.
      Parameters:
        `i` - current state
        `j` - state to reach
        `a` - action taken (given)
        `e` - event that occurs (given)
      Returns:
        Rate

    - #### reached

          public abstract States<S> reached(S i,
                          A a,
                          E e)

      Set of reachable states from state i given that action a is taken and event e occurs.
      Parameters:
        `i` - current state
        `a` - action taken
        `e` - event that occurs
      Returns:
        set of reachable states.

    - #### activeEvents

          public abstract Events<E> activeEvents(S i)

      Set of events that are active from state i given that action a is taken.
      Parameters:
        `i` - current state
      Returns:
        set of events that can occur

    - #### lumpCost

          public abstract double lumpCost(S i,
                        A a,
                        E e)

      Reward instantaneously gained in the moment when action a is taken from state i.
      Parameters:
        `i` - current state
        `a` - action taken
        `e` - event that occurs
      Returns:
        instanteneous reward.

    - #### continuousCost

          public abstract double continuousCost(S i,
                              A a,
                              E e)

      Reward obtained continuously in time during the sojourn time in state i until an action a is taken and a transition is triggered.
      Parameters:
        `i` - current state
        `a` - action taken
        `e` - event that occurs
      Returns:
        instanteneous reward.

    - #### feasibleAct

          public abstract Actions<A> feasibleAct(S i)

      Returns the set of actions available at this state. The user must implement this method.
      Parameters:
        `i` - current state
      Returns:
        set of feasible actions