Algos Expeditus

We are going to assume that the state used by the agent is a sufficient statistic of the history, in that, in order to predict the future, you only need to know about the current state of the environment.

This implies that the future is independent of the past given the present (if you have the right aggregate statistics in the present)

• Information State : sufficient statistic of history
• State s_t is Markov iff p(s_t+1|s^t, a_t) = p(s_t+1|h_t, a_t)