The universe is a Markov chain. A Markov chain is a state diagram (typically finite), wherein there is a transition matrix which governs the probability of moving from one state to another state. The universe may be considered as a continuous time Markov chain, but it appears that time is granular at a Planck time length, so the continous model may be more of an approximation.

A Markov chain can be understood as a graph, with transitions between the nodes on the graph. Each node represents a state of the system. For example, we might consider a Markov chain which is simply whether a person eat cereal in the morning or not. After careful analysis, we discover that a person will typically eat cereal with a 70% probability if they just ate cereal the day before, and that they will eat cereal with a 45% probability if they just ate something else the day before.

In this graph there are only two nodes:

1) eating cereal (70% goes to 1, 30% goes to 2)
2) eating something else (45% goes to 1, 55% goes to 2)

This Markov chain is a "gross approximation", for it accurately represents the state of the system as it is beholden by the person doing measurements, but leaves a lot of information out of the Markov chain.

For example, what if we found out that the person actually ALWAYS ate cereal as long as there was some in the box, but then would not eat any until they got some more. This still could obey the probability distributions above, but a more sophisticated Markov chain would represent more states- that is the states which include a full box of cereal, a state with a box missing one bowl of cereal, and so forth.

Now, if you get more and more refined, until you are considering the individual atomic positions of every atom in the environment, you are considering the Markov chain of the universe itself. Note that the Markov chain contains EVERYTHING in the system and OMITS the knowledge state of the observer. Or more precisely, the Markov chain contains the knowledge state of the observer as part of itself (the specific neurons that construct the memories of the individual) but omits the perceptual ability or individual's model of the world (if the individual does not know that the box of cereal got thrown out by someone else, this would be represented in the universe Markov chain as a missing cereal box, but not in their conceptual model).

Now for some primer studies on the universe as a Markov chain:

1) Is every state accessible from every other state?

It appears from quantum theory that it is possible this is the case, that is, it is possible that a particular electron can jump a large distance probabilistically. Further, an ensemble of atoms could, in theory, "teleport" to a new configuration. Thus, for a given quantum mechanical ensemble, it appears that it CAN move from any state to any other state, but most state transitions have a probability which approaches zero. Therefore, for all practical purposes, every state is NOT accessible from every other state, although theoretically it is. This is compounded by gravitational forces which do not have a probabilistic element- that is- can an atom shift its state probabilistically relative to a strong gravitational well? This is perhaps an unknown question. However, it appears to be impossible that an entire planet could "jump" to a new orbital position.

2) So which future states are actually accessible to a physical system?

If you observe the macroscopic world, by and large it behaves deterministically- that is- most of the world is completely constrained by the immediate previous history of the system as a whole. For a particular state of the Markov chain- there are actually only a few possible future states which are in reality accessible (at least on a macroscopic level). These possible futures are constrained by the physical laws we know, love, and live within.

3) So why is there a plural "states" available to the future of the Markov chain rather than a singular "state"?

There are a number of physical situations which allow for there to exist multiple futures. These are at symmetry breaking events, such as the folding of the space time continuum during early cosmological expansion, and at "amplified" events which can carry into the future, such as synapsis firing within neural structures, the nondeterministic mutation of DNA on a molecular level, the shattering of objects, the edges of tipping points, so called "chaotic" systems, and others.

4) What are the other nodes on the graph of the Universe Markov chain?

The nodes on the graph represent every physically available configuration which the universe can exist within. Thus, they represent every possible configuration of matter, including every possible configuration of matter which represents yourself. The universe does not enter most of these nodes obviously, but moves from node to node obeying the physical laws of the system as they are defined. If you visualize the graph as a huge piece of graph paper, you can imagine the universe moving from lattice point to lattice point, with possible branches at each intersection, but it ends up following a single trajectory.

5) What do you mean "every possible configuration of matter, including yourself"?

You, as an individual, can exist in many different states- you have experienced a huge number of these states during your life. If we consider the nodes to exist at a time differential of Planck time then you have lived 10e43 configurations times the number of seconds you have been alive. For a 25 year old you have experienced roughly 10e51 states, unless you keep doing the same things over and over, then it is fewer. But also, there exists potentially every other life you could have lived, which is a large, but finite, number of potential states. Note that these states represent your perception of the world in that your memories are contained within the state. Thus, your same biochemstry passing through a different series of events would have a different set of states than the life you have experienced to this time, simply because your perception would be different, and your internal representation of reality would also differ.

6) Is the universe a recursive object?

Yes, the universe is a recursive object in which a Markov transition probability matrix is applied to each state iteratively, projecting into the future. The Markov transition probability matrix is determined by physical law. Note that the universe did not begin with multiple states- nor did it exist, as far as we can tell, with multiple states in the past. The universe follows a specific trajectory which is recursively defined into the past.

7) Does the universe have an absorption state?

An absorption state in a Markov chain is a specific state which has a 100% probability of moving back to itself. Once a Markov chain enters such a state, it can not escape this state. If the universe collapses back on itself, it might be understood that the final moment is an absorption state, and that then the Universe gets stuck forever in a single point or something. However, it is unknown what the future gravitational trajectory of the universe is. For specific parts of the universe, these specific sections can be considered to some degree to be absorption states. For example, a black hole sucks in all matter and the universe is unable to move to a new state which is not a black hole- HOWEVER, the suspected stream of virtual particles around the black hole which diminish its mass would mean that even a black hole is not truly an absorption state, even for a specific subsection of the universe. If history truly DID repeat itself, then it might be considered that the recurring series of events is in effect a form of absorption state. However, it is difficult to imagine a set of historical events which would deterministically lead back to the same setup over and over, for there are numerous branching points along the time series which would allow the events to be changed. However, it seems conceivable an alternative universe could exist where the participants could get locked in such a loop, although hopefully they would not be cognizant of such a state.

8) If the universe is continuously growing and shrinking from a point, what does this mean for the Markov chain?

If the universe repeatedly "blows up" from nothing it would mean that the Markov chain is repeatedly being run through, over and over. In this case, it might make sense to consider the probability of a specific configuration as a limiting distribution to be technically accurate. However, for the participants in a particular universe it would make sense to consider the universe from the starting point of their particular chain, not going backwards to prior universes which have no impact on the probability of later universes. (presumably)

9) Since Google utilizes a Markov chain, is Google representative of the Universe?

Every physical set of conditional relationships is a subset of the universe itself, so Google does not share any special qualities with a relationship to the universe, besides being a part of it. Any specific abstraction of the universe is a useful tool for understanding that portion of it, but one must always be suspect that critical factors of understanding are being omitted from the model and that it does not truly represent what you think it represents either probabilistically or as a subset.

10) What do you mean by this?

For example, imagine you are watching a stream of digits coming out of a device, and you see 15296.. You may assume these digits are completely stochastic, when in fact you are seeing a series which are from the digits of PI. The relative probabilities of the future digits are completely constrained and deterministic, because your model of the digits (as being stochastic) is inaccurate.

11) Is the Universe a "birth and death" Markov chain?

A "birth and death" Markov chain is a special chain wherein states move through the entire available state space until completion. If there are truly quantum multiworlds, then perhaps it is accurate to consider it as a "birth and death" Markov chain. However, we have no way of knowing if the quantum multiworld hypothesis of Everette is true or not, so we must assume that it is in fact NOT a "birth and death" Markov chain for it is not passing through every available state.

12) Is the universe an ergodic Markov chain?

An ergodic Markov chain is one which "repeats" and can move from any state to any other state through a series of steps. Since our universe is expanding, and dragging objects apart, and allowing for the light to redshift down so that it is no longer blinding, but slowly loses energy- our universe is not ergodic on a timescale within a single universe expansion. Perhaps the universe is Ergodic if it is repeatedly expanding and contracting repeatedly.