Markov property

Encyclopedia

In probability theory

and statistics

, the term

. It was named after the Russia

n mathematician

Andrey Markov

.

A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state, not on the sequence of events that preceded it. A process with this property is called a

The term

.

A Markov random field, extends this property to two or more dimensions, or to random variables defined for an interconnected network of items. An example of a model for such a field is the Ising model

.

For discrete-time processes with the Markov property, see Markov chain

.

Both the terms "Markov property" and "strong Markov property" have been used in connection with a particular "memoryless" property of the exponential distribution.

. Brownian motion

is another well-known Markov process.

on a probability space

is said to possess the Markov property if, for each and ,

where is the natural filtration

and denotes the Borel sigma-algebra on .

In the case that the process takes discrete values and is indexed by a discrete time, this can be reformulated as follows;

.

on a probability space

with natural filtration . Then is said to have the strong Markov property if, for each stopping time , conditioned on the event , the process (which maybe needs to be defined) is independent from and has the same distribution as for each .

The strong Markov property is a stronger property than the ordinary Markov property, since by taking the stopping time , the ordinary Markov property can be deduced.

for all and bounded and measurable.

computations in the context of Bayesian statistics

.

Probability theory

Probability theory is the branch of mathematics concerned with analysis of random phenomena. The central objects of probability theory are random variables, stochastic processes, and events: mathematical abstractions of non-deterministic events or measured quantities that may either be single...

and statistics

Statistics

Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....

, the term

**Markov property**refers to the memoryless property of a stochastic processStochastic process

In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...

. It was named after the Russia

Russia

Russia or , officially known as both Russia and the Russian Federation , is a country in northern Eurasia. It is a federal semi-presidential republic, comprising 83 federal subjects...

n mathematician

Mathematician

A mathematician is a person whose primary area of study is the field of mathematics. Mathematicians are concerned with quantity, structure, space, and change....

Andrey Markov

Andrey Markov

Andrey Andreyevich Markov was a Russian mathematician. He is best known for his work on theory of stochastic processes...

.

A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state, not on the sequence of events that preceded it. A process with this property is called a

*Markov process*

. The termMarkov process

In probability theory and statistics, a Markov process, named after the Russian mathematician Andrey Markov, is a time-varying random phenomenon for which a specific property holds...

**strong Markov property**is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a stopping time.The term

**Markov assumption**is used to describe a model where the Markov property is assumed to hold, such as a hidden Markov modelHidden Markov model

A hidden Markov model is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved states. An HMM can be considered as the simplest dynamic Bayesian network. The mathematics behind the HMM was developed by L. E...

.

A Markov random field, extends this property to two or more dimensions, or to random variables defined for an interconnected network of items. An example of a model for such a field is the Ising model

Ising model

The Ising model is a mathematical model of ferromagnetism in statistical mechanics. The model consists of discrete variables called spins that can be in one of two states . The spins are arranged in a graph , and each spin interacts with its nearest neighbors...

.

For discrete-time processes with the Markov property, see Markov chain

Markov chain

A Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process characterized as memoryless: the next state depends only on the current state and not on the...

.

Both the terms "Markov property" and "strong Markov property" have been used in connection with a particular "memoryless" property of the exponential distribution.

## Introduction

A stochastic process has the Markov property if the conditional probability distribution of future states of the process (conditional on both past and present values) depends only upon the present state; that is, given the present, the future does not depend on the past. A process with this property is said to be**Markovian**or a**Markov process**

. The most famous Markov process is a Markov chainMarkov process

In probability theory and statistics, a Markov process, named after the Russian mathematician Andrey Markov, is a time-varying random phenomenon for which a specific property holds...

Markov chain

A Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process characterized as memoryless: the next state depends only on the current state and not on the...

. Brownian motion

Brownian motion

Brownian motion or pedesis is the presumably random drifting of particles suspended in a fluid or the mathematical model used to describe such random movements, which is often called a particle theory.The mathematical model of Brownian motion has several real-world applications...

is another well-known Markov process.

## History

For some details of the early history of the Markov property see this brief account.## Definition

The following definition applies. An -valued stochastic processStochastic process

In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...

on a probability space

Probability space

In probability theory, a probability space or a probability triple is a mathematical construct that models a real-world process consisting of states that occur randomly. A probability space is constructed with a specific kind of situation or experiment in mind...

is said to possess the Markov property if, for each and ,

where is the natural filtration

Natural filtration

In the theory of stochastic processes in mathematics and statistics, the natural filtration associated to a stochastic process is a filtration associated to the process which records its "past behaviour" at each time...

and denotes the Borel sigma-algebra on .

In the case that the process takes discrete values and is indexed by a discrete time, this can be reformulated as follows;

.

## Strong Markov Property

Suppose that is a stochastic processStochastic process

In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...

on a probability space

Probability space

In probability theory, a probability space or a probability triple is a mathematical construct that models a real-world process consisting of states that occur randomly. A probability space is constructed with a specific kind of situation or experiment in mind...

with natural filtration . Then is said to have the strong Markov property if, for each stopping time , conditioned on the event , the process (which maybe needs to be defined) is independent from and has the same distribution as for each .

The strong Markov property is a stronger property than the ordinary Markov property, since by taking the stopping time , the ordinary Markov property can be deduced.

## Alternative Formulations

Alternatively, the Markov property can be formulated as follows;for all and bounded and measurable.

## Applications

A very important application of the Markov property in a generalized form is in Markov chain Monte CarloMarkov chain Monte Carlo

Markov chain Monte Carlo methods are a class of algorithms for sampling from probability distributions based on constructing a Markov chain that has the desired distribution as its equilibrium distribution. The state of the chain after a large number of steps is then used as a sample of the...

computations in the context of Bayesian statistics

Bayesian statistics

Bayesian statistics is that subset of the entire field of statistics in which the evidence about the true state of the world is expressed in terms of degrees of belief or, more specifically, Bayesian probabilities...

.

## See also

- Markov chainMarkov chainA Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process characterized as memoryless: the next state depends only on the current state and not on the...
- Markov blanketMarkov blanketIn machine learning, the Markov blanket for a node A in a Bayesian network is the set of nodes \partial A composed of A's parents, its children, and its children's other parents. In a Markov network, the Markov blanket of a node is its set of neighbouring nodes...
- Markov decision processMarkov decision processMarkov decision processes , named after Andrey Markov, provide a mathematical framework for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision maker. MDPs are useful for studying a wide range of optimization problems solved via...
- Causal Markov conditionCausal Markov conditionThe Markov condition for a Bayesian network states that any node in a Bayesian network is conditionally independent of its nondescendents, given its parents.A node is conditionally independent of the entire network, given its Markov blanket....
- Markov modelMarkov modelIn probability theory, a Markov model is a stochastic model that assumes the Markov property. Generally, this assumption enables reasoning and computation with the model that would otherwise be intractable.-Introduction:...
- ChapmanāKolmogorov equation