Markov property
Encyclopedia
In probability theory
and statistics
, the term Markov property refers to the memoryless property of a stochastic process
. It was named after the Russia
n mathematician
Andrey Markov
.
A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state, not on the sequence of events that preceded it. A process with this property is called a Markov process
. The term strong Markov property is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a stopping time.
The term Markov assumption is used to describe a model where the Markov property is assumed to hold, such as a hidden Markov model
.
A Markov random field, extends this property to two or more dimensions, or to random variables defined for an interconnected network of items. An example of a model for such a field is the Ising model
.
For discrete-time processes with the Markov property, see Markov chain
.
Both the terms "Markov property" and "strong Markov property" have been used in connection with a particular "memoryless" property of the exponential distribution.
. The most famous Markov process is a Markov chain
. Brownian motion
is another well-known Markov process.
-valued stochastic process
on a probability space
is said to possess the Markov property if, for each 
and 
,
where
is the natural filtration
and
denotes the Borel sigma-algebra on 
.
In the case that the process takes discrete values and is indexed by a discrete time, this can be reformulated as follows;
.
is a stochastic process
on a probability space
with natural filtration 
. Then 
is said to have the strong Markov property if, for each stopping time 
, conditioned on the event 
, the process 
(which maybe needs to be defined) is independent from 
and 
has the same distribution as 
for each 
.
The strong Markov property is a stronger property than the ordinary Markov property, since by taking the stopping time
, the ordinary Markov property can be deduced.
for all
and 
bounded and measurable.
computations in the context of Bayesian statistics
.
Probability theory
Probability theory is the branch of mathematics concerned with analysis of random phenomena. The central objects of probability theory are random variables, stochastic processes, and events: mathematical abstractions of non-deterministic events or measured quantities that may either be single...
and statistics
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
, the term Markov property refers to the memoryless property of a stochastic process
Stochastic process
In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...
. It was named after the Russia
Russia
Russia or , officially known as both Russia and the Russian Federation , is a country in northern Eurasia. It is a federal semi-presidential republic, comprising 83 federal subjects...
n mathematician
Mathematician
A mathematician is a person whose primary area of study is the field of mathematics. Mathematicians are concerned with quantity, structure, space, and change....
Andrey Markov
Andrey Markov
Andrey Andreyevich Markov was a Russian mathematician. He is best known for his work on theory of stochastic processes...
.
A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state, not on the sequence of events that preceded it. A process with this property is called a Markov process
Markov process
In probability theory and statistics, a Markov process, named after the Russian mathematician Andrey Markov, is a time-varying random phenomenon for which a specific property holds...
. The term strong Markov property is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a stopping time.
The term Markov assumption is used to describe a model where the Markov property is assumed to hold, such as a hidden Markov model
Hidden Markov model
A hidden Markov model is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved states. An HMM can be considered as the simplest dynamic Bayesian network. The mathematics behind the HMM was developed by L. E...
.
A Markov random field, extends this property to two or more dimensions, or to random variables defined for an interconnected network of items. An example of a model for such a field is the Ising model
Ising model
The Ising model is a mathematical model of ferromagnetism in statistical mechanics. The model consists of discrete variables called spins that can be in one of two states . The spins are arranged in a graph , and each spin interacts with its nearest neighbors...
.
For discrete-time processes with the Markov property, see Markov chain
Markov chain
A Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process characterized as memoryless: the next state depends only on the current state and not on the...
.
Both the terms "Markov property" and "strong Markov property" have been used in connection with a particular "memoryless" property of the exponential distribution.
Introduction
A stochastic process has the Markov property if the conditional probability distribution of future states of the process (conditional on both past and present values) depends only upon the present state; that is, given the present, the future does not depend on the past. A process with this property is said to be Markovian or a Markov processMarkov process
In probability theory and statistics, a Markov process, named after the Russian mathematician Andrey Markov, is a time-varying random phenomenon for which a specific property holds...
. The most famous Markov process is a Markov chain
Markov chain
A Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process characterized as memoryless: the next state depends only on the current state and not on the...
. Brownian motion
Brownian motion
Brownian motion or pedesis is the presumably random drifting of particles suspended in a fluid or the mathematical model used to describe such random movements, which is often called a particle theory.The mathematical model of Brownian motion has several real-world applications...
is another well-known Markov process.
History
For some details of the early history of the Markov property see this brief account.Definition
The following definition applies. An-valued stochastic processStochastic process
In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...
on a probability spaceProbability space
In probability theory, a probability space or a probability triple is a mathematical construct that models a real-world process consisting of states that occur randomly. A probability space is constructed with a specific kind of situation or experiment in mind...
is said to possess the Markov property if, for each and ,where
is the natural filtrationNatural filtration
In the theory of stochastic processes in mathematics and statistics, the natural filtration associated to a stochastic process is a filtration associated to the process which records its "past behaviour" at each time...
and
denotes the Borel sigma-algebra on .In the case that the process takes discrete values and is indexed by a discrete time, this can be reformulated as follows;
.Strong Markov Property
Suppose that is a stochastic processStochastic process
In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...
on a probability space
Probability space
In probability theory, a probability space or a probability triple is a mathematical construct that models a real-world process consisting of states that occur randomly. A probability space is constructed with a specific kind of situation or experiment in mind...
with natural filtration . Then is said to have the strong Markov property if, for each stopping time , conditioned on the event , the process (which maybe needs to be defined) is independent from and has the same distribution as for each .The strong Markov property is a stronger property than the ordinary Markov property, since by taking the stopping time
, the ordinary Markov property can be deduced.Alternative Formulations
Alternatively, the Markov property can be formulated as follows;for all
and bounded and measurable.Applications
A very important application of the Markov property in a generalized form is in Markov chain Monte CarloMarkov chain Monte Carlo
Markov chain Monte Carlo methods are a class of algorithms for sampling from probability distributions based on constructing a Markov chain that has the desired distribution as its equilibrium distribution. The state of the chain after a large number of steps is then used as a sample of the...
computations in the context of Bayesian statistics
Bayesian statistics
Bayesian statistics is that subset of the entire field of statistics in which the evidence about the true state of the world is expressed in terms of degrees of belief or, more specifically, Bayesian probabilities...
.
See also
- Markov chainMarkov chainA Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process characterized as memoryless: the next state depends only on the current state and not on the...
- Markov blanketMarkov blanketIn machine learning, the Markov blanket for a node A in a Bayesian network is the set of nodes \partial A composed of A's parents, its children, and its children's other parents. In a Markov network, the Markov blanket of a node is its set of neighbouring nodes...
- Markov decision processMarkov decision processMarkov decision processes , named after Andrey Markov, provide a mathematical framework for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision maker. MDPs are useful for studying a wide range of optimization problems solved via...
- Causal Markov conditionCausal Markov conditionThe Markov condition for a Bayesian network states that any node in a Bayesian network is conditionally independent of its nondescendents, given its parents.A node is conditionally independent of the entire network, given its Markov blanket....
- Markov modelMarkov modelIn probability theory, a Markov model is a stochastic model that assumes the Markov property. Generally, this assumption enables reasoning and computation with the model that would otherwise be intractable.-Introduction:...
- Chapman–Kolmogorov equation