Memorylessness
Encyclopedia
In probability
and statistics
, memorylessness is a property of certain probability distribution
s: the exponential distribution
s of non-negative real numbers and the geometric distributions of non-negative integers.
The property is most easily explained in terms of "waiting times". Suppose that a random variable
, X, is defined to be the time elapsed in a shop from 9 am on a certain day until the arrival of the first customer: thus X is the time a server waits for the first customer. The "memoryless" property makes a comparison between the probability distributions of the time a server has to wait from 9 am onwards for his first customer, and the time that the server still has to wait for the first customer on those occasions when no customer has arrived by any given later time: the property of memorylessness is that these distributions of "time from now to the next customer" are exactly the same.
The terms "memoryless" and "memorylessness" have sometimes been used in a slightly different way to refer to Markov process
es in which the underlying assumption of the Markov property
implies that the properties of random variables related to the future depend only on relevant information about the current time, not on information from further in the past. While these different meanings of memorylessness are connected at a deeply theoretical level, the present article describes its use in relation to probability distributions.
whose values lie in the set { 0, 1, 2, ... }.
The probability distribution of X is memoryless precisely if for any m, n in { 0, 1, 2, ... }, we have
Here, Pr(X > m + n | X ≥ m) denotes the conditional probability
that the value of X is larger than m + n, given that it is larger than or equal to m.
The only memoryless discrete probability distributions are the geometric distributions, which feature the number of independent
Bernoulli trial
s needed to get one "success", with a fixed probability p of "success" on each trial. In other words those are the distributions of waiting time in a Bernoulli process.
It does not mean that
which would be true only if the events X > 40 and X ≥ 30 were independent
, which cannot be the case.
s t and s, we have
This is similar to the discrete version except that s and t are constrained only to be non-negative real numbers instead of integer
s. Rather than counting trials until the first "success", for example, we may be marking time until the arrival of the first phone call at a switchboard.
Geometric distributions and exponential distribution
s are discrete and continuous analogs.
s, so memorylessness completely characterizes
the exponential distributions among all continuous ones.
To see this, first define the survival function
, G, as
Note that G(t) is then monotonically decreasing. From the relation
and the definition of conditional probability
, it follows that
This gives the functional equation
and solutions of this can be sought under the condition that G is a monotone decreasing function
.
The functional equation alone will imply that G restricted to rational
multiples of any particular number is an exponential function
. Combined with the fact that G is monotone, this implies that G over its whole domain is an exponential function.
Probability
Probability is ordinarily used to describe an attitude of mind towards some proposition of whose truth we arenot certain. The proposition of interest is usually of the form "Will a specific event occur?" The attitude of mind is of the form "How certain are we that the event will occur?" The...
and statistics
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
, memorylessness is a property of certain probability distribution
Probability distribution
In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....
s: the exponential distribution
Exponential distribution
In probability theory and statistics, the exponential distribution is a family of continuous probability distributions. It describes the time between events in a Poisson process, i.e...
s of non-negative real numbers and the geometric distributions of non-negative integers.
The property is most easily explained in terms of "waiting times". Suppose that a random variable
Random variable
In probability and statistics, a random variable or stochastic variable is, roughly speaking, a variable whose value results from a measurement on some type of random process. Formally, it is a function from a probability space, typically to the real numbers, which is measurable functionmeasurable...
, X, is defined to be the time elapsed in a shop from 9 am on a certain day until the arrival of the first customer: thus X is the time a server waits for the first customer. The "memoryless" property makes a comparison between the probability distributions of the time a server has to wait from 9 am onwards for his first customer, and the time that the server still has to wait for the first customer on those occasions when no customer has arrived by any given later time: the property of memorylessness is that these distributions of "time from now to the next customer" are exactly the same.
The terms "memoryless" and "memorylessness" have sometimes been used in a slightly different way to refer to Markov process
Markov process
In probability theory and statistics, a Markov process, named after the Russian mathematician Andrey Markov, is a time-varying random phenomenon for which a specific property holds...
es in which the underlying assumption of the Markov property
Markov property
In probability theory and statistics, the term Markov property refers to the memoryless property of a stochastic process. It was named after the Russian mathematician Andrey Markov....
implies that the properties of random variables related to the future depend only on relevant information about the current time, not on information from further in the past. While these different meanings of memorylessness are connected at a deeply theoretical level, the present article describes its use in relation to probability distributions.
Discrete memorylessness
Suppose X is a discrete random variableRandom variable
In probability and statistics, a random variable or stochastic variable is, roughly speaking, a variable whose value results from a measurement on some type of random process. Formally, it is a function from a probability space, typically to the real numbers, which is measurable functionmeasurable...
whose values lie in the set { 0, 1, 2, ... }.
The probability distribution of X is memoryless precisely if for any m, n in { 0, 1, 2, ... }, we have
Here, Pr(X > m + n | X ≥ m) denotes the conditional probability
Conditional probability
In probability theory, the "conditional probability of A given B" is the probability of A if B is known to occur. It is commonly notated P, and sometimes P_B. P can be visualised as the probability of event A when the sample space is restricted to event B...
that the value of X is larger than m + n, given that it is larger than or equal to m.
The only memoryless discrete probability distributions are the geometric distributions, which feature the number of independent
Statistical independence
In probability theory, to say that two events are independent intuitively means that the occurrence of one event makes it neither more nor less probable that the other occurs...
Bernoulli trial
Bernoulli trial
In the theory of probability and statistics, a Bernoulli trial is an experiment whose outcome is random and can be either of two possible outcomes, "success" and "failure"....
s needed to get one "success", with a fixed probability p of "success" on each trial. In other words those are the distributions of waiting time in a Bernoulli process.
A frequent misunderstanding
"Memorylessness" of the probability distribution of the number of trials X until the first success means thatIt does not mean that
which would be true only if the events X > 40 and X ≥ 30 were independent
Statistical independence
In probability theory, to say that two events are independent intuitively means that the occurrence of one event makes it neither more nor less probable that the other occurs...
, which cannot be the case.
Continuous memorylessness
Suppose X is a continuous random variable whose values lie in the non-negative real numbers [0, ∞). The probability distribution of X is memoryless precisely if for any non-negative real numberReal number
In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...
s t and s, we have
This is similar to the discrete version except that s and t are constrained only to be non-negative real numbers instead of integer
Integer
The integers are formed by the natural numbers together with the negatives of the non-zero natural numbers .They are known as Positive and Negative Integers respectively...
s. Rather than counting trials until the first "success", for example, we may be marking time until the arrival of the first phone call at a switchboard.
Geometric distributions and exponential distribution
Exponential distribution
In probability theory and statistics, the exponential distribution is a family of continuous probability distributions. It describes the time between events in a Poisson process, i.e...
s are discrete and continuous analogs.
The memoryless distributions are the exponential distributions
The only memoryless continuous probability distributions are the exponential distributionExponential distribution
In probability theory and statistics, the exponential distribution is a family of continuous probability distributions. It describes the time between events in a Poisson process, i.e...
s, so memorylessness completely characterizes
Characterization (mathematics)
In mathematics, the statement that "Property P characterizes object X" means, not simply that X has property P, but that X is the only thing that has property P. It is also common to find statements such as "Property Q characterises Y up to isomorphism". The first type of statement says in...
the exponential distributions among all continuous ones.
To see this, first define the survival function
Survival function
The survival function, also known as a survivor function or reliability function, is a property of any random variable that maps a set of events, usually associated with mortality or failure of some system, onto time. It captures the probability that the system will survive beyond a specified time...
, G, as
Note that G(t) is then monotonically decreasing. From the relation
and the definition of conditional probability
Conditional probability
In probability theory, the "conditional probability of A given B" is the probability of A if B is known to occur. It is commonly notated P, and sometimes P_B. P can be visualised as the probability of event A when the sample space is restricted to event B...
, it follows that
This gives the functional equation
Functional equation
In mathematics, a functional equation is any equation that specifies a function in implicit form.Often, the equation relates the value of a function at some point with its values at other points. For instance, properties of functions can be determined by considering the types of functional...
and solutions of this can be sought under the condition that G is a monotone decreasing function
Monotonic function
In mathematics, a monotonic function is a function that preserves the given order. This concept first arose in calculus, and was later generalized to the more abstract setting of order theory....
.
The functional equation alone will imply that G restricted to rational
Rational number
In mathematics, a rational number is any number that can be expressed as the quotient or fraction a/b of two integers, with the denominator b not equal to zero. Since b may be equal to 1, every integer is a rational number...
multiples of any particular number is an exponential function
Exponential function
In mathematics, the exponential function is the function ex, where e is the number such that the function ex is its own derivative. The exponential function is used to model a relationship in which a constant change in the independent variable gives the same proportional change In mathematics,...
. Combined with the fact that G is monotone, this implies that G over its whole domain is an exponential function.