Well-behaved statistic
Encyclopedia
A well-behaved statistic is a term sometimes used in the theory of statistics
to describe part of a procedure. This usage is broadly similar to the use of well-behaved
in more general mathematics. It is essentially an assumption about the formulation of an estimation procedure (which entails the specification of an estimator
or statistic
) that is used to avoid giving extensive details about what conditions need to hold. In particular it means that the statistic is not an unusual one in the context being studied. Due to this, the meaning attributed to well-behaved statistic may vary from context to context.
The present article is mainly concerned with the context of data mining
procedures applied to statistical inference
and, in particular, to the group of computationally intensive procedure that have been called algorithmic inference
.
, the property of a statistic that is of most relevance is the pivoting step which allows to transference of probability-considerations from the sample distribution to the distribution of the parameters representing the population distribution in such a way that the conclusion of this statistical inference
step is compatible with the sample actually observed.
By default, capital letters (such as U, X) will denote random variables and small letters (u, x) their corresponding realizations and with gothic letters (such as
) the domain where the variable takes specifications. Facing a sample 
, given a sampling mechanism 
, with 
scalar, for the random variable X, we have
The sampling mechanism
, of the statistic s, as a function ? of 
with specifications in 
, has an explaining function defined by the master equation:
for suitable seeds
and parameter ?
, the statistic must obey some technical properties. Namely, a statistic s is said to be well-behaved if it satisfies the following three statements:
with parameter ? the statistic
is well-behaved. The satisfaction of the above three properties is straightforward when looking at both explaining functions: 
if 
, 0 otherwise in the case of the Bernoulli random variable, and 
for the Exponential random variable, giving rise to statistics
and
Vice versa, in the case of X following a continuous uniform distribution
on
the same statistics do not meet the second requirement. For instance, the observed sample 
gives
. But the explaining function of this X is 
.
Hence a master equation
would produce with
a U sample
and a solution 
. This conflicts with the observed sample since the first observed value should result greater than the right extreme of the X range. The statistic 
is well-behaved in this case.
Analogously, for a random variable X following the Pareto distribution with parameters K and A (see Pareto example for more detail of this case),
and
can be used as joint statistics for these parameters.
As a general statement that holds under weak conditions, sufficient statistics are well-behaved with respect to the related parameters. The table below gives sufficient / Well-behaved statistics for the parameters of some of the most commonly used probability distributions.
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
to describe part of a procedure. This usage is broadly similar to the use of well-behaved
Well-behaved
Mathematicians very frequently speak of whether a mathematical object — a function, a set, a space of one sort or another — is "well-behaved" or not. The term has no fixed formal definition, and is dependent on mathematical interests, fashion, and taste...
in more general mathematics. It is essentially an assumption about the formulation of an estimation procedure (which entails the specification of an estimator
Estimator
In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule and its result are distinguished....
or statistic
Statistic
A statistic is a single measure of some attribute of a sample . It is calculated by applying a function to the values of the items comprising the sample which are known together as a set of data.More formally, statistical theory defines a statistic as a function of a sample where the function...
) that is used to avoid giving extensive details about what conditions need to hold. In particular it means that the statistic is not an unusual one in the context being studied. Due to this, the meaning attributed to well-behaved statistic may vary from context to context.
The present article is mainly concerned with the context of data mining
Data mining
Data mining , a relatively young and interdisciplinary field of computer science is the process of discovering new patterns from large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics and database systems...
procedures applied to statistical inference
Statistical inference
In statistics, statistical inference is the process of drawing conclusions from data that are subject to random variation, for example, observational errors or sampling variation...
and, in particular, to the group of computationally intensive procedure that have been called algorithmic inference
Algorithmic inference
Algorithmic inference gathers new developments in the statistical inference methods made feasible by the powerful computing devices widely available to any data analyst...
.
Algorithmic inference
In algorithmic inferenceAlgorithmic inference
Algorithmic inference gathers new developments in the statistical inference methods made feasible by the powerful computing devices widely available to any data analyst...
, the property of a statistic that is of most relevance is the pivoting step which allows to transference of probability-considerations from the sample distribution to the distribution of the parameters representing the population distribution in such a way that the conclusion of this statistical inference
Statistical inference
In statistics, statistical inference is the process of drawing conclusions from data that are subject to random variation, for example, observational errors or sampling variation...
step is compatible with the sample actually observed.
By default, capital letters (such as U, X) will denote random variables and small letters (u, x) their corresponding realizations and with gothic letters (such as
) the domain where the variable takes specifications. Facing a sample , given a sampling mechanism , with scalar, for the random variable X, we haveThe sampling mechanism
, of the statistic s, as a function ? of with specifications in , has an explaining function defined by the master equation:
for suitable seeds
and parameter ?Well-behaved
In order to derive the distribution law of the parameter T, compatible with, the statistic must obey some technical properties. Namely, a statistic s is said to be well-behaved if it satisfies the following three statements:
- monotonicity. A uniformly monotone relation exists between s and ? for any fixed seed
– so as to have a unique solution of (1); - well-defined. On each observed s the statistic is well defined for every value of ?, i.e. any sample specification
such that 
has a probability density different from 0 – so as to avoid considering a non-surjective mapping from 
to 
, i.e. associating via 
to a sample 
a ? that could not generate the sample itself; - local sufficiency.
constitutes a true T sample for the observed s, so that the same probability distribution can be attributed to each sampled value. Now, 
is a solution of (1) with the seed 
. Since the seeds are equally distributed, the sole caveat comes from their independence or, conversely from their dependence on ? itself. This check can be restricted to seeds involved by s, i.e. this drawback can be avoided by requiring that the distribution of 
is independent of ?. An easy way to check this property is by mapping seed specifications into 
s specifications. The mapping of course depends on ?, but the distribution of 
will not depend on ?, if the above seed independence holds – a condition that looks like a local sufficiency of the statistic S.
Example
For instance, for both the Bernoulli distribution with parameter p and the exponential distributionExponential distribution
In probability theory and statistics, the exponential distribution is a family of continuous probability distributions. It describes the time between events in a Poisson process, i.e...
with parameter ? the statistic
is well-behaved. The satisfaction of the above three properties is straightforward when looking at both explaining functions: if , 0 otherwise in the case of the Bernoulli random variable, and for the Exponential random variable, giving rise to statisticsand
Vice versa, in the case of X following a continuous uniform distribution
Uniform distribution (continuous)
In probability theory and statistics, the continuous uniform distribution or rectangular distribution is a family of probability distributions such that for each member of the family, all intervals of the same length on the distribution's support are equally probable. The support is defined by...
on
the same statistics do not meet the second requirement. For instance, the observed sample gives. But the explaining function of this X is .Hence a master equation
would produce witha U sample
and a solution . This conflicts with the observed sample since the first observed value should result greater than the right extreme of the X range. The statistic is well-behaved in this case.Analogously, for a random variable X following the Pareto distribution with parameters K and A (see Pareto example for more detail of this case),
and
can be used as joint statistics for these parameters.
As a general statement that holds under weak conditions, sufficient statistics are well-behaved with respect to the related parameters. The table below gives sufficient / Well-behaved statistics for the parameters of some of the most commonly used probability distributions.
| Distribution | Definition of density function | Sufficient/Well-behaved statistic |
|---|---|---|
| Uniform discrete |  |
 |
| Bernoulli |  |
 |
| Binomial |  |
 |
| Geometric |  |
 |
| Poisson |  |
 |
| Uniform continuous |  |
 |
| Negative exponential |  |
 |
| Pareto |  |
 |
| Gaussian |  |
 |
| Gamma |  |
 |