Bayesian statistics
Encyclopedia
Bayesian statistics is that subset of the entire field of statistics
in which the evidence about the true state of the world is expressed in terms of degrees of belief or, more specifically, Bayesian probabilities
. Such an interpretation is only one of a number of interpretations of probability
and there are many other statistical techiques that are not based on "degrees of belief".
is an approach to statistical inference
, that is distinct from the more traditional frequentist inference
. It is specifically based on the use of Bayesian probabilities
to summarise evidence.
s for use in Bayesian statistics has the additional feature, not present with other types of statistical techniques, of requiring the formulation of a set of prior distributions for any unknown parameters. Such prior distributions are as much part of the statistical model as the part that expresses the probability distribution
of observations given the model parameters. The specification of a set of prior distributions for a problem may involve hyperparameter
s and hyperprior distributions.
are extended in the case of Bayesian design of experiments to include the influence of prior beliefs. Importantly, the application of sequential analysis
techiques allows the outcome of earlier experiments to influence the design of the next experiment, based on the updating of beliefs as expressed by the prior and posterior distribution. Part of the problem of the design of experiments is that they should make good use of resources of all types: one example of the Bayesian design of experiments aimed at such efficiency is the multi-armed bandit problem.
includes methods for data exploration, for model validation, etc.. The use of certain modern computational techniques for Bayesian inference, specifically the various types of Markov chain Monte Carlo
techniques, have led to the need for checks, often made in graphical form, on the validity of such computations in expressing the required posterior distributions.
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
in which the evidence about the true state of the world is expressed in terms of degrees of belief or, more specifically, Bayesian probabilities
Bayesian probability
Bayesian probability is one of the different interpretations of the concept of probability and belongs to the category of evidential probabilities. The Bayesian interpretation of probability can be seen as an extension of logic that enables reasoning with propositions, whose truth or falsity is...
. Such an interpretation is only one of a number of interpretations of probability
Probability interpretations
The word probability has been used in a variety of ways since it was first coined in relation to games of chance. Does probability measure the real, physical tendency of something to occur, or is it just a measure of how strongly one believes it will occur? In answering such questions, we...
and there are many other statistical techiques that are not based on "degrees of belief".
Outline
The general set of statistical techiques can be divided into a number of activities, many of which have special "Bayesian" versions.Statistical inference
Bayesian inferenceBayesian inference
In statistics, Bayesian inference is a method of statistical inference. It is often used in science and engineering to determine model parameters, make predictions about unknown variables, and to perform model selection...
is an approach to statistical inference
Statistical inference
In statistics, statistical inference is the process of drawing conclusions from data that are subject to random variation, for example, observational errors or sampling variation...
, that is distinct from the more traditional frequentist inference
Frequentist inference
Frequentist inference is one of a number of possible ways of formulating generally applicable schemes for making statistical inferences: that is, for drawing conclusions from statistical samples. An alternative name is frequentist statistics...
. It is specifically based on the use of Bayesian probabilities
Bayesian probability
Bayesian probability is one of the different interpretations of the concept of probability and belongs to the category of evidential probabilities. The Bayesian interpretation of probability can be seen as an extension of logic that enables reasoning with propositions, whose truth or falsity is...
to summarise evidence.
Statistical modelling
The formulation of statistical modelStatistical model
A statistical model is a formalization of relationships between variables in the form of mathematical equations. A statistical model describes how one or more random variables are related to one or more random variables. The model is statistical as the variables are not deterministically but...
s for use in Bayesian statistics has the additional feature, not present with other types of statistical techniques, of requiring the formulation of a set of prior distributions for any unknown parameters. Such prior distributions are as much part of the statistical model as the part that expresses the probability distribution
Probability distribution
In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....
of observations given the model parameters. The specification of a set of prior distributions for a problem may involve hyperparameter
Hyperparameter
In Bayesian statistics, a hyperparameter is a parameter of a prior distribution; the term is used to distinguish them from parameters of the model for the underlying system under analysis...
s and hyperprior distributions.
Design of experiments
The usual considerations in the design of experimentsDesign of experiments
In general usage, design of experiments or experimental design is the design of any information-gathering exercises where variation is present, whether under the full control of the experimenter or not. However, in statistics, these terms are usually used for controlled experiments...
are extended in the case of Bayesian design of experiments to include the influence of prior beliefs. Importantly, the application of sequential analysis
Sequential analysis
In statistics, sequential analysis or sequential hypothesis testing is statistical analysis where the sample size is not fixed in advance. Instead data are evaluated as they are collected, and further sampling is stopped in accordance with a pre-defined stopping rule as soon as significant results...
techiques allows the outcome of earlier experiments to influence the design of the next experiment, based on the updating of beliefs as expressed by the prior and posterior distribution. Part of the problem of the design of experiments is that they should make good use of resources of all types: one example of the Bayesian design of experiments aimed at such efficiency is the multi-armed bandit problem.
Statistical graphics
Statistical graphicsStatistical graphics
Statistical graphics, also known as graphical techniques, are information graphics in the field of statistics used to visualize quantitative data.- Overview :...
includes methods for data exploration, for model validation, etc.. The use of certain modern computational techniques for Bayesian inference, specifically the various types of Markov chain Monte Carlo
Markov chain Monte Carlo
Markov chain Monte Carlo methods are a class of algorithms for sampling from probability distributions based on constructing a Markov chain that has the desired distribution as its equilibrium distribution. The state of the chain after a large number of steps is then used as a sample of the...
techniques, have led to the need for checks, often made in graphical form, on the validity of such computations in expressing the required posterior distributions.