Reference class forecasting
Reference class forecasting
Forecasting is the process of making statements about events whose actual outcomes have not yet been observed. A commonplace example might be estimation for some variable of interest at some specified future date. Prediction is a similar, but more general term...

is the method of predicting the future, through looking at similar past situations and their outcomes.

Reference class forcasting predicts the outcome of a planned action based on actual outcomes in a reference class of similar actions to that being forecast. The theories behind reference class forecasting were developed by Daniel Kahneman
Daniel Kahneman
Daniel Kahneman is an Israeli-American psychologist and Nobel laureate. He is notable for his work on the psychology of judgment and decision-making, behavioral economics and hedonic psychology....

 and Amos Tversky
Amos Tversky
Amos Nathan Tversky, was a cognitive and mathematical psychologist, a pioneer of cognitive science, a longtime collaborator of Daniel Kahneman, and a key figure in the discovery of systematic human cognitive bias and handling of risk. Much of his early work concerned the foundations of measurement...

. The theoretical work helped Kahneman win the Nobel Prize in Economics. The methodology and data needed for employing reference class forecasting in practice in policy, planning, and management were developed by Oxford professor Bent Flyvbjerg
Bent Flyvbjerg
Bent Flyvbjerg is the first Chair and BT Professor of Major Programme Management at Oxford University's Saïd Business School and is Founding Director of the University's BT Centre for Major Programme Management. He was previously Professor of Planning at Aalborg University, Denmark and Chair of...

 and the COWI consulting group in a joint effort. Today, reference class forecasting has found widespread use in practice in both public and private sector policy and management.

Kahneman and Tversky (1979a, b) found that human judgment is generally optimistic due to overconfidence and insufficient consideration of distributional information about outcomes. Therefore, people tend to underestimate the costs, completion times, and risks of planned actions, whereas they tend to overestimate the benefits of those same actions. Such error is caused by actors taking an "inside view," where focus is on the constituents of the specific planned action instead of on the actual outcomes of similar ventures that have already been completed.

Kahneman and Tversky concluded that disregard of distributional information, that is, risk, is perhaps the major source of error in forecasting. On that basis they recommended that forecasters "should therefore make every effort to frame the forecasting problem so as to facilitate utilizing all the distributional information that is available" (Kahneman and Tversky 1979b, p. 316).

Using distributional information from previous ventures similar to the one being forecast is called taking an "outside view". Reference class forecasting is a method for taking an outside view on planned actions.

Reference class forecasting for a specific project involves the following three steps:
  1. Identify a reference class of past, similar projects.
  2. Establish a probability distribution
    Probability distribution
    In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....

     for the selected reference class for the parameter
    Parameter from Ancient Greek παρά also “para” meaning “beside, subsidiary” and μέτρον also “metron” meaning “measure”, can be interpreted in mathematics, logic, linguistics, environmental science and other disciplines....

     that is being forecast.
  3. Compare the specific project with the reference class distribution, in order to establish the most likely outcome for the specific project.

Whereas Kahneman and Tversky developed the theories of reference class forecasting, Flyvbjerg and COWI (2004) developed the method for its practical use in policy and planning. The first instance of reference class forecasting in practice is described in Flyvbjerg (2006). This was a forecast carried out in 2004 by Ove Arup
Ove Arup
Sir Ove Nyquist Arup, CBE, MICE, MIStructE known as Ove Arup, was a leading Anglo-Danish engineer and generally considered to be one of the foremost architectural structural engineers of his time...

 of the projected capital costs for the Edinburgh Trams Line 2 proposal. The promoter's forecast estimated a cost of £320 million including allowance for contingency. Taking all available distributional information into account, based on a reference class of comparable rail projects, the reference class forecast estimated an 80th percentile value of £400 million. A report issued in August 2011 estimated that the final cost of the yet unfinished project would be over £1 billion, for a shorter tram line than the proposed Line 2.

Since the Edinburgh forecast, reference class forecasting has been applied to numerous other projects in the UK, including the £15 (US$29) billion Crossrail project in London. After 2004, The Netherlands, Denmark, and Switzerland have also implemented various types of reference class forecasting. In 2005, the American Planning Association
American Planning Association
The American Planning Association is a professional organization representing the field of city and regional planning in the United States. The APA was formed in 1978 when two separate professional planning organizations, the American Institute of Planners and the American Society of Planning...

 (APA) endorsed reference class forecasting and recommended that planners should never rely solely on conventional forecasting techniques:

"APA encourages planners to use reference class forecasting in addition to traditional methods as a way to improve accuracy. The reference class forecasting method is beneficial for non-routine projects ... Planners should never rely solely on civil engineering technology as a way to generate project forecasts" (the American Planning Association 2005).

Before this, in 2001 (updated in 2006), AACE International
AACE International
AACE International was founded in 1956 by 59 cost estimators and cost engineers during the organizational meeting of the American Association of Cost Engineering at the University of New Hampshire in Durham, New Hampshire. AACE International Headquarters is located in Morgantown, West Virginia, USA...

 (the Association for the Advancement of Cost Engineering) included Estimate Validation as a distinct step in the recommended practice of Cost Estimating (Estimate Validation is equivalent to Reference class forecasting in that it calls for separate empirical-based evaluations to benchmark the base estimate):

"The estimate should be benchmarked or validated against or compared to historical experience and/or past estimates of the enterprise and of competitive enterprises to check its appropriateness, competitiveness, and to identify improvement opportunities...Validation examines the estimate from a different perspective and using different metrics than are used in estimate preparation." (AACE International 2006)

In the process industries (e.g., oil and gas, chemicals, mining, energy, etc. which tend to dominate AACE's membership), benchmarking (i.e., "outside view") of project cost estimates against the historical costs of completed projects of similar types, including probabilistic information, has a long history (Merrow 1990).

See also

  • Reference class problem
    Reference class problem
    In statistics, the reference class problem is the problem of deciding what class to use when calculating the probability applicable to a particular case...

  • Benefit shortfall
    Benefit shortfall
    A benefit shortfall results from the actual benefits of a venture being lower than the projected, or estimated, benefits of that venture. If, for instance, a company is launching a new product or service and projected sales are 40 million dollars per year, whereas actual annual sales turn out to be...

  • Event chain methodology
    Event chain methodology
    Event chain methodology is an uncertainty modeling and schedule network analysis technique that is focused on identifying and managing events and event chains that affect project schedules...

  • Cost overrun
    Cost overrun
    A cost overrun, also known as a cost increase or budget overrun, is an unexpected cost incurred in excess of a budgeted amount due to an under-estimation of the actual cost during budgeting...

  • Optimism bias
    Optimism bias
    Optimism bias is the demonstrated systematic tendency for people to be overly optimistic about the outcome of planned actions. This includes over-estimating the likelihood of positive events and under-estimating the likelihood of negative events. Along with the illusion of control and illusory...

  • Planning fallacy
    Planning fallacy
    The planning fallacy is a tendency for people and organizations to underestimate how long they will need to complete a task, even when they have experience of similar tasks over-running. The term was first proposed in a 1979 paper by Daniel Kahneman and Amos Tversky...

  • Strategic misrepresentation
    Strategic misrepresentation
    "Strategic misrepresentation is the planned, systematic distortion or misstatement of fact—lying—in response to incentives in the budget process...

  • Forecasting
    Forecasting is the process of making statements about events whose actual outcomes have not yet been observed. A commonplace example might be estimation for some variable of interest at some specified future date. Prediction is a similar, but more general term...

  • Base rate fallacy
    Base rate fallacy
    The base rate fallacy, also called base rate neglect or base rate bias, is an error that occurs when the conditional probability of some hypothesis H given some evidence E is assessed without taking into account the "base rate" or "prior probability" of H and the total probability of evidence...

  • Financial risk
    Financial risk
    Financial risk an umbrella term for multiple types of risk associated with financing, including financial transactions that include company loans in risk of default. Risk is a term often used to imply downside risk, meaning the uncertainty of a return and the potential for financial loss...

The source of this article is wikipedia, the free encyclopedia.  The text of this article is licensed under the GFDL.