Applied information economics
Encyclopedia
Applied information economics (AIE) is a decision analysis method developed by Douglas W. Hubbard and partially described in his book How to Measure Anything: Finding the Value of Intangibles in Business (2007; 2nd ed. 2010). AIE is a method for the practical application of several proven methods from decision theory
and risk analysis
including the use of Monte Carlo method
s. However, unlike some other modeling approaches with simulations, AIE incorporates the following:
Practitioners of AIE claim that if something has an impact on an organization, it must be observable and, therefore, measurable.
AIE does tend to be somewhat more elaborate than these alternatives. But practitioners argue that it is no more complicated than analysis methods used in many other fields, as long as trained specialists are used. It also becomes more important to choose rigor over simplicity when the decisions being analyzed are much larger and riskier. The drawback to the AIE methodology is its sophistication, which requires an analytical background to understand and appreciate. This would impede adoption.
AIE as a whole, like many decision analysis and risk analysis methods, has little or no research showing the long term benefits of the method. However, AIE itself is not new method and is based on previously-developed components that have a sound theoretical basis and/or have strong empirical evidence of improving on unaided intuition or other popular decision analysis methods. Among these components are Monte Carlo simulations, calibration training, information value calculations from decision theory, and widely accepted empirical methods used for scientific measurement (see references above).
Decision theory
Decision theory in economics, psychology, philosophy, mathematics, and statistics is concerned with identifying the values, uncertainties and other issues relevant in a given decision, its rationality, and the resulting optimal decision...
and risk analysis
Risk analysis (Business)
Risk analysis is a technique to identify and assess factors that may jeopardize the success of a project or achieving a goal.This technique also helps to define preventive measures to reduce the probability of these factors from occurring and identify countermeasures to successfully deal with these...
including the use of Monte Carlo method
Monte Carlo method
Monte Carlo methods are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo methods are often used in computer simulations of physical and mathematical systems...
s. However, unlike some other modeling approaches with simulations, AIE incorporates the following:
- Calibrated probability assessmentCalibrated probability assessmentCalibrated probability assessments are subjective probabilities assigned by individuals who have been trained to assess probabilities in a way that historically represents their uncertainty. In other words, when a calibrated person says they are "80% confident" in each of 100 predictions they...
. This is a method for training estimators and experts (who are relied on for the inputs in Monte Carlo methods) to be neutrally confident about their assigned probabilities. That is, their probabilities are neither over confident (too high) nor under confident (too low). - Computing the value of additional information. AIE uses information value calculations from decision theoryDecision theoryDecision theory in economics, psychology, philosophy, mathematics, and statistics is concerned with identifying the values, uncertainties and other issues relevant in a given decision, its rationality, and the resulting optimal decision...
such as the expected value of perfect informationExpected value of perfect informationIn decision theory, the expected value of perfect information is the price that one would be willing to pay in order to gain access to perfect information....
and the value of imperfect (partial) information. Often, this is done for a large number of uncertain variables in some type of decision model or business case. The result will reveal where efforts to reduce uncertainty by making further measurements are best spent. - Empirical methods applied according to the information value of the measurement. This step is, in fact, the reason for the name of the method. Most Monte Carlo modeling experts stop modeling after the first (uncalibrated) probability estimates from experts and there is usually little emphasis on further measurements with empirical methods. Since AIE computes the value of additional information, measurement can be selective and focused. This step often results in a very different set of measurement priorities than would otherwise have been used.
- Various optimization methods including modern portfolio theoryModern portfolio theoryModern portfolio theory is a theory of investment which attempts to maximize portfolio expected return for a given amount of portfolio risk, or equivalently minimize risk for a given level of expected return, by carefully choosing the proportions of various assets...
. MPT and other methods are applied to determine ideal risk and return positions for a set of alternatives.
Practitioners of AIE claim that if something has an impact on an organization, it must be observable and, therefore, measurable.
Comparison to other methods
AIE differs in several ways from other popular methods of decision analysis:- Unlike the accounting-style business caseBusiness caseA business case captures the reasoning for initiating a project or task. It is often presented in a well-structured written document, but may also sometimes come in the form of a short verbal argument or presentation. The logic of the business case is that, whenever resources such as money or...
or cost benefit analysis, it does not rely entirely on point estimates of uncertain values. Since it uses the Monte Carlo methodMonte Carlo methodMonte Carlo methods are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo methods are often used in computer simulations of physical and mathematical systems...
, uncertainty can be modeled explicitly. - Unlike most decision-theory analysis of uncertain decisions, the AIE model starts with "calibrating" the estimators.
- The calculation of the value of information is used to guide further measurement efforts.
- AIE converts all values to economic terms so that powerful financial and economic optimization methods can be employed. This is unlike some methods that rely mostly on subjective scores.
AIE does tend to be somewhat more elaborate than these alternatives. But practitioners argue that it is no more complicated than analysis methods used in many other fields, as long as trained specialists are used. It also becomes more important to choose rigor over simplicity when the decisions being analyzed are much larger and riskier. The drawback to the AIE methodology is its sophistication, which requires an analytical background to understand and appreciate. This would impede adoption.
Limitations
While there are multiple articles in industry periodicals and government sources (see below) referencing applied information economics, there are few in academic literature. In addition, the following limitations apply:- Until enough experts are fully trained in this method, its complexity might limit its adoption by managers accustomed to traditional cost-benefit analysis.
- Like traditional cost-benefit analysis it does not guarantee that some important factor won't be excluded if nobody thinks of adding it. It simply ensures that the factors included are at least calibrated estimates (not overconfident or underconfident) and that further reduction of uncertainty (by measurement efforts) is optimized and applied to the right factors.
- Some of the same limitations as Monte Carlo simulations intervene. For example, if some variables are (unknown to the analysts) covariant, then the Monte Carlo would generate a distribution of results that does not fit reality.
- Calibration removes certain systemic human estimate biases, but not all of them. It is a significant improvement on Monte Carlos models that have no calibration for initial estimates, but there is no guarantee that other biases of experts will not be introduced into the model.
- It uses MPT in a modified and limited manner. Since many business investments are evaluated as opportunities arise, and not in a large "batch", they are often evaluated one at a time. The only persistently used component of MPT is the investment boundary, modified for the assessment of single investments of various sizes.
AIE as a whole, like many decision analysis and risk analysis methods, has little or no research showing the long term benefits of the method. However, AIE itself is not new method and is based on previously-developed components that have a sound theoretical basis and/or have strong empirical evidence of improving on unaided intuition or other popular decision analysis methods. Among these components are Monte Carlo simulations, calibration training, information value calculations from decision theory, and widely accepted empirical methods used for scientific measurement (see references above).
See also
- Choice ModellingChoice ModellingChoice modelling attempts to model the decision process of an individual or segment in a particular context. Choice modelling may also be used to estimate non-market environmental benefits and costs....
- Decision theoryDecision theoryDecision theory in economics, psychology, philosophy, mathematics, and statistics is concerned with identifying the values, uncertainties and other issues relevant in a given decision, its rationality, and the resulting optimal decision...
- Behavioral economics
- Information economicsInformation economicsInformation economics or the economics of informationis a branch of microeconomic theory that studies how information affects an economy and economic decisions. Information has special characteristics. It is easy to create but hard to trust. It is easy to spread but hard to control. It...
- Monte Carlo methodMonte Carlo methodMonte Carlo methods are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo methods are often used in computer simulations of physical and mathematical systems...
- Calibrated probability assessmentCalibrated probability assessmentCalibrated probability assessments are subjective probabilities assigned by individuals who have been trained to assess probabilities in a way that historically represents their uncertainty. In other words, when a calibrated person says they are "80% confident" in each of 100 predictions they...
External links
- U.S. Government study on AIE http://www.cio.gov/documents/aie_report_final.pdf
- Federal CIO Council comparison of AIE and balanced scorecardBalanced scorecardThe Balanced Scorecard is a strategic performance management tool - a semi-standard structured report, supported by proven design methods and automation tools, that can be used by managers to keep track of the execution of activities by the staff within their control and to monitor the...
http://www.cio.gov/documents/PM_Lessons_Learned_Final_Report.pdf - Environmental Protection Agency AIE study about optimizing desktop replacement http://www.federalelectronicschallenge.net/resources/docs/aie_desktop.pdf
- Reference by Balance Scorecard Institute http://www.balancedscorecard.org/bscit/prm.html#aie and definition by same http://www.balancedscorecard.org/basics/definitions.html
- A book that discusses AIE within the context of Sarbanes Oxley http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471771996,descCd-tableOfContents.html
- Information Week article "ROI in the Real World" includes survey results on use of various methods, including AIE http://www.informationweek.com/679/79iuroi2.htm
- Dept. of Homeland Security Cybersecurity article discusses the AIE method https://buildsecurityin.us-cert.gov/daisy/bsi/articles/knowledge/business/684.html
- Applied Information Economics (AIE): Not just calculate, scientifically MEASURE IT http://www.iasahome.org/web/home/featurememberarticle
- Enterprise Architecture Benefit Analysis: Applied Information Economics (AIE) connects those SMART dots http://www.iasahome.org/c/portal/layout?p_l_id=PUB.1.398