Model risk
Encyclopedia
In finance
, model risk is the risk involved in using models to value financial securities. Rebonato considers alternative definitions including:
Rebonato defines model risk as "the risk of occurrence of a significant difference between the mark-to-model value of a complex and/or illiquid instrument, and the price at which the same instrument is revealed to have traded in the market."
Wrong model
Model implementation
Model usage
Buraschi and Corelli formalise the concept of 'time inconsistency' with regards to no-arbitrage
models that allow for a perfect fit of the term structure of the interest rates. In these models the current yield curve
is an input so that new observations on the yield curve
can be used to update the model at regular frequencies. They explore the issue of time-consistent and self-financing strategies in this class of models. Model risk affects all the three main steps of risk management
: specification, estimation and implementation.
The volatility smile
Derman believes that products whose value depends on a volatility smile
are most likely to suffer from model risk. He writes "I would think it’s safe to say that there is no area where model risk is more of an issue than in the modeling of the volatility smile."
Correlation modelling
Gennheimer investigates the model risk present in pricing basket default derivatives. He prices these derivatives with various copulas and concludes that "... unless one is very sure about the dependence structure governing the credit basket, any investors willing to trade basket default products should imperatively compute prices under alternative copula specifications and verify the estimation errors of their simulation to know at least the model risks they run."
Buraschi, Porchia and Trojani (2010) propose a framework for intertemporal portfolio choice when the covariance matrix of returns is stochastic. An important contribution of this framework is that it allows to derive optimal portfolio implications for economies in which the degree of correlation across different industries, countries, and asset classes is time-varying and stochastic.
Implementation
Testing
Model averaging
Rantala (2006) mentions that "In the face of model risk, rather than to base decisions on a single selected ”best” model, the modeller can base his inference on an entire set of models by using model averaging."
Position limits and valuation reserves
Kato and Yoshiba discuss qualitative and quantitative ways of controlling model risk. They write "From a quantitative perspective, in the case of pricing models, we can set up a reserve to allow for the difference in estimations using alternative models. In the case of risk measurement models, scenario analysis can be undertaken for various fluctuation patterns of risk factors, or position limits can be established based on information obtained from scenario analysis."
Taleb wrote when describing why most new models that attempted to correct the inadequacies of the Black–Scholes model failed to become accepted:
"Traders are not fooled by the Black–Scholes–Merton model. The existence of a 'volatility surface' is one such adaptation. But they find it preferable to fudge one parameter, namely volatility, and make it a function of time to expiry and strike price, rather than have to precisely estimate another."
However Cherubini and Della Lunga describe the disadavantages of parsimony in the context of volatility and correlation modelling. Using an excessive number of parameters may induce overfitting
while choosing a severely specified model may easily induce model misspecification and a systematic failure to represent the future distribution.
Model risk does not only exit for complex financial contracts. Frey (2000) presents a study of how market illiquidity is a source of model risk.
He writes "Understanding the robustness of models used for hedging and risk-management purposes with respect to the assumption of perfectly liquid markets is therefore an important issue in the analysis of model risk in general."
Convertible bonds, mortgage backed securities and high-yield bonds can often be illiquid and difficult to value. Hedge funds that trade these securities can be exposed to model risk when calculating monthly NAV for its investors.
Model risk premium
Fender and Kiff (2004) note that holding complex financial instruments such as CDOs "translates into heightened dependence on these assumptions and, thus, higher model risk. As this risk should be expected to be priced by the market, part of the yield pick-up obtained relative to equally rated single obligor instruments is likely to be a direct reflection of model risk."
Difficulty of quantifying model risk
To measure the risk induced by a model it has to be compared to an alternative model. To correctly specify the model risk you have to know an accurate model. However, accurate models are often hard to find. As we cannot always find an accurate model we need a benchmark model giving a close approximation to our data. The problem is how to choose this benchmark model. If we knew the model closest to our data we could easily use this model for risk measurement and model risk would no longer be a problem. Thus, finding a suitable benchmark model is a serious problem. However, it will hardly be possible to overcome this problem when discussing model risk.
Finance
"Finance" is often defined simply as the management of money or “funds” management Modern finance, however, is a family of business activity that includes the origination, marketing, and management of cash and money surrogates through a variety of capital accounts, instruments, and markets created...
, model risk is the risk involved in using models to value financial securities. Rebonato considers alternative definitions including:
- After observing a set of prices for the underlying and hedging instruments, different but identically calibrated models might produce different prices for the same exotic product.
- Losses will be incurred because of an ‘incorrect’ hedging strategy suggested by a model.
Rebonato defines model risk as "the risk of occurrence of a significant difference between the mark-to-model value of a complex and/or illiquid instrument, and the price at which the same instrument is revealed to have traded in the market."
Types of model risk
Burke regards failure to use a model (instead over-relying on expert judgement) as a type of model risk. Derman describes various types of model risk that arise from using a model:Wrong model
- Inapplicability of model.
- Incorrect model specification.
Model implementation
- Programming errors.
- Technical errors.
- Use of numerical approximations.
Model usage
- Implementation Risk.
- Data issues.
- Calibration errors.
Sources of model risk
Interest rate modellingBuraschi and Corelli formalise the concept of 'time inconsistency' with regards to no-arbitrage
Arbitrage
In economics and finance, arbitrage is the practice of taking advantage of a price difference between two or more markets: striking a combination of matching deals that capitalize upon the imbalance, the profit being the difference between the market prices...
models that allow for a perfect fit of the term structure of the interest rates. In these models the current yield curve
Yield curve
In finance, the yield curve is the relation between the interest rate and the time to maturity, known as the "term", of the debt for a given borrower in a given currency. For example, the U.S. dollar interest rates paid on U.S...
is an input so that new observations on the yield curve
Yield curve
In finance, the yield curve is the relation between the interest rate and the time to maturity, known as the "term", of the debt for a given borrower in a given currency. For example, the U.S. dollar interest rates paid on U.S...
can be used to update the model at regular frequencies. They explore the issue of time-consistent and self-financing strategies in this class of models. Model risk affects all the three main steps of risk management
Financial risk management
Financial risk management is the practice of creating economic value in a firm by using financial instruments to manage exposure to risk, particularly credit risk and market risk. Other types include Foreign exchange, Shape, Volatility, Sector, Liquidity, Inflation risks, etc...
: specification, estimation and implementation.
The volatility smile
Derman believes that products whose value depends on a volatility smile
Volatility Smile
In finance, the volatility smile is a long-observed pattern in which at-the-money options tend to have lower implied volatilities than in- or out-of-the-money options. The pattern displays different characteristics for different markets and results from the probability of extreme moves...
are most likely to suffer from model risk. He writes "I would think it’s safe to say that there is no area where model risk is more of an issue than in the modeling of the volatility smile."
Correlation modelling
Gennheimer investigates the model risk present in pricing basket default derivatives. He prices these derivatives with various copulas and concludes that "... unless one is very sure about the dependence structure governing the credit basket, any investors willing to trade basket default products should imperatively compute prices under alternative copula specifications and verify the estimation errors of their simulation to know at least the model risks they run."
Buraschi, Porchia and Trojani (2010) propose a framework for intertemporal portfolio choice when the covariance matrix of returns is stochastic. An important contribution of this framework is that it allows to derive optimal portfolio implications for economies in which the degree of correlation across different industries, countries, and asset classes is time-varying and stochastic.
Mitigating model risk
Theoretical basis- Considering key assumptions.
- Considering simple cases and their solutions (model boundaries).
- Parsimony.
Implementation
- Pride of ownership.
- Disseminating the model outwards in an orderly manner.
Testing
- Stress testingStress testing (software)In software testing, stress testing refers to tests that determine the robustness of software by testing beyond the limits of normal operation. Stress testing is particularly important for "mission critical" software, but is used for all types of software...
and backtestingBacktestingBacktesting is the process of evaluating a strategy, theory, or model by applying it to historical data. Backtesting can be used in situations like studying how a trading method would have performed in past stock markets or how a model of climate and weather patterns would have matched past...
. - Try to simulate model risk.
- Avoid letting small issues snowball into large issues later on.
Model averaging
Rantala (2006) mentions that "In the face of model risk, rather than to base decisions on a single selected ”best” model, the modeller can base his inference on an entire set of models by using model averaging."
Position limits and valuation reserves
Kato and Yoshiba discuss qualitative and quantitative ways of controlling model risk. They write "From a quantitative perspective, in the case of pricing models, we can set up a reserve to allow for the difference in estimations using alternative models. In the case of risk measurement models, scenario analysis can be undertaken for various fluctuation patterns of risk factors, or position limits can be established based on information obtained from scenario analysis."
Examples of model risk mitigation
ParsimonyTaleb wrote when describing why most new models that attempted to correct the inadequacies of the Black–Scholes model failed to become accepted:
"Traders are not fooled by the Black–Scholes–Merton model. The existence of a 'volatility surface' is one such adaptation. But they find it preferable to fudge one parameter, namely volatility, and make it a function of time to expiry and strike price, rather than have to precisely estimate another."
However Cherubini and Della Lunga describe the disadavantages of parsimony in the context of volatility and correlation modelling. Using an excessive number of parameters may induce overfitting
Overfitting
In statistics, overfitting occurs when a statistical model describes random error or noise instead of the underlying relationship. Overfitting generally occurs when a model is excessively complex, such as having too many parameters relative to the number of observations...
while choosing a severely specified model may easily induce model misspecification and a systematic failure to represent the future distribution.
Model risk properties and implications
Illiquid product model riskModel risk does not only exit for complex financial contracts. Frey (2000) presents a study of how market illiquidity is a source of model risk.
He writes "Understanding the robustness of models used for hedging and risk-management purposes with respect to the assumption of perfectly liquid markets is therefore an important issue in the analysis of model risk in general."
Convertible bonds, mortgage backed securities and high-yield bonds can often be illiquid and difficult to value. Hedge funds that trade these securities can be exposed to model risk when calculating monthly NAV for its investors.
Model risk premium
Fender and Kiff (2004) note that holding complex financial instruments such as CDOs "translates into heightened dependence on these assumptions and, thus, higher model risk. As this risk should be expected to be priced by the market, part of the yield pick-up obtained relative to equally rated single obligor instruments is likely to be a direct reflection of model risk."
Difficulty of quantifying model risk
To measure the risk induced by a model it has to be compared to an alternative model. To correctly specify the model risk you have to know an accurate model. However, accurate models are often hard to find. As we cannot always find an accurate model we need a benchmark model giving a close approximation to our data. The problem is how to choose this benchmark model. If we knew the model closest to our data we could easily use this model for risk measurement and model risk would no longer be a problem. Thus, finding a suitable benchmark model is a serious problem. However, it will hardly be possible to overcome this problem when discussing model risk.
Case studies
- Natwest - Interest rate options and swaptions - incorrect model specification.
- Bank of Tokyo/Mitsubushi - Interest rate options and swaptions.
- LTCM - lack of stress testing - Crouhy, Galai and Mark.
- Barclays de Zoete Wedd (BZW) - Mispriced currency options.