Backtesting
Encyclopedia
Backtesting is the process of evaluating a strategy, theory, or model by applying it to historical data. Backtesting can be used in situations like studying how a trading method would have performed in past stock markets or how a model of climate and weather patterns would have matched past measurements. A key element of backtesting that differentiates it from other forms of historical testing is that backtesting calculates how a strategy would have performed if it had actually been applied in the past. This requires the backtest to replicate the conditions of the time in question in order to get an accurate result. Backtesting is a common and methodologically accepted approach to research, however a high or successful correlation
between a backtested strategy and historical results can never prove a theory correct, since past results do not necessarily indicate future results. In other words, things are always changing, but in a world where yesterday bears some resemblance to today, backtesting can be a useful tool of analysis and prediction.
Backtesting can be applied to any set of historical data, but it is most common in the social and natural sciences where processes produce measurable data, take a relatively long time, and are chaotic enough to suggest a statistical approach.
In the August 2010 issue of Stocks, Futures and Options Magazine it was confirmed that Louis B. Mendelsohn
was the first person to introduce backtesting in trading software for the personal computer in 1983.
A key element of backtesting that differentiates it from other forms of historical testing is that backtesting calculates how a strategy would have performed if it had actually been applied in the past. This requires the backtest to replicate the market conditions of the time in question in order to get an accurate result. Examples of these market conditions include screening/buying/selling stocks that no longer exist, or using market index compositions as they were in the past, rather than current compositions. Due to the expense of obtaining these data sets, backtesting has historically been performed by institutions and professional money managers. With the advent of electronic trading and more accessible online databases, however, basic backtrading has become an option for casual traders as well and may be included as part of an investor's online brokerage account.
Various types of capital market strategies can be backtested, such as asset allocation strategies, stock screening strategies, and trading strategies. Other types of strategies are less amenable to backtesting, such as programmed trading strategies for buying or selling large quantities of a stock at the best prices by spreading the trade over a period of hours, days or weeks. This is because the act of selling large quantities of an individual issue affects the trading price for that issue, resulting in a feedback loop. Since the feedback loop is the effect being studied, backtesting is inappropriate for such strategies.
s. For example, in composing a new theory of hurricane formation, a model could be backtested against actual conditions that preceded and accompanied real hurricanes. If the model accurately forecast the location, strength, trajectory and duration of a past event, it would gain credibility for future predictions. With the field of climate modeling, backtesting plays a particularly important role because changes to the global climate happen on a scale for which it is prohibitively slow to just 'wait and see'. Doing so could take decades or centuries to even begin to evaluate new ideas. Backtesting, then, allows the hundreds and thousands of years of past climate data to be employed as benchmarks for new theories. Afterall, if a model could not accurately predict what you know to have already happened, it probably is not worth holding onto going forward.
Correlation
In statistics, dependence refers to any statistical relationship between two random variables or two sets of data. Correlation refers to any of a broad class of statistical relationships involving dependence....
between a backtested strategy and historical results can never prove a theory correct, since past results do not necessarily indicate future results. In other words, things are always changing, but in a world where yesterday bears some resemblance to today, backtesting can be a useful tool of analysis and prediction.
Backtesting can be applied to any set of historical data, but it is most common in the social and natural sciences where processes produce measurable data, take a relatively long time, and are chaotic enough to suggest a statistical approach.
In the August 2010 issue of Stocks, Futures and Options Magazine it was confirmed that Louis B. Mendelsohn
Louis B. Mendelsohn
Louis B. Mendelsohn, a pioneer in the application of personal computers to the financial markets, is President and Chief Executive Officer of Market Technologies, LLC, which he founded in 1979 to develop technical analysis trading software for use by commodity futures traders.- Biography...
was the first person to introduce backtesting in trading software for the personal computer in 1983.
Backtesting in finance and economics
In the application of backtesting techniques to capital markets, backtesting is a specific type of historical testing that determines the performance of the strategy if it had actually been employed during past periods and market conditions. Since backtesting uses real-world data, it has advantages over testing with synthesized data sets. While backtesting does not allow one to predict how a strategy will perform under future conditions, its primary benefit lies in understanding the vulnerabilities of a strategy as it encountered real-world conditions of the past. This enables the designer of a strategy to "learn from their mistakes" without actually having to make them with actual money.A key element of backtesting that differentiates it from other forms of historical testing is that backtesting calculates how a strategy would have performed if it had actually been applied in the past. This requires the backtest to replicate the market conditions of the time in question in order to get an accurate result. Examples of these market conditions include screening/buying/selling stocks that no longer exist, or using market index compositions as they were in the past, rather than current compositions. Due to the expense of obtaining these data sets, backtesting has historically been performed by institutions and professional money managers. With the advent of electronic trading and more accessible online databases, however, basic backtrading has become an option for casual traders as well and may be included as part of an investor's online brokerage account.
Various types of capital market strategies can be backtested, such as asset allocation strategies, stock screening strategies, and trading strategies. Other types of strategies are less amenable to backtesting, such as programmed trading strategies for buying or selling large quantities of a stock at the best prices by spreading the trade over a period of hours, days or weeks. This is because the act of selling large quantities of an individual issue affects the trading price for that issue, resulting in a feedback loop. Since the feedback loop is the effect being studied, backtesting is inappropriate for such strategies.
Backtesting in climate modeling
Backtesting plays a critical role in the evaluation of weather and climate modelClimate model
Climate models use quantitative methods to simulate the interactions of the atmosphere, oceans, land surface, and ice. They are used for a variety of purposes from study of the dynamics of the climate system to projections of future climate...
s. For example, in composing a new theory of hurricane formation, a model could be backtested against actual conditions that preceded and accompanied real hurricanes. If the model accurately forecast the location, strength, trajectory and duration of a past event, it would gain credibility for future predictions. With the field of climate modeling, backtesting plays a particularly important role because changes to the global climate happen on a scale for which it is prohibitively slow to just 'wait and see'. Doing so could take decades or centuries to even begin to evaluate new ideas. Backtesting, then, allows the hundreds and thousands of years of past climate data to be employed as benchmarks for new theories. Afterall, if a model could not accurately predict what you know to have already happened, it probably is not worth holding onto going forward.