The Lincoln index is a statistical measure used in several fields to estimate

Estimator

In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule and its result are distinguished....

 the number of cases that have not yet been observed, based on two independent sets of observed cases. It is also sometimes known as the Lincoln-Petersen method.

Applications

Consider two observers who separately count the different species of plants or animals in a given area. If they each come back having found 100 species but only 5 particular species are found by both observers, then each observer clearly missed at least 95 species (that is, the 95 that only the other observer found). Thus, we know that both observers miss a lot. On the other hand, if 99 of the 100 species each observer found had been found by both, it is fair to expect that they have found a far higher percentage of the total species that are there to find.

The same reasoning applies to mark and recapture

Mark and recapture

Mark and recapture is a method commonly used in ecology to estimate population size. This method is most valuable when a researcher fails to detect all individuals present within a population of interest every time that researcher visits the study area...

. If some animals in a given area are caught and marked, and later a second round of captures is done: the number of marked animals found in the second round can be used to generate an estimate of the total population.

Another example arises in computational linguistics

Computational linguistics

Computational linguistics is an interdisciplinary field dealing with the statistical or rule-based modeling of natural language from a computational perspective....

 for estimating the total vocabulary of a language. Given two independent samples, the overlap between their vocabularies enables a useful estimate of how many more vocabulary items exist but did not happen to show up in either sample. A similar example involves estimating the number of typographical errors remaining in a text, from two proofreaders' counts.

Formulation

The Lincoln Index formalizes this phenomenon. If E1 and E2 are the number of species (or words, or other phenomena) observed by two independent methods, and S is the number of observations in common, then the Lincoln Index is simply

L = E1×E2/S

If S = 0 (that is, if there is no overlap), then the Lincoln Index is undefined and no estimate is justified. This can arise if the observers only find a small percentage of the actual species (perhaps by not looking hard enough or long enough), if the observers are using methods that are not statistically independent (for example if one looks only for large creatures and the other only for small), or in other circumstances.

Limitations

The Lincoln Index is merely an estimate. For example, the species in a given area could tend to be either very common or very rare, or tend to be either very hard or very easy to see. Then it would be likely that both observers would find a large share of the common species, and that both observers would miss a large share of the rare ones. Such distributions would throw off the consequent estimate. However, such distributions are unusual for natural phenomena, as suggested by Zipf's Law).

T. J. Gaskell and B. J. George propose an enhancement of the Lincoln Index that claims to reduce bias.

Further reading

• Lincoln, F. C. (1930). "Calculating Waterfowl Abundance on the Basis of Banding Returns". United States Department of Agriculture Circular, 118, 1–4.
• Petersen, C. G. J. (1896). "The Yearly Immigration of Young Plaice Into the Limfjord From the German Sea", Report of the Danish Biological Station (1895), 6, 5–84.