Texas sharpshooter fallacy
Encyclopedia
The Texas sharpshooter fallacy is a logical fallacy in which pieces of information that have no relationship to one another are called out for their similarities, and that similarity is used for claiming the existence of a pattern. This fallacy is the philosophical/rhetorical application of the multiple comparisons
problem in statistics, and apophenia
in cognitive psychology. It is related to the clustering illusion
, which refers to the tendency in human cognition
to interpret patterns in randomness where none actually exist.
The name comes from a joke about a Texan
who fires some shots
at the side of a barn, then paints a target
centered on the biggest cluster of hits and claims to be a sharpshooter
.
To illustrate, suppose that a researcher flips a coin ten times to determine whether or not the coin is balanced. The probability of that researcher seeing the coin come up heads all ten flips is 1 in 1,024 — if it were to happen it would be reasonable to assume that the coin is indeed biased. However, suppose instead that the researcher has 5,000 coins, and performs the same ten-flip test on each of them. Though each coin should only have a ten-head probability of 1 in 1,024, it is overwhelmingly likely (with a probability of 99.24%) that, if this ten-flip test is performed 5,000 times, at least one such time will have a coin coming up heads for all ten flips by pure chance alone. Therefore, the alleged finding of an imbalanced coin is a nearly inevitable outcome of the testing methodology itself. If the researcher indeed finds one such ten-head occurrence in 5,000 trials and claims that the coin involved in the ten-head occurrence is imbalanced, the researcher would be committing a Texas sharpshooter fallacy. (Note that, if the coin is indeed balanced, then subsequent testing with that same coin will fail to reveal an unusual occurrence of heads.)
The fallacy is characterized by a lack of specific hypothesis prior to the gathering of data, or the formulation of a hypothesis only after data has already been gathered and examined. Thus, it typically does not apply if one had an ex ante, or prior, expectation of the particular relationship in question before examining the data. For example one might, prior to examining the information, have in mind a specific physical mechanism implying the particular relationship. One could then use the information to give support or cast doubt on the presence of that mechanism. Alternatively, if additional information can be generated using the same process as the original information, one can use the original information to construct a hypothesis, and then test the hypothesis on the new data. See hypothesis testing. What one cannot do is use the same information to construct and test the same hypothesis (see hypotheses suggested by the data) — to do so would be to commit the Texas sharpshooter fallacy.
Multiple comparisons
In statistics, the multiple comparisons or multiple testing problem occurs when one considers a set of statistical inferences simultaneously. Errors in inference, including confidence intervals that fail to include their corresponding population parameters or hypothesis tests that incorrectly...
problem in statistics, and apophenia
Apophenia
Apophenia is the experience of seeing meaningful patterns or connections in random or meaningless data.The term was coined in 1958 by Klaus Conrad, who defined it as the "unmotivated seeing of connections" accompanied by a "specific experience of an abnormal meaningfulness", but it has come to...
in cognitive psychology. It is related to the clustering illusion
Clustering illusion
The clustering illusion refers to the tendency erroneously to perceive small samples from random distributions to have significant "streaks" or "clusters", caused by a human tendency to underpredict the amount of variability likely to appear in a small sample of random or semi-random data due to...
, which refers to the tendency in human cognition
Cognition
In science, cognition refers to mental processes. These processes include attention, remembering, producing and understanding language, solving problems, and making decisions. Cognition is studied in various disciplines such as psychology, philosophy, linguistics, and computer science...
to interpret patterns in randomness where none actually exist.
The name comes from a joke about a Texan
Texas
Texas is the second largest U.S. state by both area and population, and the largest state by area in the contiguous United States.The name, based on the Caddo word "Tejas" meaning "friends" or "allies", was applied by the Spanish to the Caddo themselves and to the region of their settlement in...
who fires some shots
Gunshot
A gunshot is the discharge of a firearm, producing a mechanical sound effect and a chemical gunshot residue. The term can also refer to a gunshot wound caused by such a discharge. Multiple discharges of a firearm or firearms are referred to as gunfire. The word can connotate either the sound of a...
at the side of a barn, then paints a target
Bullseye (target)
The bullseye, or bull's-eye, is the centre of a target , and by extension the name given to any shot that hits the bullseye...
centered on the biggest cluster of hits and claims to be a sharpshooter
Marksman
A marksman is a person who is skilled in precision, or a sharpshooter shooting, using projectile weapons, such as with a rifle but most commonly with a sniper rifle, to shoot at long range targets...
.
Structure
The Texas sharpshooter fallacy often arises when a person has a large amount of data at their disposal, but only focuses on a small subset of that data. Random chance may give all the elements in that subset some kind of common property (or pair of common properties, when arguing for correlation). If the person fails to account for the likelihood of finding some subset in the large data with some common property strictly by chance alone, that person is likely committing a Texas Sharpshooter fallacy.To illustrate, suppose that a researcher flips a coin ten times to determine whether or not the coin is balanced. The probability of that researcher seeing the coin come up heads all ten flips is 1 in 1,024 — if it were to happen it would be reasonable to assume that the coin is indeed biased. However, suppose instead that the researcher has 5,000 coins, and performs the same ten-flip test on each of them. Though each coin should only have a ten-head probability of 1 in 1,024, it is overwhelmingly likely (with a probability of 99.24%) that, if this ten-flip test is performed 5,000 times, at least one such time will have a coin coming up heads for all ten flips by pure chance alone. Therefore, the alleged finding of an imbalanced coin is a nearly inevitable outcome of the testing methodology itself. If the researcher indeed finds one such ten-head occurrence in 5,000 trials and claims that the coin involved in the ten-head occurrence is imbalanced, the researcher would be committing a Texas sharpshooter fallacy. (Note that, if the coin is indeed balanced, then subsequent testing with that same coin will fail to reveal an unusual occurrence of heads.)
The fallacy is characterized by a lack of specific hypothesis prior to the gathering of data, or the formulation of a hypothesis only after data has already been gathered and examined. Thus, it typically does not apply if one had an ex ante, or prior, expectation of the particular relationship in question before examining the data. For example one might, prior to examining the information, have in mind a specific physical mechanism implying the particular relationship. One could then use the information to give support or cast doubt on the presence of that mechanism. Alternatively, if additional information can be generated using the same process as the original information, one can use the original information to construct a hypothesis, and then test the hypothesis on the new data. See hypothesis testing. What one cannot do is use the same information to construct and test the same hypothesis (see hypotheses suggested by the data) — to do so would be to commit the Texas sharpshooter fallacy.
Examples
- A Swedish study in 1992 tried to determine whether or not power lines caused some kind of poor health effects. The researchers surveyed everyone living within 300 meters of high-voltage power lines over a 25-year period and looked for statistically significant increases in rates of over 800 ailments. The study found that the incidence of childhood leukemia was four times higher among those that lived closest to the power lines, and it spurred calls to action by the Swedish government. The problem with the conclusion, however, was that the number of potential ailments, i.e. over 800, was so large that it created a high probability that at least one ailment would exhibit statistically significant difference just by chance alone. Subsequent studies failed to show any links between power lines and childhood leukemia, neither in causation nor even in correlation.
- Attempts to find cryptograms in the works of William ShakespeareShakespeare authorship questionImage:ShakespeareCandidates1.jpg|thumb|alt=Portraits of Shakespeare and four proposed alternative authors.|Oxford, Bacon, Derby, and Marlowe have each been proposed as the true author...
, which tended to report results only for those passages of ShakespeareWilliam ShakespeareWilliam Shakespeare was an English poet and playwright, widely regarded as the greatest writer in the English language and the world's pre-eminent dramatist. He is often called England's national poet and the "Bard of Avon"...
for which the proposed decoding algorithm produced an intelligible result. This could be explained as an example of the fallacy because passages which do not match the algorithm have not been accounted for. - Attempts to find cryptograms in the BibleBible codeThe Bible code , also known as the Torah code, is a purported set of secret messages encoded within the text Hebrew Bible and describing prophesies and other guidance regarding the future. This hidden code has been described as a method by which specific letters from the text can be selected to...
, and the Quran CodeQuran CodeThe Quran code is a series of numerical patterns claimed to exist in the text of the Quran, mostly using the number 19 as a common denominator...
. - This fallacy is often found in modern-day interpretations of the quatrainQuatrainA quatrain is a stanza, or a complete poem, consisting of four lines of verse. Existing in various forms, the quatrain appears in poems from the poetic traditions of various ancient civilizations including Ancient Greece, Ancient Rome, and China; and, continues into the 21st century, where it is...
s of NostradamusNostradamusMichel de Nostredame , usually Latinised to Nostradamus, was a French apothecary and reputed seer who published collections of prophecies that have since become famous worldwide. He is best known for his book Les Propheties , the first edition of which appeared in 1555...
. Nostradamus' quatrains are often liberally translated from the original (archaic) French, stripped of their historical context, and then applied to support the conclusion that Nostradamus predicted a given modern-day event, after the event actually occurred. For instance, the Nostradamus lines that supposedly predicted 9/11 were taken from three separate and unrelated passages and a fictional line was added. - An xkcdXkcdxkcd is a webcomic created by Randall Munroe. The comic's tagline describes it as "a webcomic of romance, sarcasm, math, and language." It has been recognized in such mainstream media as The Guardian and The New York Times....
comic illustrates this fallacy at work in a fictional announcement that green jelly beans cause acne.
See also
- Accuracy and precisionAccuracy and precisionIn the fields of science, engineering, industry and statistics, the accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity's actual value. The precision of a measurement system, also called reproducibility or repeatability, is the degree to which...
- Anthropic principleAnthropic principleIn astrophysics and cosmology, the anthropic principle is the philosophical argument that observations of the physical Universe must be compatible with the conscious life that observes it. Some proponents of the argument reason that it explains why the Universe has the age and the fundamental...
- Availability heuristicAvailability heuristicThe availability heuristic is a phenomenon in which people predict the frequency of an event, or a proportion within a population, based on how easily an example can be brought to mind....
- Confirmation biasConfirmation biasConfirmation bias is a tendency for people to favor information that confirms their preconceptions or hypotheses regardless of whether the information is true.David Perkins, a geneticist, coined the term "myside bias" referring to a preference for "my" side of an issue...
- PostdictionPostdictionAccording to critics of paranormal beliefs, postdiction is an effect of hindsight bias that explains claimed predictions of significant events, such as plane crashes and natural disasters...
Related logical fallacies
- Cum hoc ergo propter hoc
- Post hoc ergo propter hocPost hoc ergo propter hocPost hoc ergo propter hoc, Latin for "after this, therefore because of this," is a logical fallacy that states, "Since that event followed this one, that event must have been caused by this one." It is often shortened to simply post hoc and is also sometimes referred to as false cause,...
- Correlative based fallacies
- Moving the goalpostMoving the goalpostMoving the goalposts is a metaphor meaning changing the target of a process or competition by one side in order to gain advantage.-Etymology:...
, a related fallacy used to obtain the opposite conclusion.