Pairwise comparison
Encyclopedia
Pairwise comparison generally refers to any process of comparing entities in pairs to judge which of each entity is preferred
, or has a greater amount of some quantitative property
. The method of pairwise comparison is used in the scientific study of preference
s, attitudes, voting systems, social choice, public choice, and multiagent AI systems. In psychology
literature, it is often referred to as paired comparison.
Prominent psychometrician L. L. Thurstone first introduced a scientific approach to using pairwise comparisons for measurement in 1927, which he referred to as the law of comparative judgment
. Thurstone linked this approach to psychophysical theory developed by Ernst Heinrich Weber
and Gustav Fechner
. Thurstone demonstrated that the method can be used to order items along a dimension such as preference or importance using an interval-type scale.
The agent prefers x over y: "x > y" or "xPy"
The agent prefers y over x: "y > x" or "yPx"
The agent is indifferent between both alternatives: "x = y" or "xIy"
, called the law of comparative judgment, is more aptly regarded as a measurement model. The Bradley–Terry–Luce (BTL) model (Bradley & Terry, 1952; Luce, 1959) is often applied to pairwise comparison data to scale preferences. The BTL model is identical to Thurstone's model if the simple logistic function
is used. Thurstone used the normal distribution in applications of the model. The simple logistic function varies by less than 0.01 from the cumulative normal ogive
across the range, given an arbitrary scale factor.
In the BTL model, the probability that object j is judged to have more of an attribute than object i is:
Preference
-Definitions in different disciplines:The term “preferences” is used in a variety of related, but not identical, ways in the scientific literature. This makes it necessary to make explicit the sense in which the term is used in different social sciences....
, or has a greater amount of some quantitative property
Quantitative property
A quantitative property is one that exists in a range of magnitudes, and can therefore be measured with a number. Measurements of any particular quantitative property are expressed as a specific quantity, referred to as a unit, multiplied by a number. Examples of physical quantities are distance,...
. The method of pairwise comparison is used in the scientific study of preference
Preference
-Definitions in different disciplines:The term “preferences” is used in a variety of related, but not identical, ways in the scientific literature. This makes it necessary to make explicit the sense in which the term is used in different social sciences....
s, attitudes, voting systems, social choice, public choice, and multiagent AI systems. In psychology
Psychology
Psychology is the study of the mind and behavior. Its immediate goal is to understand individuals and groups by both establishing general principles and researching specific cases. For many, the ultimate goal of psychology is to benefit society...
literature, it is often referred to as paired comparison.
Prominent psychometrician L. L. Thurstone first introduced a scientific approach to using pairwise comparisons for measurement in 1927, which he referred to as the law of comparative judgment
Law of comparative judgment
The law of comparative judgment was conceived by L. L. Thurstone. In modern day terminology, it is more aptly described as a model that is used to obtain measurements from any process of pairwise comparison...
. Thurstone linked this approach to psychophysical theory developed by Ernst Heinrich Weber
Ernst Heinrich Weber
Ernst Heinrich Weber was a German physician who is considered one of the founders of experimental psychology.Weber studied medicine at Wittenberg University...
and Gustav Fechner
Gustav Fechner
Gustav Theodor Fechner , was a German experimental psychologist. An early pioneer in experimental psychology and founder of psychophysics, he inspired many 20th century scientists and philosophers...
. Thurstone demonstrated that the method can be used to order items along a dimension such as preference or importance using an interval-type scale.
Overview
If an individual or organization expresses a preference between two mutually distinct alternatives, this preference can be expressed as a pairwise comparison. If the two alternatives are x and y, the following are the possible pairwise comparisons:The agent prefers x over y: "x > y" or "xPy"
The agent prefers y over x: "y > x" or "yPx"
The agent is indifferent between both alternatives: "x = y" or "xIy"
Probabilistic models for pairwise comparison
In terms of modern psychometric theory, Thurstone's approachThurstone scale
In psychology, the Thurstone scale was the first formal technique for measuring an attitude. It was developed by Louis Leon Thurstone in 1928, as a means of measuring attitudes towards religion. It is made up of statements about a particular issue, and each statement has a numerical value...
, called the law of comparative judgment, is more aptly regarded as a measurement model. The Bradley–Terry–Luce (BTL) model (Bradley & Terry, 1952; Luce, 1959) is often applied to pairwise comparison data to scale preferences. The BTL model is identical to Thurstone's model if the simple logistic function
Logistic function
A logistic function or logistic curve is a common sigmoid curve, given its name in 1844 or 1845 by Pierre François Verhulst who studied it in relation to population growth. It can model the "S-shaped" curve of growth of some population P...
is used. Thurstone used the normal distribution in applications of the model. The simple logistic function varies by less than 0.01 from the cumulative normal ogive
Ogive
An ogive is the roundly tapered end of a two-dimensional or three-dimensional object.-Applied physical science and engineering:In ballistics or aerodynamics, an ogive is a pointed, curved surface mainly used to form the approximately streamlined nose of a bullet or other projectile.The traditional...
across the range, given an arbitrary scale factor.
In the BTL model, the probability that object j is judged to have more of an attribute than object i is:
-
where
is the scale location of object 
; 
is the inverse logitLogitThe logit function is the inverse of the sigmoidal "logistic" function used in mathematics, especially in statistics.Log-odds and logit are synonyms.-Definition:The logit of a number p between 0 and 1 is given by the formula:...
function. For example, the scale location might represent the perceived quality of a product, or the perceived weight of an object.
The BTL is very closely related to the Rasch modelRasch modelRasch models are used for analysing data from assessments to measure variables such as abilities, attitudes, and personality traits. For example, they may be used to estimate a student's reading ability from answers to questions on a reading assessment, or the extremity of a person's attitude to...
for measurement.
Thurstone used the method of pairwise comparisons as an approach to measuring perceived intensity of physical stimuli, attitudes, preferences, choices, and values. He also studied implications of the theory he developed for opinion polls and political voting (Thurstone, 1959).
Transitivity of pairwise comparisons
For a given decision agent, if the information, objective, and alternatives used by the agent remain constant, then it is generally assumed that pairwise comparisons over those alternatives by the decision agent are transitive. Most agree upon what transitivity is, though there is debate about the transitivity of indifference. The rules of transitivity are as follows for a given decision agent.
If xPy and yPz, then xPz
If xPy and yIz, then xPz
If xIy and yPz, then xPz
If xIy and yIz, then xIz
This corresponds to (xPy or xIy) being a total preorder, P being the corresponding strict weak order, and I being the corresponding equivalence relationEquivalence relationIn mathematics, an equivalence relation is a relation that, loosely speaking, partitions a set so that every element of the set is a member of one and only one cell of the partition. Two elements of the set are considered equivalent if and only if they are elements of the same cell...
.
Probabilistic models require transitivity only within the bounds of errors of estimates of scale locations of entities. Thus, decisions need not be deterministically transitive in order to apply probabilistic models. However, transitivity will generally hold for a large number of comparisons if models such as the BTL can be effectively applied.
Argument for intransitivity of indifference
Some contend that indifference is not transitive. Consider the following example. Suppose you like apples and you prefer apples that are larger. Now suppose there exists an apple A, an apple B, and an apple C which have identical intrinsic characteristics except for the following. Suppose B is larger than A, but it is not discernible without an extremely sensitive scale. Further suppose C is larger than B, but this also is not discernible without an extremely sensitive scale. However, the difference in sizes between apples A and C is large enough that you can discern that C is larger than A without a sensitive scale. In psychophysical terms, the size difference between A and C is above the just noticeable differenceJust noticeable differenceIn psychophysics, a just noticeable difference, customarily abbreviated with lowercase letters as jnd, is the smallest detectable difference between a starting and secondary level of a particular sensory stimulus...
('jnd') while the size differences between A and B and B and C are below the jnd.
You are confronted with the three apples in pairs without the benefit of a sensitive scale. Therefore, when presented A and B alone, you are indifferent between apple A and apple B; and you are indifferent between apple B and apple C when presented B and C alone. However, when the pair A and C are shown, you prefer C over A.
Preference orders
If pairwise comparisons are in fact transitive in respect to the four mentioned rules, then pairwise comparisons for a list of alternatives (A1, A2, A3, ..., An−1, and An) can take the form:
- A1(>XOR=)A2(>XOR=)A3(>XOR=) ... (>XOR=)An−1(>XOR=)An
For example, if there are three alternatives a, b, and c, then the possible preference orders are:
If the number of alternatives is n, and indifference is not allowed, then the number of possible preference orders for any given n-value is n! . If indifference is allowed, then the number of possible preference orders is the number of total preorders. It can be expressed as a function of n:
where S2(n, k) is the Stirling number of the second kind.
Applications
One important application of pairwise comparisons is the widely-used Analytic Hierarchy ProcessAnalytic Hierarchy ProcessThe Analytic Hierarchy Process is a structured technique for organizing and analyzing complex decisions. Based on mathematics and psychology, it was developed by Thomas L...
, a structured technique for helping people deal with complex decisions. It uses pairwise comparisons of tangible and intangible factors to construct ratio scales that are useful in making important decisions.
See also
- Law of comparative judgmentLaw of comparative judgmentThe law of comparative judgment was conceived by L. L. Thurstone. In modern day terminology, it is more aptly described as a model that is used to obtain measurements from any process of pairwise comparison...
- Potentially all pairwise rankings of all possible alternativesPotentially all pairwise rankings of all possible alternativesPotentially all pairwise rankings of all possible alternatives is a method for multi-criteria decision making or conjoint analysis based on decision-makers’ preferences as expressed using pairwise rankings of alternatives....
(PAPRIKA) method - PROMETHEEProméthéeProméthée is an opera in three acts by the French composer Gabriel Fauré. The libretto, by Jean Lorrain and Ferdinand Hérold, is based on the Greek myth of Prometheus. Although designated a tragédie lyrique, the opera resists easy categorisation. It was intended as a large-scale work with spoken...
pairwise comparison method - ScalingScalingScaling may refer to:* Scaling , a linear transformation that enlarges or diminishes objects* Reduced scales of semiconductor device fabrication processes...
Sources
- Sequence in OEIS
- Sequence in OEIS
- Y. Chevaleyre, P.E. Dunne, U. Endriss, J. Lang, M. Lemaître, N. Maudet, J. Padget, S. Phelps, J.A. Rodríguez-Aguilar, and P. Sousa. Issues in Multiagent Resource Allocation. Informatica, 30:3–31, 2006.
