Arrow's impossibility theorem

Arrow's impossibility theorem is a key result in social choice showing that no rank-order method for collective decision-making can behave rationally or coherently. Specifically, any such rule violates independence of irrelevant alternatives, the principle that a choice between $$A$$ and $$B$$ should not depend on the quality of a third, unrelated option $$C$$.

The result is most often cited in election science and voting theory, where $$C$$ is called a spoiler candidate. In this context, Arrow's theorem can be restated as showing that no ranked voting rule can eliminate the spoiler effect.

Some methods are more susceptible to spoilers than others. Plurality and instant-runoff in particular are highly sensitive to spoilers, often manufacturing them even in situations where they are not forced. By contrast, Condorcet methods minimize the possibility of spoilers. In other words, a ranked voting system can always be made limiting them to rare  situations called Condorcet paradoxes. As a result, the practical consequences of the theorem are debatable, with Arrow noting "Most [ranked] systems are not going to work badly all of the time. All I proved is that all can work badly at times."

Rated methods are not affected by Arrow's theorem or IIA failures. Arrow initially asserted the information provided by these systems was meaningless, and therefore could not be used to prevent paradoxes, leading him to overlook them. However, he and other authors would later recognize this to have been a mistake, with Arrow admitting systems based on cardinal utility (such as score and approval voting) are not subject to his theorem.

Background
Arrow's theorem falls under the branch of welfare economics and ethics called social choice theory. This field deals with aggregating preferences and beliefs to make fair or accurate decisions. The goal of social choice is to identify a social choice function—a procedure that determines which of two outcomes or options is better, according to all members of a society—that satisfies the properties of rational behavior.

Such a social ordering function can be any way to aggregate information or preferences from a group; this procedure can be a market, a voting system, a country's constitution, or even a moral or ethical framework. Arrow's theorem therefore generalizes the voting paradox discovered earlier by Condorcet, proving such paradoxes exist for any possible collective decision-making procedure that relies only on orderings of different options.

History
Arrow's theorem is named after economist and Nobel laureate Kenneth Arrow, who demonstrated it in his doctoral thesis and popularized it in his 1951 book.

Arrow's work is remembered as much for its pioneering methodology as its direct implications. Arrow's axiomatic approach provided a framework for proving facts about all conceivable social choice rules at once, contrasting with the earlier approach of investigating such rules one by one.

Among the most important axioms of rational choice is independence of irrelevant alternatives, which says that when deciding between $$A$$ and $$B$$, our opinion about some irrelevant option $$C$$ should not affect our decision. Arrow's theorem shows this is not possible without relying on further information, such as rated ballots (rejected by some strict behaviorists).

Non-degenerate systems
As background, it is typically assumed that any non-degenerate (that is, actually useful) voting system satisfies the principles of:


 * Non-dictatorship—the system depends on more than one voter's ballot. It can also be taken as defining social choices (depend on >1 person), as opposed to individual choices.
 * This weakens anonymity (one vote, one value), i.e. every voter should be treated equally.

In proofs, non-imposition is usually replaced with the stronger assumption of Pareto efficiency, i.e. if every voter provides the exact same ranking, the system will output that same ranking. (If voters unanimously support A over B, A should win.) Introducing Pareto efficiency allows skipping the case of an anti-dictatorship (a system that always does the opposite of what the dictator says).
 * Non-imposition—it is possible for any candidate to win, given some combination of votes (the social choice function is onto). 
 * This weakens neutrality (free and fair elections), i.e. every candidate should be treated equally.
 * Universal domain—the social welfare function is a total function over the domain of all possible preferences (not a partial function).
 * In other words, the system cannot simply "give up" and refuse to elect a candidate for some sets of ballots.

Arrow's original statement of the theorem mistakenly said the theorem assumed monotonicity (positive association), i.e. increasing the rank of an outcome should not make them lose. However, this assumption is not needed or used in the proof, except to derive the weaker Pareto efficiency axiom, and so as a result it can be dropped. While originally considered an obvious requirement of any practical system, ranked choice-runoff fails this criterion. Arrow later corrected his statement of the theorem to include runoffs and other perverse voting rules.

The Marquis de Condorcet proved the case for voting methods where in a two-candidate race between $$A$$ and $$B$$, the candidate with more votes will win; his proof of this is called the voting paradox.

Independence of irrelevant alternatives (IIA)
The IIA condition is an important assumption governing rational choice. The axiom says that adding irrelevant—i.e. rejected—options should not affect the outcome of a decision. From a practical point of view, the assumption prevents electoral outcomes from behaving erratically in response to the arrival and departure of candidates.

Arrow defines IIA slightly differently, by stating that the social preference between alternatives $$A$$ and $$B$$ should only depend on the individual preferences between $$A$$ and $$B$$. In other words, we should not be able to go from $$A \succ B$$ to $$B \succ A$$ by changing preferences over some irrelevant alternative, e.g. whether $$A \succ C$$. This is equivalent to the above statement about independence of spoiler candidates, which can be proven by using the standard construction of a placement function.

Intuitive argument
Condorcet's example is already enough to see the impossibility of a fair ranked voting system, given stronger conditions for fairness than what Arrow's theorem assumes. Suppose we have three candidates, A, B, and C, and that there are three voters with preferences as follows:

If C is chosen as the winner, it can be argued that any fair voting system would say B should win instead, since two voters (1 and 2) prefer B to C and only one voter (3) prefers C to B. However, by the same argument A is preferred to B, and C is preferred to A, by a margin of two to one on each occasion. Thus, even though each individual voter has consistent preferences, the preferences of society are contradictory: A is preferred over B which is preferred over C which is preferred over A.

Due to this example, some authors credit Condorcet with having given an intuitive argument that presents the core of Arrow's theorem. However, Arrow's theorem is substantially more general and even applies to many "unfair" systems for making decisions, which give some voters more influence than others or favor some candidates.

Formal statement
Let $A$ be a set of alternatives. A preference on $A$ is a complete and transitive binary relation on $A$ (sometimes called a total preorder), that is, a subset $R$ of $A × A$ satisfying:


 * 1) (Transitivity) If $(a, b)$ is in $R$ and $(b, c)$ is in $R$, then $(a, c)$ is in $R$,
 * 2) (Completeness) At least one of $(a, b)$ or $(b, a)$ must be in $R$.

The element $(a, b)$ being in $R$ is interpreted to mean that alternative $a$ is preferred to alternative $b$. This situation is often denoted $$\mathbf a \succ \mathbf b$$ or $aRb$. Denote the set of all preferences on $A$ by $Π(A)$.

Let $N$ be a positive integer. An ordinal (ranked) social welfare function is a function


 * $$ \mathrm{F} : \Pi(A)^N \to \Pi(A) $$

which aggregates voters' preferences into a single preference on $A$. An $N$-tuple $(R1, …, RN) ∈ Π(A)N$ of voters' preferences is called a preference profile. Arrow's impossibility theorem states that there is no social welfare function satisfying all three of the conditions listed below:


 * Pareto efficiency
 * If alternative $a$ is preferred to $b$ for all orderings $R1, …, RN$, then $a$ is preferred to $b$ by $F(R1, R2, …, RN)$. This axiom is not needed, but simplifies the proof by avoiding a tedious-case-by-case analysis that provides no more intuition.


 * Non-dictatorship
 * There is no individual $i$ whose preferences always prevail. That is, there is no $i ∈ {1, …, N}$ such that for all $(R1, …, RN) ∈ Π(A)N$ and all $a$ and $b$, when $a$ is preferred to $b$ by $Ri$ then $a$ is preferred to $b$ by $F(R1, R2, …, RN)$.


 * Independence of irrelevant alternatives
 * For two preference profiles $(R1, …, RN)$ and $(S1, …, SN)$ such that for all individuals $i$, alternatives $a$ and $b$ have the same order in $Ri$ as in $Si$, alternatives $a$ and $b$ have the same order in $F(R1, …, RN)$ as in $F(S1, …, SN)$.

Formal proof
Arrow's proof used the concept of decisive coalitions.

Definition:


 * A subset of voters is a coalition.
 * A coalition is decisive over an ordered pair $$(x, y)$$ if, when everyone in the coalition ranks $$x \succ_i y$$, society overall will always rank $$x \succ y$$.
 * A coalition is decisive if and only if it is decisive over all ordered pairs.

Our goal is to prove that the decisive coalition contains only one voter, who controls the outcome—in other words, a dictator.

The following proof is a simplification taken from Amartya Sen and Ariel Rubinstein. The simplified proof uses an additional concept:


 * A coalition is weakly decisive over $$(x, y)$$ if and only if when every voter $$i$$ in the coalition ranks $$x \succ_i y$$, and every voter $$j$$ outside the coalition ranks $$y \succ_j x$$, then $$x \succ y$$.

Thenceforth assume that the social choice system satisfies unrestricted domain, Pareto efficiency, and IIA. Also assume that there are at least 3 distinct outcomes. $$ $$

By Pareto, the entire set of voters is decisive. Thus by the group contraction lemma, there is a size-one decisive coalition—a dictator.

Proofs using the concept of the pivotal voter originated from Salvador Barberá in 1980. The proof given here is a simplified version based on two proofs published in Economic Theory.

We will prove that any social choice system respecting unrestricted domain, unanimity, and independence of irrelevant alternatives (IIA) is a dictatorship. The key idea is to identify a pivotal voter whose ballot swings the societal outcome. We then prove that this voter is a partial dictator (in a specific technical sense, described below). Finally we conclude by showing that all of the partial dictators are the same person, hence this voter is a dictator.

For simplicity we have presented all rankings as if there are no ties. A complete proof taking possible ties into account is not essentially different from the one given here, except that one ought to say "not above" instead of "below" or "not below" instead of "above" in some cases. Full details are given in the original articles.

Part one: There is a "pivotal" voter for B over A
Say there are three choices for society, call them A, B, and C. Suppose first that everyone prefers option B the least: everyone prefers A to B, and everyone prefers C to B. By unanimity, society must also prefer both A and C to B. Call this situation profile 0.

On the other hand, if everyone preferred B to everything else, then society would have to prefer B to everything else by unanimity. Now arrange all the voters in some arbitrary but fixed order, and for each i let profile i be the same as profile 0, but move B to the top of the ballots for voters 1 through i. So profile 1 has B at the top of the ballot for voter 1, but not for any of the others. Profile 2 has B at the top for voters 1 and 2, but no others, and so on.

Since B eventually moves to the top of the societal preference as the profile number increases, there must be some profile, number k, for which B first moves above A in the societal rank. We call the voter k whose ballot change causes this to happen the pivotal voter for B over A. Note that the pivotal voter for B over A is not, a priori, the same as the pivotal voter for A over B. In part three of the proof we will show that these do turn out to be the same.

Also note that by IIA the same argument applies if profile 0 is any profile in which A is ranked above B by every voter, and the pivotal voter for B over A will still be voter k. We will use this observation below.

Part two: The pivotal voter for B over A is a dictator for B over C
In this part of the argument we refer to voter k, the pivotal voter for B over A, as the pivotal voter for simplicity. We will show that the pivotal voter dictates society's decision for B over C. That is, we show that no matter how the rest of society votes, if pivotal voter ranks B over C, then that is the societal outcome. Note again that the dictator for B over C is not a priori the same as that for C over B. In part three of the proof we will see that these turn out to be the same too. In the following, we call voters 1 through k − 1, segment one, and voters k + 1 through N, segment two. To begin, suppose that the ballots are as follows:


 * Every voter in segment one ranks B above C and C above A.
 * Pivotal voter ranks A above B and B above C.
 * Every voter in segment two ranks A above B and B above C.

Then by the argument in part one (and the last observation in that part), the societal outcome must rank A above B. This is because, except for a repositioning of C, this profile is the same as profile k − 1 from part one. Furthermore, by unanimity the societal outcome must rank B above C. Therefore, we know the outcome in this case completely.

Now suppose that pivotal voter moves B above A, but keeps C in the same position and imagine that any number (even all!) of the other voters change their ballots to move B below C, without changing the position of A. Then aside from a repositioning of C this is the same as profile k from part one and hence the societal outcome ranks B above A. Furthermore, by IIA the societal outcome must rank A above C, as in the previous case. In particular, the societal outcome ranks B above C, even though Pivotal Voter may have been the only voter to rank B above C. By IIA, this conclusion holds independently of how A is positioned on the ballots, so pivotal voter is a dictator for B over C.

Part three: There exists a dictator
In this part of the argument we refer back to the original ordering of voters, and compare the positions of the different pivotal voters (identified by applying parts one and two to the other pairs of candidates). First, the pivotal voter for B over C must appear earlier (or at the same position) in the line than the dictator for B over C: As we consider the argument of part one applied to B and C, successively moving B to the top of voters' ballots, the pivot point where society ranks B above C must come at or before we reach the dictator for B over C. Likewise, reversing the roles of B and C, the pivotal voter for C over B must be at or later in line than the dictator for B over C. In short, if kX/Y denotes the position of the pivotal voter for X over Y (for any two candidates X and Y), then we have shown


 * kB/C ≤ kB/A ≤ kC/B.

Now repeating the entire argument above with B and C switched, we also have


 * kC/B ≤ kB/C.

Therefore, we have


 * kB/C = kB/A = kC/B

and the same argument for other pairs shows that all the pivotal voters (and hence all the dictators) occur at the same position in the list of voters. This voter is the dictator for the whole election.

Interpretation and practical solutions
Arrow's theorem establishes that no ranked voting rule can always satisfy independence of irrelevant alternatives, but it says nothing about the frequency of spoilers. This led Arrow to remark that "Most systems are not going to work badly all of the time. All I proved is that all can work badly at times."

Attempts at dealing with the effects of Arrow's theorem take one of two approaches: either accepting his rule and searching for the least spoiler-prone methods, or dropping his assumption of ranked voting to focus on studying rated voting rules.

Minimizing IIA failures: Majority-rule methods
The first set of methods economists have studied are the majority-rule methods, which limit spoilers to rare situations where majority rule is self-contradictory, and uniquely minimize the possibility of a spoiler effect among rated methods. Arrow's theorem was preceded by the Marquis de Condorcet's discovery of cyclic social preferences, situations where majority rule is logically inconsistent. Condorcet believed voting rules should satisfy both independence of irrelevant alternatives and the majority rule principle, i.e. if most voters rank Alice ahead of Bob, Alice should defeat Bob in the election.

Unfortunately, as Condorcet proved, this rule can be self-contradictory (intransitive), because there can be a rock-paper-scissors cycle with three or more candidates defeating each other in a circle. Thus, Condorcet proved a weaker form of Arrow's impossibility theorem long before Arrow, under the stronger assumption that a voting system in the two-candidate case will agree with a simple majority vote.

Condorcet methods avoid the spoiler effect in non-cyclic elections, where candidates can be chosen by majority rule. Political scientists have found such cycles to be empirically rare, suggesting they may be of limited practical concern. Spatial voting models also suggest such paradoxes are likely to be infrequent or even non-existent.

Left-right spectrum
Duncan Black showed his own remarkable result, the median voter theorem. The theorem proves that if voters and candidates are arranged on a left-right spectrum, Arrow's conditions are compatible, and all of them will be met by any rule satisfying Condorcet's majority-rule principle.

More formally, Black's theorem assumes preferences are single-peaked: a voter's happiness with a candidate goes up and then down as the candidate moves along some spectrum. For example, in a group of friends choosing a volume setting for music, each friend would likely have their own ideal volume; as the volume gets progressively too loud or too quiet, they would be increasingly dissatisfied.

If the domain is restricted to profiles where every individual has a single-peaked preference with respect to the linear ordering, then social preferences are acyclic. In this situation, Condorcet methods satisfy a wide variety of highly-desirable properties.

Unfortunately, the rule does not generalize from the political spectrum to the political compass, a result called the McKelvey-Schofield Chaos Theorem. However, a well-defined median and Condorcet winner do exist if the distribution of voters on the ideological spectrum is rotationally symmetric. In realistic cases, when voters' opinions follow a roughly-symmetric distribution such as a normal distribution or can be accurately summarized in one or two dimensions, Condorcet cycles tend to be rare.

Generalized stability theorems
The Campbell-Kelly theorem shows that Condorcet methods are the most spoiler-resistant class of ranked voting systems: whenever it is possible for some ranked voting system to avoid a spoiler effect, a Condorcet method will do so. In other words, replacing a ranked-voting method with its Condorcet variant (i.e. elect a Condorcet winner if they exist, and otherwise run the method) will sometimes prevent a spoiler effect, but never cause a new one.

In 1977, Ehud Kalai and Eitan Muller gave a full characterization of domain restrictions admitting a nondictatorial and strategyproof social welfare function. These correspond to preferences for which there is a Condorcet winner.

Holliday and Pacuit devised a voting system that provably minimizes the number of candidates who are capable of spoiling an election, albeit at the cost of occasionally failing vote positivity (though at a much lower rate than seen in instant-runoff voting).

Eliminating IIA failures: Rated voting
As shown above, the proof of Arrow's theorem relies crucially on the assumption of ranked voting, and is not applicable to rated voting systems. As a result, systems like score voting and graduated majority judgment pass independence of irrelevant alternatives. These systems ask voters to rate candidates on a numerical scale (e.g. from 0–10), and then elect the candidate with the highest average (for score voting) or median (graduated majority judgment).

While Arrow's theorem does not apply to graded systems, Gibbard's theorem still does: no voting game can be straightforward (i.e. have a single, clear always-best strategy), so the informal dictum that "no voting system is perfect" still has some mathematical basis.

Meaningfulness of cardinal information
Arrow's framework assumed individual and social preferences are orderings or rankings, i.e. statements about which outcomes are better or worse than others. Taking inspiration from the strict behaviorism popular in psychology, some philosophers and economists rejected the idea of comparing internal human experiences of well-being. Such philosophers claimed it was impossible to compare the strength of preferences across several people who disagreed; Sen gives as an example that it would be impossible to know whether Nero's choice to begin the Great Fire of Rome was right or wrong, because while it killed thousands of Romans, it had the positive effect of allowing Nero to expand his palace.

Arrow originally agreed with these positions and rejected cardinal utility, leading him to focus his theorem on preference rankings. However, he later reversed this opinion, admitting scoring methods can provide useful information that allows them to evade his theorem. Similarly, Amartya Sen first claimed interpersonal comparability is necessary for IIA, but later argued it would only require "rather limited levels of partial comparability" to hold in practice.

Balinski and Laraki dispute the necessity of any genuinely cardinal information for rated voting methods to pass IIA. They argue the availability of a common language with verbal grades is sufficient for IIA by allowing voters to give consistent responses to questions about candidate quality. In other words, they argue most voters will not change their beliefs about whether a candidate is "good", "bad", or "neutral" simply because another candidate joins or drops out of a race.

John Harsanyi noted Arrow's theorem could be considered a weaker version of his own theorem and other utility representation theorems like the VNM theorem, which generally show that rational behavior requires consistent cardinal utilities. Harsanyi and Vickrey each independently derived results showing such interpersonal comparisons of utility could be rigorously defined as individual preferences over the lottery of birth.

Kaiser and Oswald conducted an empirical review of four decades of research including over 700,000 participants who provided self-reported measures of utility, with the goal of identifying whether people "have a sense of an actual underlying scale for their innermost feelings". They found that responses to such questions were consistent with all expectations of a well-specified quantitative measure. Furthermore, they were highly predictive of important decisions (such as international migration and divorce), moreso than even standard socioeconomic variables such as income and demographics. Ultimately, the authors concluded "this feelings-to-actions relationship takes a generic form, is consistently replicable, and is fairly close to linear in structure. Therefore, it seems that human beings can successfully operationalize an integer scale for feelings".

These results have led to the rise of implicit utilitarian voting approaches, which model ranked-choice procedures as approximations of the utilitarian rule (i.e. score voting).

Nonstandard spoilers
Behavioral economists have shown individual irrationality involves violations of IIA (e.g. with decoy effects), suggesting human behavior might cause IIA failures even if the voting method itself does not. However, past research on similar effects has found their effects are typically small, and such psychological spoiler effects can occur regardless of the electoral system in use. Balinski and Laraki discuss techniques of ballot design that could minimize these psychological effects, such as asking voters to give each individual candidate a verbal grade (e.g. "bad", "neutral", "good", "excellent").

Esoteric solutions
In addition to the above practical resolutions, there exist unusual (less-than-practical) situations where Arrow's conditions can be satisfied.

Supermajority rules
Supermajority rules can avoid Arrow's theorem at the cost of being poorly-decisive (i.e. frequently failing to return a result). In this case, a threshold that requires a $$2/3$$ majority for ordering 3 outcomes, $$3/4$$ for 4, etc. does not produce voting paradoxes.

In spatial (n-dimensional ideology) models of voting, this can be relaxed to require only $$1-e^{-1}$$ (roughly 64%) of the vote to prevent cycles, so long as the distribution of voters is well-behaved (quasiconcave). This can be seen as providing some justification for the common requirement of a $$2/3$$ majority for constitutional amendments, which is sufficient to prevent cyclic preferences in most situations.

Uncountable voter sets
Fishburn shows all of Arrow's conditions can be satisfied for uncountable sets of voters given the axiom of choice; however, Kirman and Sondermann showed this requires disenfranchising almost all members of a society (eligible voters form a set of measure 0), leading them to refer to such societies as "invisible dictatorships".

Common misconceptions
Arrow's theorem is not related to strategic voting, which does not appear in his framework, although the theorem does have important implications on strategic voting (being used as a lemma to prove Gibbard's theorem). The Arrovian framework of social welfare assumes all voter preferences are known and the only issue is in aggregating them.

Contrary to a common misconception, Arrow's theorem deals with the limited class of ranked-choice voting systems, rather than voting systems as a whole.