The Trouble With Ranked-Choice Voting

By Corey Cook*
January 6, 2012
Corey Cook responds to Rich DeLeon's rebuttal to his Urbanist article on ranked-choice voting.

In the December issue of The Urbanist, we published an article by University of San Francisco Associate Professor Corey Cook that questioned the impact of ranked-choice voting on San Francisco elections. Recently, San Francisco State Professor Emeritus Rich DeLeon asked us if he could present another point of view. His reply appears in the previous post, and Professor Cook has in turn responded with a follow-up, below.


I very much appreciate reading Professor Rich DeLeon’s response to my article on the recent municipal elections in San Francisco. Thank you to SPUR for encouraging this dialogue – I think the “author meets critic” format is a good one for instigating a reasoned and analytic debate. Unfortunately, in this case, I think Professor DeLeon’s critique rests on a host of flawed assumptions and tortured interpretations of my analysis of the election that border on the disingenuous and eludes the substantive issues. The purpose of my piece was simply to offer a short appraisal of the mayoral election and the use of ranked choice voting in the November contests. Space and time limitations precluded the type of rigorous analysis Professor DeLeon would have preferred (the article was written the weekend after the election and limited to 800 words) and I am pleased to respond here to his critique with both greater specificity and clarity because I think Professor DeLeon disagree to a far lesser extent than he imagines.

At the outset, and in the interests of full disclosure, I should state that I am neither a proponent nor an opponent of ranked choice voting, despite Professor DeLeon’s efforts to label me a “critic”. My own view is that there are no perfect voting systems and that there are tradeoffs inherent in each. I provided testimony in defense of ranked choice voting in the recent lawsuit brought against the city because I thought the data were clearly supportive of the city’s position, and my research on the recent elections in San Francisco, San Leandro, Oakland, and Berkeley suggests both substantial advantages to ranked choice voting and remaining challenges in the implementation of this voting system. I am not a partisan of either the “pro” or “anti” camps (apparently to the dismay of both).

To quote Professor DeLeon from his 2005 white paper, “it is important for both advocates and critics of this electoral reform to study whether IRV actually works in practice the way some have predicted it would based mainly on theory.” Unfortunately, it appears that to those who helped initiate the national movement for ranked choice voting, anyone who offers anything short of full-throated support is branded a critic and dismissed with misdirections and ad hominem arguments. This has the unfortunate consequence of limiting the discussion to the more extreme voices on both sides – those for repeal and those for maintaining RCV.

As I stated at the beginning of the piece, “by most objective measures”, ranked choice voting held up quite well in November. The results in all three citywide races were clear, there were proportionally less invalidated ballots than in previous elections, and a higher percentage of voters fully used the three rankings. This is indeed worthy of praise, though I don’t know about “breaking out the champagne” as Professor DeLeon suggests. The Department of Elections was wise to begin reporting the ranked choice counts the day after the election, and, as Professor DeLeon states in his conclusion, some candidates and political and non-political groups in town did their part to inform voters about ranked choice voting. It seems that Professor DeLeon would have me stop with that statement.

Instead, I express my perspective that the 2011 election, particularly the mayoral contest, was deeply flawed. The flaws, in my view, included low voter interest and participation and a rather tedious summer of interminable debates and candidate forums which largely yielded minimal substance succeeded by an ugly final month. I do not attempt to make assertions about the independent effects of ranked choice voting and repeatedly state (as the subtitle captures) that the relative effects of ranked choice voting in relation to these matters is “difficult to answer.” I think it is impossible to reach any definitive conclusions based on a single election. But I think that it is important to study whether ranked choice voting is indeed “a necessary reform for repairing our broken democracy” as Professor DeLeon, Chris Jerdonek, and Steven Hill write in their 2006 editorial.

In his letter to the Berkeley City Council urging adoption of Instant Runoff Voting, Professor DeLeon echoes the arguments proffered in favor of adoption in San Francisco: “IRV will insure that elected representatives have majority voter support. IRV will reward positive, issue-based campaigns, discourage negative campaigning, and promote coalition-building. IRV will require only one election rather than two, thus maximizing turnout (runoff elections typically draw fewer voters) and minimizing costs.” He then goes on to offer four additional arguments, one of which is that “ IRV will help to expand voter choice, activate voter interest, encourage greater (but kinder & gentler) political competition, and restore legitimacy to a political system…” I am unsure how else to “study whether IRV actually works in practice the way some have predicted it” without looking at the overall election.

I respond here to each of Professor DeLeon’s five criticisms of my article.

1. Ranked choice voting and popular opinion.

First, I ask whether ranked choice voting accurately captures individual voters’ preferences. This should be our primary concern about any voting system (a voting system is simply a method of aggregating individual preferences). As Professor DeLeon states, there are many possible ways of analyzing this critical question, including “an assessment of how accurately the observed voting patterns reflected the city’s diverse and complex demography.” I agree with him that this would be a worthwhile study, but not one that I could execute in 800 words or in the four days after an election as Professor DeLeon states he “might reasonably expect.” Thank goodness he wasn’t on my tenure committee.

Instead, I chose to analyze the nearly 200,000 ballots in the race to look at two types of voting behaviors: overvotes (ballots in which voters make errors that might invalidate their vote) and the numbers of rankings used by voters. The reason I look at these two measures is fairly straightforward. If voters’ ballots are invalidated at relatively high rates or disproportionately across groups, and/or if voters are not fully expressing their sincere preferences by exploiting the fullest potential of the ballot, this might suggest that the current system of implementing ranked choice voting is not accurately tallying individual voters’ sincere preferences. This methodology is a direct replication of a 2008 study by Neely and Cook that is, to my understanding, the only peer reviewed academic publication on ranked choice voting in San Francisco.

The data from the 2011 election reveal that 1.2% of voters cast an overvote in one of the three citywide contests. This rate is lower than in the past, and yet still higher than in typical “vote for one” candidate contests. Professor DeLeon wonders “how much lower… can it possibly be?” According to a study by Kimball and Kropf (2005), the mean overvote rate in gubernatorial races in their study was 0.17%. These scholars find that “overvotes are almost entirely a function of ballot features and voting technology” and are not related to demography. Overvotes are higher in counties using “connect the arrow” systems used in San Francisco, and yet they are “substantially less common in counties using the error correction feature” used in San Francisco (Kimball and Kropf, 2005:526). So we don’t have an apples-to-apples comparison and reasonable people can disagree about whether 1.2% is substantially higher than 0.17%. For comparison sake, rates of overvotes in 2010 were 1.2% in San Francisco, 0.9% in Oakland, and 0.4% in San Leandro. This strikes me as significant and worthy of discussion about how to mitigate voter error. But unlike overvotes in the races studied by Kimball and Kropf (2005), we do find that errors have been correlated with demographic characteristics of the population (see Neely and Cook, 2008). So it seems reasonable to discern whether there are things San Francisco can do to reduce their occurrence and ensure that some voters are not systematically less likely to cast valid ballots and perhaps learn from across the bay.

My concern with the number of rankings expressed by voters is similarly straightforward and is directly a test of the proposition that ranked choice voting “expand(s) voter choice” in practice as well as in theory. The number of fully-ranked ballots in this election was 73% in the Mayoral, and 52% in the DA race and 43% in the Sheriff race. If this accurately reflects voters’ preferences, the system is working well. If not, then less so. What do the data tell us?

The proportion of voters only voting for only one candidate in the mayoral race was 16%, compared with 27% in the DA’s race and 38% in the Sheriff’s race. 9% of voters ranked only one candidate in all three contests. Now, there are many explanations for this – that voters did not have enough information about other candidates, that voters found no other candidates acceptable, that voters were unaware about the option to rank three candidates, that voters were persuaded by a “vote for one” endorsement (like that of the city’s largest newspaper) or that they were confused by the various political actors spouting inaccuracies about ranked choice voting, among other explanations. I presume, based on previous studies, that it’s a bit of each, but do not hazard a guess beyond that. I would note that, as with overvotes, the incidence of ranking fewer than three choices is not randomly distributed. Data show that voters in the southeastern neighborhoods were more likely to rank only one candidate.

But as Professor DeLeon rightly notes, more research on this is needed and I make no conclusions about why this is the case, only to state that it is both important and “difficult to answer”. For what it’s worth, in the 2005 exit poll conducted by Neely, Blash, and Cook that Professor DeLeon cites, 31% of voters who ranked less than three state that they didn’t know enough about the other candidates, 21% say that no others were acceptable, and 9% say they didn’t know they could rank three.

Just a note about terminology. Professor DeLeon twice calls “misleading” my use of the term “bullet-voting” to characterize voters who vote only for one candidate “because political scientists and campaign managers typically use the term to mean a form of tactical voting in which a voter is encouraged to vote only for his or her preferred candidate.” This is not consistent with my reading of the political science literature which does not distinguish, as I do not, between tactical and sincere votes for a single candidate. Richard Niemi (in his seminal paper) calls bullet voting “voting for only one’s most preferred candidate” (Niemi, 1984). Similarly, Bullock and MacManus define bullet voting as “Voting for only one candidate out of a list of candidates” (Bullock and MacManus 1993). In more recent years, scholars have used bullet voting and “single shot voting” interchangeably (see Bowler and Yoshinaka, 2007; Bowler and Farrell, 2001; Zimmerman, 1994). And to appease any students who might be reading this, good old Wikipedia also does not distinguish between sincere and tactical votes for a single candidate in its description of bullet voting.

2. Ranked choice voting and turnout

Professor DeLeon and I agree that turnout in 2011 was lower than in previous mayoral contests, save 2007. He takes issue with my use of the term “abysmal”, which is certainly fair. He is right that turnout in 2011 was only 3.2% lower than in 2003 and 2.5% lower than in 1999 and higher than in Gavin Newsom’s largely uncontested re-election in 2007. I might mention, however, that observers in 1999 and 2003 complained about low turnout and that in both December runoffs, turnout increased. In 2003, turnout went from 45.7 in November to 54.5% in December. In 1999, turnout went from 45.0% to 48.8%. I suppose I used the term “abysmal” because I was particularly impressed by this group of candidates – the acting mayor, three citywide electeds, and six current and former members of the Board of Supervisors, including its president. And of course the top seven finishers would all have been “firsts”: first elected Chinese-American, Japanese-American, Latino, or openly gay mayor. If that’s not enough, the ballot propositions included two competing pension reform measures and a statement on school assignment policy. I might have expected more voter interest. I thought it was abysmal.

Professor DeLeon seemingly takes issue with why I compare 2011 to past mayoral contests, and yet “dismiss as anomalous” the 2007 race which he deems “truly abysmal by San Francisco standards”. Here’s why I exclude 2007. In 2007, there were three candidate races and 11 propositions. The closest of the three candidate races was won by 47.5%. Gavin Newsom’s nearest competitor scored 6.3% of the vote. District Attorney Kamala Harris ran completely unopposed. Sheriff Michael Hennessy won with just under ¾ of the votes cast. As far as I can tell, there was no significant money spent by any of these challengers nor had any held prior elective office. The ballot propositions were even less interesting than these essentially uncontested races. The most controversial was Proposition E, which would have required the Mayor to participate in question time before the Board of Supervisors. In fact, voters who came to the polls were less likely to stick around and participate on the ballot propositions in that year (93.4% of voters who cast ballots, on average, voted on the propositions in 2007 compared with 93.8% who did so in 2011 suggesting that ballot measures were even less of a draw). So I don’t regard 2007 as “abysmal” turnout relative to what was on the ballot. My own judgment, for what it is worth, is that 2011 was a much more interesting election and that 42% turnout is far more problematic given the nature of the races.

In an August editorial, Professor DeLeon writes, “This election, as I see it, is about choosing how San Francisco will be governed as a progressive city through economic hard times. It is about making the transition from a strident politics of ideology to a more traditional politics of interest and identity. And it is about the capacity of local government to take care of business and the capacity of business to take care of San Francisco.” Given these high stakes, it would appear that 42% turnout would be considered “abysmal”. But, I suppose, reasonable people can disagree about this point.

More significantly, I did not mean to imply that ranked choice voting is “the culprit behind” the low turnout. As I explicitly state in the piece, turnout was low because the costs of voting were high (owing to the large field of most liberal Democrats), and the race was perceived as not particularly close. I would add to that the general sense of voters that things in San Francisco were headed in the right direction. Professor DeLeon dismisses my use of a rational voter framework to explain voter turnout due to the existence of “habitual voters.” I don’t doubt that Professor DeLeon knows this voting behavior literature far better than I so I defer to his judgment on this, but in my reading, habitual voters can be understood using reasonable assumptions of voter rationality (see Geys, 2006 for a review of the literature).

In any case, in their strategic planning, the campaigns had estimated that turnout would approach 50 percent. It was far lower and I think it is an interesting empirical question for future research. But the overall effects of RCV on turnout are not empirically discernible from my perspective – positive, negative, or otherwise. Actually, a piece that Professor Francis Neely and I published several years ago shows that the rate of “undervotes” in Board of Supervisorial elections is lower using ranked choice voting. That is to say that voters participating in the election were less likely to leave those contests blank than in similar elections. Again, it’s unclear whether this brought people to the polls or merely encouraged them to continue to the downballot races once there, but is interesting nonetheless. The impression I intended to give in regards to turnout was not that ranked choice voting is the culprit, but rather that the race was a largely undifferentiated contest involving huge amounts of money and incredible amounts of information, and yet voters seemed not to respond to the historic election in the way I might have expected.

3. Ranked choice voting and information costs

I do think it is the case that ranked choice voting imposes greater informational costs on voters. Ranking three choices takes a lot of information. It seems that Professor DeLeon would agree. As he wrote in the August editorial, “under the city's ranked-choice voting system, the voters will need to do more political homework much earlier than in the past, because this election will be a one-day sale without the option of a later runoff election simplifying choice, for good or ill, by whittling the 16 down to two.” As noted above, the single greatest explanation for why voters did not rank all three candidates in 2005 was that the voter “did not know enough” about the other candidates. And that study shows a strong, statistically-significant relationship between the number of rankings voters express and their perceptions of the ease of the ranking task.

In this section Professor DeLeon raises the issue of whether the limit on articulating three rankings is too stringent and links to the judicial decision on the matter. As noted above, I provided expert testimony on precisely this issue, so the statement that “Professor Cook doesn’t consider this side of the debate in his assessment” is somewhat amusing as I spent several months and dozens of hours working pro bono for the city to empirically defend precisely that side of the debate! In that case, I examined past election results in San Francisco to investigate the plaintiff’s assertions that voters were “disenfranchised” by the limit on three choices. I did not agree then, and do not agree now, that voters were “disenfranchised” and that the consequence of adding rankings would include a more complex and confusing ballot. I think the judge’s decision in that case was exactly right. However, the empirical argument I made rested on the evidence that relatively few voters ranked three choices and of those, relatively few had their ballots discontinued. In the 2011 mayoral race, 72.7% of voters ranked three choices and 22.5 percent of those ballots were exhausted (meaning that they did not include Ed Lee or John Avalos (the two final candidates). These data do not change my opinion about the “disenfranchisement” argument. But these data do suggest that perhaps in this particular contest some voters might have taken advantage of the opportunity to rank more candidates had that been an option and thus, RCV might have more precisely reflected voter preferences.

4. Ranked choice voting and negative campaigning

I am disappointed about Professor DeLeon’s characterization of my writing in this section because I think that he and I actually do not disagree at all on this point. I argue that the 2011 mayoral election was “generally uneventful for the better part of a year (and) became exceptionally nasty in the final month.” I do not state, as he would have me, that “negative campaigning reached new heights of vituperative meanness under RCV. (italics mine). Rather, I state that this campaign got uncommonly nasty down the home stretch in comparison to earlier in the campaign. I did not mean to suggest a comparison to all other races but to the earlier period.

My point was to simply state, as Professor DeLeon does in his analysis of the exit poll results in 2005, that there is no evidence that, at least as it relates to the top tier candidates, RCV reduces negative campaigning. The 2002 ballot argument states that “Previous runoff elections have seen excessive negative campaigning and ‘hit’ pieces. Such mudslinging is common when the field is reduced to two candidates, and candidates can win by attacking their lone opponent rather than attracting voters.” The point I make about ranked choice voting is simply that despite the promises of its proponents, RCV only appears to discourage negativity against those lower in the rankings and/or before the candidate ordering becomes more clear.

If Professor DeLeon wants to count this as “at least half a point” in favor of RCV, for whatever reason, great. I was surprised to find myself characterized as “regretful” about the negativity of the campaign. The political science literature on negative campaigns is mixed – some scholars argue that voters get better information through attack ads, others that negativity turns off voters and dampens turnout. I leave that to folks far smarter than me to sort out – I’m agnostic on how to “score” this.

5. Ranked choice voting and “majority rule”

In the final section of my piece, I write about the complicated political mandates that might emerge from close ranked choice elections. I make four statements:

  • “In 15 of the 18 ranked-choice contests held so far in San Francisco, the winning candidate did not receive a majority of the votes cast. Mayor Ed Lee only appeared on 43.9 percent of ballots. Sheriff-elect Ross Mirkarimi appeared on 46.9 percent. Their “majorities” were secured in relation to their nearest competitors and rested upon on tens of thousands of ballots that were eliminated early in the counting rounds because they did not include second or third choices. These elections did not simulate a majority runoff.”
  • “All of the winners on election night received the legal mandate to govern.”
  • “It is likely, given the margin of victory, that the vast majority of voters will see these outcomes as legitimate (unlike what appears to have happened in Oakland, where a mayor who did not win a majority now faces a lack of support).”
  • And, “particularly for a mayor, there is an advantage to securing a majority electoral coalition when it comes time to govern… The jury remains out on whether ranked-choice voting facilitates this.”

I had to reread my own piece after reading Professor DeLeon’s astonishingly intemperate and wildly inaccurate interpretations of what I actually wrote. There was insufficient space to fully develop each of these points in print, so I appreciate the opportunity to explain them further and clear up any confusion, but I am surprised by Professor DeLeon’s caricature of those four statements and concluding ad hominem argument.

He first claims that I do not “accept the charter language defining a ‘majority’ winner under RCV as the candidate receiving a majority of continuing votes.” Presumably my first statement that “all of the winners on election night received the legal mandate to govern” suggests my acceptance of the charter language. Every one of the 56 winners of ranked choice elections in the bay area legitimately won their races. Every one.

Then, Professor DeLeon repeats the meme that “For Cook, a ‘majority’ (in so many words) is at least 50 percent plus one of total votes cast in a given election, and that is that, no matter how many more votes might be cast for the winner in the typically high-turnout November RCV elections than in the typically low-turnout December runoff elections.”

Actually, I am just referring again to the proponents’ ballot argument, which reads that it “fulfill(s) the goal of electing majority winners without the inconvenience of a second election. The ‘instant’ runoff works much like December’s ‘delayed’ runoff. Voters indicate their favorite candidate, just like now…By doing it in one election, we produce winners who have a majority of the vote and save millions of tax dollars” (italics mine). I was making the quite simple mathematical point that in the majority of cases, the winners do not “have a majority of vote”, but rather have the majority of continuing ballots. I make no predictions about whether Ed Lee, George Gascon, or Ross Mirkarimi would have received more or less votes in December had there been a runoff in place, as I don’t presume to know whether this year would be like the seven times that turnout declined between November and December or the three that it increased during the past 35 years. But I do know that Ed Lee received the least votes of any elected mayor at least since 1975. He was a top three choice of less than 100,000 voters. No other mayor in 35 years, elected by RCV or two stage runoff, won with less than 100,000 votes.

Next, he accuses me of “(denying) governing legitimacy to majority winners under current RCV”, and taking a “gratuitous swipe at Oakland’s new mayor, Jean Quan, challenging the legitimacy of her election.”

Despite the intentionally explosive language, I most certainly do not deny the legitimacy of those elected under RCV or challenge the legitimacy of Jean Quan’s election. I was in fact arguing the opposite. Jean Quan was legitimately elected mayor of Oakland. She won the election because she was preferred by those who voted in the election. My analysis of ballot image data show that she was the Condorcet winner in the race: she was preferred one-on-one to every candidate in the race. I have repeated this time and again over the past year in many different contexts and media. In a blog post for SPUR before the election, I wrote “Though (Don Perata) led the first place tally by 9 percentage points, he appeared on 8 percent fewer ballots than Jean Quan. Head to head, she was preferred to him. It wasn’t a fluke, she wasn’t lucky. She was preferred by voters.”

Rather, the point that I made was that it is my sense that a not insignificant number of Oakland residents do not view her election as having been legitimate. The misconception that her victory was somehow tainted or the result of superior gamesmanship of the voting system, is, in my opinion startlingly common. To be fair, my analysis rests on purely qualitative rather than quantitative data. But even a cursory review of newspaper articles, local political blogs, the statements of those collecting signatures for the recall effort, or even simple conversations with my neighbors in Oakland convinces me that a segment of the population does not agree with Professor DeLeon and me about the legitimacy of her election. Heck, just a couple of weeks ago, an article in the Laney College paper says “Many people feel that Quan was not elected fairly.” Partly, I think that was the result of the long delay between the announcement of first place votes on Tuesday night and the ranked choice tally on Friday. In fact, Mayor Quan made this exact point in a panel we were on together a few weeks ago.

Unfortunately, Professor DeLeon does not address the central argument I am making in this section, that “the jury remains out” whether ranked choice contests are more or less effective in conferring a working governing mandate than other voting systems. I would argue that Mayor Quan would have been better off politically had she scored a victory in a head-to-head matchup with Perata. But that’s not unique to RCV. When Oakland Mayor Ron Dellums won the June 2006 primary election with 50.2% of the vote (thus ending the race and precluding a typical November runoff between the top vote getters) a narrative emerged that he had “narrowly won”. In fact, he was 18 points ahead of his nearest competitor. He would have been better off winning 60-40 in the runoff and building more political capital, in my estimation. As I wrote, particularly for a mayor, there is an advantage to securing a majority electoral coalition when it comes time to govern, and the bigger the better. Mayors will undoubtedly engage the electorate again – whether in contesting charter amendments, bond measures, and statutory propositions, endorsing sympathetic candidates, or in her or his re-election. I suspect that at least in some cases, victorious candidates would have been better served had they achieve a majority of support of those coming to the polls rather than a majority of continuing ballots. I regard this as an open empirical question and have discussed precisely this issue with several colleagues over the years with no simple resolution, thus my safe conclusion that “the jury remains out.”

Disappointingly, Professor DeLeon concludes his critique with his judgment that my “assessment of RCV would be more credible as a fair and objective analysis if (I) had actually acknowledged and engaged those who offer other perspectives on these issue.” This is both a specious and wildly inaccurate claim. Professor DeLeon is certainly correct that as my article was a short opinion piece, I did not explicitly include the perspectives of activists on either side of the electoral reform debate and find them very capable of articulating their own views. Rather, my intent was to offer my nuanced perspective that raised questions rather than answered them. I look forward to a robust and objective discussion of these important issues in keeping with Professor DeLeon’s previous call for such analysis.

Read Corey Cook’s original article in The Urbanist >>

Read Rich DeLeon’s rebuttal >>

* Corey Cook is an associate professor at the University of San Francisco, where he teaches American politics and conducts research on election results and political geography in California.