Debunking the Bradley Effect
By Nate Silver NEWSWEEK
With Barack Obama having moved into a statistically significant lead in most public polling, and the Democrats' strongest issue—the economy—likely to remain tops on the electorate's mind between now and Nov. 4, commentators are asking how John McCain can win the election. The best answer that many of them have come up with? The "Bradley effect."
The Bradley effect is named after Tom Bradley, the former Los Angeles mayor who, in 1982, narrowly lost a bid to become California's governor after having led substantially in the polls. The same pattern reflected itself in other instances involving African-American candidates: Douglas Wilder underperformed his polling in 1989 (but still narrowly won the Virginia governor's race), as did David Dinkins in the New York mayoral race that same year. The theory goes that, in these races, white voters wanted to appear politically correct by telling pollsters they were going to vote for a black candidate when, in fact, they were not prepared to do so.
Will Obama be the next victim? I say no. Examples like Bradley and Wilder are nearly a quarter of a century old, and there's no proof that the Bradley effect still exists.
The Bradley effect is named after Tom Bradley, the former Los Angeles mayor who, in 1982, narrowly lost a bid to become California's governor after having led substantially in the polls. The same pattern reflected itself in other instances involving African-American candidates: Douglas Wilder underperformed his polling in 1989 (but still narrowly won the Virginia governor's race), as did David Dinkins in the New York mayoral race that same year. The theory goes that, in these races, white voters wanted to appear politically correct by telling pollsters they were going to vote for a black candidate when, in fact, they were not prepared to do so.
Will Obama be the next victim? I say no. Examples like Bradley and Wilder are nearly a quarter of a century old, and there's no proof that the Bradley effect still exists.
Take the recent study by Harvard fellow Daniel Hopkins, who examined the performances of African-American candidates in major electoral races from the 1980s through the present day. Hopkins found that the Bradley effect did exist during the '80s and early '90s. But it dissipated sometime thereafter; recent black candidates like Deval Patrick and Harold Ford Jr. have performed almost exactly as their polls predicted ahead of time. Hopkins theorized that this was because many hot-button racial issues, like crime and welfare, had been taken off the table by the centrist reforms of the Clinton administration.
Then there are this year's primaries. Everyone remembers New Hampshire, when nearly all polls predicted a big win for Obama, but Hillary Clinton emerged victorious. That was a bad day for the pollsters—and for Obama, who underperformed the Pollster.com composite average by 9 points. (Still, it is not clear that there was evidence of the Bradley effect at work here. Contributing factors to Obama's loss may have included his "nice enough" comment, Senator Clinton's teary moment in the diner—and a simultaneous GOP primary, which allowed McCain to pick off some Obama voters who thought their guy was safely ahead.) What fewer remember is what happened two weeks later in South Carolina. In that case, the Pollster projection had Obama winning by 15 points—but he won by 29. That 14-point error was actually of greater magnitude than the mistake in New Hampshire, if less noticeable because the polls hadn't picked the wrong horse.
South Carolina was not the only state in which Obama overperformed his polls. They significantly underestimated Obama's margin in essentially every Southern state, including Virginia, Georgia and North Carolina, as well as a couple of states outside the South, like Wisconsin, Indiana and Oregon. On balance, the polling during the primaries underestimated Obama's support by 3.3 points when compared to the Pollster averages in those states. And yet, a belief in the Bradley effect persists. Why? People are confusing voters exhibiting racist behavior with voters lying about their intentions to pollsters.
Then there are this year's primaries. Everyone remembers New Hampshire, when nearly all polls predicted a big win for Obama, but Hillary Clinton emerged victorious. That was a bad day for the pollsters—and for Obama, who underperformed the Pollster.com composite average by 9 points. (Still, it is not clear that there was evidence of the Bradley effect at work here. Contributing factors to Obama's loss may have included his "nice enough" comment, Senator Clinton's teary moment in the diner—and a simultaneous GOP primary, which allowed McCain to pick off some Obama voters who thought their guy was safely ahead.) What fewer remember is what happened two weeks later in South Carolina. In that case, the Pollster projection had Obama winning by 15 points—but he won by 29. That 14-point error was actually of greater magnitude than the mistake in New Hampshire, if less noticeable because the polls hadn't picked the wrong horse.
South Carolina was not the only state in which Obama overperformed his polls. They significantly underestimated Obama's margin in essentially every Southern state, including Virginia, Georgia and North Carolina, as well as a couple of states outside the South, like Wisconsin, Indiana and Oregon. On balance, the polling during the primaries underestimated Obama's support by 3.3 points when compared to the Pollster averages in those states. And yet, a belief in the Bradley effect persists. Why? People are confusing voters exhibiting racist behavior with voters lying about their intentions to pollsters.
There is little doubt Obama is losing some votes due to his race; a recent Associated Press survey suggested that as many as 6 percent of the electorate may be voting against Obama because he is black. But that's not what the Bradley effect is about. As long as those prejudiced voters are telling pollsters that they're going to vote for McCain, their sentiments will be reflected accurately in the polling. The Bradley effect emerges when voters tell pollsters one thing and then do another at the ballot booth.
So the question is why, if a voter does not intend to vote for Obama, would he or she feel compelled to lie about it? There are perfectly legitimate reasons not to vote for Obama; a voter who wanted to vote against him because of his race would have little trouble rationalizing his vote. If a voter felt compelled to lie to a pollster, he might tell them that he was voting against Obama because of his inexperience or his liberal politics—when, in fact, he was voting against him because of his race. But the pollster would still tally the vote correctly in the McCain column. By contrast, in cases where the Bradley effect existed, including Bradley's race itself, the black candidate was as much or more experienced than the white opponent. So voters found it harder to excuse their racism and may have misstated their voting intention to pollsters as a result.
This is not to imply that there aren't any relationships between race and polling errors. In a study I conducted of undecided voters at ++fivethirtyeight.com++,<> I found that a majority (between 60-65 percent) of white undecideds broke for Clinton during the Democratic primaries. And essentially all black undecideds broke for Obama. (Clinton performed worse among blacks than could be explained through the behavior of undecided voters alone, suggesting the possible presence of a reverse Bradley effect.)
So the question is why, if a voter does not intend to vote for Obama, would he or she feel compelled to lie about it? There are perfectly legitimate reasons not to vote for Obama; a voter who wanted to vote against him because of his race would have little trouble rationalizing his vote. If a voter felt compelled to lie to a pollster, he might tell them that he was voting against Obama because of his inexperience or his liberal politics—when, in fact, he was voting against him because of his race. But the pollster would still tally the vote correctly in the McCain column. By contrast, in cases where the Bradley effect existed, including Bradley's race itself, the black candidate was as much or more experienced than the white opponent. So voters found it harder to excuse their racism and may have misstated their voting intention to pollsters as a result.
This is not to imply that there aren't any relationships between race and polling errors. In a study I conducted of undecided voters at ++fivethirtyeight.com++,<> I found that a majority (between 60-65 percent) of white undecideds broke for Clinton during the Democratic primaries. And essentially all black undecideds broke for Obama. (Clinton performed worse among blacks than could be explained through the behavior of undecided voters alone, suggesting the possible presence of a reverse Bradley effect.)
In this respect, however, the undecided voters weren't behaving any differently from decided voters, since Clinton had won a solid majority of whites during the primaries and Obama dominated among African-Americans. So we might be able to tell something by looking at the composition of undecided voters in a poll; if the undecideds are mostly older, non-college-educated whites, the smart bet is that the majority of them will break for McCain. If they're younger college graduates, most of them will probably go to Obama.
Even if McCain were to win a great majority of undecided voters—there is evidence that most of the undecideds come from strong McCain demographics—there are other polling artifacts that might serve to counteract this. For example most pollsters do not call cell-phone-only voters—a young, urban, Obama-friendly demographic. And most pollsters err on the conservative side with their turnout models, assuming that the composition of the electorate looks like 2004—when, in fact, there were massive surges in participation in the primaries among groups like young voters and Latinos, who will go mostly for Obama.
With so many "X factors" like race, cell phones and turnout, there is probably an extra margin of error this year. And polls aren't terrifically accurate to begin with. But there is no reason to conclude that the polls are systematically overestimating Obama's support; the reverse is at least as likely to be true. McCain, in all likelihood, will need to win this election fair and square—which means that he has his work cut out for him.
Even if McCain were to win a great majority of undecided voters—there is evidence that most of the undecideds come from strong McCain demographics—there are other polling artifacts that might serve to counteract this. For example most pollsters do not call cell-phone-only voters—a young, urban, Obama-friendly demographic. And most pollsters err on the conservative side with their turnout models, assuming that the composition of the electorate looks like 2004—when, in fact, there were massive surges in participation in the primaries among groups like young voters and Latinos, who will go mostly for Obama.
With so many "X factors" like race, cell phones and turnout, there is probably an extra margin of error this year. And polls aren't terrifically accurate to begin with. But there is no reason to conclude that the polls are systematically overestimating Obama's support; the reverse is at least as likely to be true. McCain, in all likelihood, will need to win this election fair and square—which means that he has his work cut out for him.
No comments:
Post a Comment