Dan Payne’s analysis of the Presidential race in today’s Boston Globe illustrates why he was a bad fit for the Deval Patrick campaign, which he left soon before Deval blew the lid off the 2006 Democratic gubernatorial primaries. Payne repeatedly cites, while also chastising himself for citing, poll numbers without any serious analysis of the correlation between pre-election polling and final results.

To analogize between a political campaign and a military one, tactics like endorsements and advertising are like long-range bombing: all they do is “soften up” the populace and provide the potential for votes. But warplanes and artillery do not capture territory: for that you need “boots on the ground,” which in the electoral context means real people making phone calls and canvassing door-to-door to convert general support into real votes.

Political analysts like Dan Payne are biased towards covering the “air war” because it’s sexy and easy to see. But a more accurate way to interpret poll data is to weight them by the presence of ground troops. Sudden shifts in popular numbers are unlikely to show up in real votes without a large and well-organized volunteer base to realize those gains. (The analysis is somewhat different when the numbers are static, in which case the leader will win, all else being equal). As I’d written earlier, Obama pulled the organizational structure out of Massachusetts, and Deval Patrick’s supporters just couldn’t cover the ground fast enough to capitalize on the sudden shift in public sentiment.

White homeowners afraid of a black family moving into their neighborhoods often encourage the would-be seller to pull the home off the market. “I’m not racist,” they explain, “but other people are. And all of our home prices will suffer.”

These homeowners are perpetuating bigotry, and so are voters who won’t cast their ballots for a woman or African-American because of “electability.”

Stop wondering whether America is “ready” for a non-white or non-male person to be President. Wonder if YOU are ready. Then cast your vote for the best candidate.

Physicists understand that observation can change the thing being studied. Perhaps observer effect partially explains the pollsters’ poor predictions (“Stunned by N.H., pollsters regroup to seek answers,” Jan 10). Maybe the very act of publishing the polls changed the vote. If voters were truly undecided to the last minute, as many appeared to be, they’d want to prolong the race by voting for the underdog — which, according to the polls, was Obama in Iowa and Clinton in New Hampshire.

On the other hand, cognitive scientists also believe that humans have a natural bias to see patterns where none exist — which, in the case of pollsters, pundits, and journalists, leads to the need to find a story to “explain” statistics, whether polls or vote results. We’re also biased towards dramatic, human stories: it’s a lot more interesting to attribute Clinton’s victory to the “choked up” episode than a complex mix of more prosaic factors like operational effectiveness.

But then, with the exception of Mitt Romney, well-oiled machines are rarely photogenic.

Published in the Boston Globe on 12 Jan 2008.