Thursday, December 14, 2017

The Meaning of the Alabama Election

As Doug Jones himself pointed out in his victory speech Tuesday night, his win over Roy Moore was, as much as anything, a great victory for Alabama.  For the first time in roughly half a century, a progressive Democrat had won a statewide election there, following in the tradition of Hugo Black, John Sparkman, Lister Hill, and 1950s Governor Jim Folsom. (Although Sparkman and Hill were segregationists, they were New Deal liberals on everything else and helped pass a lot of important legislation.)  Meanwhile, the victory signals today’s Democratic Party that all is not lost, and that the elections next year might turn out much better than expected.  Exactly why Jones won, however, is being misunderstood, because so many people do not understand the numerical dynamics of American politics and off-year elections.
Black turnout is frequently being credited for Jones’s victory, and it played a part.  I have seen references to a “surge” in black voter turnout.  That, however, is somewhat misleading.  Considerably fewer black (and white) people voted for Jones, as a matter of fact, than voted for Hillary Clinton a year ago.  And even the relatively good turnout of black voters on Tuesday was not the main reason that Jones won.  As I hope to make clear, the figures leave no doubt about this at all.
We easily forget that the distribution of political power in our country is heavily impacted by people’s failure to vote.   Last year only 55.4% of the voting-age population cast ballots in what has turned out to be one of the more consequential elections in American history.  Yet as we shall see in a moment, even that low figure is much higher than turnout in off-year elections such as 2014 or (in three states) 2017.  And we need to understand that to understand what happened in Alabama.
To illustrate this point, I want to compare presidential and off-year voting in two hotly contested purple states, North Carolina and Virginia.  Alabama, of course, is not usually a purple state, but it momentarily became one last Tuesday, when the electorate at the polls divided almost evenly. 
The North Carolina votes for President in 2012 [sic] and for Senate in 2014 provide a very simple control, because although Barack Obama defeated Mitt Romney in the former year and Republican Tom Tillis beat incumbent Kay Hagan in 2014, both elections were extremely close, indicating that differences in turnout affected both sides about equally.  North Carolinians, it turns out, cast 4,448,786 votes for President in 2012, and just 2,915,281 for the Senate in 2014.  That represents a 35% drop in total turnout—both sides, in short, lost about 1/3 of their votes.  

 Turning to a more recent (and relevant) comparison, we find that purple Virginia cast 3,984,631 votes for President in 2016, with Hillary Clinton winning by a comfortable five percentage points, and just 2,614,282 in 2017—an almost exactly equal decline of 34%.  When we look at the votes for the Democratic and Republican gubernatorial candidates, however, we find an unequal decline.  Republican Ed Gillespie’s vote was 66% of Donald Trump’s, while Democrat Ralph Northam’s was a full 71% of Hillary Clinton’s. (The reason both parties showed declines smaller than the overall 34% decline in turnout is that minor parties won several percentage points of the 2016 vote, but only 1% of the 2017 vote.)  Northam held on to a higher percentage of Clinton’s vote than Gillespie did of Trump’s—and this allowed him to win by a very comfortable margin of 54%-45%.

            What happened in Alabama Tuesday followed the same pattern, but on a mind-boggling scale.  Here are the results of the Alabama presidential elections of 2012 and 2016, and the Senate election this week.

2012 Presidential election
Romney 1,255,925 61%
Obama   795,696 38%

              2016 Presidential election
Trump   1,318,255 62.1%
Clinton: 729,547 34.4%

2016 Senate election
Shelby (R) 1,335,104
Crumpton (D) 748,709

              2017 Senate election
Moore 652,300
Jones: 673,236

         Comparing the two presidential results, it is not surprising that Barack Obama was a significantly more popular candidate in Alabama than Hillary Clinton, or, sadly, that Donald Trump did somewhat better than Mitt Romney.  But moving to the Senate, what happened yesterday suddenly becomes very clear indeed.  Neither black turnout nor Democratic turnout overall, it seems to me, could reasonably be said to have “surged” compared to 2017, since Jones didn’t get as many votes as Crumpton. But he totaled 90% of Crumpton’s vote, which is, indeed, an extraordinary figure compared to a more expected 65% off-year total of the previous presidential election.  Because of the peculiar circumstances of this race—the candidacy of Roy Moore and the related incumbency of Donald Trump—it looks at first glance as if the Democratic Party turned out an extra 15% of its votes, compared to what normally would have been expected.
That in itself, however, would not have won the election for Jones. Equally striking is the collapse of Republican vote, which was almost only 49% of what it was last year.  While the Democrats turned out 15% more votes than expected, the Republicans turned out 16% less.  And since the Republican vote last year was much, much larger than the Democratic one, the Republican decline was almost twice as responsible for Jones’s victory than the Democrats’ success in turning out more votes than expected.  Had it not been for the evident antipathy of a substantial number of Republicans towards Roy Moore, he still would have won.

But that is not all. This analysis assumes that no significant number of voters cast ballots for Trump in 2016 and Jones in 2017.  Evidence suggests, however, that that is not true.  First, polls show that a good many Republican women crossed over to vote for Jones.  And secondly, at least one estimate—which I received over the phone from a reliable source, but unfortunately can’t document—is that black turnout this year was only 80% of what it was last year, less than the overall Democratic figure of 90% of last year. 

The Democrats won one of the reddest states in the nation because the Republicans fielded one of the weakest candidates in modern history, a 21st-century theocrat who was reported to have pursued and molested teenagers many years ago, while they fielded a very strong one (as Jones in his victory speech showed himself to be.)  Jones’s win is a great result that shows there is a limit to what moderate Republicans, including southern ones, will swallow.  But it is certainly not clear that Moore’s defeat portends any general swing towards the Democrats in red states—or indicates any ability on the Democrats’ part to get the 45% of the eligible population that never votes to go to the polls.  The vote vindicated the Republican establishment, and one of the big losers Tuesday was Steve Bannon.

Friday, December 08, 2017

Searching for truth

This week I read a remarkable book by a 30-something named Seth Stephens-Davidowitz, Everybody Lies.  Seth, which I shall call him for obvious reasons, has a graduate degree in economics and writes for the New York Times, but he is not mainly interested in purely economic issues any more.  Instead he is interested on what “Big Data”—the enormous amounts of digital data that go along with our new way of life—can tell us about various aspects of our life.  Seth has a very quick mind, a flair for simple, punchy sentences, and a sense of humor.  His work reminds me somewhat of the early work by Bill James, the founder of modern sabermetrics, or statistical baseball analysis, with whom he clearly feels some affinity.  The book has a lot to say about different areas of American life, but I will focus on what he has to say about two sensitive topics, sex and politics. (Religion, apparently, will have to wait at least until his next book.)

Seth’s findings about sex illustrate the difficulties of matching public morality and real human behavior.  Both the political right and the political left have very definite ideas nowadays about relations between the sexes (and between consenting adults of the same sex), but neither one of them seems to match what the bulk of the population is doing.  Seth uses two main sources: statistics on google searches, which we can all access, it turns out, using a readily available app called google trend, and statistics he acquired from, one of the nation’s leading pornography sites.  He points out that google searches are especially useful because people will ask google questions that they literally would not ask any human being face to face.  (Thinking about this carefully, I realized I had done this more than once myself.)  People lie on Facebook all the time—one reason for the title of the book—but they give an accurate idea of what they are worried about on google.  That leads to interesting findings.

Using search data from, one of the leading pornography sites, Seth estimates the gay male population of the country to amount to 4-5% of all men—a much lower estimate that Alfred Kinsey’s 10%, which was probably skewed, he explains, by his sample, which included a good many prison inmates.  This is a very straightforward calculation: about 5% of all porn searches by men on pornhub ask for gay porn. The search data also suggests that there are roughly as many gay men in red states as in blue.  Other data from google, as we shall see, tends to confirm that there are far more men in the closet in red states than blue.

What do people worry about concerning sex?  Men, few will be surprised to learn, are very concerned about the size of their penises, and often wonder how they might be increased.  Straight women, on the other hand, are much more likely to complain about excessively large penises than excessively small ones.  Women’s concerns about their appearance, on the other hand, have shifted.  The posterior has replaced the breasts as the main source of anxiety during the last decade, and women are more likely to worry that their rear end isn’t big enough.  I couldn’t help but wonder, although Seth never brings this up, whether this might have had something to do with one of the most visible women in the United States, the last First Lady, who was well endowed in this respect.  A great many women are also worried about certain bodily odors, but I shall leave the details of those concerns to those who want to read the book for themselves.  There is one very encouraging piece of data from porn searches that more people should take to heart.  Just as the New Yorker facebook movie page, where I now contribute, tells us that there is no movie so bad that it does not have at least a few fans out there, it turns out that there is no aspect of physical appearance, including obesity, that does not excite some measurable portion of the population.  In fact, Seth has commented in an interview I read that if if men and women were willing to use dating sites to search for what they really wanted, instead of what society tells them they should want, they would be a lot happier.

I was very struck, however, by the data google provides on sex within relationships. Relatively few people seem to ask google why their partners want so much sex; instead, they (and especially women) often ask why their partner seems to want so little.  A surprisingly popular question among women is, “how do I tell if my husband is gay?”—and this question is especially popular in the deep red South, perhaps (see above) because there are quite a few closeted gay men there.  I did get the feeling that the sexual revolution and new courtship customs for 20-somethings may have done a lot of harm to marital sex.  People, especially better-educated people, now tend not to get married until the most intensely sexual phase of their relationship is over.  I also wondered about the effect of infinite, free, readily available hard core pornography on the sex lives of young Americans.  It does not seem to have improved it, and indeed, I know from other sources that there is a measurable population now, including some women, who are so addicted to masturbation while watching it that they can no longer do the real thing.

For whatever reason, Seth had much less to say about what big data tells us about women’s sexuality, but he does report one very politically incorrect fact.   I quote: “Fully 25 percent of female searches for straight porn emphasize the pain and/or humiliation of the woman—‘painful anal crying,’ ‘public disgrace,’ and ‘extreme brutal gangbang,’ for example.  Five percent look for nonconsensual sex—‘rape’ or ‘forced’ sex—even though those videos are banned on PornHub. And search rates for all these terms are at least twice as common among women as among men.”  I will leave it my readers to absorb that data on their own—except to say that those findings would not have surprised my favorite author on relationships and sex, the late Nancy Friday, whose compilations of female sexual fantasies include quite a few along those lines.

Now let us turn to politics—where Seth gives us only one major finding, but one which has really set me thinking.

That finding has to do with race in politics, and its apparent impact in each of the last presidential elections.  During the last year and a half we have all had occasion to ask whether racism is in fact a much more powerful force in the US than we had come to believe. The answer is yes.

Although Seth is an Xer, not a Boomer, he shares the Boomer view of 50 years ago that words do not kill, and does not sugar coat his message.  I won’t either.  The main instrument he uses to measure racism in the US is searched using the word “nigger,” of which there are about seven million every year in the US.  That is comparable to the number of searches for “economist” or “migraine.” Sadly,. 150 years after the Civil War and 50 after the great civil rights acts, no other racial or ethnic prejudice in the US remotely compares to racism.  I quote: “Searches for ‘nigger jokes’ are seventeen times more common than searches for ‘kike jokes,’ ‘gook jokes,’ ‘spic jokes,’ ‘chink jokes,’ and ‘fag jokes’ combined.” [emphasis added.]  Such searches tend to spike whenever black Americans are in the news—most notably, of course, during the presidential elections of 2008 and 2012.  And when the searches were broken geographically, a clear pattern emerged. There was a negative correlation in 2008 and 2012 between the number of searches for “nigger” in a given area and the vote for Barack Obama, and a positive correlation in 2016 between the number of those searches and the vote for Donald Trump.  And those areas included upstate New York, western Pennsylvania, eastern Ohio, West Virginia (all those, actually, part of a contiguous region), industrial Michigan, and rural Illinois, as well as Appalachia and parts of Louisiana and Mississippi.  Overall, comparing Obama’s vote in 2008 and 2012 to John Kerry’s in 2004, Seth estimated that racism had cost Obama about four percentage points in the national vote.  When he and a collaborator first wrote an article presenting those results, quite a few academic journals refused to publish it, but one eventually did.

Now there are some possibly complicating factors which Seth might have paid more attention to.  In these mostly white areas, Obama’s race cost him perhaps 4% of the national vote.  But his race seems to have earned some additional votes elsewhere, not only from his fellow black Americans, but also from young people who were inspired to vote by his status as the first major non-white presidential candidate.  Analysis of the 2016 vote has shown that many of those voters did not turn out to vote for Hillary Clinton.  We are dealing here, it seems to me, with an effect of partisanship. A candidate who appeals to his or her party’s activists and seems out of the mainstream, such as Obama or Trump, will draw more votes from his own side, but may be less effective at winning votes from the other side.  That dynamic, if it exists, will make it very difficult for any candidate to be elected with more than a narrow majority.

The other huge question this data raised in my mind, however, was the role sexism in the 2016 election.  I went to google trend myself, and I did searches for “Hillary Clinton” and “bitch.”  Searches for Clinton’s name alone, clearly, would not distinguish supporters from opponents.  Now the time line for those searches showed huge spikes during two recent years—2008 and 2016, when Clinton was running for President.  The 2008 spike took place in the early spring, at the height of the primary season, and the 2016 one took place in the fall.  The largest number of 2016 hits occurred in some of the same places the ones indicating racism did—in Mississippi, Louisiana, Kentucky, and Indiana  Pennsylvania, and Indiana, but also in Wisconsin and Minnesota.  But when I used another feature of the program and compared the numbers to searches for “migraines,” the term that had been about as frequent as the racist searches when Obama was running they were much less numerous.  A few other abusive search terms turned up even fewer hits.  Unless I simply failed to think of the most revealing search, it seems that overt sexism is now much less of a factor in our politics than overt racism.  Clinton seems to have lost critical states not because so many people hated her, but because too few people were moved to come out and vote for her.

A good deal of Everybody Lies is devoted to exploring uses and potential uses of big data and what it might do to us.  Seth effectively describes one of its pitfalls, the pitfall of dimensionality, which has led to some misleading articles tying particular genes to particular human characteristics or diseases.  Essentially, there are so many thousands of variables within our chromosomes that, given a large sample of human beings with a particular problem, chance will produce an apparent match or two.  But those will be random matches with no predictive value for the future. Everybody Lies does not seem to have been very widely reviewed. I recommend it.