June 24, 2010

Confounded Divorce ("Why Oh Why Can't We Have a Better Press Corps?" Dept.)

Attention conservation notice: 1000+ words about how I am irritated by journalists being foolish, and about attempts at causal inference on social networks. As novel as a cat meowing or a car salesman scamming.

I have long thought that most opinion writers could be replaced, to the advantage of all concerned, by stochastic context-free grammars. Their readers would be no less well-informed about how the world is and what should be done about it, would receive no less surprise and delight at the play of words and ideas, and the erstwhile writers would be free to pursue some other trade, which did not so corrode their souls. One reason I feel this way is that these writers habitually make stuff up because it sounds good to them, even when actual knowledge is attainable. They have, as a rule, no intellectual conscience. Yesterday, therefore, if you had told me that one of their number actually sought out some social science research, I would have applauded this as a modest step towards a better press corps.

Today, alas, I am reminded that looking at research is not helpful, unless you have the skills and skepticism to evaluate the research. Exhibit A is Ross "Chunky Reese Witherspoon Lookalike" Douthat, who stumbled upon this paper from McDermott, Christakis, and Fowler, documenting an association between people getting divorced and those close to them in the social network also getting divorced. Douthat spun this into the claim that "If your friends or neighbors or relatives get divorced, you're more likely to get divorced --- even if it's only on the margins --- no matter what kind of shape your marriage is in." It should come as no surprise that McDermott et al. did not, in any way whatsoever, try to measure what shape peoples' marriages were in.

Ezra Klein, responding to Douthat, suggests that the causal channel isn't making people who are happy in their marriages divorce, but leading people to re-evaluate whether they are really happily married, by making it clear that there is an alternative to staying married. "The prevalence of divorce doesn't change the shape your marriage is in. It changes your willingness to face up to the shape your marriage is in." (In other words, Klein is suggesting that many people call their marriages "happy" only through the mechanism of adaptive preferences, a.k.a. sour grapes.) Klein has, deservedly, a reputation for being more clueful than his peers, and his response shows a modicum of critical thought, but he is still relying on Ross Douthat to do causal inference, which is a sobering thought.

Both of these gentlemen are assuming that this association between network neighbors' divorces must be due to some kind of contagion — Douthat is going for some sort of imitation of divorce as such, Klein is looking to more of a social learning process about alternatives and their costs. Both of them ignore the possibility that there is no contagion here at all. Remember homophily: People tend to be friends with those who are like them. I can predict your divorce from your friends' divorces, because seeing them divorce tells me what kind of people they are, which tells me about what kind of person you are. From the sort of observational data used in this study, it is definitely impossible to say how much of the association is due to homophily and how much to contagion. (The edge-reversal test they employ does not work.) It seems to be impossible to even say whether there is any contagion at all.*

To be clear, I am not castigating columnists for not reading my pre-prints; on balance I'm probably happier that they don't. But the logical issue of running together influence from friends and inference from the kind of friends you have is clear and well known. (Our contribution was to show that you can't escape the logic through technical trickery.) One would hope it would have occurred to people to ponder it before calling for over-turning family law, or saying, in effect, "You should stay together, for the sake of your neighbors' kids". I also have no problem with McDermott et al. investigating this. It's a shame that their data is unable to answer the causal questions, but without their hard work in analyzing that data we wouldn't know there was a phenomenon to be explained.

I hope it's obvious that I don't object to people pontificating about whatever they like; certainly I do enough of it. If people can get paying jobs doing it, more power to them. I can even make out a case why ideologically committed opinionators have a role to play in the social life of the mind, like so. It's a big complicated world full of lots of things which might, conceivably, matter, and it's hard to keep track of them all, and figure out how one's principles apply** — it takes time and effort, and those are always in short supply. Communicating ideas takes more time and effort and skill. People who can supply the time, effort and skill to the rest of us, starting from more or less similar principles, thereby do us a service. But only if they are actually trustworthy — actually reasoning and writing in good faith — and know what they are talking about.

(Thanks, of a kind, to Steve Laniel for bringing this to my attention.)

*: Arbitrarily strong predictive associations of the kind reported here can be produced by either mechanism alone, in the absence of the other. We are still working on whether there are any patterns of associations which could not be produced by homophily alone, or contagion alone. So far the answer seems to be "no", which is disappointing.

**: And sometimes you reach conclusions so strange or even repugnant that the principles they followed from come into doubt themselves. And sometimes what had seemed to be a principle proves, on reflection, to be more like a general rule, adapted to particular circumstances. And sometimes one can't articulate principles at all. All of this, too, could and should be part of our public conversation; but let me speak briefly in the main text.

(Typos corrected, 26 June)

Manual trackback: The Monkey Cage.

Networks; The Running Dogs of Reaction

Posted by crshalizi at June 24, 2010 20:45 | permanent link

Three-Toed Sloth:   Hosted, but not endorsed, by the Center for the Study of Complex Systems