See, this is why we can’t have a nice talk with the public about scientific research.
First, a statistical analysis of some elements of the National Longitudinal Study of Adolescent Health (known as Add Health) gets published in the new issue of Social Science Research, under the rubric “Personal traits, cohabitation, and marriage.” Then, it gets picked up by major news outlets like Atlantic Monthly and Slate because, you know, Science! and Relationships! and Page Views!
According to the news outlets, science – or should we say Science! – says that the existence of certain traits makes it more likely that you will end up finding a partner for marriage. (Shades of the Science News Cycle.)
Here’s the problem: A closer reading of the original study reveals that some convoluted statistical meta-analysis may or may not disclose some correlation between marriageability and the subjectively described traits of physical attractiveness, personality, and grooming. Here’s another problem: How do you operationally define attractiveness, personality, and grooming for the purposes of this study? You don’t. Instead, you rely on your literature review to mention that other studies have found that inter-rater reliability is not a problem, so you can assert that it’s not a problem here.
Here’s yet another problem: The authors of this study tout their inquiry into the effects of attractiveness, personality, and grooming as a novel breakthrough for their study. Well, no. Interviewers conducting the third wave of Add Health (2001-2002) were asked to rate each of their interviewees on five-point scales for attractiveness, personality, and grooming. The authors of the current study are mining the available data, not innovating or breaking new ground.
So, despite caveats and qualifications from the authors of the study, and some vague conclusions about some ill-defined traits, we end up with headlines like: “New Study Reveals Secret to Romantic Success, and It Doesn’t Sound So Hard to Achieve.”
Our Internet-driven attention deficit disorder continues to degrade the discourse between researchers and the public. Is there a solution? Or is it too late?