Did You Marry A Cheater? Or, 4 Ways to Tell A Study Is Baloney
"Engaged in February, Married in June, Affair in September?"
That's how the press release opened. It went on to claim that a survey had shown that men who propose in February are more likely to cheat on their spouses. Per the release:
A new survey from extra-marital dating site AshleyMadison.com has revealed the month you get engaged could determine the success of your marriage! According to the survey of 7,000 members, a whopping 57% of men on the site proposed to their wives during the month of February.
December has the most proposals for the general population, so it was no surprise that the bulk of women using the affair service were proposed to in December (34%). The men's responses, however, lead AshleyMadison.com founder and CEO Noel Biderman to draw some conclusions as to the reason why:
"There is something about a February engagement which implies the man bowed to pressure during the 'most romantic month of the year.' Rather than set the engagement at their own schedule, these men may have rushed the proposal in order to keep their partners or families happy. The fact that they're using AshleyMadison does not mean that the marriage will end in divorce. While these unfaithful spouses may have jumped the gun on their marriage, they are now using an affair as a life raft to help stay afloat."
The press release went on to identify the top three engagement months for male users of the popular extra-marital dating site: February, December and June. Man, I always knew all those friends of mine with engagements in February were totally doomed!
"Cheaters" via Shutterstock.
Or not. Sure, this is a survey of 7,000 respondents, looking at engagement month and cheating. But does it mean that people getting engaged in February, December, and June are going to face a cheating spouse? It's time we looked at this thing with a skeptical eye. And this study has a lot to tell us. In particular, it's got all of the rules that allow you to determine whether the study you're looking at is total baloney.
Show Me the Data
And I mean all the data. What does the information this study provided (it's all up there in the press release) tell you?
- 57% of the men on the site proposed to (or were proposed to by) their spouses in February
- 34% of the women on the site proposed to (or were proposed to by) their spouses in December
- 7,000 people took the survey
- The top three engagement months for cheatin' hearts were February, December, and June
Here's what it doesn't tell you:
- How did the percentages break down for those three months? How did they break down by sex? By age?
- What is the average marriage length of the people who responded, what year did they get engaged?
- And more important: how does this compare to engagements overall? How does this compare to marriages overall?
As we're about to see, this survey is not exactly random. When you are looking at a report of a study like this, look at what they say, and evaluate it carefully. Are the data complete? Are they under specific conditions? Do they involve specific scenarios? For example, how do the months February, December, and June rank in terms of total number of engagements?
This varies pretty widely over time. December is always the highest in percent of engagements, but sometimes February comes in second, and sometimes July does. And the numbers I found only go back for the past five years, the variation could be even higher than that.
Study Your Sample
So they got 7,000 people to take a survey. That's all well and good, but who ARE these people? What are their demographics? What are their age ranges? Because it turns out this group of people is pretty biased. The overwhelming majority of people who took the survey were male, mid-forties, white, and college educated (based on the site statistics Alexa). These are by no means the only people in the world who cheat.
And where did they get their sample? The samples were all obtained from the website AshleyMadison. Yes, THAT AshleyMadison, the infamous cheating website. So before we even get into the matter of engagement months, we already know that these people, who have already registered on the site after all, are pretty committed to cheating.
What this study lacks is a control group, a group of people who have not cheated on their spouses and are unlikely to do so. While this study can tell you that 57 percent of cheating men on their site got engaged in February, they cannot tell you how exactly this compares to non-cheating men, or cheating men elsewhere. This study has some pretty extreme sample bias, and as such can only really tell you about AshleyMadison users, and not, as the title proudly claims, that all men who propose in February are more likely to cheat.
When you see reports of science in the news, especially scientific studies done with human surveys, keep an eye out for this kind of bias. Are you sure we're all attracted to women or men in red, as a recent study claimed? What if everyone the researchers asked was a white college student? Doesn't that change your concept of "all of us"? And if you see a study looking at engagement months making people more likely to cheat, and only ask cheaters -- is that really any better?
Consider the Conclusions
The authors of the study say that if men get engaged in February they must have bowed to pressure due to the intensely romantic nature of the impending Valentine's Day, didn't really want to get married, and that's why they are cheating now. But is that really the case? Here's my issue with this claim: they never asked this question. They never asked any of the sampled people why they were interested in cheating, why they got married in the first place, and if they got married without really wanting to.
In fact, if you look at real studies that have been done on why men and women may engage in extramarital relationships, you will find that not really being into the relationship in the first place is not on the list. What does make the list are things like: marital dissatisfaction, self-esteem, and suspecting your partner of an affair -- and these factors are influenced by things like how religious you are and whether your spouse is pregnant at the time. Does the month in which you got engaged predict any of these factors? Well, no study has asked that.
Follow the Money
While some studies on extramarital affairs are funded by respectable institutions like the National Science Foundation, this one was funded by…AshleyMadison, the same site they got their people to take the survey from. A site which markets primarily as a matching system for people already in relationships is going to tell you which men are likely to cheat?
While they may indeed have an "in" here, they will never have an unbiased sample of the population, and are pretty heavily influenced to report what they want to be the case. This is not a study performed by scientists for the sake of answering a question. It's a study performed by a website for the sake of marketing.
So when you see a study like this, look at it carefully. Where did it come from? Who conducted the study? Why did they conduct the study in the first place? Are these people scientists, a company, someone else?
Obviously these are not the only ways to tell if a study such as this one (I can't bring myself to call it "scientific," it'd be an insult to good science) is any good or not. But this is a really great example of some of the most common red flags, which you can look for the next time you see a study claiming to tell you how all men or women behave.
And do you have to worry about infidelity if you got engaged in February? Probably not any more than someone who got engaged in August. Unless your spouse is already on AshleyMadison, but then I think your engagement month is probably the least of your concerns.
Scicurious is a blogger at Scientific American The Scicurious Brain and Scientopia's Neurotic Physiology, where this post first appeared. She has a PhD in Physiology, and is currently a postdoctoral researcher. She loves science, and so should you.