The "Agar Plate" of Science Writing

In her Wired review of last year's developing story about sexual harassment in science, Sarah Scoles describes academia as an "agar plate" of such behavior. The scandals, she tells us, are expressions of an environment that "doesn’t merely permit such transgressions" but actually fosters them. We might say it cultivates bad behavior; there is a culture of harassment. That's the narrative that we can expect to see developing in 2017 and that I think it is important to approach with some skepticism.

Scoles' argument does not rely merely on anecdotes, but on what Christina Richey, the chair of the Committee for the Status of Women in Astronomy of the American Astronomical Society, explicitly presents as "social science" (see video 406.01, at 11:10). Scoles reports as follows:

In a 2015 survey of astronomers, 32 percent of respondents reported experience of verbal harassment in their current job specifically because of their gender. Around 9 percent reported physical harassment.

Although she fails to cite her source explicitly, I have confirmed that she is referring to Richey's 2015 workplace climate survey. It is important to keep in mind that, not only has this survey not yet been published, over a year after announcing her results, Richey still refuses to discuss its methodology and results or make a write up available to people, like me, who are interested in understanding them better.

But even if we accepted the survey as valid, there are serious problems with the statement Scoles is using it to support. Scoles is drawing the 32% figure from this slide:

What she fails to say is that the 32% actually breaks down into "rarely", "sometimes" and "often", which she gives equal weight and counts merely as yes/no on whether they've been verbally harassed. (As I pointed out in a previous post, Richey plays up this sort of thing even more. The way she interprets her results we'd probably be told that astronomers are verbally harassed "32% of the time".) To see why this is a bad interpretation, consider a hypothetical result where no one had answered "sometimes" or "often", but 32% had answered "rarely" and 68% had (as it actually happened) answered "never". On Scoles' logic this result would support the very same statement. But a much more accurate (and hopeful) statement would of course be: "In a 2015 survey of astronomers, 68% reported never experiencing verbal harassment and the rest reported experiencing it only rarely."

What the CSWA survey actually (or at least also) found is that under 2% of respondents reported experiencing gender-based verbal harassment often and 11% reported experiencing it only sometimes. In the case of gender-based physical harassment, less than 1% reported experiencing it often and less than 2% reported experiencing it just sometimes; wholly 97% of respondents reported never or rarely experiencing physical harassment. Scoles' statement only becomes true when we include all those who said they experienced it "rarely", which one might of course say in response to a survey even if one's experience says that verbal and physical harassment is the exception rather than the rule.

That is, it is what one would answer if one worked in an academic environment that has enough freedom to permit* occasional bad behavior, but which is precisely not an "agar plate" for a culture of sexual harassment. I am really hoping that 2017 will bring us much less of this sort of misreporting of already dubious survey results. Bad science and bad journalism, after all, are agar plates, to be sure, for bad policy.** Let's try to keep our Petri dishes clean this year, shall we?

_______
*No one uses this in the sense that such behavior is allowed or approved of. What is meant is that conditions exist that let people do certain things and, as Scoles emphasizes, makes it difficult to punish them afterwards. It's important to keep in mind that a free society intentionally establishes such conditions even in the case of murder. There is plenty of freedom to physically commit a murder in our society, and it is relatively difficult to be convicted of the crime. The dystopian vision presented in the movie Minority Report is, of course, the antithesis of this society; here, murders are prevented as "pre-crimes", and would-be murderers are jailed without trial for something they haven't yet done.
**Including the informal policies that guide the judgement of individuals in networks. In a previous post, I've written about Bryan Gaensler's somewhat disconcerting practice of keeping students from opportunities to get supervision from professors that he has heard rumors about. Interestingly, when Gaensler tweeted Scoles' story, it was the "agar plate" line that he found most quotable.