CSWA, AAS, and I

[10/10/2016 at 15:36: I've updated this post with more precise time markers in the video, added some charts from the presentation which might be helpful, and fixed various typos and such.]

I was going to leave this subject for later, but Andrew Gelman's recent post about the journalistic coverage of research results for which no written report exists has inspired me to move the issue forward a bit.

It was always my intention to take a look at the bigger picture of sexual harassment in science, astronomy in particular. This will require a few posts about both the policy environment around harassment and the scientific studies of the problem. In an important sense, I am trying to understand what it's like to be a woman in astronomy today. But before I get into the details, I need to tell a story about my dealings with the Committee for the Status of Women in Astronomy (CSWA) of the American Astronomical Society (AAS).

Personally, I find the way I've been treated not just strange but outrageous. But I'll let you be the judge of this.

As background, you need to know that in November of 2015, Christina Richey, the chair of the CSWA, received the Harold Masursky Award at the AAS Division of Planetary Sciences meeting in Washington, DC. Instead of the customary brief acceptance speech, she asked to be allowed to present the preliminary results of the CSWA Survey on Workplace Climate. Her presentation is available as a video at the conference website. Here she promised to begin "the most uncomfortable conversation we've ever had", i.e., to broach the subject of sexual harassment in astronomy, and it is my attempt to join that conversation that my story is about.

I will probably write a more detailed post about her presentation some other time. To understand my story, the most important claims are made over four minutes that begin about 13 minutes into her presentation. Here [13:55] she explains the way she has displayed the results on a "TSA style" color scale, from yellow to red, where red, of course, is most severe. "There is no blue," she emphasizes. But that's only because her slides don't plot the data that would have to be presented in blue, i.e., all those respondents who did not report the relevant behavior. I brought this issue up on my other blog back in February. [And took it up here too.]

After this she tells us that "over 40% of the time, people were hearing sexist remarks from their supervisors." This is just false. It is a straightforward misinterpretation of her own data.

The 44% that the slide shows is a proportion of the sample (i.e., some 187 out of 426 respondents); it is not a measure of frequency with which "people were hearing sexist remarks". Indeed, according to the survey, 56% never hear such language (the missing "blue" bars) and 25% hear it rarely. 15% hear it "sometimes" and only 4% hear it "often". Presumably hearing such language 40% of the time would count as hearing it "often", so her statement applies to only 4% of the sample. This is just very bad social science. One hopes that Dr. Richey interprets her astronomical observations with greater care.

At about 15:30[15:10], she presents an important slide that says that 57% reported being verbally harassed because of their gender. She characterizes this as "over half", which 57% of course is. But as a number of people noticed* when the presentation made news again in January, the slide may say 57% but seems to show 32%. That is, the bar on the chart doesn't seem to add up to 242 out of 426 respondents. I also blogged about this at the time.

Indeed, I wasn't just blogging about it at the time. I was also tweeting about this issue. After all, as Richey emphasized in her talk, it's a conversation we have to have. I blogged about it, I tweeted about it, and wrote a number of emails to Richey asking for clarification.

In fact, my detailed engagement with the CSWA study began with a well-stuck jab by someone on Twitter. Based on a column by Meg Urry, the then-president of the AAS, I had tweeted that we didn't know the actual extent of the harassment problem in astronomy. At the same time, another tweet (using the same hashtag) announced that over 75% of women, people of color, and LGBTQ astronomers experience harassment, citing a piece in Forbes by Ethan Siegel. A twitterzen named "Grant" took a screenshot of the juxtaposition of the two tweets and remarked about the "Twitter timing". Ouch!

(I've written about all this in detail here.)

Well, I looked at Siegel's post and noticed something odd. It seemed that "over 75%" had experienced all the relevant behaviors, regardless of severity. The only reason for this that I could think of would be if the sample had overwhelming self-selection bias, so that only [mainly] people who had experienced the worst kind of harassment [or none at all] had participated in the survey at all. It is reasonable to assume, after all, that they would also have experienced less severe behaviors as part of the severe behavior.

Siegel had cited the CSWA survey, so I contacted Christina Richey by email to ask about it. She did not answer my mail, but it didn't take long before Siegel's post had been updated with corrected numbers and a link to the slides Richey had used in her November presentation. This was before anyone had noticed that the 57% verbal harassment figure was wrong. But once it was brought to my attention, I sent Richey another mail, to which she also did not respond. But this time she quietly corrected her slides: she uploaded a new PDF file with the 57% figure corrected to 32% to the same URL [compare slide 5 with 15:10 of the DPS video], so that Siegel's post now linked to a source that said something other than what he quoted it for. Remember that this is after he had changed the post at her request to reflect the 57% figure.

I wrote a post about this too. By this time I had given up on communicating with Richey and her co-author, Kate Clancy, both of whom were ignoring my mails and had blocked me on Twitter. (I am no longer on Twitter.) With this act of what I took to be simple dishonesty (changing a dated document to correct an error without acknowledging it), I decided to leave the subject for a while. Indeed, I decided to take a break from all social media, including blogging.

While all this was going on, I was being told that a paper was being published in the spring (of 2016), so I decided to save my critical energies for a reading of that paper. But it never came. In September of this year, I decided to reach out to Richey again, in part because the promised paper had not been published, with the passing of both spring and summer. (At present, Clancy has tweeted, it is being held up in the review process at Plos-One.)

But this time I had decided to treat her, not as a private citizen [and public scientist], but as an officer of the AAS, namely, the appointed chair of the Committee for the Status of Women in Astronomy. I wrote to her on September 15, reminding her that I've been trying to get a hold of her for some time, and then said that

I’ve had some issues [with the CSWA study], both about your methods and the interpretation of your results, but they are difficult to resolve in the absence of a write up. I understand that the paper has been under review for some time, which suggests that you have a draft version you could share. I would very much appreciate having a look at it.

After I had not heard back from her for 24 hours, I wrote to Christine Jones, the president of the AAS, explaining the situation, and requesting her help "to open a line of communication" with Richey (whom I cc'ed). Three hours later, I received a mail from a press officer of the AAS. Curiously, he did not acknowledge my mail to Jones. Instead, he said the following:

My colleague Christina Richey tells me you’ve been trying to reach her ... As NASA Deputy Program Scientist for the just-launched OSIRIS-REx asteroid-sample-retrieval mission, Dr. Richey has been extremely busy lately, so I’m writing in her stead. ... A detailed report on the survey results, including descriptions of the methods used to gather and analyze the survey data, is currently working its way through the publication process. ... Please be patient while the review and editorial process runs its course ...

Some correspondence back and forth followed, in which I tried to explain why this wasn't really acceptable (especially given the dodgy behavior with the altered slides). It ended essentially where I started: no line of communication about the widely publicized results of the CSWA study would be opened, although the slides would continue to be cited by journalist to support the claim that "academia is rife with harassment of all sorts." Any critical engagement with this claim would just have to wait until the "publication process" was finished.

I agree with Andrew Gelman's advice to journalists:

Next time someone sends you a press release and you’re thinking of running the story, first contact the organization and ask to see the written report. If they say they don’t have a report, it’s simple: Either don’t run the study, or run a report that is appropriately dripping with skepticism, including the phrase "for which the organization refused to supply a written report" as many times as possible.

Indeed, it gets a bit worse here. The AAS ethics statement says that:

Data and research results should be recorded and maintained in a form that allows review, analysis, and reproduction by others. It is incumbent on researchers involved in large, publicly-supported studies to make results available in a timely manner.

Fabrication of data or selective reporting of data with the intent to mislead or deceive is unethical, unacceptable and fraudulent, as is the appropriation of unpublished data or research results from others without permission and attribution.

It should be recognized that honest error is an integral part of the scientific enterprise. It is not unethical to be wrong, provided that errors are promptly acknowledged and corrected when they are detected.

I quoted this to the AAS's press officer and ultimately asked him to guide me towards the resources that the AAS has to resolve ethics disputes. I asked specifically if there is an ombudsperson. I was told that there isn't and there was no need for one and that there would now be no further communication. My objections to this last email have gone unanswered for a couple of weeks now. It's the first time I've experienced a press officer of a professional organization unilaterally dismissing an ethics complaint, rather than passing it on to someone with the authority to make a determination.

And that's as good a place to end this story as any, I think. It is where I stand with the Committee for the Status of Women in Astronomy of the American Astronomical Society.

________
*Here I need to acknowledge the very helpful contributions of a twitterzen "B" (@ticobas).