Error and Blame, part 4

"Coming to recognize you are wrong is like coming to recognize you are sick. You feel bad long before you admit you have any of the symptoms and certainly long before you are willing to take your medicine." (Norman Maclean)

[Part 1 here]
[Part 3 here]

I have found that sensemaking scholars in general have a hard time engaging with direct criticism. It is as if telling them that they're wrong about something is merely a signal to them that you're not the sort of person they should be talking to. Many years ago, one young sensemaking scholar seriously told me that, unlike me, he was brought up to see the good in people before the bad. He suggested I start by pointing out all the good things in his paper before offering my criticisms. In what has turned out to be a longer series of posts than I originally anticipated, I'm beginning to understand my feelings about that reaction to my work.

Consider Karl Weick's own response. He has offered only two dismissive "rejoinders" (see here and here) to my critiques. He's probably said less than 500 words on the matter. Granted, I've been pretty hard on the guy. I've called him a plagiarist and a poor scholar. I've accused him of getting a great many empirical facts wrong, in part because he can't distinguish between evidence and anecdote. I've even suggested he can't tell an anecdote from a joke. It's understandable that he's unhappy with me, but it is simply unacceptable that, as a leading figure in his field, he does not directly address criticism of his work. If I'm right, he owes it to his readers to acknowledge it; if I'm wrong he owes it to my readers to set me straight. My reflections about his analysis of the Mann Gulch disaster is making it clear to me why I think this is so important.

In his analysis he blames the disaster on "multiple failures of leadership". His only source of facts about the event—Norman Maclean's Young Men and Fire, which he distorts beyond recognition—reaches the opposite conclusion, namely, that the tragedy was the result of a "conflagration" of cultural and natural, organizational and infernal factors, for which no one could be held responsible, and about which nothing, at the end of the day, could have reasonably been done. Weick's source says "the men did not panic"; Weick says they did. Weick's source says that their leader performed brilliantly; Weick says he lacked wisdom. The truth is that the young men were unlucky and their leader was a tragic figure. Wagner Dodge suffered the loss of thirteen young men under his command—an experience that is unimaginable to most of us, especially those of us who have chosen less physically dangerous work.

Tim Vogus has offered a reason sensemaking scholars don't think my criticism should be acknowledged: "the right/wrong frame that you apply to thinking about emotional ambivalence," he tells me, "is unnecessarily limited and restricts our ability to integrate work on mindfulness across the setting[s we are interested in]". At the end of his remarks, he expresses the hope that his response "helps to advance the conversation and moves us away from 'tussles' and binary thinking in favor of carefully contextualized arguments and collaborative synthesis." That is, it's the very idea that there is a "right" and "wrong" here that he thinks is counterproductive. He believes that there will always be some "context" in which, say, Weick's analysis might hold up. He thinks I should be looking for a "collaborative synthesis" of my understanding of events with Weick's, not a correction of his mistakes. He thinks, I guess, that I should begin with everything Weick gets right.*

It's just that, for all intents and purposes, Weick is completely wrong about what happened in Mann Gulch. I mean this in the sense that only saying that the whole thing took place in a snowstorm in the Swiss Alps could be worse. He's wrong about what happened and why it happened. He's wrong about why it happened because he's wrong about what happened. Indeed, one suspects that since he wanted the disaster to have resulted from a "collapse of sensemaking" he distorted his so-called "data" (which not he but Maclean collected) to fit his theory. He cooked the data. He fabricated events to suit his hypothesis. Whether he did this intentionally, or was just enormously careless, doesn't matter. It's the worst thing you can do as a scholar,** especially when you are sitting in judgment where lives were lost.

"Can't you take it down a notch, Thomas?" I imagine some will say. Well, remember that I'm not using language that is any stronger than Weick himself uses when talking about the firefighters. Weick uses the "right/wrong binary" to call Wag Dodge a failure as a leader. I'm using the same binary to say that Weick is a failure as a scholar. Dodge had to defend his actions before a tribunal at an inquest, and live with them the rest of his life. Maclean worked on the problem until his death, too, stopped him. It took eight months, by contrast, to get Weick's paper from conception to publication, and it has since been widely celebrated as the victory of his style over stuffy, "academic" scholarship. (See John Van Maanen's famous argument with Jeremy Pfeffer.)

One last thing: Weick ends his paper by invoking the authority of his source, stopping "just about where Maclean would want us to end." On the contrary, I think Maclean would have wanted the conversation to go on at least long enough to discover Weick's errors. Indeed, if my sense of Maclean's intellectual courage is correct, he would have wanted us to go on long enough discover Maclean's errors too. But since Weick presents his analysis as entirely supported by Maclean's, which he then gets wrong, we have not even begun that critical part of the process of interpretation. After all, Weick went public with his analysis only a few months after he had begun reading Maclean's book. And then it's as if he wants to declare his paper the last word on the matter.

_________________
*This is very reminiscent of the regard that some social scientists have for the work of Malcolm Gladwell. I wrote about this back in 2010. As Teppo Felin pointed out in that discussion, Kathleen Sutcliffe and Timothy Wintermute made the connection to Weick's work explicitly almost ten years ago.
**[Update, 03/04/15: I'm sure this strong claim is debatable and I'm very willing to have that debate. I definitely think it's worse to fabricate your data than to plagiarise another's prose; and it's worse to massage your data than to patchwrite your theory. When I say that what Weick has done is "the worst thing" a scholar can do I mean, in part, that the most serious error in social science is to mischaracterise what the people you have studied have done. But I also mean something else. It's pretty bad to publish as your own something you haven't written (i.e., to commit plagiarism) for the sake of advancing your career; but surely it's worse to publish a result that is not supported by your data (i.e., to engage in fabrication) for the sake promoting your theory. In the first case, you're misleading your employer about your abilities. In the second, you are misleading all of us about how the world works.]