Wednesday, March 25, 2015

Error and Blame, part 2

[Part 1 here]

In his 1993 ASQ piece on the Mann Gulch disaster, Karl Weick attributed the deaths of thirteen young men to "multiple failures of leadership", specifically, a lacking "attitude of wisdom" that manifested itself in the "stubborn belief" that they would have the fire out by morning. In his 1996 EAQ application of this analysis to the problems of educational administrators, he said that "Better leadership, mindful that small events conceal the potential for interactive complexity, would have encouraged the jumpers to adopt the attitude of wisdom and simultaneously believe and doubt that they were facing a 10:00 fire." In both cases, that is, a scholar is criticising the work of a leader. And the criticism here is, of course, quite serious: he is blaming the leader for thirteen deaths. He is saying that better leadership could have saved them.

On Monday, I argued that, in making this case, Weick gets the basic facts wrong. It is simply not true that the firefighters held on to their belief that the fire would be out by morning. In a certain sense, it wasn't even a belief; it was simply their job to do everything they could to have it under control by morning. And one of the first decisions that was made in the Gulch that day was to give up the idea that this goal could be accomplished, precisely because they did not "ignore clues" (as Weick claims) about the impending danger.

So what?

That's a question I seriously hear quite often from sensemaking scholars. On this view, it doesn't matter that Weick got the story wrong because he never meant it as an empirical, factual account of the events in the Gulch; he meant it as an "allegory" for business leaders and public administrators. Never mind that Weick presents the story as true. Never mind that he spends two and a half pages describing "the methodology" used by Norman Maclean to construct his account (resulting in the book Young Men and Fire), concluding by urging us to "take these data seriously". Never mind that he later brags that his paper "was read by a forest ranger, who passed the article to firefighting friends, who asked me how I had come up with my analysis, which they thought was better than their own investigations.” (Weick 2007: 15). For some scholars the important question is whether their analyses generate advice that leaders find useful, not whether the events described actually happened.

What these scholars forget is something that Maclean was very conscious of when writing his book. At one point he reflects upon everything he has learned about the disaster, and how long it took to get it right. (Let's remember that even at the time of his death he was not satisfied. The book was published posthumously.)

It's different with me now from when I first started climbing Mann Gulch. Now I carry inside me part of the purgation of its tragedy. It is the part of me and the tragedy that knows more about forests and fires because of this forest fire. If now the dead of this fire should awaken and I should be stopped beside a cross, I would no longer be nervous if asked the first and last question of life, How did it happen? (Maclean 1992, page 87)

At the end of the book he says that to write it he had to "enlarge [his] knowledge and spirit so [he] could accompany young men whose lives [he] might have lived on their way to death" (page 300). He tells us that, after he rose from the ashes of his escape fire, Wagner Dodge, their leader, "had his own brief tragedy to live, which in some ways must be considered a part of this tragedy."

Weick, by contrast, had spent no time in the Gulch at all. His analysis "flowed from a single book [he] consumed while acting as an armchair ethnographer." Indeed, the time Weick spent on his analysis can be plotted quite precisely. Maclean's book was published in 1992, but Weick's analysis was spurred by an approaching lecture he had agreed to do in April 1993. "The normally smooth trajectory of developing a lecture," Weick cheerfully explains, "was interrupted by the basic fact that I had nothing to talk about." But at around this time, it seems, he was discussing the book in a book club. That was enough to let him write his lecture "in a preliminary fashion", which he was then invited, by Bob Sutton, to publish in ASQ. From the time of the lecture to the time of publication, in December 1993, only eight months passed. On this basis, then, Weick gives us an account of a panicked crew under incompetent leadership.

I wonder. Would he be nervous if the dead of that fire should stop him by a cross and ask him what he thinks happened to them—why they died? What, I wonder, would he say to Wagner Dodge?

[For those who want the background, it begins with this post back in December.
I've linked all the relevant posts together in a chain.]

Monday, March 23, 2015

Error and Blame, part 1

Karl Weick's work on wildland firefighters—the Mann Gulch disaster in particular—has had an enormous influence on sensemaking scholarship. Sensemaking scholarship, in turn, has had an enormous influence on both organisation theory and organisational practice. One of the things I worry about, therefore, is whether Weick's empirical analyses are right, i.e., whether our theorising about sensemaking and our recommendations for practice have a reliable basis. While it is sometimes suggested that Weick isn't really making empirical claims, that he is only trying to "get people to think", it is important to keep in mind that his ability to get people thinking is grounded in their believing his stories, at least for the sake of argument. As Barbara Czarniawska has suggested, we must "suspend disbelief" when reading him—we have to "trust" him. And many people do "think differently" after reading Weick. My question is whether that's a good thing.

Here's one reason I don't trust Weick's work on Mann Gulch. Weick has always argued that one of the reasons that the thirteen men died in that disaster was that they held a "stubborn belief" that they would have the fire out by the next morning. In his 1996 EAQ piece he puts it as follows:

The person responsible for spotting landing zones remarked that the jumpers would have the fire under control by 10:00 the next morning, which led the firefighters to call this a 10:00 fire. Later in the day, clues, such as increasing flame size, more erratic swirling of flames, and louder noise, were ignored because they did not fit the expectation that the fire would be out within hours. (569)

This simply doesn't square with the facts we find in Weick's source. (Keep in mind that at the time of writing, Weick only knew as much about the Mann Gulch disaster as he had learned from Norman Maclean's book Young Men and Fire, which we can read for ourselves.) Maclean tells us that at the time of the Mann Gulch fire, "Smokejumpers were still so young that they referred to all fires they jumped on as 'ten o'clock fires'" (page 19, my emphasis); that is, this was not an erroneous assessment of an actual fire, but the underlying attitude of the firefighters. Indeed, it turns out that this wasn't just a question of inexperience; in the 1930s the US Forest Service instituted what is known as "the 10:00 a.m. policy" (see Donovan, Rideout and Omi 1999, page 99). I haven't been able to find a place in Maclean's account where the firefighters "ignore clues" about the fire, and it's pretty much agreed that the fire "turned to murder", tragically, at the very moment when the firefighters couldn't see it. It wasn't the stubbornness of their beliefs that got in the way but the topography of the land. Finally, and perhaps most importantly, reading Maclean, we find out that the firefighters had abandoned their belief that they would have the fire out by morning after an hour in the Gulch*, as a reaction to the conditions they found there, which their leader (as Weick actually notes) described as a "death trap". To say, as Weick does, that they were not "mindful that small events conceal the potential for interactive complexity" (page 569) is simply false.

On Wednesday, I want to take this critique a step further. Both in this 1996 EAQ piece and his original 1993 ASQ piece, Weick blames the loss of thirteen lives on a "failure of leadership", specifically the lack of an "attitude of wisdom" in the crew's foreman, Wag Dodge. But Maclean explicitly tells us that "the disgraced officer's plot", while it makes for entertaining movies, does not give us any insight into what actually happened in the Gulch. It merely combines "small broken pieces of truth" with a "worn-out literary convention" (Maclean 1992, page 155). One of the reasons I worry about the reliability of our scholarship is that empirical error can lead us to blame the wrong people, or simply to blame people who could have done nothing differently. In my view, that's what Weick did in the Mann Gulch case.

[Continues here]

*"They were not in that high state of bliss they had been in when they expected to have the fire out by tomorrow morning ... attacking the fire from the rear would make the job last longer ..." (Maclean 1992, page 66).

Friday, March 20, 2015

A Very Rich Topic

(I keep underestimating the task in my discussion with Tim Vogus. Indeed, it's a bit worrying to notice the analogy to the firefighters in Mann Gulch, who, Weick tells us, believed that they'd have the fire out by morning, a belief that contributed to their demise. Fortunately, I don't agree with Weick on that point, but I do keep thinking I'll have things wrapped up in my next post. It now looks like I'll be continuing the topic well into next week. This post is just preliminaries.)

In his "Fighting Fires in Educational Administration" (EAQ, vol. 32, no 4, 1996), Karl Weick proposes to apply what he knows about wildland firefighting to the work of educational administrators. Making what he calls a "simple and shameless generalisation", he says that "the way in which wildland firefighters preclude failure when they fight fires in forests has direct relevance to the way in which educational [administrators] can preclude failure when they [deal with problems] in schools" (566). The idea of "precluding failure" establishes the link to high-reliablity organisations; reliability is simply is the systematic preclusion of failure. When I talk about organising writing processes like HROs, I'm simply saying that authors do well to be reliable.

But there are many pitfalls in such an analogy. In order for Weick's argument for the "direct relevance" of firefighters to administrators to hold, he has to do a number of difficult things well. First, he has to get the work of the firefighters themselves right. Second, he has to understand the work of educational administrators. Third, he has establish the right connections; he has to correctly identify which parts of the firefighting are like which parts of administration. Moreover, on both sides of the analogy, he has to both understand the concrete particulars of the work and adduce appropriate abstract generalisations. In his case, the particulars are drawn from his work on the 1949 Mann Gulch disaster, in which thirteen firefighters lost their lives.

It is important to keep in mind that, in a certain sense, we all have the same authority to interpret the Mann Gulch disaster that Weick has, since his analysis (in his famous 1993 ASQ paper) is based solely on his reading of Norman Maclean's book Young Men and Fire. I have read that book alongside his analysis and found several critical issues. I suppose that those of us who work at a university also have the same experiential access to educational administration that Weick does; in any case, he does not suggest in his EAQ paper that his thoughts about the topic are grounded in a particular empirical study. This is an ideal critical situation: we're going to be engaging with Weick's interpretation of a book that we have as much access to as he does, from the perspective of similar life experiences, i.e., experiences with university administration. It's a bit like the conversation one imagines firefighters could have about Maclean's book. It can happen at a very high level.

Now, it just so happens that I that I think Weick's analysis of Mann Gulch is wrong. Also, as I've pointed out before, Weick himself seems somewhat ambivalent about his own analysis, and therefore draws contradictory lessons. This equivocation is, interestingly, itself part of the recommendation he makes in the paper he wrote with Tim, where they argue that we do well to foster "emotional ambivalence" through "complex and contradictory" job design. Next week I'm going to try to explain what I think academics can really learn from Mann Gulch, both as leaders and as scholars. It really is a very rich topic.

[Continues here]

Monday, March 16, 2015

Ambivalence and Contradiction

In his response to my critique, Tim Vogus advises us not to dismiss "the idea of complex and contradictory jobs as one possible mechanism for eliciting emotional ambivalence and, in turn, sustaining ... mindful organising". He says that this is an empirical question, and suggests that there are, in fact,

actual highly reliable organizations that design work in precisely this way. For example, in wildland firefighting there is the so-called LCES structure (e.g., Weick, 1996) that balances the faith in capabilities to detect weak signals of changing conditions and respond swiftly to them via lookouts and communication links. That embeds hope in the system. At the same time escape routes and safety zones are also in place. These simultaneously instill doubt in the system in the form of a recognition that things can fall apart rapidly and unexpectedly.

Part of me doesn't want to grant that this is really an "empirical" dispute. After all, even if we take Weick's account at face value, I just don't see the "emotional ambivalence" in having both lookouts and escape routes, which is to say, I don't see how these aspects of the job design of firefighters might be contradictory. Consider, by comparison, the design of writing processes. I generally recommend that people choose to write about something they are quite confident they know and to keep their inner critic at a distance while writing. This could be interpreted as a kind of "faith" in your ability as a writer. But then I also suggest planning to let your inner critic read the text at a later time in search of grammatical mistakes and logical errors. This could be interpreted as "instilling doubt". But is there really any contradiction here? Am I really recommending that writers be ambivalent about their work? Surely, not. I am recommending that they do both things resolutely, namely, writing and then reading their text. That is, even if I accept Weick's empirical description of the LCES structure, it does not force me to accept his logical conclusions.

Now, Tim is quick to point out that universities, where the writing processes I help to design go on, are hardly "high-reliability organisations". But is the comparison really as specious as he suggests? I mean, the very article he cites to show that HROs do actually organise their work in "complex and contradictory" ways—namely, Weick's 1996 paper in the Educational Administration Quarterly—proposes to apply the lessons of firefighters to the work of educational administrators. Weick's paper, in fact, encourages us to take those administrators' language about "putting out fires" much more literally than they probably do themselves. If they're going to talk that way, he says, they should organise their work like firefighters.

On Wednesday, I'm going to conclude this engagement with the notion of emotional ambivalence by looking at Weick's work on firefighters. I have some differences of opinion there too.

[Continues here]

Friday, March 13, 2015

Practical, Theoretical, Empirical, Normative

Last week, Randall Westgren reminded me of old idea of mine with his comment on my post about "empirical questions". While it's going to come off a bit "philosophical" (I don't know why I keep apologising for this quirk about my ideas) it begins with a very common-sense observation: though it may be true, as Heidegger suggested, that "science is the theory of the real", social science is also a "theory of practice". That is, "the real" of social science is our entirely practical, workaday reality, i.e., the place where social life goes on. Social science "theorises" this practical reality, which is to say, it turns it into an "object", indeed, an empirical object.

What Randy reminded me of is that this is really two different ways of "othering" the notion of theory. We can distinguish "theoretical" concerns either from "practical" ones or from "empirical" ones. Being a science depends on enforcing both distinctions. And we can then ask how we establish them in our writing. My standard outline of a social science paper offers a neat way of doing this.

The background section (and the first paragraph of the introduction) should present the practical context that the object of your research figures in. (Yes, I'm using the word "figure" advisedly here; the back-ground is the foil for your object.) It is an entirely "factual", but not quite "empirical", description of the world (or the age) in which we live. It is not empirical precisely in the sense in which your results section is empirical. It, too, makes claims about "the facts of the matter", but instead of basing them on publicly available authoritative sources (that can be cited), it bases them on your data, i.e., information that is, in an important sense, only directly available ("given") to you, the author of the paper.

Between the background and the analysis, we have the theory section and the methods section, which in an important sense construct your sense of the "empirical". The theory shapes our expectations of the object; the method makes it visible (so that it can disappoint our expectations, i.e., so that we can learn). But there's another "other" to your empiricism, namely, your normativity. And that's what the implications section is for. This is where you tell us what we should do in the light of the truth of what you are saying. In many cases, you are telling us how society should change. That is, you are proposing new norms for the practices you have theorised and then studied empirically.

Keeping your practical, theoretical, empirical and normative concerns distinct can be very helpful to your writing. Use your outline to define a space for writing and remember: space is what keeps everything from piling up in the same place.

Wednesday, March 11, 2015

Freedom and Telepathy

"The idea of the intangibility of a mental state ... is of the greatest importance? Why is it intangible? Isn't it because we refuse to count what is tangible about our state as part of the specific state which we are postulating?" (Wittgenstein)

Once you've given yourself the time and the space you need to write—once you've coordinated the here and now of your writing moment—writing is, in a certain sense, "easy". You have your entire vocabulary to draw on, and since you are, for the moment, alone and no one is watching, you can say whatever you want. The words won't even refuse to be combined in ungrammatical ways. Consider, by contrast, the mason or the carpenter, whose work is forever governed by the laws of physics. Sure, your pen or computer has to obey the laws of physics, but your words are free. It is no more difficult to write them down than to think them.

Perhaps this is why Roland Barthes thinks of writing as a sublime kind of freedom. And why Stephen King calls it, almost without metaphor or irony, a kind of telepathy. Because the materials of writing exert so little resistance against our choices, because words are almost made of nothing, are weightless and colourless (in the sense that their colour does not, normally, affect their meaning), we forget that they—the words—are what we are making our writing out of. Indeed, we forget that we are actually making something—sentences, paragraphs—not just doing something—writing. We think that writing is just the act of meaning, an entirely abstract activity. We think it is intangible.

Against this, let's remember James Randi's remark about Uri Geller. "If he's using his mind to bend those spoons," said Randi, "he's doing it the hard way." Geller also claimed to be telepathic. Now, in both cases, Geller was probably very intentionally concealing "what is tangible" about his act from the audience, namely, the important work that his hands were doing in bending the spoons. It was, in short, a trick. (I'm told he's now openly performing the trick as such; he has stopped calling himself a mystic.) To think of writing as some remarkable species of freedom, or a kind of telepathy, is, really, to think of it as a kind of magic. It is a refusal to count "what is tangible" about the activity as part of the specific activity we are doing. In reality, writing is just another thing we do with our hands. In really good writing, of course, like that of Barthes and King, that trick just happens to be concealed.

Monday, March 09, 2015