Why is the problem of obesity so damned tricky?

1. Why is the problem of obesity so damned tricky?

 

A lot of us are fat. Fatness increased dramatically in the 20th century, especially since the 1980's in the US.  We don't like being fat in general.  When it becomes extreme, body fat usually doesn't look good to us and can lead to health problems, diminished longevity, and restricted quality of life.  Some distributions of body fat are worse than others for health or for appearance, but at some point it eventually looks bad to us and detracts from our lives.  

 

1. Just Theories are Well Communicated and Popular but Wrong

 

If we find fat so problematic, why haven't we simply found good solutions for it and begun to apply them?  Most people can lose some body fat temporarily by some combination of depriving themselves of some particular kinds of preferred foods, trying to force themselves to eat less,  and trying to force themselves to be more active.  Most people end up going back to their original body fat levels, or greater, after a few weeks, months, or years of that effort.    There is no lack of simplistic just theories about why this is happening:

 

Fat people are just lazy and not exercising enough self-control over their eating and exercise

Fat people are just gluttons who eat too much

Fat people are just eating too much carbohydrate

Fat people are just eating too much starch

Fat people are just eating too much sugar

Fat people are just eating too much fat

Fat people are just maintaining their weight at a genetic set point

 

Although each of those ideas is wrong in its own particular way, and they are each perhaps partly right in some sense,  it is the just part that is most wrong of all.  The just theory is a special kind of problem in itself that is part of the reason fatness is so difficult to address.

 

Here's my central claim: scientific consensus is not just a matter of certain opinions winning out over others, the problem of obesity is understood in broad form as a scientific consensus even though many the details are complex and some of our knowledge of the factors is undoubtedly incomplete.  There is a rough scientific consensus about various aspects of obesity across fields of physiology, nutrition, psychology, neuroscience, and medicine.  It is not just the confusing mess that appears in popular media.

 

I also claim: this consensus understanding is not what is published in most diet, fitness, and weight control books.  In fact, a sizable number of popular books on diet and weight control conflict dramatically with the scientific consensus, but are marketed as if they were new and revolutionary new findings.    This has contributed heavily to the confusion and so added to the problem. 

 

 

B. The Scientific Consensus is Poorly Communicated

 

In the stories that journalists tell to try to communicate their own interpretations of the obesity and health research, they have often engaged the science compellingly but not faithfully.  We now know some very useful things that are not well communicated, or are even ignored or denied by popular authors with their own agendas, or get lost in the media confusion:

 

1. The theory that people can control their weight by exercising self-control at each eating or activity decision is simply wrong.  

 

No one has that much self-control against an environment that constantly tempts them.  The psychological study of self-control reveals that variation in impulsiveness is indeed a factor in all sorts of problems, including obesity, and that people can often learn skills to compensate for impulsiveness, but these do not rely on individual acts of self-control.  Self-control is always finite, and we successfully compensate for impulsiveness by not just exercising self-control in each decision, but also even more importantly by altering our environment and altering our habits so that we reduce the need to exercise our always finite capacity for self-control.

 

2. All of the theories of specific  macronutrients causing  us to become fat are simply wrong.

 

The theories of metabolic advantages of particular diets (low carb, low fat, vegan, paleo, etc.) have all been falsified scientifically so far.  Obesity is in general at the population level not caused by everyone's metabolism being "broken" by certain foods nor is it "fixed" by eating certain foods. 

 

We know for a fact for example that low carb diets do not cause people to lose weight by any special "fat burning mode" as is often claimed in popular diet books.   We know that simply eating a lot of fat or a lot of sugar while keeping energy intake constant does not cause us to gain more body fat.  We know for a fact that the theory of obesity being caused by insulin levels rising due to eating  too much starch or sugar is simply wrong.  People become fat when they take in too much energy regardless of the source and they lose body fat when they take in less energy regardless of the sources they eat less of. 

 

That doesn't mean people respond equally to every kind of diet.  It's just that the reasons are not metabolic.  The reasons have to do with reward, satiety, and sometimes individual differences.  But the metabolic differences between different macronutrient strategies is based on thinking that has already been tested and falsified.  

 

Also  this wouldn't have to be true necessarily.  It isn't a logical necessity that foods have no powerful differential effect on absorption and conversion to fat and fat storage.  It's not just a matter of thermodynamics or conversation of mass and energy.  It is possible that our biology might have allowed us to take in some kids of nutrient, extract energy or chemicals from it, and excrete most of it without gaining body fat.  But it turns out to be false.  The reason is not conservation of mass and energy, the reason that our biology is particularly efficient in using nutrients and in storing the surplus energy. 

 

We might truly have discovered some  differential metabolic effects that prevent us from processing certain foods into fat (and that is exactly what many popular authors have claimed) but … it turns out that it doesn't seem to be true in general, and certainly not as a reliable  way to lose body fat permanently.  The small differential effects of nutrients on metabolism are sometimes used to good effect in short term efforts at bodybuilding and fitness, but they are not a reliable approach to obesity in general.

 

3. Consequently, No macronutrient restriction strategy is a best practice diet for everyone. 

 

Framing the problem of fatness as if we  need to choose between popular branded diets is more often part of the problem than part of the solution.  Branded diets are generally based on a particular just theory.  

 

Debates over low carb, low fat, low sugar, low starch, low glycemic, high fiber, vegan, paleo, and so on are mostly based on asking the wrong question: "what specific food is making me fat?" 

 

Cutting out problematic "trigger" foods can be helpful for particular people, but not for metabolic reasons.  Also cutting out entire classes of food can help at least temporarily lose weight.  This is also not for metabolic reasons.  This is because we tend to reduce intake more than we compensate, at least for a while, when we cut out entire classes of food. 

 

Some of those strategies work better than others in the short run.  But it turns out with those macronutrient restriction strategies that it really doesn't matter which class we chose to cut out in the long run. 

 

People who successfully reduce intake by restricting carbohydrates and people who successfully reduce intake by eating vegan or by eating "paleo" are all losing weight because they are taking in less energy, not because of metabolic advantages. 

 

So long as we can sustain the calorie deficit with that strategy, we lose body fat.  That's wonderful for the people who end up with a strategy they can sustain, and we can find some success stories for many different strategies.  Strategies that restrict entire classes of nutrients end up being unsustainable for most people  in the long run though and the claims often made about any of these having unique metabolic advantages for weight control have been soundly falsified.  

 

4. People do not regulate their weight to a particular inherited set point.

 

This is the opposite problem from the panacea solutions.  This one reinforces our tendency to give up.  Fatness and leanness often run in families, but not because our genes have a built in weight that we are fated to maintain.  It is because of all of the various factors that go into activity, reward, impulse control, individual metabolism, and habit formation, how those interact with specific environments, and because we often inherit things in addition to our genes.  Animals in their native ecological niche tend to regulate their weight very tightly and people who try to lose weight often end up back at the same weight.  But these are as much a result of stable aspects of their environments as stability in their weight regulation.  When we re-engineer out environments we end up altering weight regulation.  The efficiency of our biology and the stability of our dispositions are powerful factors in making obesity a difficult problem but they do not  make it impossible to solve. 

 

I'm going to try to do more than just add my own personal just theory to the already confusing and conflicting list.  What I'm going to try to do is navigate the available evidence to show what is going on and make sense of why the problem is so difficult and what people are doing when they do manage to succeed.

 

3. The Backlash Culture Against Obesity Does More Harm Than Good

 

So the "obesity epidemic," as it has often been called, has led to a culture of backlash against fatness.  By that I mean a commonly shared negative attitude toward fatness and toward fat people.  We have mobilized mightily against the problem in all sorts of ways.  Some of those have unfortunately probably made the problem worse. 

 

The backlash against fatness might have been a good thing if it had led to a problem solving culture that recognized the biological, psychological, cultural, and economic dimensions problem realistically, helped us understand it, and began providing realistic solutions.  Obesity researchers have sometimes attempted to offer realistic solutions, especially focusing on preventing obesity in children where we can potentially have the most effect.   

 

But realistic problem solving is definitely not most of what has happened so far.   What we have instead is:

 

1. The problem remains: most people who try to lose body fat end up going from diet to diet or from one exercise program to another , succeed for a while, and then give up.  Eventually many give up on the problem entirely as hopeless.  In effect, we often give up on ourselves.  This is especially true when we buy in to the popular misconception that obesity reflects a simple failure of willpower.  Fat people are often stigmatized as lazy and they feel like failures, which generally makes the problem worse rather than better.

 

2. The food industry exploits the situation by creating and marketing niche "diet" and "health" products that supposedly address the problem but which rely on outdated theories, popular misconceptions, and strategically selective interpretations of research.  The market for high density rewarding "diet" foods for example was created to exploit medical advice to eat less food by eating less fat.  The advice was oversimplified and the new market for diet foods probably added to the problem significantly by exploiting it.  Their goal is selling more food product rather than improving health, which most often turn out to be conflicting priorities.     They become part of the problem by flooding us with false solutions and misleading information. 

 

3. The fitness industry exploits the situation by creating complex dietary and exercise programs that are oriented to short term bodybuilding or fitness goals and then marketing those as solutions to obesity. 

 

4. Popular authors tout their range of idiosyncratic interpretations and solutions, each trying to make sense of the confusion but they mostly end up increasing it.  Popular diet just theories by journalists and doctors very often end up inadvertently creating even more confusion.  

 

So the backlash culture in these various dimensions,  far from helping to lessen the problem, has exacerbated and perpetuated it in several specific ways:

 

• It increases the difficulty of the problem for individuals psychologically, by fostering confusion and then discouragement and self-loathing.  By emphasizing willpower in unrealistic ways, and relying on solutions that require extraordinary short term efforts, we make people less able to succeed rather than more able to succeed in the long run.

 

• It adds to choice confusion when we try to make decisions and are led astray by the marketing of product rather than decisions in our own best interests.  Unless things like "diet foods" and "bootcamps" and "extreme weight loss" are themselves actually viable solutions to obesity (which they are not in general) they become new sources of both confusion and discouragement.

 

• It reinforces the epistemological problem of cynicism.  Thinking we are all scientific experts  who are well situated to simply choose between the popular diets and popular theories that seem most plausible to us prevents us from learning from the actual research and prevents us from moving toward more realistic solutions.  We become easy prey for every scheme that comes along. 

D. Science Cynicism and Overly Broad Mistrust of Expertise

 

Journalists writing about the problem of obesity have responded to this ongoing confusion by either claiming we don't know what causes obesity or by trying to promote a particular new finding.  The problem is very complex in some ways, but it is very misleading to say that we don't know enough about it to move toward better solutions.  The journalists who say we don't know the answer are a minor concern though, they are at least sincerely trying to be good communicators of the science and clear up the confusion.  The ones who are really dangerous are the grandiose prophets of false information. 

 

What we usually have is a theory that catches the interest of the journalist and then they confirm it by gathering information and telling stories that reinforce their own theory.  This is a compelling and powerful way to communicate, but it offers no assurance that they are getting the science right.  And while we sometimes find out about truly revolutionary ideas this way, as it turns out, these "sciencey" storytellers are most often getting as much of the science wrong as they are getting right.   

 

These "sciencey" journalists often end up asking bad questions and then we end up talking about the wrong things.  I think this is a big part of the problem and the reason I have written this book.  I hope to describe what we know about the problem of fatness without falling into the common trap of arrogance and confirmatory bias, but of course I can only argue my own perspective.  I could be wrong about the scientific consensus, but I will do my best to communicate it as faithfully as I can. 

 

The unconstrained confirmatory bias by journalists wouldn't be so bad by itself in some cases, since they do sometimes bring information to light that was not previously well known.  At least some people would benefit from some of this information.    But these folks often then read some scientific literature to try to bolster their articles and books and I feel they often do a hatchet job on it.  They can't reconcile their idiosyncratic theories with the existing evidence so they start claiming that the evidence that conflicts with their own theory was due to "corrupt researchers."  This ad hominem strategy has been a highly successful one for some authors, but it is an unfair way to argue and much worse, it plays into general cultural cynicism of science.  That part is extremely bad for the rest us.  It means we perceive the science as arbitrary or a matter of choosing sides based on what sounds plausible or what seems to work for some people.  It feeds the popular trend for imagining that everyone is naturally capable of evaluating scientific evidence by virtue of "common sense." 

 

What often happens, I believe, is that we end up mistrusting expertise, which is in itself a very serious problem.   That fact that we find so many false or misleading claims of expertise makes the problem of finding, trusting, and interpreting legitimate expertise even more of a challenge.    Nutrition and health sometimes involve on complex technical knowledge and interpreting rich patterns of seemingly conflicting evidence.  This kind of interpretative skill and knowledge is not something we possess as part of our "common sense."  It is something that requires not only understanding "critical thinking" in general, but also requires domain-specific knowledge in the specific fields in question. 

 

The backlash culture and especially cynicism of real expertise has fed bias and arrogance by various popular journalists who then set the tone for popular conversations in unproductive ways.  Although they may have a lot of charts and graphs and selectively cite and interpret a lot of research, rarely do journalists engage the research deeply and competently and systematically and put it into context.  Rarely do they make a distinction between different kinds of evidence and different quality of evidence in order to do a skilled job or drawing general conclusions from the existing science. 

 

The biggest problem with many of these popular authors is their grandiosity.  They think they've discovered a pattern that everyone else has missed or they have some limited success with some people and they proclaim to the world that they have the solution.  Grandiosity exploits uncertainty and cynicism to produce cults of misinformation that are self-perpetuating and extremely difficult to address with reasoning and evidence.  

 

 

Summary of Why the Big Fat Problem is So Tricky

 

Summary of broad factors making the Big Fat Problem so nasty:

 

Problem Factor	Class of Factor	Associated Bad Thinking
We tend to think in terms of simple actionable heuristics we can act upon when we make decisions.  What food is causing me to get fat, so I can stop eating it and lose weight?  What foods should I eat to lose weight instead?  Our need for actionable heuristics makes us especially vulnerable to just theories, and just theories of complex outcomes are typically wrong.	Oversimplification	"Obesity is just a matter of eating more of … or eating less of  …"
The science is often distorted into oversimplified advice.  The scientific consensus relies on expertise to understand and is hard to communicate in terms of actionable steps so it gets lost in the confusion of just theories.   We end up systematically asking the wrong questions and arguing without regard to the existing patterns of evidence.	Oversimplification	"One study constantly contradicts the previous one, so it's all really useless information, we should just pick the authors we agree with and follow their advice"
Our collective cultural assault on fatness has led to a backlash culture that makes things worse and leads to additional problems.	Unintended Consequences	"Being fat is really bad, stop being so lazy!  Just do this… "
Journalistic arrogance and widespread science cynicism exploit the problem of ubiquitous expertise	Knowledge Cynicism	"We're all scientific experts, including me!  Forget the scientific consensus, this sounds like a good way, follow me!"
"Sciencey" storytelling journalism	Poor science communication	"Telling stories about research and researchers is a great way to learn the science!  The details of weighing evidence are boring and irrelevant and people aren't able to handle it."
Industry interests driven by economic and business concerns that often end up conflicting with public health or individual interests of consumers	Economically Driven Consumer Culture	"People citing research in their ads and branding their products for health are generally offering valuable new choices for us."

Book Review: "Are We All Scientific Experts Now?"

Highly recommended.  Brief book with a very high level argument relying a lot on his experience but I think it has some deep insights into how science is done, our shifting conception of how it is done, and the way it is communicated. 

Three big ideas are covered in this book.  First, a three wave conception of how science has been conceived since the mid-20th century:

(1) idealistic deference to science focusing selectively on fairy tale versions of its history and overly focusing on the heroic successes,

(2) cynical deconstruction of science as nothing special and decloaking of experts by adopting a symmetry of explanations regarding what is true and what is false, making science just another storytelling activity and focusing selectively on failures and controversies and social and political influences on science, and

(3) a more realistic wave that accepts the messy processes of doing scientific work and the requirements for doing it well along with the ethos that makes sincere inquiry into nature legitimately special. This is where we need to be now in his view, and I agree strongly with him.
The second wave Collins associates with the problem of "default expertise," the idea that since it is nothing special we can all directly understand scientific ideas and primary sources, learn from them, and make competent and useful evaluations of them, without any specialist knowledge or skills.  He ultimately finds the concept untenable and a notion drawn from significantly overreaching the real insights that were drawn from the second wave that revolted against the privileged scientific priesthood.

Second big idea, a model of different kinds of expertise and its relationship to tacit knowledge, recognizing especially the difference between things we pick up ourselves and things we pick up by interacting deeply and systematically with other people.

Third big idea, distance lends enchantment.  The farther we get from being researchers directly discussing their work with each other in a field day to day, the more the conversation changes and becomes more idealized regarding how it might be applied and the farther we get from actually understanding it.

My summary from the review:

"We are all ubiquitous experts but we are not all scientific experts in the sense that not only are we not all domain specialists in the fields we want to think about but we don't even widely share the ethos of inquiry that is common to scientists. Even specialist scientists vary in how highly they value aspects of that ethos.

The conclusion is that it is vital to both protect the scientific ethos and have a more realistic understanding of the kinds of expertise and the messy processes that make it work and cloud our communications about it when we go to make decisions from scientific work.

This kind of nuanced, important thinking about science and expertise is a wonderful gift from Collins that I truly hope we don't squander."

Full review on Amazon:

http://www.amazon.com/review/R29W32NZEDCBIA/ref=cm_cr_rdp_perm

Book Review: The Blind Spot by William Byers

The Blind Spot: Science and the Crisis of Uncertainty

William Byers

Princeton University Press, 2011

Full review on Amazon.

Interesting food for thought, a reason for some humility, and an argument for perspectivism

I got some real value out of this book because it made me think more deeply about an important and fundamental philosophical problem, the problem of knowability.  How well do we directly know the universe we live in, and how well can even our best explanations really help us grasp it?  In the Western intellectual tradition, the question goes all the way back to the ancients, and we still frame it in much the same way they did.  The problem is fairly obvious to any reasonably reflective person I think, but I suspect most people tend to assume it is either an illusion or something of little consequence.
 
We have come to understand a massive number of details about our existence in various ways through science and we have discovered that with a bit of ingenuity, we can use mathematics to describe and predict the behavior of real things remarkably well.  This gives many of us the sense that we can grasp just about anything about our existence in the same way.  Afterall, from the modern humanistic point of view, what else is there besides our scientific causal explanations, our mathematical models, and our various superstitions? 

Byers seems to be playing something like the mischievous role of Ian Malcolm from Jurassic Park here in some ways. He is telling us that we shouldn't be so certain that we really understand everything the way we think we do.  Nature will always surprise us.  And he is not saying this because he thinks we should believe in miracles or because science doesn't work as well as we think.  He is saying it because science and mathematics cover the universe as we experience it rather like a lumpy carpet covers a smooth floor.  You can push down the lumps, but they always pop up again somewhere else. 

Our ability to explain our existence doesn't quite map to the totality of that existence in any uniform way.  We end up taking different perspectives on the same things in order to understand what we perceive.  This need to take different perspectives is "ambiguity." Our sense of certainty about what we understand, the perception of smooth areas of carpet, is not an illusion, but neither is it absolute, perfectly generallizable, not even entirely objective.  We tend to ignore ambiguity when we see it because it makes us feel uncomfortable.  Byers claims this discomfort creates a permanent blindspot in our perception of our own existence. 

The central theme of the book is that there are many different expressions of ambiguity in nature, and that they are all expressions of the same underlying limitation in our ability to grasp nature in terms of concepts and symbols.  It isn't just that there are some problems more difficult than others, it may be, according to Byers, that some difficulties are impossible to resolve permanently, they will always appear again in another form whenever we grasp them.  Blindspot tells us that something about the territory makes it fundamentally unmappable in any complete and consistent sense.  This, for Byers, drives us to keep trying to explain by grappling with the ambiguity in nature.

 
Byers makes some fascinating points and uses a lot of good examples from mathematics and science to make his points.  Most of the examples will be familiar to avid readers of science and math.  They include the usual suspects such as Godel's Incompleteness, Heisenberg's Uncertainty, wave/particle duality, the problem of the objective and subjective perspectives, intentionality, self-reference, and so on. Byers also draws on his own field of mathematics for some less well trodden examples such as real number theory and differential vs. integral calculus. 

While this book is definitely worthwhile in my opinion, I don't quite share the same excitement as some of the other reviewers because I also found it very repetitive and for me personally it sometimes seems to jump from point to point without really taking on important points in the detail they deserve.  I think this impression I get comes from Byers background in mathematics.  He will often start to talk about something I find really interesting, like the problem of intentionality or the problem of mental causation, but quickly turn it back into a logical question instead of exploring the scientific or philosophical questions in more detail.  I suppose my criticism is that Byers seems to be a "math" but not a "polymath" and the ambitious topic he has taken on may require someone with deep understanding of many different fields, since the thesis is that the blindspot is not just a quirk of mathematics but of our symbolic and conceptual abilities in general.

Ultimately I think the claim Byers is making is basically that the perception of clear understanding of nature is a local phenomenon not a global one because we can never get rid of the problem of incompatible but valid perspectives.  We can make the carpet perfectly smooth in an area, but we can never make the whole carpet completely smooth.  The "consilience" of nature's laws will never have a final expression in one scheme because we can't ever reconcile the different viewpoints that: (1) accurately describe situations, (2) are each self-consistent, and (3) are not only different but incompatible with each other. 

From my perspective, I think Byers has identified the problem in a legitimate way, but his logical arguments don't really convince me that the problem of perspectives is forever irreconcileable.  I agree with him that we should take the problem(s) more seriously than we do, since we usually wave this sort of problem away through various tactics.  I also agree that we should incorporate the challenge of multiple perspectives in our thinking.  It is not clear to me that the lumps are an intrinsic part of nature, or that they are all manifestations of the same blind spot, and Byers doesn't seem to me to exhibit the depth and breadth of scientific knowledge across the various relevant disciplines to make that point stick, but it does seem reasonable to think of them that way until we have actually smoothed them out and to take Byers' claim seriously as at least a sensible policy and a reason for humility.

Book Review - Apollo Root Cause Analysis: A New Way of Thinking (?)

Book Review - Apollo Root Cause Analyis: A New Way of Thinking Book by Dean Gano, Apollonian Publications, 2008. Review by Todd I. Stark Link to Amazon review.

Very useful principles for preventing bad things from happening, April 11, 2011 By
Todd I. Stark "Cellular Wetware plus Books"

This review is from: Apollo Root Cause Analysis: A New Way of Thinking (Paperback)

How can we prevent bad things from happening, and how can a formal method help us in this quest? That's the topic of this book.

"By understanding the cause and effect principle and creating a RealityChart, your understanding of what constitutes reality will be changed forever ... allow you to see a reality that was previously beyond your comprehension." (Apollo Root Cause Analysis, p. 3)

"... the problem of a linear language in a nonlinear world while accommodating the simple human mind has been a challenge for the ages. I believe this challenge has been met with Apollo Root Cause Analysis." (ARCA, p. 176)

The importance of the topic covered by this book is immense, so for my purposes, I'll forgive the author his amusing enthusiasm for this own method as seen in the quotes above and try to determine what may actually be truly useful in it rather than dwell on what may or may not be unique about it. I will be referring to this book and to the Apollo method in this review. The method is also implemented with associated RealityCharting(tm) software which is outside the scope of this review, except where explicitly mentioned.

Problem solving well is one of if not the most critical factor for human success across a wide range of activities. One big part of problem solving is explaining *why* something unexpected happened. I you think about it, anytime something unexpected happens, we generally want to know why. We do this naturally, automatically, and effortlessly. We can't stop ourselves from doing it to at least some extent. Sometimes the explanation seems obvious and sometimes it seems very elusive. Even when the explanation seems obvious, there is often more going on of importance than we realized. We do this seemingly because especially if it's a bad thing that happened, understanding why it happened potentially helps us predict whether it will happen again and perhaps prevent it from happening again. That's important to us as individuals as well as to organizations. It's so important that we already do it all the time. We also tend to assume that we're very good at it, I think. My experience is in agreement with the author of this book, people are in general not nearly as good at solving problems as we think we are, at least when the problem become complex and involve multiple people and organizations.

We have a powerful natural ability to make sense of events that happen around us by identifying people, places, things, and weaving a story around how they interact with each other. (1) The power of storytelling to make sense of events is also its tragic downfall when we need to understand how things happen in great detail. Our temptation to tell stories actually gets in the way of understanding what specific things need to happen in order for other things to happen, and that's the sort of understanding that is needed in root cause analysis. Getting past our natural but distracting abilities to use the right tools effectively is the greatest value of an effective formal method. We often look to formal methods to systematize our thinking in general, but problem solving outside the academic realm of worked problems is often not amendable to being constrained by formal methods. Instead, formal methods provide their real benefit in forcing us to look past our own blind spots at key points during problem solving. To his credit, the author of this book seems to have grasped this important point very well and applied it effectively.

The most problematic blind spots in our thinking are those that aren't avoided by intelligence, education, or domain-specific expertise, skills or knowledge. The most problematic blind spots are actually natural abilities that serve us well most of the time. One of the most important examples is that we tend to use storytelling to construct a plausible sequence of events leading to an outcome, putting people into the center of the action and making the events meaningful to us. There are a number of reasons why this natural and compelling process is a bad idea in problem solving.

First, since a story starts with the putative "root cause" and then proceeds to its effects, it tends to assume a single cause. Events don't have a single cause. If our goal is to prevent bad things from happening, we'd rather identify as many relevant causes as possible, and potentially address each one as makes sense.

Second, storytelling tends to center around characters, their motives, and the things they do. The motives and behaviors of people are often the things we have the least reliable control over in problem solving. We'd prefer to find things we can control better if at all possible, and save human choices and our interpretation of human motives as a last resort.

Third, we can usually tell multiple different stories about the same events, depending on what information we emphasize and how we emphasize it, which in turn depends on our motivations and perspective. This means that the choice of which "root cause" we focus on tends to be more political and subjective than a result of careful analysis.

Our storytelling is natural and compelling but it tends to be more to make sense of events than to understand the details of what causes what.

So effective Root Cause Analysis tools and methods should discourage storytelling and instead search the prerequisite conditions and sequences of events that lead to the results we want to prevent. That's the message of this book and its method. So how well does it accomplish this?

Why a formal process?

Causal analysis (or what is sometimes more specifically referred to as "root cause analysis") is the more formal process of doing what we tend to do naturally, try to figure out why events diverged from what we expected. Why do we need a formal process for doing this, if we already do it naturally? Because often we don't do it very well. The very fact that we do it naturally means that we will tend to make *confident guesses* about why things happened. Often more confident than accurate. This far, the author is on solid ground. The author doesn't refer to it explicitly, but there is a very large literature on the cognitive science of decision making that supports the author's claim that people tend to jump confidently to unwarranted conclusions when they first try to explain causation. (2, 3, 6)

One very important reason for a formal process is to systematize the data gathering and analysis process in order to help compensate for our natural biases that lead us to make confident but inaccurate guesses. The value of formal vs. informal processes is debated widely in the decision science literature, since much of our innate intelligence is the result of automatic non-conscious processes that are opaque to us in our own thinking. (5) However in general I think it is a pretty safe conclusion that a good formal process often helps us focus attention on things that we would not otherwise have seen to see past some of our own blind spots. That's the first reasonable rationale for causal analysis methods like Apollo Root Cause Analysis (ARCA) and its primary tool, RealityCharting(tm). Simply having a process to focus us on causes rather than stories is useful in itself.

Why a formal group process?

There are various reasons for using a formal process in organizations. Unexpected events often affect more than one person, often more than one person is part of the problem, and often more than one person has information or perspective needed to figure out what happened. So causal analysis very often becomes a group process. In addition to the reasons for using a formal process in general (to get around our individual blind spots), it is also important in groups in order to help avoid "group blind spots" such as the principles of behavior in groups studied by psychologists. We are often tempted us to go along with the consensus, to protect our own ideas, to react initially negatively to new ideas, and other tendencies that can negatively impact the problem solving process in groups. (6) A good formal group process can help get around our group blind spots just as it can help us get around our individual blind spots. In spite of the well-established problems created by "groupthink" and other group dynamics studied in social psychological, under optimal conditions, groups can often perform significantly better than individuals on some kinds of problems. (7)

The author makes a key point about group processes that he says defies conventional wisdom (p. 9). He implies essentially that the principles of good problem solving are simple and domain-general so should be taught to everyone, whereas domain expertise is deeper and necessarily differs more between people, so "subject matter experts" should also be at hand. The conventional wisdom he says is that problem solving is entirely "inherent to the subject at hand," or what I would call domain-specific, ignoring the value of domain-general principles in problem solving. I don't know how much this really defies conventional wisdom, but I agree with him completely and I think this is an important principle. It is a major part of the rationale for involving more people in a collaborative group problem solving process rather than just pulling in a few experts. If the process is good, and this principle is valid, then it has deep implications for improving problem solving in organizations by broadening involvement and investment in the process itself. That brings us to the real question at hand, whether the author's method accomplishes this objective.

Why Apollo?

So far my description here is I think pretty much in line with the author's rationale. Now we come to the real meat. How good is this particular method at getting around our individual and group blind spots, compared to other methods? That's the significance the author claims for this book and for his method, so that's what we really need to know to evaluate this book. The book covers a lot of ideas that are very important to know, but many are common knowledge to experienced problem solvers, so I am not going to focus on those. What I will focus on is what is supposed to be special about ARCA and RealityCharting(tm) in particular.

According to the author, the key distinction between his formal method and all others is that all others are ways of categorizing causes and schemes for voting on the best way to categorize them, while his method discerns (and RealityCharting displays) the actual relationship between different causes, along with the evidence for each cause. (p. 193)

All viable methods of "root cause analysis" in groups involve taking chains of events that would be too numerous and related in too complex a manner to envision in a common way by everyone involved in the process of they were simply described in words. Visual representations of trees of causes play a key role in all formal methods of "root cause analysis," including ARCA.

The trick to understanding the underlying message of this book and grasping the uniqueness claimed for the method is to understand exactly what he means by the *relationships between causes* and how the method forces you to think about evidence for causes. I don't personally favor the author's explanation for why his method is unique. His rules of causality seem awkward to me and I think there is a slightly different and clearer way to think of how his method works.

First, every effect is also a cause. There is no distinction between causes and effects except at the endpoints. Feedback loops of causes that have effects that ultimately lead back to the original cause are common in nature as well as human design and this method has no problem handling them. We start with the effect we are trying to explain and end with causes that either do not need to be explained or cannot yet be explained. This seems fairly straightforward to me for a causal chain, I'm not sure why the author feels it is unique to his method. I suppose it may be that commonly used methods tend to de-emphasize this aspect of causality.

Second, the method forces you to identify both actions and conditions that had to be in place for the action to cause the effect. This isn't useful for causal modeling as much as it is a useful trick for helping to identify all of the causes that might be relevant to solving the problem. Breaking the "Why?" question into an action and a set of conditions is seemingly somewhat unique to this method and helps avoid thinking solely in terms of actions or solely in terms of things that were in place at the time or characteristics of the situation. The theory is that it is usually *easier* to see actions that happened and less obvious what conditions had to be in place as well, but it is often *more useful* to identify the conditions because they tend to be more predictable and more controllable.

Third, the method forces you to have evidence for each cause, and takes an unusual and perhaps mildly questionable empiricist slant here. It distinguishes "sensed" from "inferred" evidence. "Sensed" of course refers to direct observation by someone, with as little interpretation as possible. "Inferred" refers to anything else. Assumptions and opinions represent doubts and must be investigated further, and just about anything that someone doesn't observe directly themselves has some doubt associated with it, whereas direct observation is automatically given very high credibility by the rules of the methodology.

This is a slightly odd sort of slant from my perspective because the credibility of eyewitness testimony in general is often in doubt (8), the reliability of inference is often not obvious, and because in my experience, hypothesis testing is particularly useful and important problem solving tool. Storytelling, much maligned and minimized in the ARCA method for good reason, is built on inferences from observations. However so do scientific theories and mathematical models arise from inferences. (9) The main difference is that science and math involve systematic linking and pragmatic testing of inferences rather than just weaving meaningful narratives. The author frequently asserts in different ways that storytelling negatively impacts problem solving, and defers to direct observation. I would instead argue that both are in doubt, and that their relative strength is not absolute based on source, but depends upon how inferences and observations support each other. One contemporary philosopher usefully compares the relationship to that of a crossword puzzle. (10) It is notable that the ARCA method and reality charting tool have no trouble accommodating different conceptions of the reliability of evidence.

Strengths

1. The philosophy behind the method is extremely inclusive organizationally. The idea is not to find the putative "true causes" but to gather enough information to produce effective solutions. So the method encourages as many people as possible to participate as fully as possible, rather than to argue over the right causes to focus on. That is perhaps the greatest strength of this method, its formal incorporation of diverse perspectives, allowing everyone to be heard and every perspective considered while discouraging the usual drama of competing narratives. Different definitions of the problem are managed by starting from different primary causes and creating separate charts. Since the framing of the problem often guides or constrains the search for a solution, I consider this a strength of this method.

2. ARCA explicitly includes the source and specifics of evidence for each cause, helping to identify and prioritize information gathering tasks to further validate assumptions and opinions, or reproduce observations. You are strongly encouraged to provide evidence for every cause, a process that I think can lead to a far more comprehensive yet focused data gathering effort than other methods.

3. ARCA allows for the explicit representation of causal feedback loops, which can be important factors in understanding what is happening. Many causal charts make this important relationship very difficult.

4. The technique of treating each effect as a cause and subjecting each to requirements for evidence, and explicit follow-up for finding further causes, intermediate causes, information required, or explicit reasons for stopping is a very powerful aid to causal thinking and to me is the core of both the method and the associated software.

Limitations and Criticisms

1. Potential overreliance on unreliable eyewitness testimony vs. discouragement of useful inferences. The process of hypothesis generation is a particular kind of inference where we consider more alternatives than we suggest, and as a result we understand that they are explicitly hypotheses. There is experimental evidence that generating hypotheses ourselves leads to a more realistic appraisal of their strength and less false confidence than inferences provided from other people. (cf. Derek Koehler) Other research supports the notion that our causal judgments are themselves inferences that often result from mental simulation. [This is a comment regarding the method in the book, which downplays the role of "inference" vs. "direct observation." It is a minor point in practice because there is nothing in the RealityCharting tool that forces you to evaluate sources of evidence in a particular way.]

2. Overemphasis on uniqueness of the method. This mostly refers to the tone in which the book is written, which I often found distracting and annoying, wherein it often makes the book sound like a sales presentation for the method and the tool rather than a treatise on effective problem solving. This point goes along with the scarcity of credit to other sources that for me would have greatly enriched my appreciation of the rationale for the method. The author relies too heavily on himself as an authority for his rationale to rate this a five star book.

3. Restriction of causal statements to very brief verb-noun and noun-verb format rather than sentences. I think this has some value in fostering clarity if you manage to find a phrase that everyone happens to understand in the same way. Still, in practice I found that it can be time consuming to come up with these compact ways of expressing causes, and that they were easier to misinterpret than full sentences would be. [This is also a potential issue in the tool because the rule checking tries to enforce it. However, it can be disabled if desired.]

4. Does not allow you to visibly categorize causes except into actions vs. conditions. This is an attribute of both the method and the associated software. This "limitation" is the intentional result of the author's core principle that he feels distinguishes his method in particular: categorizing causes is relatively useless compared to relating them. Categorizing causes is suggested in ARCA only if you are stuck finding plausible candidate causes and want to try thinking in terms of categories in order to help find candidates. Since this is intentional, it is a limitation only in the sense that someone familiar with other methods may run into it and have to think differently in order to use the method as intended.

Given that so far every substantive criticism I've had of this method is easily accommodated by minor changes to the method (as well as in the associated tool, with the exception of categorizing causes) I found this to be a particularly versatile method. It captures a number of key principles of problem solving, especially in groups, and provides practical and effective ways of compensating for our most problematic blind spots and biases. For the most part, I think understanding the principles of ARCA and RealityCharting would probably enhance most other methods as well as the method standing on its own.

A formal method itself is no substitute for improving the reflective intelligence of each problem solver, but I think this method could go a long way in any organization toward getting people to think more effectively, and especially for helping them communicate their best thinking to each other. I found very little to disagree with and much of value in this book and found the method easy to understand and apply.

References

(1) Brian Boyd on art and storytelling as biological adaptations which derive from play.

"On the Origin of Stories: Evolution, Cognition, and Fiction," Brian Boyd, 2009, Belknap Press of Harvard University Press

(2) David Perkins discusses the significance of domain-general principles for effective thinking, and why they can't be replaced completely by intelligence or expertise. A plausible research-based discussion of why we need help getting around our built-in individual blind spots.

"Outsmarting IQ: The Emerging Science of Learnable Intelligence," David Perkins, 1995, Simon and Schuster

(3) The psychological study of how we tend to answer questions of the form "Why ... ?" is called attribution theory. Probably a result of our bias toward storytelling explanations, most attribution theory has attempted to address attributions of people in social situations and sometimes products in the case of consumer research. A much smaller amount of research has been done on the perception of causality in other ways, such as in building causal explanations, and only a tiny amount has made its way into popularly accessible books.

"Causal attribution: from cognitive processes to collective beliefs," Miles Hewstone , Wiley-Blackwell

(4) There are not many modern books that deal relatively broadly with causal modeling from both a philosophical and mathematical perspective. One of the few is Pearl's text.

"Causality: Models, reasoning, and inference," J. Pearl, 2000, Cambridge University Press

(5) The non-conscious processes in realistic problem solving is addressed by a fairly sizeable body of technical literature by Dijksterhuis, Bargh, Gollwitzer, and others, and introduced in an accessible and practical way by Gary Klein.

"Sources of Power: How People Make Decisions," 1999, Gary Klein, MIT Press

(6) The problems exacerbated by group behavior under non-optimal conditions are introduced accessibly and briefly yet very broadly in:

"The Psychology of Judgment and Decision Making," Scott Plous, 1993, Mcgraw-Hill.

(7) The positive potential of groups under optimal conditions is discussed in
"Group Problem Solving" by Patrick Laughlin, 2011, Princeton University Press.

The common belief that direct observation is inherently more reliable than inference is based on two ideas that are at best only partially true: (1) the assumption that reports based on observation does not significantly depend on memory, inference, or explanation, and so is automatically free of distortions, biases, or storytelling, and (2) the assumption that inferences are roughly equivalently reliable under a wide range of conditions.

(8) On the reconstructive nature of memory for events, see:
"Searching for Memory: The Brain, The Mind, and the Past," Daniel L. Schacter (1997), Basic Books

(9) On the nature of inference and especially scientific inference, see:

"Error and Inference: Recent Exchanges on Experimental Reasoning, Reliability, and the Objectivity and Rationality of Science," Mayo and Spanos (2010), Cambridge University Press

(10) The useful crossword puzzle metaphor for the relationship of observation and inference is introduced in:

"Evidence and Inquiry: A Pragmatist Reconstruction of Epistemology," Susan Haack (2009), Prometheus Books

What, If Anything, Can Skeptics Say About Science? - some thoughts

The link below is to a thought-provoking post by Daniel Loxton about the relationship of science and skepticism. My thoughts are in reply #27, and I'm also posting them here. If you're interested in such things as I am, I also draw your attention to Jim Lippard's interesting epistemological observations in reply #10.

http://skepticblog.org/2009/12/22/what-if-anything-can-skeptics-say-about-science/

Summarizing Daniel's heuristics ...

1) Where both scientific domain expertise and expert consensus exist, skeptics are (at best) straight science journalists.

2) Where scientific domain expertise exists, but not consensus, we can report that a controversy exists — but we cannot resolve it.

3) Where scientific domain expertise and consensus exist, but also a denier movement or pseudoscientific fringe, skeptics can finally roll up their sleeves and get to work.

4) Where a paranormal or pseudoscientific topic has enthusiasts but no legitimate experts, skeptics may perform original research, advance new theories, and publish in the skeptical press.

I'd summarize my response by saying that I think these are good categories, but we mainly know them after-the-fact. They don't really get to the heart of what it means to be an effective doubter.

My thoughts in response:

I do find the idea of having heuristics for applying critical thinking appealing, but I’m uneasy about this particular very broad set and framework. There’s some question begging that seems inevitable when we draw up neat categories for observations.

Specifically, as heretical as it may perhaps seem to some, I don’t know that I agree that skepticism means a “science-based epistemology.” I think it means more a heavily empirical epistemology: observe and guess and test, rather than theorize and predict. Clearly, theory and prediction do play a central role in _science_, but not neccessarily in _skepticism_ pe se. To me they are closely related but not the same thing.

These categories in the post seem in part based on the underlying notion that expertise and epistemic value are closely related. To me, expertise does not have a straightforward simple relationship with our knowledge of the underlying phenomena. For one thing, it takes us in two different directions at once: (1) refined expertise organizes our knowledge of a domain along very specific lines – thus its power – and this also causes us to treat true anomalies as outliers to be ignored, and (2) expertise also makes us better able to see finer distinctions that lead to new discoveries.

So to me _expertise_ does contribute greatly to scientific discovery, but expert _consensus_ does not neccessarily define the underlying phenomena or by itself merit a different approach to experimentation. It is in the areas where we have the strongest expert consensus that the most interesting anomalies arise. It is often in testing the least likely conjectures, the ones outside the expert consensus, that we make the most interesting discoveries.

Before the discovery of metamaterials, there was an almost unanimous consensus that em radiation could not be guided around objects except in science fiction. The discovery had to be made by experts who could understand the significance of the discovery and had the tools for isolating it, but still it violated the expert consensus. Examples like this are rare, but I think well established, showing dramatically the two divergent ways that expertise influences epistemic value.

Dealing with the problem of interpreting an anomaly, if we knew ahead of time what the relevant domain of expertise was, and how it affected our understanding of the observations, we would already have largely solved the problem, thus the question begging of dealing with claims differently based on their relationship to the expert consensus, especially assuming that the expert consensus renders moot the scientific value of applying expertise to studying a putative anomaly.

I would argue that skeptics are at their best domain-general observers and experts in various areas of protocol and experimentalism and avid students of past lessons learned in studying anomalous claims in general. Consequently I think they are best engaged across the board investigating the circumstances of interesting claims – making use of scientific domain experts … knowing the expert consensus and taking it as the default … but not relying on the expert consensus by assuming it always makes anomalies less likely.

As a personal preference, I don’t think skeptics should be only in the job of confirming the consensus, but also in the job of questioning it reasonably.

kind regards,
Todd

The Expected Unexpected, a review of The Black Swan by Taleb

The Expected Unexpected - or, What can we learn from White Swans?

A review of The Black Swan, by Nassim Nicholas Taleb

Review by Todd I. Stark, 12/20/2009

Link to review on Amazon.

I came to this book expecting a clever but flawed argument for intellectual laziness or superficial thinking, another popular argument for "gut" or "intuition" or "Zen." Or perhaps a slick Gladwell-esque treatment of randomness. Perhaps a popularization of postmodernism or neo-pragmatism applied to financial markets, or a "Thriving on Chaos" (Tom Peters) for the 21st century.

This book is none of those things. Instead I found myself immersed in a very intriguing and deep intellectual journey into the roots of applied statistics and empirical science that had me thinking and taking notes prolifically. As readable as it is, this is not (or should not be) a quick read. Taleb is an erudite scholar but he uses his scholarship in the service of ideas rather than to accumulate impressive footnotes. His lectures are conversational but convey great weight.

I do find his tone somewhat arrogant in spots, oddly so. The half dozen or so people he finds interesting are worthy dinner companions, the rest of the world of intellegent mortals are pretty much fools who he chooses to stereotype and parody. Anyway, that's the impression I get. Contempt for audience usually works against an author. Still, very few modern authors can combine technical knowledge, originality, and readability the way Taleb can, and this to me kept me reading even when I imagined I was one of the many targets of the author's contempt.

Enough on style and impression, the content of this book is what makes it merit five stars. The core idea here is that we are creatures who quickly and easily sort things into categories and tell stories to make sense of them. Narration comes unbidden to us, but not skillful abstract thinking. This much of course we have heard before from the heuristics and biases school and behavioral economics.

Taleb's contribution is to point out a particularly broad implication of this principle, that our knowledge rapidly degrades when rare events are consequential. We explain them away and miss their importance. At the same time, we overestimate the impact of rare events for arbitrary reasons when they really have no consequence.

That would be of mostly academic interest except for one thing. The key to Taleb's overall argument is his claim that the impact of rare events is domain-specific. That means we can learn to distinguish domains where: (a) classical risk statistics apply ("Mediocristan") from (b) those where rare events and winner-take-all competitions dominate ("Extremistan").

In "Mediocristan" domains, classical decision theory in principle should help us (although Taleb seems to feel that these domains are few and far between among things we really care about). In "Extremistan," Taleb advises, we should take steps to protect ourselves from rare adverse events and use diverse aggressive risk taking to exploit rare positive events.

The idea here is that we are betting that something rare will eventually impact us in these domains, even though we can't know what specifically it will be. The point is that in domains where likelihood can't really be calculated, we should focus our efforts instead on the impact rather than guessing at the probability. Taleb doesn't make this argument lightly, he explains the limits in our predictive ability in an understandable way yet he also takes heed of technical details in his arguments.

"Extremistan" consists of all of the domains where scaling and nonlinear accumulations occur, or rapid deviations from small differences in initial conditions. In line with complexity theorists, Taleb finds these effects pretty much everywhere of interest to social scientists, economists, and people in general. He suggests that we might be able to use scalable non-linear maths to get a rough idea of what to expect in some domains, but that we need to remain careful of the illusion of mathematical predictive power.

Biological values like height and weight and average life expectancy are areas where the normal curve applies, and perhaps mean failure rate of parts in engineering. But when important things like money or fame or success can accumulate virtually without limit and often for arbitrary reasons like contagion, the assumptions of Gaussian distributions are just not relevant. At least that's how I understand Taleb's claim that the normal distribution is a great intellectal fraud: it is applied confidently to things where it has no place being applied.

With one strange but perhaps unavoidable exception, Taleb usually follows his own advice. He considers theories for storytelling purposes and explanation but doesn't take them seriously or depend on them for his argument. He does use storytelling quite a bit to make his point, even though his argument is largely about the unreliability of our stories. He seems to be saying that we automatically make sense of events through stories, so it takes a story to help us understand the argument against relying on storytelling.

I suppose there is a touch of the postmodernist skepticism of narratives in The Black Swan, but the possibility of having specific strategies for dealing differently with specific domains is an intriguing and welcome update to that tradition, bringing it perhaps more in line with the critical rationalism and conjectures and refutations of Karl Popper and the scientific skepticism of the classical pragmatists than with the modern cynics. Taleb is not rejecting theory entirely, but he certainly prefers direct experiment and aggressive risk taking (in strategic areas) to bland assumptions of predictable statistical likelihood and the reliability of knowledge.

For me this book ties together a lot of diverse ideas very nicely in an original and interesting way, and yet stays on target with its message.

Very impressive effort and a very worthwhile read.