Todd's Review of Range by David Epstein

Range: Why Generalists Triumph in a Specialized World
by David Epstein
New York, Riverhead Books, 2019

Review by Todd I. Stark

This is a beautifully written and well justified discussion of the various specific things that are not taken into account by the widespread cultural emphasis on early specialization for success and our popular model of performance in terms of domain-specific expertise.

This takes the form of a single conclusion which I would paraphrase as: "we need to be able to play and explore widely and to color outside the lines for a while in order to become very good at solving the difficult problems we later encounter.  But our cultural obsession with specialization pushes counter to that." 

There is a constant tension of the author's confidence in his conclusion that generalists are uniquely valuable and desperately needed and his recognition that he is fighting an almost Quixotic uphill battle against powerful cultural trends and incentives for specialization. 

What he means by specialization and the factors closely tied to it:

1. Head Start: Encouraging children from an early age to narrowly pursue things they seem talented at or have an interest in.
    
2. Domain-Specificity: Training with heavy emphasis on the specific narrow range skills we know we will need in the target environment and assuming far transfer of skills from other activities will be limited or non-existent.
    
3. Disciplinary Focus: Viewing learning as consisting of accumulating facts and theories specific to a particular field or subfield of study in order to become highly skilled at working in that narrow field.
    
4. Persistence:  The idea that we should identify a passion early and stick with it no matter what because it’s what we’re good at and enjoy and so can become successful at it if we manage to persist.
    
5. Fast and Efficient Short-Term Learning:  The assumption that we are learning better when we feel familiar with the material quickly and that we are then learning more efficiently. 
   
   
    
   Against those powerful and popular specialization factors, Epstein presents several compelling lines of evidence:
    

1. Domain-Specificity varies with Kind vs. Wicked Learning:  The argument for early specialization and domain-specificity is based on the observation that we need a long period of deliberate practice to accumulate the patterns and skills specific to performing in that specific activity and that practicing or exploring other activities is unlikely to do anything helpful for our performance in our specialty.  Epstein counters that on closer inspection we find a crucial distinction between different kinds of domains and learning environments, where in some of them deliberate practice reliably makes us better but in others deliberate practice either helps much less or can even make us perform more poorly under some conditions.  So not all domains or learning environments are equally specific and the head start is not equally helpful in all activities. 
2. Creative Performance comes from early exploration and interdisciplinary learning:  Given the domain-specific view of expertise we tend to assume that in order for someone to perform at a high level in any activity, since they need expertise, they need to specialize in that activity.  Epstein counters that when we focus specifically on creative performance, we find that deep expertise can be invaluable but is not enough.  In order to come up with truly novel solutions to problems we need to make use of analogies that cross different domains while sharing deep structural similarities.  That means being familiar with a wider range of ideas and ways of thinking than just those in our specialty, and so Epstein says creative performance is found more in people with broader backgrounds.  Epstein argues that outstanding creative performance also tends to be associated with early exploration of different activities more than with early specialization. 
3. The Efficiency We Perceive from Narrow Immersion is Very Often Illusory: We tend to assume that when we feel more familiar with the activity or material that we are learning it.  That’s part of the strong intuitive appeal for immersion in an activity comes from, it feels like we are learning more when we are more immersed.  Epstein argues that the evidence from learning research show quite often exactly the opposite, that the learning we think we are doing under conditions of immersion is either much less or much shorter lived than we assume.  Robert Bjork’s concept of “desirable difficulty” in learning and the evidence base behind it plays a central role in this argument.   This, Epstein argues, tells us that “slow learning” which helps us make new connections between a wider range of experiences is much more conducive to learning in the long run than fast, efficient learning from immersion in a narrow subject matter.
4. Match Quality is Not Necessarily the Same as Early Passion: Part of the argument for early specialization is based on the assumption that people have certain interests and talents from early on and if they can find something that matches them well and start early, they can align their passion with a successful career in that activity.   Epstein argues that what we know about lifespan development tells us that people’s passions are not so fixed or narrow and finds a number of cases of exceptionally successful people who spent their lives exploring and trying different things before finding a match that was truly satisfying and successful for them.   

Range is an appeal to encourage exploration in our lives from early on and for experimenting and experiencing broadly in our learning, even though it may seem to be inefficient or slow.  Epstein does not deny the immense value of long deliberate specialized practice in “kind” domains or the value of having deep specialized experience in some areas, but he has also made a passionate and well-argued case for making better use of a completely different dimension of performance.  A dimension rooted in longer term developmental outcomes, more exploratory or playful learning, and an ongoing search for ever better matches between our interests and abilities and our activities. 

Review on Amazon:
https://www.amazon.com/review/R3I57PTPZXSYU4/ref=cm_cr_srp_d_rdp_perm?ie=UTF8

Book Review:  Ungifted by Scott Barry Kaufman

Todd I. Stark  5/29/2013

Link to review on Amazon

Intelligence turns out to be a difficult topic, for reasons that aren’t at all obvious at first. Our understanding of mental ability has been captured in several independent threads of research that are surprisingly oblivious of each other for the most part.  Our stereotypes of the gifted and the ungifted often miss the details of what is going on.  The study of individual differences in general, while useful, doesn’t just de-emphasize, but actually systematically misses some of the most important things going on when people become exceptionally successful contributors.  

The author of Ungifted is well situated to make an important contribution to our understanding of intelligence.  He has made a deep academic study of a wide span of existing research programs, he has worked directly in collaboration with many of the leading researchers in several related fields and he has passionately engaged these ideas since childhood when he became  painfully aware of the academic sorting process for giftedness, and he himself is a wonderful example of many of the principles that emerge in his new book.  This is not another book that just starts out with a vague progressive vision of education and ability that everyone is a potential “genius” and then fills it in with wishful thinking.  No, this is a book that dives very deeply and realistically into the literature of psychometrics, heritability, cognitive neuroscience, and expertise.  It looks closely at patterns from the span of phenomena of human differences, including savantism, prodigy, autism, schizophrenia, personality, g factor, motivation, and creativity.

Ungifted is so compelling, rich, and significant a book for me that most of it was well worn by the end of the first day it arrived. What makes this book so rich is the unique combination of personal passion, strong but not intrusive scholarship, deep domain expertise, clear analysis, and all tied together with an original new and constructive perspective. Hard to ask for more than that from a non-fiction book. Every chapter is a stimulating lesson in an important topic that brings together a broad range of data, ever heading toward the book’s conclusion, nothing less than a complete rethinking of human intellectual ability, consistent with considerable evidence gathered along the way.

The most impressive and distinctive aspect of the author's thinking is his consistent and effective use of perspective-taking. In each case where a controversy is identified, each side is explored with great depth and sympathy to understand what its advocates understand that those on the other side seem to miss. It isn't difficult to do that for the side of an argument we agree with but it is an impressive achievement to do it for different sides and then to synthesize the perspectives into an overlapping understanding. This ability is particularly relevant to topics like IQ, giftedness, and learning disabilities, where sharp controversy shapes conversations at every turn.

Ungifted is about how we conceive of mental ability in general. Some people manage to accomplish much more with their mind than others do. There’s no escaping that basic observation, nor the fact that it has immense significance for political and educational thinking. What is the difference that makes a difference? The answers we get depend on the kinds of questions we ask.

Ungifted is in part the story of how the important questions have changed over time and why. There seem to be two tragic errors that we’ve fallen into historically. For one, we’ve often ignored the differences between us and tried to force everyone into cookie-cutter educational molds that well serve only a minority of the people they are intended to serve. Most of us appreciate this personal plight, as does the author in his "subjective" voice and personal experience throughout the book.

The other tragic error is ironically the reverse. In trying to appreciate the differences between people we’ve taken the opposite extreme of becoming obsessed with stable, predictive individual differences. We test ourselves and compare ourselves with each other and we look for the numbers that tell us who deserves what because we assume we are identifying potential. The author appreciates the motives and sometimes successes of this approach when done well, but the focus of Ungifted is on how we can do better.

It is the tragedy of our obsession with individual differences that Ungifted in the author's "objective" voice most eloquently addresses. We assume that testing people against each other will tell us how to best teach each person by identifying what makes people different. Ungifted describes in great detail and punctuated by the author’s own personal life story why that well-intended approach has failed us time and time again.

It isn’t simply as many politically motivated accounts would have it, that IQ has no meaning, or is too culture bound, or just measures test taking ability. IQ and similar kinds of tests when given and interpreted intelligently provide a useful and well-validated way of identifying the lion’s share of variation in human intellectual ability across a wide range of situations and this has some very real correlations with meaningful life outcomes. The problem is not that the tests are useless but that they have come to be misconstrued as if they measure a single stable ability that resides in each person and predicts what that person is capable of accomplishing. People who do well in IQ tests do tend to be smart people in general. But so are many people who do more poorly in IQ tests, and doing well in IQ tests doesn’t provide any guarantee that we also have persistence, motivation, or other qualities so important to making the best of our abilities.

A number of theorists have made useful additions to the body of well validated tests, but these are still tests of static abilities and they don't solve the most basic problem identified in Ungifted. The reasons the individual differences approach has failed us in the broad global sense that we have tried to apply it are that while effectively explaining variation between people in the same population, it has not taken broader environments into account, it has not taken the course of development into account, and it has not taken the dynamic differences into account that make the most difference in human lives over time.

A number of fundamental misunderstandings of heritability, genetics, development, and psychometric research have been exploited, often through politically motivated movements, to obscure the larger vision of intellectual ability. Ungifted makes a serious bid to help correct these fundamental misunderstandings.

Do some people have more of a critical trait or traits from the start, or do some people learn more from their experience, and in either case how much can we influence our abilities over the course of our life? The traditional dialectic of nature vs. nurture seems to have its own unavoidable groove in our thinking that we rarely manage to escape. Yet as long as we have been studying human ability scientifically, there has been evidence that the dichotomy is inadequate. 21st century research has given us some useful insights into the specifics. Ungifted summarizes the most important lessons from a wide range of data about human abilities, and the author is particularly careful to distinguish his subjective passion for the subject (which is often in evidence) from his more detached coverage of the data in the various fields.

Ungifted starts out with a concise summary of principles of the 21st century picture of development, setting the background for the rest of the book. The concept of traits is explored, and the patterns by which they develop.

Then we have a tour of the ways we have tried to measure human mental ability and our reasonable motivations for the various testing innovations over time. From the measurement of ability, we then see how measurement slides into the sorting of people into categories for practical purposes and the allocation of finite educational resources. We see the well-motivated practical reasons for labeling people, but we also get a sense of the often tragic real world limitations of that way of thinking.

We are introduced in an understandable but expert way to the real strengths and weaknesses of IQ testing, and to the best available model of how its scales map to specific cognitive abilities. This prepares us to begin to understand the different ways that "giftedness" has been defined and measured and why we are still so far behind where we need to be to cultivate the best in every individual. We also are given enough background to begin to appreciate the unique challenges of being different, whether perceived as higher or lower in ability than others around us. Ungifted artfully blends a sympathetic understanding of the needs of researchers and testers with those of teachers, and the diverse individuals just striving to do their best.

Then we are introduced to the core concept that distinguishes this developmental reframing of human mental ability, the concept of engagement. Engagement brings together the factors that distinguish the learnable, experiential aspects of intelligence from those that seem particularly stable. Engagement is the hinge that swings the big door to individual potential. Engagement is built on a number of factors that are particularly malleable and context-dependent, so it is of particular interest to the way education is done. Once we have a sense of what engagement is about, we take a fresh look at human abilities in terms of what is known about their development over time. The role of engagement over time in development begins to become clearer as we tour through the critical concepts of intelligence, creativity, talent, and expertise, and begin to see how they each relate to development over time. At the end, the key points learned along the way are summarized to give the outline of a new theory of intelligence.

For me the theory of intelligence introduced here is distinguished by two critical characteristics: (1) it emphasizes what happens in the individual over time rather than differences between people, and in so doing draws on different kinds of data, and (2) it is synthetic in spirit, emphasizing multiple ways of achieving the same outcomes by drawing on different resources, rather than looking for additional ways of distinguishing the abilities of different people. These two characteristics make this theory very different from most of the alternatives that are intended to address some of the same gaps in our intelligence models, alternatives such as “multiple intelligences,” “emotional intelligence,” and so on. Rather than just identifying more things that we think might be missed by IQ testing, and turning them into new sources of labeling and categorizing, the personal developmental theory of intelligence assumes that there are many different components to be identified, but places them into an overarching biological framework where ability is developed over time by identifying, selecting, modifying, and constructing niches suited to the thriving of the individual.

Several important shifts of emphasis emerge:

1.      Away from reliance on studying stable individual  differences and toward the details of person-centered development

We have focused primarily on measuring stable individual differences, whether “general intelligence” or other kinds of “intelligence” or personality in order to support research and allocate finite educational resources.  What this approach misses is the details of development within each individual over time.  It turns out that these two approaches, individual differences and person-centered, are not just different but produce incompatible results.  So this is a very important source of new information about how mental ability arises.  It is common for authors to point out that nature and nurture are an archaic dichotomy and that it is the interaction that matters, but the details are generally left vague.  There’s a need for specific research programs that focus on the development of ability over time.  The developmental approach is not just a detail, it is a separate source of crucial empirical data.

2.      Away from viewing the positive manifold of abilities on tests (“g”)  as a single ability in each person, and toward understanding its value in conveniently capturing most of the variation an array of mental abilities that collectively underlie many different kinds of tests.    

3.      Away from seeing intelligence as a number or even a trait, and toward seeing it as the adaptive fit between the individual and their environment by finding, selecting, shaping, and creating niches they can thrive in.

4.      Away from focusing on cognitive skills in isolation, and toward consideration of the motivations, strategies, and experience that turn those skills into practical abilities over time.

5.      Away from the focus on being able to measure abilities that represent potential in a brief test, and toward finding the best way to engage and cultivate each person.  Active engagement with the world and ability are inseparably intertwined in a mutual feedback process over time.

6.      Away from focus solely on controlled cognitive processes underlying reasoning, and toward better understanding of the role of both controlled and spontaneous processes in intelligence.  Being smart involves flexible use both controlled and spontaneous mental processes, and using the right resources when needed, rather than relying on one to the exclusion of the other.

7.      Away from fixed rules about how long it should take to become good at something or what special levels of ability provide thresholds, and toward seeing our “readiness for engagement” as a better indicator of potential.

8.  Away from seeing intelligence and expertise as independent (and one inborn and the other learned), and toward seeing the overlaps between them as cognitive abilities,personality, and motivation support the acquisition of expertise, and cognitive expertise is part of what shows up in testing for ability.  One of the most remarkable findings in the book that links intelligence and expertise is that chunking in memory, a key aspect of organizing knowledge in memory involved in expertise, activates the brain structures involved in fluid reasoning, a central component measured by tests.  

The resulting view of intelligence doesn’t see everyone as equal by any means, it seems unavoidable that some people will not be able to excel at some things relative to other people.  Stable individual differences do not somehow disappear because we shift emphasis to the person and their development.  However we do begin to see the real value of changing the nature of education to focus on the fit between people and niches rather than selecting people for special treatment via brief tests and subjective judgments of merit.  Several practical working examples of programs that successfully accomplish this change are described in the book.

I place this book alongside two others in a trilogy that for me represents a broad understanding of the nature of mental ability as it is best envisioned at the current time.

1. "Surpassing Ourselves" is about how outstanding performers learn differently.  It shows expertise as a distinctive way of learning rather than just an endpoint of domain specialization. By comparing the same person over time as they develop, rather than just comparing novices with experts, we get an understanding of the process by which people become smarter.  This is very similar to the shift made with intelligence in Ungifted, but applied specifically to expertise and shows again how the shift to a process perspective captures additional important information.  Surpassing ourselves also introduces  the reinvestment perspective, which looks at the growth of ability in terms of becoming more efficient over time and then reinvesting the saved time and energy in new learning, which seems to be a big part of how people who ultimately become exceptional learn differently from those who do not.
http://www.amazon.com/Surpassing-Ourselves-Inquiry-Implications-Expertise/dp/0812692055/ref=sr_1_1?s=books&ie=UTF8&qid=1369837497&sr=1-1&keywords=surpassing+ourselves

2. "Outsmarting IQ" is about learnable intelligence.  It shows how experience, stable cognitive abilities, and strategies work together to navigate realms of knowledge and let us apply our knowledge.  It gives a good account of the role of strategies, including learnable strategies and cognitive expertise, in trading off between cognitive abilities in order to make the best of our existing stable cognitive abilities. 
http://www.amazon.com/Outsmarting-IQ-Learnable-Intelligence-ebook/dp/B001D1Y8Z8/ref=sr_1_1?s=books&ie=UTF8&qid=1369837528&sr=1-1&keywords=outsmarting+iq

3. "Ungifted" combines the intrapersonal process perspective with a developmental model and an overall hierarchical model of cognitive abilities, consistent with the other two books in this list and yet going way beyond them to explain in much greater detail where “IQ” and other tests scores come from and what they tell us, and how stable cognitive abilities, expertise, creativity, and personality all interact to produce intelligence. 
http://www.amazon.com/Ungifted-Intelligence-Redefined-ebook/dp/B00B3M3UME/ref=sr_1_1?s=books&ie=UTF8&qid=1369837581&sr=1-1&keywords=ungifted+intelligence+redefined

Update 5/29/13 1PM:  Link to abbreviated GoodReads version of review:

Ungifted: Intelligence Redefined by Scott Barry Kaufman
My rating: 5 of 5 stars




View all my reviews

The Power of Expertise

The Power of Expertise

Who Ya Gonna Call?

Let’s say you’re working on your computer and it starts acting strangely. You get errors that don’t understand or it crashes for no apparent reason. If you aren’t sure what to do at first, where will you look for help? You might perform a web search for the symptoms to see if it’s a known problem and other people have solved it before you. You might run some diagnostic program or an antivirus scan because those are the tools you happen to have.

If you can’t fix it easily and you aren’t confident with computers you’ll probably start looking for help from another person at some point. Who? If it were me, I probably wouldn’t head down to the local college and find the top honors student or someone in the local Mensa chapter. I probably wouldn’t look for someone with great SAT scores or someone really good at Sudoku or even a master electrician. I’d look for someone with a lot of experience with computers and a proven track record fixing them. I’d look for an expert, and an expert specifically in that area, not just a smart person or an expert in a related area.

I stacked the deck a little bit with this question, because I picked a problem that is probably going to be technical in nature. That is, it seems like it will require some specialized knowledge to solve because it involves computers which are complicated devices that are a little mysterious to the average person and far less so for someone who has worked extensively with them.

It turns out, though, that my guess is pretty accurate for a wide range of fields, not just highly technical ones. Knowledge about the job turns out to be a far better predictor of performance than how high our IQ is or any other general disposition, not just in certain kinds of jobs but across a wide range from complex technical work to manual labor.[1] Just as I’d rather have a computer expert help me rather than my friend with an astronomical IQ, in most cases I’d prefer someone who has job experience rather than someone very smart but inexperienced. And I can point to research evidence that supports my preference.

Seeing Differently vs. Seeing More

Even in many areas where we would tend to expect pure reasoning ability to play a large role, it turns out that on average experience tends to win out consistently over any more general ability or measurement we have come up with.

The research that inspired the modern study of expertise began with the game of chess. Think about chess for just a moment. Chess is an activity with a small number of relatively simple rules. Yes chess has the reputation for being a difficult game. But that’s not because chess is hard to play. Nearly anyone can learn the game. It’s because we soon discover that differences in individual ability are immense.

The difference between someone who plays chess for fun who doesn’t study the game seriously, and an average tournament player, is like night and day. It doesn’t seem like much of a competition most of the time. The difference between an average tournament player and a strong one is just as large, which is why there is a rating system.

Ratings allow people of similar ability to play relatively evenly, or to estimate handicaps as they do in golf. The difference between a strong player and a master is similarly imposing as is that between the master and a grandmaster, and between the average grandmaster and a world champion.

How can a game with a handful of simple rules end up with people playing at such astronomical differences in ability? This was the question that intrigued early researchers trying to figure out how people solve problems. The obvious answer is that the stronger players must be seeing more on the board. But what are they seeing differently?

When most of us look at the chess board we see a collection of pieces in different places that are allowed to move in particular ways. We know what we have to do to win; we have to trap the king. We also know some ways to accomplish that. For example we can capture the opponent’s pieces so we have a bigger army, and we can harass the opponent’s pieces so that they are forced into a less defensible position, allowing us to attack the king. Everyone who plays the game, even for fun, knows these things. Still most of us pretty much have to guess at how to get from some arbitrary position to that result.

If I move here, I’ll attack this piece, but how do I know that my opponent doesn’t have some better move in response that is even stronger? More insidiously, is that move by my opponent actually setting up a surprise for me later? If so, what are my options? These kinds of considerations quickly lead to the very intuitive notion that being better at chess is really about calculation, about being able to imagine a lot of different moves, and what might happen if we made them, and keeping track of all that imagining. The better player must be seeing more moves on the board, figuring out what the options are more accurately, and then predicting the outcome.

This is indeed how early chess software played the game well. It looked at the possible moves, looked at the possible responses to each move, evaluated the resulting positions, and chose the move that seemed to give the best outcome based on what the opponent was able to do. The trouble was that trying to do this more than a couple of moves ahead turned out to be a very demanding calculation. More demanding than even the most powerful computers could handle. Researchers were curious as to whether seeing more moves in their mind is really what good players were doing.

Maybe the human brain is really that much more powerful at calculation than we thought. Or maybe the brain is doing something else entirely?

In a pioneering study of chess players in the 1940’s[2] a Dutch psychologist found the surprising answer. I say his work was pioneering not just because it was early but because it led to entire fields of research based upon it and validating his basic findings. The most compelling and surprising findings:

...Weaker players examined the same number of moves as stronger players, and equally thoroughly (!)

...Stronger players could recognize an actual game position far better than weaker players.

...Stronger players were just as bad as weaker players at recognizing an arbitrary configuration of pieces.

This may not seem so earthshattering at first, but think about the implications. Experts at chess consistently beat weaker players, but without examining more moves and without examining the outcomes of those moves more thoroughly. They aren’t “looking ahead more” and they aren’t “reasoning better” and they aren’t even remembering more in general. They do remember more about chess in a sense but not because they have a better memory. And looking ahead is important, but not by keeping track of moves. Their ability is a result of their mind being better trained to remember chess configurations in particular and to use that knowledge quickly and efficiently to evaluate moves.

So what are chess experts seeing that the rest of us aren’t? They aren’t seeing more moves ahead, they are seeing the board in terms of chess configurations instead of seeing it in terms of individual pieces. Their mind has been trained to see meaningful configurations of pieces instead of individual moves. They are not seeing more per se, they are seeing differently. They are seeing in terms of larger and more meaningful groupings. Experts with extended experience acquire a larger number of more complex patterns and use these new patterns to store knowledge about which actions should be taken in similar situations.[3]

The result is profound. We have a game where a few simple rules results in an incalculably large number of possible sequences of moves. But we become good at this game of many, many moves not by thinking about more moves but by thinking in terms of larger patterns: patterns of pieces rather than movements by individual pieces.

Through practice, chess masters have trained their mind to recognize the unique meaningful patterns that apply to their game. Further, the ability to learn to recognize new patterns (along with a huge capacity to remember them) seems to be something we all possess, not just chess masters. It is a fundamental principle of learning, at least learning to be a chess expert.

Even more interesting, we don’t recognize this as knowledge, in the sense of things we recognize that we know. I know that I know some things. I know that I know all sorts of facts like the capital of some of the U.S. states and the number of sides in a triangle and Newton’s formula relating force and mass and acceleration. These sorts of things are considered explicit knowledge.[4]

Chess masters can’t write down most of the patterns they know, both because those patterns are so vast and because they use them without thinking about them. The patterns they learn become part of their chess intuition in a manner of speaking. A common technical term for this is tacit knowledge.[5] We use tacit knowledge in our thinking without realizing that we are using it. This is why it took focused research to discover what was going on in the minds of chess masters.

Tacit knowledge becomes part of our perception. Chess masters see the board differently; for example they often immediately see positions as good or bad without having to do the kind of analysis that the rest of us would have to rely upon.[6]

Tacit knowledge is also used automatically in our thinking. When chess masters guess at the best move in a given position, their guess is informed by their vast database of tacit knowledge, so it is very different from the guess made by a weaker player. Experts make better guesses in their area of expertise. This is what I mean by their “chess intuition” above.

Trained Intuition and Better Guesses

You might be wondering at this point why I’ve spent so much time talking about chess experts. Or you may have guessed the answer. The most interesting conclusions from the research on chess masters are by no means limited to chess masters. Very similar or consistent results have been obtained across a staggeringly wide variety of fields from physical pursuits like wrestling and ballet to intellectual subjects like calculus and philosophy to artistic activities like painting and violin playing, to a wide variety of everyday jobs, to oddball activities like picking the winners at the horse races.[7] Even among scientists, where the role of abstract reasoning is particularly central and the subject matter particularly challenging, productivity doesn’t seem to be predicted on the whole by supposed general ability measures such as IQ.[8]

The chess findings are a particularly useful rhetorical device here because chess seems like it should be so dependent on reasoning and analysis. It turns out that experts analyze chess positions with the help of a vast mental database of chess configurations that apply without any recognition that they know them. The resulting perception and memory of the board just seems natural to them as a result of practice. Examined closely, in spite of its natural appearance for some people, the effortlessness of deep expertise seems to be an extreme kind of skill acquisition[9] far more than an expression of talent.

Even if you interpret all of these findings from different fields very conservatively, collectively they still tell us something of tremendous importance about how we become good at things. We modify the way we perceive the activity. In effect, we train our intuition about the activity.

In all of these activities, researchers have found that time spent in the activity lets us acquire a new way of perceiving patterns in that activity that let us transcend the limits of our working memory and sequential reasoning capacity. That’s why expertise consistently outperforms IQ or working memory capacity or other general measures as a predictor of performance in virtually every activity that has been studied so far. And expertise is not just specialized knowledge or skills; it is also more importantly an accumulation of organized tacit knowledge that lets us make better guesses.



[1] (Hunter, 1986)

[2] (de Groot, 1965)

[3] This has been the most common interpretation of the chess research findings amongst expertise researchers, based on the influential theory of Chase and Simon. (Chase & Simon, 1973), (Simon & Chase, 1973)

[4] “Explicit knowledge,” basically just means things we know that can be easily identified and written down. The descriptor declarative is sometimes used as well, meaning that we can declare it.

[5] In contrast to “explicit knowledge,” this is often referred to as “tacit knowledge,” meaning things we know but we can’t easily express, especially things that support action. Tacit knowledge is usually assumed to be useful for doing things more than for taking part in our conscious reasoning processes. The descriptor procedural is sometimes also used for tacit knowledge because we think of it as involving procedures for doing things rather than declarations about things. For this reason, a common rule of thumb is that tacit knowledge refers to “know how” whereas explicit knowledge refers to “know that” (i.e. I know that grass is green). The casual rule of thumb is troublesome because we don’t really know how we do those things we call procedural, the usage of the word “know” in “know how” is very different than the word “know” in “know that.”

[6] Following the pioneering chess research, research into other areas reinforced the same finding: expert performance depends heavily on a large accumulated memory of patterns that give us a different “intuitive perceptual orientation” to tasks. “Experts can ‘see’ what challenges and opportunities a particular situation without affords.” (and without doing any analysis) (Perkins, 1995, p. 82)

[7] One of the leading and best known figures in the study of expertise is K. Anders Ericsson, whose research encompasses a particularly wide range of fields. An excellent and accessible overview of work in diverse areas of expertise research is Ericsson’s edited collection: The Road to Excellence (Ericsson, 1996).

[8] (Taylor, 1975)

[9] (Proctor & Dutta, 1995), (VanLehn, 1996)

Does adding men to a group make the group dumber?

Does adding men to a group make the group dumber?

New preliminary finding reported at the Harvard Business Review:

 Adding more women to a group may make the group smarter.

Previous related finding:

 Collective intelligence of groups is roughly independent of intelligence of individual members

Previous assumption:

 A more diverse group is better than a less diverse group.

Surprising new possible implication:

 Gender could potentially be more important than diversity in collective intelligence of a group, with women adding to group intelligence and men detracting from it.

Limitations:

 This is a preliminary finding not yet a robust one. It has been reported in two studies by the same team under a limited range of conditions.

 Collective intelligence is not the only thing of importance in group problem solving.

 The effect of gender may be through process factors that potentially could be achieved in other ways as well if isolated.

 The effect has not been tested very far yet at the extremes.

 Measured of collective intelligence are not as standardized as measures of individual intelligence, which are themselves of mixed value in actual problem solving.

Additional thoughts:

This is a preliminary finding but it seems plausible to me and if it is borne out robustly then combined with the previous finding that collective intelligence is roughly independent of individual intelligence of group members, this seems to mean that (1) group process is more important to collective intelligence than individual insights, and (2) that group process depends strongly on gender.

The first still amazes me but I think it may be true, and if so, the second one seems even more plausible. My impression is that there is a lot less constructive interaction between men than between women in group processes in general.

My own speculation is that while both experience a mixture of task and relationship tension in groups, women leverage the relationship tension more constructively, whereas men tend to align more quickly when they agree and to stonewall more strongly when they disagree. Men usually seem to be more likely to drift toward a goal of satisficing (coming to the first satisfactory solution) and then disengaging, whereas women seem to interact in a more prolonged way. I'm guessing that this contributes to collective intelligence as it is being measured here in some way.

refs:

Article on MindHacks: http://mindhacks.com/2011/06/24/a-dose-of-female-intelligence/

Interview at HBR: http://hbr.org/2011/06/defend-your-research-what-makes-a-team-smarter-more-women/ar/1

Chart (The Female Factor): http://hbr.org/2011/06/defend-your-research-what-makes-a-team-smarter-more-women/sb1

Book Review - Apollo Root Cause Analysis: A New Way of Thinking (?)

Book Review - Apollo Root Cause Analyis: A New Way of Thinking Book by Dean Gano, Apollonian Publications, 2008. Review by Todd I. Stark Link to Amazon review.

Very useful principles for preventing bad things from happening, April 11, 2011 By
Todd I. Stark "Cellular Wetware plus Books"

This review is from: Apollo Root Cause Analysis: A New Way of Thinking (Paperback)

How can we prevent bad things from happening, and how can a formal method help us in this quest? That's the topic of this book.

"By understanding the cause and effect principle and creating a RealityChart, your understanding of what constitutes reality will be changed forever ... allow you to see a reality that was previously beyond your comprehension." (Apollo Root Cause Analysis, p. 3)

"... the problem of a linear language in a nonlinear world while accommodating the simple human mind has been a challenge for the ages. I believe this challenge has been met with Apollo Root Cause Analysis." (ARCA, p. 176)

The importance of the topic covered by this book is immense, so for my purposes, I'll forgive the author his amusing enthusiasm for this own method as seen in the quotes above and try to determine what may actually be truly useful in it rather than dwell on what may or may not be unique about it. I will be referring to this book and to the Apollo method in this review. The method is also implemented with associated RealityCharting(tm) software which is outside the scope of this review, except where explicitly mentioned.

Problem solving well is one of if not the most critical factor for human success across a wide range of activities. One big part of problem solving is explaining *why* something unexpected happened. I you think about it, anytime something unexpected happens, we generally want to know why. We do this naturally, automatically, and effortlessly. We can't stop ourselves from doing it to at least some extent. Sometimes the explanation seems obvious and sometimes it seems very elusive. Even when the explanation seems obvious, there is often more going on of importance than we realized. We do this seemingly because especially if it's a bad thing that happened, understanding why it happened potentially helps us predict whether it will happen again and perhaps prevent it from happening again. That's important to us as individuals as well as to organizations. It's so important that we already do it all the time. We also tend to assume that we're very good at it, I think. My experience is in agreement with the author of this book, people are in general not nearly as good at solving problems as we think we are, at least when the problem become complex and involve multiple people and organizations.

We have a powerful natural ability to make sense of events that happen around us by identifying people, places, things, and weaving a story around how they interact with each other. (1) The power of storytelling to make sense of events is also its tragic downfall when we need to understand how things happen in great detail. Our temptation to tell stories actually gets in the way of understanding what specific things need to happen in order for other things to happen, and that's the sort of understanding that is needed in root cause analysis. Getting past our natural but distracting abilities to use the right tools effectively is the greatest value of an effective formal method. We often look to formal methods to systematize our thinking in general, but problem solving outside the academic realm of worked problems is often not amendable to being constrained by formal methods. Instead, formal methods provide their real benefit in forcing us to look past our own blind spots at key points during problem solving. To his credit, the author of this book seems to have grasped this important point very well and applied it effectively.

The most problematic blind spots in our thinking are those that aren't avoided by intelligence, education, or domain-specific expertise, skills or knowledge. The most problematic blind spots are actually natural abilities that serve us well most of the time. One of the most important examples is that we tend to use storytelling to construct a plausible sequence of events leading to an outcome, putting people into the center of the action and making the events meaningful to us. There are a number of reasons why this natural and compelling process is a bad idea in problem solving.

First, since a story starts with the putative "root cause" and then proceeds to its effects, it tends to assume a single cause. Events don't have a single cause. If our goal is to prevent bad things from happening, we'd rather identify as many relevant causes as possible, and potentially address each one as makes sense.

Second, storytelling tends to center around characters, their motives, and the things they do. The motives and behaviors of people are often the things we have the least reliable control over in problem solving. We'd prefer to find things we can control better if at all possible, and save human choices and our interpretation of human motives as a last resort.

Third, we can usually tell multiple different stories about the same events, depending on what information we emphasize and how we emphasize it, which in turn depends on our motivations and perspective. This means that the choice of which "root cause" we focus on tends to be more political and subjective than a result of careful analysis.

Our storytelling is natural and compelling but it tends to be more to make sense of events than to understand the details of what causes what.

So effective Root Cause Analysis tools and methods should discourage storytelling and instead search the prerequisite conditions and sequences of events that lead to the results we want to prevent. That's the message of this book and its method. So how well does it accomplish this?

Why a formal process?

Causal analysis (or what is sometimes more specifically referred to as "root cause analysis") is the more formal process of doing what we tend to do naturally, try to figure out why events diverged from what we expected. Why do we need a formal process for doing this, if we already do it naturally? Because often we don't do it very well. The very fact that we do it naturally means that we will tend to make *confident guesses* about why things happened. Often more confident than accurate. This far, the author is on solid ground. The author doesn't refer to it explicitly, but there is a very large literature on the cognitive science of decision making that supports the author's claim that people tend to jump confidently to unwarranted conclusions when they first try to explain causation. (2, 3, 6)

One very important reason for a formal process is to systematize the data gathering and analysis process in order to help compensate for our natural biases that lead us to make confident but inaccurate guesses. The value of formal vs. informal processes is debated widely in the decision science literature, since much of our innate intelligence is the result of automatic non-conscious processes that are opaque to us in our own thinking. (5) However in general I think it is a pretty safe conclusion that a good formal process often helps us focus attention on things that we would not otherwise have seen to see past some of our own blind spots. That's the first reasonable rationale for causal analysis methods like Apollo Root Cause Analysis (ARCA) and its primary tool, RealityCharting(tm). Simply having a process to focus us on causes rather than stories is useful in itself.

Why a formal group process?

There are various reasons for using a formal process in organizations. Unexpected events often affect more than one person, often more than one person is part of the problem, and often more than one person has information or perspective needed to figure out what happened. So causal analysis very often becomes a group process. In addition to the reasons for using a formal process in general (to get around our individual blind spots), it is also important in groups in order to help avoid "group blind spots" such as the principles of behavior in groups studied by psychologists. We are often tempted us to go along with the consensus, to protect our own ideas, to react initially negatively to new ideas, and other tendencies that can negatively impact the problem solving process in groups. (6) A good formal group process can help get around our group blind spots just as it can help us get around our individual blind spots. In spite of the well-established problems created by "groupthink" and other group dynamics studied in social psychological, under optimal conditions, groups can often perform significantly better than individuals on some kinds of problems. (7)

The author makes a key point about group processes that he says defies conventional wisdom (p. 9). He implies essentially that the principles of good problem solving are simple and domain-general so should be taught to everyone, whereas domain expertise is deeper and necessarily differs more between people, so "subject matter experts" should also be at hand. The conventional wisdom he says is that problem solving is entirely "inherent to the subject at hand," or what I would call domain-specific, ignoring the value of domain-general principles in problem solving. I don't know how much this really defies conventional wisdom, but I agree with him completely and I think this is an important principle. It is a major part of the rationale for involving more people in a collaborative group problem solving process rather than just pulling in a few experts. If the process is good, and this principle is valid, then it has deep implications for improving problem solving in organizations by broadening involvement and investment in the process itself. That brings us to the real question at hand, whether the author's method accomplishes this objective.

Why Apollo?

So far my description here is I think pretty much in line with the author's rationale. Now we come to the real meat. How good is this particular method at getting around our individual and group blind spots, compared to other methods? That's the significance the author claims for this book and for his method, so that's what we really need to know to evaluate this book. The book covers a lot of ideas that are very important to know, but many are common knowledge to experienced problem solvers, so I am not going to focus on those. What I will focus on is what is supposed to be special about ARCA and RealityCharting(tm) in particular.

According to the author, the key distinction between his formal method and all others is that all others are ways of categorizing causes and schemes for voting on the best way to categorize them, while his method discerns (and RealityCharting displays) the actual relationship between different causes, along with the evidence for each cause. (p. 193)

All viable methods of "root cause analysis" in groups involve taking chains of events that would be too numerous and related in too complex a manner to envision in a common way by everyone involved in the process of they were simply described in words. Visual representations of trees of causes play a key role in all formal methods of "root cause analysis," including ARCA.

The trick to understanding the underlying message of this book and grasping the uniqueness claimed for the method is to understand exactly what he means by the *relationships between causes* and how the method forces you to think about evidence for causes. I don't personally favor the author's explanation for why his method is unique. His rules of causality seem awkward to me and I think there is a slightly different and clearer way to think of how his method works.

First, every effect is also a cause. There is no distinction between causes and effects except at the endpoints. Feedback loops of causes that have effects that ultimately lead back to the original cause are common in nature as well as human design and this method has no problem handling them. We start with the effect we are trying to explain and end with causes that either do not need to be explained or cannot yet be explained. This seems fairly straightforward to me for a causal chain, I'm not sure why the author feels it is unique to his method. I suppose it may be that commonly used methods tend to de-emphasize this aspect of causality.

Second, the method forces you to identify both actions and conditions that had to be in place for the action to cause the effect. This isn't useful for causal modeling as much as it is a useful trick for helping to identify all of the causes that might be relevant to solving the problem. Breaking the "Why?" question into an action and a set of conditions is seemingly somewhat unique to this method and helps avoid thinking solely in terms of actions or solely in terms of things that were in place at the time or characteristics of the situation. The theory is that it is usually *easier* to see actions that happened and less obvious what conditions had to be in place as well, but it is often *more useful* to identify the conditions because they tend to be more predictable and more controllable.

Third, the method forces you to have evidence for each cause, and takes an unusual and perhaps mildly questionable empiricist slant here. It distinguishes "sensed" from "inferred" evidence. "Sensed" of course refers to direct observation by someone, with as little interpretation as possible. "Inferred" refers to anything else. Assumptions and opinions represent doubts and must be investigated further, and just about anything that someone doesn't observe directly themselves has some doubt associated with it, whereas direct observation is automatically given very high credibility by the rules of the methodology.

This is a slightly odd sort of slant from my perspective because the credibility of eyewitness testimony in general is often in doubt (8), the reliability of inference is often not obvious, and because in my experience, hypothesis testing is particularly useful and important problem solving tool. Storytelling, much maligned and minimized in the ARCA method for good reason, is built on inferences from observations. However so do scientific theories and mathematical models arise from inferences. (9) The main difference is that science and math involve systematic linking and pragmatic testing of inferences rather than just weaving meaningful narratives. The author frequently asserts in different ways that storytelling negatively impacts problem solving, and defers to direct observation. I would instead argue that both are in doubt, and that their relative strength is not absolute based on source, but depends upon how inferences and observations support each other. One contemporary philosopher usefully compares the relationship to that of a crossword puzzle. (10) It is notable that the ARCA method and reality charting tool have no trouble accommodating different conceptions of the reliability of evidence.

Strengths

1. The philosophy behind the method is extremely inclusive organizationally. The idea is not to find the putative "true causes" but to gather enough information to produce effective solutions. So the method encourages as many people as possible to participate as fully as possible, rather than to argue over the right causes to focus on. That is perhaps the greatest strength of this method, its formal incorporation of diverse perspectives, allowing everyone to be heard and every perspective considered while discouraging the usual drama of competing narratives. Different definitions of the problem are managed by starting from different primary causes and creating separate charts. Since the framing of the problem often guides or constrains the search for a solution, I consider this a strength of this method.

2. ARCA explicitly includes the source and specifics of evidence for each cause, helping to identify and prioritize information gathering tasks to further validate assumptions and opinions, or reproduce observations. You are strongly encouraged to provide evidence for every cause, a process that I think can lead to a far more comprehensive yet focused data gathering effort than other methods.

3. ARCA allows for the explicit representation of causal feedback loops, which can be important factors in understanding what is happening. Many causal charts make this important relationship very difficult.

4. The technique of treating each effect as a cause and subjecting each to requirements for evidence, and explicit follow-up for finding further causes, intermediate causes, information required, or explicit reasons for stopping is a very powerful aid to causal thinking and to me is the core of both the method and the associated software.

Limitations and Criticisms

1. Potential overreliance on unreliable eyewitness testimony vs. discouragement of useful inferences. The process of hypothesis generation is a particular kind of inference where we consider more alternatives than we suggest, and as a result we understand that they are explicitly hypotheses. There is experimental evidence that generating hypotheses ourselves leads to a more realistic appraisal of their strength and less false confidence than inferences provided from other people. (cf. Derek Koehler) Other research supports the notion that our causal judgments are themselves inferences that often result from mental simulation. [This is a comment regarding the method in the book, which downplays the role of "inference" vs. "direct observation." It is a minor point in practice because there is nothing in the RealityCharting tool that forces you to evaluate sources of evidence in a particular way.]

2. Overemphasis on uniqueness of the method. This mostly refers to the tone in which the book is written, which I often found distracting and annoying, wherein it often makes the book sound like a sales presentation for the method and the tool rather than a treatise on effective problem solving. This point goes along with the scarcity of credit to other sources that for me would have greatly enriched my appreciation of the rationale for the method. The author relies too heavily on himself as an authority for his rationale to rate this a five star book.

3. Restriction of causal statements to very brief verb-noun and noun-verb format rather than sentences. I think this has some value in fostering clarity if you manage to find a phrase that everyone happens to understand in the same way. Still, in practice I found that it can be time consuming to come up with these compact ways of expressing causes, and that they were easier to misinterpret than full sentences would be. [This is also a potential issue in the tool because the rule checking tries to enforce it. However, it can be disabled if desired.]

4. Does not allow you to visibly categorize causes except into actions vs. conditions. This is an attribute of both the method and the associated software. This "limitation" is the intentional result of the author's core principle that he feels distinguishes his method in particular: categorizing causes is relatively useless compared to relating them. Categorizing causes is suggested in ARCA only if you are stuck finding plausible candidate causes and want to try thinking in terms of categories in order to help find candidates. Since this is intentional, it is a limitation only in the sense that someone familiar with other methods may run into it and have to think differently in order to use the method as intended.

Given that so far every substantive criticism I've had of this method is easily accommodated by minor changes to the method (as well as in the associated tool, with the exception of categorizing causes) I found this to be a particularly versatile method. It captures a number of key principles of problem solving, especially in groups, and provides practical and effective ways of compensating for our most problematic blind spots and biases. For the most part, I think understanding the principles of ARCA and RealityCharting would probably enhance most other methods as well as the method standing on its own.

A formal method itself is no substitute for improving the reflective intelligence of each problem solver, but I think this method could go a long way in any organization toward getting people to think more effectively, and especially for helping them communicate their best thinking to each other. I found very little to disagree with and much of value in this book and found the method easy to understand and apply.

References

(1) Brian Boyd on art and storytelling as biological adaptations which derive from play.

"On the Origin of Stories: Evolution, Cognition, and Fiction," Brian Boyd, 2009, Belknap Press of Harvard University Press

(2) David Perkins discusses the significance of domain-general principles for effective thinking, and why they can't be replaced completely by intelligence or expertise. A plausible research-based discussion of why we need help getting around our built-in individual blind spots.

"Outsmarting IQ: The Emerging Science of Learnable Intelligence," David Perkins, 1995, Simon and Schuster

(3) The psychological study of how we tend to answer questions of the form "Why ... ?" is called attribution theory. Probably a result of our bias toward storytelling explanations, most attribution theory has attempted to address attributions of people in social situations and sometimes products in the case of consumer research. A much smaller amount of research has been done on the perception of causality in other ways, such as in building causal explanations, and only a tiny amount has made its way into popularly accessible books.

"Causal attribution: from cognitive processes to collective beliefs," Miles Hewstone , Wiley-Blackwell

(4) There are not many modern books that deal relatively broadly with causal modeling from both a philosophical and mathematical perspective. One of the few is Pearl's text.

"Causality: Models, reasoning, and inference," J. Pearl, 2000, Cambridge University Press

(5) The non-conscious processes in realistic problem solving is addressed by a fairly sizeable body of technical literature by Dijksterhuis, Bargh, Gollwitzer, and others, and introduced in an accessible and practical way by Gary Klein.

"Sources of Power: How People Make Decisions," 1999, Gary Klein, MIT Press

(6) The problems exacerbated by group behavior under non-optimal conditions are introduced accessibly and briefly yet very broadly in:

"The Psychology of Judgment and Decision Making," Scott Plous, 1993, Mcgraw-Hill.

(7) The positive potential of groups under optimal conditions is discussed in
"Group Problem Solving" by Patrick Laughlin, 2011, Princeton University Press.

The common belief that direct observation is inherently more reliable than inference is based on two ideas that are at best only partially true: (1) the assumption that reports based on observation does not significantly depend on memory, inference, or explanation, and so is automatically free of distortions, biases, or storytelling, and (2) the assumption that inferences are roughly equivalently reliable under a wide range of conditions.

(8) On the reconstructive nature of memory for events, see:
"Searching for Memory: The Brain, The Mind, and the Past," Daniel L. Schacter (1997), Basic Books

(9) On the nature of inference and especially scientific inference, see:

"Error and Inference: Recent Exchanges on Experimental Reasoning, Reliability, and the Objectivity and Rationality of Science," Mayo and Spanos (2010), Cambridge University Press

(10) The useful crossword puzzle metaphor for the relationship of observation and inference is introduced in:

"Evidence and Inquiry: A Pragmatist Reconstruction of Epistemology," Susan Haack (2009), Prometheus Books

A lifetime learning approach, more than a course in "critical thinking"

Review of --> Critical Thinking: Learn the Tools the Best Thinkers Use

by Richard Paul

A lifetime learning approach, more than a course in "critical thinking."

I enjoyed this book and got a lot out of it. For such a high level book it isn't easy reading because it requires you to think about your own thinking while you are reading it in order to get full value out of it. If you skip that exercise, in my opinion, you will not appreciate the value of this book. So if you want to grow from what the authors are offering, be prepared to take a long term vision of your own abilities and to do a lot of work. In spite of the somewhat misleading title of this book, it is not really about what most people would call "critical thinking," it is a much deeper and more useful view of thinking abilities in general.

This book presents a relatively accurate (in my opinion) and very usable model of how human reasoning works in practice and how we get better at it. In addition it offers many simple but practical drills for identifying how well you are thinking and what sorts of things would be good to work on. So I do recommend this book.

Having gotten that positive impression from my reading, I also noticed the many very negative reviews here and that made me pause and think a bit. It may be helpful to expand on what I think it good about this book and what the negative reviewers are seeing.

(...)

Link to complete review on Amazon

Review of "Big Brain" by Lynch and Granger - an education in brain evolution and development

Big Brains: The Origins and Future of Human Intelligence
by Gary Lynch and Richard Granger
Palgrave Macmillan, 2009

There is a lot of very clearly and cleverly presented original material on brains and their evolution and development in this book. It is one of those books that constantly makes you think and has enough depth to read several times fruitfully. I highly recommend it.

See my full review on Amazon here.

A classic battle of politicized science: Kamin vs. Eysenck

My review of the 1981 book: Intelligence: The Battle for the Mind by Hans Eysenck and Leon Kamin can be found on Amazon here.

I found this old book a fascinating look at the politics of intelligence at its most extreme prior the publication of The Bell Curve. In spite of all the friction generated by The Bell Curve that left a misleading impression in the minds of many people, there was actually a general consensus over most of the technical claims regarding intelligence and intelligence testing.

By the time The Bell Curve was written, the definition of intelligence in psychometric terms was well established as were the moderate correlations with academic success and some kinds of occupational success, and the heritability numbers for IQ from several twins studies. It was also pretty well accepted that there were group differences in scores as well as individual differences.

However, the implications of all of these things was more in question than ever because in contrast to some of the basic assumptions in The Bell Curve, heritability was known be highly variable from population to population, IQ was a reasonably good predictor of an important but narrow range of abilities mainly related to literacy and certain kinds of reasoning, and more was unknown than known about the reasons and implications of the group differences.

In the Kamin vs. Eysenck book, we see Eysenck focusing on things that for the most part are not at controvesial at all, and not taking an extreme hereditarian view. His politics are subtle, you can still read echoes of the earlier hereditarian view in his chapters. He talks about people's "capacity" and emphasizes how environment is important in developing intelligence, but implies that people still reach limits determined by heredity in some sense.

Kamin on the other hand reveals his own politics far less subtly by accusing nearly every individual differences researcher of some kind of bias or racism and by looking for anomalies and assuming fraud throughout the entire range of psychometric testing research.

A very instructive account in the dynamics of how scientists interact (or not!) around controversial research programs depending on the way they express their own biases.

See the full review here.

Book Review: David Perkins' must-read brilliant map of human thinking ability and its improvement

Review of David Perkins’ “Outsmarting IQ: The Emerging Science of Learnable Intelligence,” 1995, Free Press.

Link to review on Amazon --> http://www.amazon.com/review/R3AYGZV7G7AUTO/ref=cm_cr_rdp_perm

Profound Thinking By Example

This is the single best book I’ve come across on the potential for improving human thinking ability. I give it my highest recommendation; I think it should be read by everyone interested in problem solving, decision making, and human abilities in general. It is amazingly broad in its coverage of data, profoundly deep in its treatment of specific lines of relevant evidence, and ingenious in its vision of the future.

What impressed me most about this book is that the author, David Perkins, demonstrates the power of deep reflective thinking by his own example in the organization and treatment of evidence throughout this book, in his critical treatment of his own evidence and ideas, in his creative original ideas, and in his effective consolidation and filtering of massive amounts of research. Showing how asking the right questions can help us understand seemingly contradictory data about intelligence, Perkins gives an engaging plausibility proof for the kind of reflective intelligence he argues for in this book.

The Concept of Realms of Thinking

To give away the ending, the book culminates in a model of problem solving ability based on the metaphor of a map. Human thinking ability results from learning our way around. Navigation is fundamental to all sorts of human thinking. Perkins suggests that all intelligent human thinking results from navigation of various kinds, which can be thought of in terms of levels of realms. Perkins organizes the realms in an overall map or “mindscape” from the lowest level of specific contexts of thinking to the highest level dealing with thinking itself.

In learning to solve problems we not only learn our way around physical realms geographically, but we learn our way around specific contexts we find ourselves in such as the realm of buying a house or the realm of choosing a career. We learn our way around different situations like resolving conflicts or making purchases in general. We learn our way around professional fields like law, physics, and mathematics, and areas of technical expertise such as probability and statistics, game theory, and business. We learn our way around the use of tools. We learn our way around various basic kinds of challenges like problem solving, decision making, planning, and learning. Finally, at Perkins’ top level, which he calls thinking dispositions, and we learn our way around thinking itself in terms of the qualities and attitudes that make it more or less effective.

Perhaps the central thrust of this book is that in organizing human problem solving areas into navigational realms, Perkins is not just providing a training map for learning problem solving skills a million different areas, he is also making a case for the learning the critical skills of navigation itself.

Perkins’ realms are very similar to the traditional concept of domains of expertise, but different in one critically important way: realms emphasize the central skills of navigation rather than just the use of repetition or rote memorization or even just the use of deliberate practice. The concept of realms makes it more explicit that all areas of ability that we learn share some commonality in terms of key skills and attitudes we need for navigation itself.

It is learning to be a better navigator; in all realms of human thinking and not just certain subset of them; that is the central message of Perkins’ book. This is encapsulated in his concept of “reflective intelligence.” Reflective intelligence is the aspect of intelligence that can be most improved for the greatest effect across the range of all realms of thinking. Perkins reviews a number of different attempts to improve human thinking and makes various suggestions based on their results regarding specific kinds of changes that can be made to educational curricula in order to teach children to be better navigators in all areas.

Getting Perspective on Intelligence through 3 Dimensions

In giving away Perkins’ final model, I’ve skipped over two very important and interesting aspects: his argument for the model he uses and for the prospect of learnable intelligence through better navigation, and his predictions for important areas of the evolution of learnable intelligence.

The bulk of Outsmarting Intelligence deals tightly with the subject of the title, the legacy of how intelligence has been envisioned and researched so far. Perkins deals in equally deep, reflective, careful, and often fascinating manner with: (1) the evidence for a single common problem solving ability from psychometric data, (2) the evidence showing us how novices think differently from experts, and (3) the evidence showing us what happens when we try to learn general skills and rules for solving problems in general and how computers solve problems.

From these three bodies of evidence, Perkins derives three corresponding dimensions of human intelligence: (1) a neural intelligence dimension which respects what psychometric data gets right and is most closely associated with what we typically assume IQ tests are measuring, (2) an experiential intelligence dimension which respects what expertise research data gets right, and (3) a reflective intelligence dimension which respects what we have learned about metacognition and from the various programs that have tried to teach thinking skills in general.

Neural intelligence, Perkins concludes, is a real dimension of human ability and very important in some situations especially, but it is simply the wrong target for attempts at improvement for various reasons.

Experiential intelligence represents most of our actual problem solving abilities in practice.
Faced with novel and complex situations where we have no relevant experience, our neural intelligence gives us our best chance at solving the challenges presented. But once we have been acquiring experience in an area, a difference in expertise will make people better problem solvers in that area than will a difference in general intelligence.

So experiential intelligence and neural intelligence work together to make us the generally good problem solvers that we are in most situations: neural intelligence helps us deal with novelty and complexity, and experiential intelligence helps us acquire the knowledge and skills we need to deal with specific domains.

So the obvious question is: what role does reflective intelligence play and why does Perkins consider it so important?

The Significance of Reflective Intelligence

Perkins reviews various lines of research into the wide variety of situations where otherwise powerful problem solving abilities seem to fail us in systematic ways. He looks at social psychological effects, cognitive shortcuts, and so on, similar to other reviews of blind spots in human thinking by many other authors except that Perkins attempts to characterize these foibles specifically in terms of side effects of our experiential intelligence.

Perkins suggests that the human mind is mostly akin to a pattern seeker and pattern-driven problem solving engine and as a result its weaknesses are also those we would expect from a pattern-driven process. The human mind often tends to be hasty, narrow, fuzzy, and sprawling.
HASTY. The goal of a pattern seeking intelligence is to find the right response that most closely matches the current situation rather than making an exhaustive search. As a result, our experiential intelligence tends to mislead us to jump to hasty conclusions when the situation is an unusual variation of a known situation.

NARROW. As a result of efficiently seeking patterns we have already seen, the domain-specificity of expertise tends to make us think in narrow ways when we think we have grasped the situation rather than to broaden our thinking.

FUZZY. Part of the power of pattern-matching is that we can so often generalize the lessons from one situation to another similar one. In situations where the appearance is very similar but the underlying principles are different, again our pattern matching effectiveness leads to mistakes: we overgenerallize from our experience.

SPRAWLING. When a pattern-seeking process does not have a single clear path to follow, as often happens in very complex situations, it will tend to follow one path after another and keep switching back and forth rather than working toward an overall goal.

Experiential intelligence, Perkins concludes, is an elegant system for long-term moderate success. When situations are new to us or complex, we get help from our neural intelligence and we have also learned various tricks for getting around our weaknesses, and these are largely accounted for in reflective intelligence. Reflective intelligence represents realms where we think about our own thinking in order to avoid settling on hasty conclusions, to broaden our thinking beyond the initial scope we assumed, to use precision to distinguish similar looking but different things, and to stay on track when notice we are sprawling.

This explains why reflective intelligence is so important to us in tricky situations where we have inadequate experience and where experience misleads us. But it also helps explain, in Perkins’ view, why reflective intelligence is so important for us to learn to be better thinkers in general. Neural intelligence does not replace experiential intelligence, it tends to reinforce it.

When we don’t have experience, neural intelligence helps us grasp the situation, but when we do have experience, we tend to use our neural intelligence to reinforce what our experience already tells us. That’s one big reason why genius is not simply high IQ. That’s why reflective intelligence is so important, it is the tool we use to remind us of the weak points in our own thinking and help us compensate for them regardless of our experience and general intelligence. The abilities and traits we need in order to overcome our blind spots are learnable. A large and crucial aspect of intelligence is learnable.

Existing Approaches: How they Compare

There are various approaches to teaching reflective intelligence, and Perkins reviews the best known and the best studied among them such as Project Intelligence, Reuven Feuerstein’s Instrumental Enrichment, Edward de Bono’s CORT, and Matthew Lipman’s Philosophy for Children, and others, reviewing their approaches and their results and comparing and contrasting them in order to get a sense of what it takes to enhance reflective intelligence.

One of the things that distinguishes Perkins as a deep reflective thinker himself is that he anticipates, researches, and deals fairly with opposition to his arguments. The very idea of learnable intelligence has in the past come under attack from several angles such as past failures of various programs which tried to teach improved thinking, the implications of expertise and psychometric research data, the apparent weakness of general methods for problem solving, and the challenge of transfer between learning domains. Perkins addresses each of these concerns in turn, resulting in a very persuasive case for the very real improvability of intelligence through changes in education.

The Future of Learnable Intelligence

Toward the end of the book, Perkins reveals the ingenuity of his vision through his discussion of several areas for the future evolution of reflective intelligence: areas which ended up being (remarkable for a book written in 1995) accurate predictions of areas that have since become central areas of interest for science and human improvement in general:

1. Intelligence can become distributed -- good thinking depends upon artifacts to offload the limitations of our attention and memory, and we can use our symbol systems and tools to help us keep track of things we could not track individually. This is a wonderful general description of how we are attempting to use computer networks to help us manage complexity (as opposed to some of the more superficial books in recent years which imply that networks somehow replace rather than enhance individual thinking).

2. Intelligence can embrace complexity -- through information visualization tools, effective use of classification, tagging, and finding things by meaning, consolidation, filtering, the mathematical tools for finding large scale patterns in complex phenomena, and by eliminating narrow information silos, we can use our intelligence to solve increasingly complex problems.

3. Intelligence can be dialectical -- this means raising the level of thinking from lower level more concrete concerns to higher order patterns by recognizing the properties specific to complex systems. Perkins offers Peter Senge’s “The Fifth Discipline” and Murray Gell-Mann’s “The Quark and the Jaguar” as exemplifying ways of understanding dialectical intelligence.

Perkins covers a massive amount of data about intelligence and problem solving, summarizes it effectively, and applies it to a practical, powerfully supported, and exceptionally understandable approach to improving human life by teaching ourselves to be more intelligent. Thinking well in general is an unnatural act but we can learn to do it. All that is left is for us to overcome the ideological and political barriers. This book would make a wonderful, gentle manifesto for that grand effort.