Showing posts with label IQ. Show all posts
Showing posts with label IQ. Show all posts

Wednesday, May 29, 2013

Book Review:  Ungifted by Scott Barry Kaufman

Todd I. Stark  5/29/2013

Link to review on Amazon

Intelligence turns out to be a difficult topic, for reasons that aren’t at all obvious at first. Our understanding of mental ability has been captured in several independent threads of research that are surprisingly oblivious of each other for the most part.  Our stereotypes of the gifted and the ungifted often miss the details of what is going on.  The study of individual differences in general, while useful, doesn’t just de-emphasize, but actually systematically misses some of the most important things going on when people become exceptionally successful contributors.  

The author of Ungifted is well situated to make an important contribution to our understanding of intelligence.  He has made a deep academic study of a wide span of existing research programs, he has worked directly in collaboration with many of the leading researchers in several related fields and he has passionately engaged these ideas since childhood when he became  painfully aware of the academic sorting process for giftedness, and he himself is a wonderful example of many of the principles that emerge in his new book.  This is not another book that just starts out with a vague progressive vision of education and ability that everyone is a potential “genius” and then fills it in with wishful thinking.  No, this is a book that dives very deeply and realistically into the literature of psychometrics, heritability, cognitive neuroscience, and expertise.  It looks closely at patterns from the span of phenomena of human differences, including savantism, prodigy, autism, schizophrenia, personality, g factor, motivation, and creativity.

Ungifted is so compelling, rich, and significant a book for me that most of it was well worn by the end of the first day it arrived. What makes this book so rich is the unique combination of personal passion, strong but not intrusive scholarship, deep domain expertise, clear analysis, and all tied together with an original new and constructive perspective. Hard to ask for more than that from a non-fiction book. Every chapter is a stimulating lesson in an important topic that brings together a broad range of data, ever heading toward the book’s conclusion, nothing less than a complete rethinking of human intellectual ability, consistent with considerable evidence gathered along the way.

The most impressive and distinctive aspect of the author's thinking is his consistent and effective use of perspective-taking. In each case where a controversy is identified, each side is explored with great depth and sympathy to understand what its advocates understand that those on the other side seem to miss. It isn't difficult to do that for the side of an argument we agree with but it is an impressive achievement to do it for different sides and then to synthesize the perspectives into an overlapping understanding. This ability is particularly relevant to topics like IQ, giftedness, and learning disabilities, where sharp controversy shapes conversations at every turn.

Ungifted is about how we conceive of mental ability in general. Some people manage to accomplish much more with their mind than others do. There’s no escaping that basic observation, nor the fact that it has immense significance for political and educational thinking. What is the difference that makes a difference? The answers we get depend on the kinds of questions we ask.

Ungifted is in part the story of how the important questions have changed over time and why. There seem to be two tragic errors that we’ve fallen into historically. For one, we’ve often ignored the differences between us and tried to force everyone into cookie-cutter educational molds that well serve only a minority of the people they are intended to serve. Most of us appreciate this personal plight, as does the author in his "subjective" voice and personal experience throughout the book.

The other tragic error is ironically the reverse. In trying to appreciate the differences between people we’ve taken the opposite extreme of becoming obsessed with stable, predictive individual differences. We test ourselves and compare ourselves with each other and we look for the numbers that tell us who deserves what because we assume we are identifying potential. The author appreciates the motives and sometimes successes of this approach when done well, but the focus of Ungifted is on how we can do better.

It is the tragedy of our obsession with individual differences that Ungifted in the author's "objective" voice most eloquently addresses. We assume that testing people against each other will tell us how to best teach each person by identifying what makes people different. Ungifted describes in great detail and punctuated by the author’s own personal life story why that well-intended approach has failed us time and time again.

It isn’t simply as many politically motivated accounts would have it, that IQ has no meaning, or is too culture bound, or just measures test taking ability. IQ and similar kinds of tests when given and interpreted intelligently provide a useful and well-validated way of identifying the lion’s share of variation in human intellectual ability across a wide range of situations and this has some very real correlations with meaningful life outcomes. The problem is not that the tests are useless but that they have come to be misconstrued as if they measure a single stable ability that resides in each person and predicts what that person is capable of accomplishing. People who do well in IQ tests do tend to be smart people in general. But so are many people who do more poorly in IQ tests, and doing well in IQ tests doesn’t provide any guarantee that we also have persistence, motivation, or other qualities so important to making the best of our abilities.

A number of theorists have made useful additions to the body of well validated tests, but these are still tests of static abilities and they don't solve the most basic problem identified in Ungifted. The reasons the individual differences approach has failed us in the broad global sense that we have tried to apply it are that while effectively explaining variation between people in the same population, it has not taken broader environments into account, it has not taken the course of development into account, and it has not taken the dynamic differences into account that make the most difference in human lives over time.

A number of fundamental misunderstandings of heritability, genetics, development, and psychometric research have been exploited, often through politically motivated movements, to obscure the larger vision of intellectual ability. Ungifted makes a serious bid to help correct these fundamental misunderstandings.

Do some people have more of a critical trait or traits from the start, or do some people learn more from their experience, and in either case how much can we influence our abilities over the course of our life? The traditional dialectic of nature vs. nurture seems to have its own unavoidable groove in our thinking that we rarely manage to escape. Yet as long as we have been studying human ability scientifically, there has been evidence that the dichotomy is inadequate. 21st century research has given us some useful insights into the specifics. Ungifted summarizes the most important lessons from a wide range of data about human abilities, and the author is particularly careful to distinguish his subjective passion for the subject (which is often in evidence) from his more detached coverage of the data in the various fields.

Ungifted starts out with a concise summary of principles of the 21st century picture of development, setting the background for the rest of the book. The concept of traits is explored, and the patterns by which they develop.

Then we have a tour of the ways we have tried to measure human mental ability and our reasonable motivations for the various testing innovations over time. From the measurement of ability, we then see how measurement slides into the sorting of people into categories for practical purposes and the allocation of finite educational resources. We see the well-motivated practical reasons for labeling people, but we also get a sense of the often tragic real world limitations of that way of thinking.

We are introduced in an understandable but expert way to the real strengths and weaknesses of IQ testing, and to the best available model of how its scales map to specific cognitive abilities. This prepares us to begin to understand the different ways that "giftedness" has been defined and measured and why we are still so far behind where we need to be to cultivate the best in every individual. We also are given enough background to begin to appreciate the unique challenges of being different, whether perceived as higher or lower in ability than others around us. Ungifted artfully blends a sympathetic understanding of the needs of researchers and testers with those of teachers, and the diverse individuals just striving to do their best.

Then we are introduced to the core concept that distinguishes this developmental reframing of human mental ability, the concept of engagement. Engagement brings together the factors that distinguish the learnable, experiential aspects of intelligence from those that seem particularly stable. Engagement is the hinge that swings the big door to individual potential. Engagement is built on a number of factors that are particularly malleable and context-dependent, so it is of particular interest to the way education is done. Once we have a sense of what engagement is about, we take a fresh look at human abilities in terms of what is known about their development over time. The role of engagement over time in development begins to become clearer as we tour through the critical concepts of intelligence, creativity, talent, and expertise, and begin to see how they each relate to development over time. At the end, the key points learned along the way are summarized to give the outline of a new theory of intelligence.

For me the theory of intelligence introduced here is distinguished by two critical characteristics: (1) it emphasizes what happens in the individual over time rather than differences between people, and in so doing draws on different kinds of data, and (2) it is synthetic in spirit, emphasizing multiple ways of achieving the same outcomes by drawing on different resources, rather than looking for additional ways of distinguishing the abilities of different people. These two characteristics make this theory very different from most of the alternatives that are intended to address some of the same gaps in our intelligence models, alternatives such as “multiple intelligences,” “emotional intelligence,” and so on. Rather than just identifying more things that we think might be missed by IQ testing, and turning them into new sources of labeling and categorizing, the personal developmental theory of intelligence assumes that there are many different components to be identified, but places them into an overarching biological framework where ability is developed over time by identifying, selecting, modifying, and constructing niches suited to the thriving of the individual.

Several important shifts of emphasis emerge:

1.      Away from reliance on studying stable individual  differences and toward the details of person-centered development


We have focused primarily on measuring stable individual differences, whether “general intelligence” or other kinds of “intelligence” or personality in order to support research and allocate finite educational resources.  What this approach misses is the details of development within each individual over time.  It turns out that these two approaches, individual differences and person-centered, are not just different but produce incompatible results.  So this is a very important source of new information about how mental ability arises.  It is common for authors to point out that nature and nurture are an archaic dichotomy and that it is the interaction that matters, but the details are generally left vague.  There’s a need for specific research programs that focus on the development of ability over time.  The developmental approach is not just a detail, it is a separate source of crucial empirical data.

2.      Away from viewing the positive manifold of abilities on tests (“g”)  as a single ability in each person, and toward understanding its value in conveniently capturing most of the variation an array of mental abilities that collectively underlie many different kinds of tests.    

3.      Away from seeing intelligence as a number or even a trait, and toward seeing it as the adaptive fit between the individual and their environment by finding, selecting, shaping, and creating niches they can thrive in.

4.      Away from focusing on cognitive skills in isolation, and toward consideration of the motivations, strategies, and experience that turn those skills into practical abilities over time.

5.      Away from the focus on being able to measure abilities that represent potential in a brief test, and toward finding the best way to engage and cultivate each person.  Active engagement with the world and ability are inseparably intertwined in a mutual feedback process over time.

6.      Away from focus solely on controlled cognitive processes underlying reasoning, and toward better understanding of the role of both controlled and spontaneous processes in intelligence.  Being smart involves flexible use both controlled and spontaneous mental processes, and using the right resources when needed, rather than relying on one to the exclusion of the other.

7.      Away from fixed rules about how long it should take to become good at something or what special levels of ability provide thresholds, and toward seeing our “readiness for engagement” as a better indicator of potential.

8.  Away from seeing intelligence and expertise as independent (and one inborn and the other learned), and toward seeing the overlaps between them as cognitive abilities,personality, and motivation support the acquisition of expertise, and cognitive expertise is part of what shows up in testing for ability.  One of the most remarkable findings in the book that links intelligence and expertise is that chunking in memory, a key aspect of organizing knowledge in memory involved in expertise, activates the brain structures involved in fluid reasoning, a central component measured by tests.  

The resulting view of intelligence doesn’t see everyone as equal by any means, it seems unavoidable that some people will not be able to excel at some things relative to other people.  Stable individual differences do not somehow disappear because we shift emphasis to the person and their development.  However we do begin to see the real value of changing the nature of education to focus on the fit between people and niches rather than selecting people for special treatment via brief tests and subjective judgments of merit.  Several practical working examples of programs that successfully accomplish this change are described in the book.

I place this book alongside two others in a trilogy that for me represents a broad understanding of the nature of mental ability as it is best envisioned at the current time.

1. "Surpassing Ourselves" is about how outstanding performers learn differently.  It shows expertise as a distinctive way of learning rather than just an endpoint of domain specialization. By comparing the same person over time as they develop, rather than just comparing novices with experts, we get an understanding of the process by which people become smarter.  This is very similar to the shift made with intelligence in Ungifted, but applied specifically to expertise and shows again how the shift to a process perspective captures additional important information.  Surpassing ourselves also introduces  the reinvestment perspective, which looks at the growth of ability in terms of becoming more efficient over time and then reinvesting the saved time and energy in new learning, which seems to be a big part of how people who ultimately become exceptional learn differently from those who do not.

http://www.amazon.com/Surpassing-Ourselves-Inquiry-Implications-Expertise/dp/0812692055/ref=sr_1_1?s=books&ie=UTF8&qid=1369837497&sr=1-1&keywords=surpassing+ourselves

2. "Outsmarting IQ" is about learnable intelligence.  It shows how experience, stable cognitive abilities, and strategies work together to navigate realms of knowledge and let us apply our knowledge.  It gives a good account of the role of strategies, including learnable strategies and cognitive expertise, in trading off between cognitive abilities in order to make the best of our existing stable cognitive abilities. 

http://www.amazon.com/Outsmarting-IQ-Learnable-Intelligence-ebook/dp/B001D1Y8Z8/ref=sr_1_1?s=books&ie=UTF8&qid=1369837528&sr=1-1&keywords=outsmarting+iq

3. "Ungifted" combines the intrapersonal process perspective with a developmental model and an overall hierarchical model of cognitive abilities, consistent with the other two books in this list and yet going way beyond them to explain in much greater detail where “IQ” and other tests scores come from and what they tell us, and how stable cognitive abilities, expertise, creativity, and personality all interact to produce intelligence. 
http://www.amazon.com/Ungifted-Intelligence-Redefined-ebook/dp/B00B3M3UME/ref=sr_1_1?s=books&ie=UTF8&qid=1369837581&sr=1-1&keywords=ungifted+intelligence+redefined

Update 5/29/13 1PM:  Link to abbreviated GoodReads version of review:

Ungifted: Intelligence RedefinedUngifted: Intelligence Redefined by Scott Barry Kaufman
My rating: 5 of 5 stars




View all my reviews

Monday, August 29, 2011

The Power of Expertise

The Power of Expertise

Who Ya Gonna Call?

Let’s say you’re working on your computer and it starts acting strangely. You get errors that don’t understand or it crashes for no apparent reason. If you aren’t sure what to do at first, where will you look for help? You might perform a web search for the symptoms to see if it’s a known problem and other people have solved it before you. You might run some diagnostic program or an antivirus scan because those are the tools you happen to have.

If you can’t fix it easily and you aren’t confident with computers you’ll probably start looking for help from another person at some point. Who? If it were me, I probably wouldn’t head down to the local college and find the top honors student or someone in the local Mensa chapter. I probably wouldn’t look for someone with great SAT scores or someone really good at Sudoku or even a master electrician. I’d look for someone with a lot of experience with computers and a proven track record fixing them. I’d look for an expert, and an expert specifically in that area, not just a smart person or an expert in a related area.

I stacked the deck a little bit with this question, because I picked a problem that is probably going to be technical in nature. That is, it seems like it will require some specialized knowledge to solve because it involves computers which are complicated devices that are a little mysterious to the average person and far less so for someone who has worked extensively with them.

It turns out, though, that my guess is pretty accurate for a wide range of fields, not just highly technical ones. Knowledge about the job turns out to be a far better predictor of performance than how high our IQ is or any other general disposition, not just in certain kinds of jobs but across a wide range from complex technical work to manual labor.[1] Just as I’d rather have a computer expert help me rather than my friend with an astronomical IQ, in most cases I’d prefer someone who has job experience rather than someone very smart but inexperienced. And I can point to research evidence that supports my preference.

Seeing Differently vs. Seeing More

Even in many areas where we would tend to expect pure reasoning ability to play a large role, it turns out that on average experience tends to win out consistently over any more general ability or measurement we have come up with.

The research that inspired the modern study of expertise began with the game of chess. Think about chess for just a moment. Chess is an activity with a small number of relatively simple rules. Yes chess has the reputation for being a difficult game. But that’s not because chess is hard to play. Nearly anyone can learn the game. It’s because we soon discover that differences in individual ability are immense.

The difference between someone who plays chess for fun who doesn’t study the game seriously, and an average tournament player, is like night and day. It doesn’t seem like much of a competition most of the time. The difference between an average tournament player and a strong one is just as large, which is why there is a rating system.

Ratings allow people of similar ability to play relatively evenly, or to estimate handicaps as they do in golf. The difference between a strong player and a master is similarly imposing as is that between the master and a grandmaster, and between the average grandmaster and a world champion.

How can a game with a handful of simple rules end up with people playing at such astronomical differences in ability? This was the question that intrigued early researchers trying to figure out how people solve problems. The obvious answer is that the stronger players must be seeing more on the board. But what are they seeing differently?

When most of us look at the chess board we see a collection of pieces in different places that are allowed to move in particular ways. We know what we have to do to win; we have to trap the king. We also know some ways to accomplish that. For example we can capture the opponent’s pieces so we have a bigger army, and we can harass the opponent’s pieces so that they are forced into a less defensible position, allowing us to attack the king. Everyone who plays the game, even for fun, knows these things. Still most of us pretty much have to guess at how to get from some arbitrary position to that result.

If I move here, I’ll attack this piece, but how do I know that my opponent doesn’t have some better move in response that is even stronger? More insidiously, is that move by my opponent actually setting up a surprise for me later? If so, what are my options? These kinds of considerations quickly lead to the very intuitive notion that being better at chess is really about calculation, about being able to imagine a lot of different moves, and what might happen if we made them, and keeping track of all that imagining. The better player must be seeing more moves on the board, figuring out what the options are more accurately, and then predicting the outcome.

This is indeed how early chess software played the game well. It looked at the possible moves, looked at the possible responses to each move, evaluated the resulting positions, and chose the move that seemed to give the best outcome based on what the opponent was able to do. The trouble was that trying to do this more than a couple of moves ahead turned out to be a very demanding calculation. More demanding than even the most powerful computers could handle. Researchers were curious as to whether seeing more moves in their mind is really what good players were doing.

Maybe the human brain is really that much more powerful at calculation than we thought. Or maybe the brain is doing something else entirely?

In a pioneering study of chess players in the 1940’s[2] a Dutch psychologist found the surprising answer. I say his work was pioneering not just because it was early but because it led to entire fields of research based upon it and validating his basic findings. The most compelling and surprising findings:

...Weaker players examined the same number of moves as stronger players, and equally thoroughly (!)

...Stronger players could recognize an actual game position far better than weaker players.

...Stronger players were just as bad as weaker players at recognizing an arbitrary configuration of pieces.

This may not seem so earthshattering at first, but think about the implications. Experts at chess consistently beat weaker players, but without examining more moves and without examining the outcomes of those moves more thoroughly. They aren’t “looking ahead more” and they aren’t “reasoning better” and they aren’t even remembering more in general. They do remember more about chess in a sense but not because they have a better memory. And looking ahead is important, but not by keeping track of moves. Their ability is a result of their mind being better trained to remember chess configurations in particular and to use that knowledge quickly and efficiently to evaluate moves.

So what are chess experts seeing that the rest of us aren’t? They aren’t seeing more moves ahead, they are seeing the board in terms of chess configurations instead of seeing it in terms of individual pieces. Their mind has been trained to see meaningful configurations of pieces instead of individual moves. They are not seeing more per se, they are seeing differently. They are seeing in terms of larger and more meaningful groupings. Experts with extended experience acquire a larger number of more complex patterns and use these new patterns to store knowledge about which actions should be taken in similar situations.[3]

The result is profound. We have a game where a few simple rules results in an incalculably large number of possible sequences of moves. But we become good at this game of many, many moves not by thinking about more moves but by thinking in terms of larger patterns: patterns of pieces rather than movements by individual pieces.

Through practice, chess masters have trained their mind to recognize the unique meaningful patterns that apply to their game. Further, the ability to learn to recognize new patterns (along with a huge capacity to remember them) seems to be something we all possess, not just chess masters. It is a fundamental principle of learning, at least learning to be a chess expert.

Even more interesting, we don’t recognize this as knowledge, in the sense of things we recognize that we know. I know that I know some things. I know that I know all sorts of facts like the capital of some of the U.S. states and the number of sides in a triangle and Newton’s formula relating force and mass and acceleration. These sorts of things are considered explicit knowledge.[4]

Chess masters can’t write down most of the patterns they know, both because those patterns are so vast and because they use them without thinking about them. The patterns they learn become part of their chess intuition in a manner of speaking. A common technical term for this is tacit knowledge.[5] We use tacit knowledge in our thinking without realizing that we are using it. This is why it took focused research to discover what was going on in the minds of chess masters.

Tacit knowledge becomes part of our perception. Chess masters see the board differently; for example they often immediately see positions as good or bad without having to do the kind of analysis that the rest of us would have to rely upon.[6]

Tacit knowledge is also used automatically in our thinking. When chess masters guess at the best move in a given position, their guess is informed by their vast database of tacit knowledge, so it is very different from the guess made by a weaker player. Experts make better guesses in their area of expertise. This is what I mean by their “chess intuition” above.

Trained Intuition and Better Guesses

You might be wondering at this point why I’ve spent so much time talking about chess experts. Or you may have guessed the answer. The most interesting conclusions from the research on chess masters are by no means limited to chess masters. Very similar or consistent results have been obtained across a staggeringly wide variety of fields from physical pursuits like wrestling and ballet to intellectual subjects like calculus and philosophy to artistic activities like painting and violin playing, to a wide variety of everyday jobs, to oddball activities like picking the winners at the horse races.[7] Even among scientists, where the role of abstract reasoning is particularly central and the subject matter particularly challenging, productivity doesn’t seem to be predicted on the whole by supposed general ability measures such as IQ.[8]

The chess findings are a particularly useful rhetorical device here because chess seems like it should be so dependent on reasoning and analysis. It turns out that experts analyze chess positions with the help of a vast mental database of chess configurations that apply without any recognition that they know them. The resulting perception and memory of the board just seems natural to them as a result of practice. Examined closely, in spite of its natural appearance for some people, the effortlessness of deep expertise seems to be an extreme kind of skill acquisition[9] far more than an expression of talent.

Even if you interpret all of these findings from different fields very conservatively, collectively they still tell us something of tremendous importance about how we become good at things. We modify the way we perceive the activity. In effect, we train our intuition about the activity.

In all of these activities, researchers have found that time spent in the activity lets us acquire a new way of perceiving patterns in that activity that let us transcend the limits of our working memory and sequential reasoning capacity. That’s why expertise consistently outperforms IQ or working memory capacity or other general measures as a predictor of performance in virtually every activity that has been studied so far. And expertise is not just specialized knowledge or skills; it is also more importantly an accumulation of organized tacit knowledge that lets us make better guesses.



[1] (Hunter, 1986)

[2] (de Groot, 1965)

[3] This has been the most common interpretation of the chess research findings amongst expertise researchers, based on the influential theory of Chase and Simon. (Chase & Simon, 1973), (Simon & Chase, 1973)

[4] “Explicit knowledge,” basically just means things we know that can be easily identified and written down. The descriptor declarative is sometimes used as well, meaning that we can declare it.

[5] In contrast to “explicit knowledge,” this is often referred to as “tacit knowledge,” meaning things we know but we can’t easily express, especially things that support action. Tacit knowledge is usually assumed to be useful for doing things more than for taking part in our conscious reasoning processes. The descriptor procedural is sometimes also used for tacit knowledge because we think of it as involving procedures for doing things rather than declarations about things. For this reason, a common rule of thumb is that tacit knowledge refers to “know how” whereas explicit knowledge refers to “know that” (i.e. I know that grass is green). The casual rule of thumb is troublesome because we don’t really know how we do those things we call procedural, the usage of the word “know” in “know how” is very different than the word “know” in “know that.”

[6] Following the pioneering chess research, research into other areas reinforced the same finding: expert performance depends heavily on a large accumulated memory of patterns that give us a different “intuitive perceptual orientation” to tasks. “Experts can ‘see’ what challenges and opportunities a particular situation without affords.” (and without doing any analysis) (Perkins, 1995, p. 82)

[7] One of the leading and best known figures in the study of expertise is K. Anders Ericsson, whose research encompasses a particularly wide range of fields. An excellent and accessible overview of work in diverse areas of expertise research is Ericsson’s edited collection: The Road to Excellence (Ericsson, 1996).

[8] (Taylor, 1975)

[9] (Proctor & Dutta, 1995), (VanLehn, 1996)

Saturday, September 11, 2010

A classic battle of politicized science: Kamin vs. Eysenck

My review of the 1981 book: Intelligence: The Battle for the Mind by Hans Eysenck and Leon Kamin can be found on Amazon here.

I found this old book a fascinating look at the politics of intelligence at its most extreme prior the publication of The Bell Curve. In spite of all the friction generated by The Bell Curve that left a misleading impression in the minds of many people, there was actually a general consensus over most of the technical claims regarding intelligence and intelligence testing.

By the time The Bell Curve was written, the definition of intelligence in psychometric terms was well established as were the moderate correlations with academic success and some kinds of occupational success, and the heritability numbers for IQ from several twins studies. It was also pretty well accepted that there were group differences in scores as well as individual differences.

However, the implications of all of these things was more in question than ever because in contrast to some of the basic assumptions in The Bell Curve, heritability was known be highly variable from population to population, IQ was a reasonably good predictor of an important but narrow range of abilities mainly related to literacy and certain kinds of reasoning, and more was unknown than known about the reasons and implications of the group differences.

In the Kamin vs. Eysenck book, we see Eysenck focusing on things that for the most part are not at controvesial at all, and not taking an extreme hereditarian view. His politics are subtle, you can still read echoes of the earlier hereditarian view in his chapters. He talks about people's "capacity" and emphasizes how environment is important in developing intelligence, but implies that people still reach limits determined by heredity in some sense.

Kamin on the other hand reveals his own politics far less subtly by accusing nearly every individual differences researcher of some kind of bias or racism and by looking for anomalies and assuming fraud throughout the entire range of psychometric testing research.

A very instructive account in the dynamics of how scientists interact (or not!) around controversial research programs depending on the way they express their own biases.

See the full review here.

Saturday, July 31, 2010

Book Review: David Perkins' must-read brilliant map of human thinking ability and its improvement

Review of David Perkins’ “Outsmarting IQ: The Emerging Science of Learnable Intelligence,” 1995, Free Press.

Link to review on Amazon --> http://www.amazon.com/review/R3AYGZV7G7AUTO/ref=cm_cr_rdp_perm

Profound Thinking By Example

This is the single best book I’ve come across on the potential for improving human thinking ability. I give it my highest recommendation; I think it should be read by everyone interested in problem solving, decision making, and human abilities in general. It is amazingly broad in its coverage of data, profoundly deep in its treatment of specific lines of relevant evidence, and ingenious in its vision of the future.

What impressed me most about this book is that the author, David Perkins, demonstrates the power of deep reflective thinking by his own example in the organization and treatment of evidence throughout this book, in his critical treatment of his own evidence and ideas, in his creative original ideas, and in his effective consolidation and filtering of massive amounts of research. Showing how asking the right questions can help us understand seemingly contradictory data about intelligence, Perkins gives an engaging plausibility proof for the kind of reflective intelligence he argues for in this book.

The Concept of Realms of Thinking

To give away the ending, the book culminates in a model of problem solving ability based on the metaphor of a map. Human thinking ability results from learning our way around. Navigation is fundamental to all sorts of human thinking. Perkins suggests that all intelligent human thinking results from navigation of various kinds, which can be thought of in terms of levels of realms. Perkins organizes the realms in an overall map or “mindscape” from the lowest level of specific contexts of thinking to the highest level dealing with thinking itself.

In learning to solve problems we not only learn our way around physical realms geographically, but we learn our way around specific contexts we find ourselves in such as the realm of buying a house or the realm of choosing a career. We learn our way around different situations like resolving conflicts or making purchases in general. We learn our way around professional fields like law, physics, and mathematics, and areas of technical expertise such as probability and statistics, game theory, and business. We learn our way around the use of tools. We learn our way around various basic kinds of challenges like problem solving, decision making, planning, and learning. Finally, at Perkins’ top level, which he calls thinking dispositions, and we learn our way around thinking itself in terms of the qualities and attitudes that make it more or less effective.

Perhaps the central thrust of this book is that in organizing human problem solving areas into navigational realms, Perkins is not just providing a training map for learning problem solving skills a million different areas, he is also making a case for the learning the critical skills of navigation itself.

Perkins’ realms are very similar to the traditional concept of domains of expertise, but different in one critically important way: realms emphasize the central skills of navigation rather than just the use of repetition or rote memorization or even just the use of deliberate practice. The concept of realms makes it more explicit that all areas of ability that we learn share some commonality in terms of key skills and attitudes we need for navigation itself.

It is learning to be a better navigator; in all realms of human thinking and not just certain subset of them; that is the central message of Perkins’ book. This is encapsulated in his concept of “reflective intelligence.” Reflective intelligence is the aspect of intelligence that can be most improved for the greatest effect across the range of all realms of thinking. Perkins reviews a number of different attempts to improve human thinking and makes various suggestions based on their results regarding specific kinds of changes that can be made to educational curricula in order to teach children to be better navigators in all areas.

Getting Perspective on Intelligence through 3 Dimensions

In giving away Perkins’ final model, I’ve skipped over two very important and interesting aspects: his argument for the model he uses and for the prospect of learnable intelligence through better navigation, and his predictions for important areas of the evolution of learnable intelligence.

The bulk of Outsmarting Intelligence deals tightly with the subject of the title, the legacy of how intelligence has been envisioned and researched so far. Perkins deals in equally deep, reflective, careful, and often fascinating manner with: (1) the evidence for a single common problem solving ability from psychometric data, (2) the evidence showing us how novices think differently from experts, and (3) the evidence showing us what happens when we try to learn general skills and rules for solving problems in general and how computers solve problems.

From these three bodies of evidence, Perkins derives three corresponding dimensions of human intelligence: (1) a neural intelligence dimension which respects what psychometric data gets right and is most closely associated with what we typically assume IQ tests are measuring, (2) an experiential intelligence dimension which respects what expertise research data gets right, and (3) a reflective intelligence dimension which respects what we have learned about metacognition and from the various programs that have tried to teach thinking skills in general.

Neural intelligence, Perkins concludes, is a real dimension of human ability and very important in some situations especially, but it is simply the wrong target for attempts at improvement for various reasons.

Experiential intelligence represents most of our actual problem solving abilities in practice.
Faced with novel and complex situations where we have no relevant experience, our neural intelligence gives us our best chance at solving the challenges presented. But once we have been acquiring experience in an area, a difference in expertise will make people better problem solvers in that area than will a difference in general intelligence.

So experiential intelligence and neural intelligence work together to make us the generally good problem solvers that we are in most situations: neural intelligence helps us deal with novelty and complexity, and experiential intelligence helps us acquire the knowledge and skills we need to deal with specific domains.

So the obvious question is: what role does reflective intelligence play and why does Perkins consider it so important?

The Significance of Reflective Intelligence

Perkins reviews various lines of research into the wide variety of situations where otherwise powerful problem solving abilities seem to fail us in systematic ways. He looks at social psychological effects, cognitive shortcuts, and so on, similar to other reviews of blind spots in human thinking by many other authors except that Perkins attempts to characterize these foibles specifically in terms of side effects of our experiential intelligence.

Perkins suggests that the human mind is mostly akin to a pattern seeker and pattern-driven problem solving engine and as a result its weaknesses are also those we would expect from a pattern-driven process. The human mind often tends to be hasty, narrow, fuzzy, and sprawling.
HASTY. The goal of a pattern seeking intelligence is to find the right response that most closely matches the current situation rather than making an exhaustive search. As a result, our experiential intelligence tends to mislead us to jump to hasty conclusions when the situation is an unusual variation of a known situation.

NARROW. As a result of efficiently seeking patterns we have already seen, the domain-specificity of expertise tends to make us think in narrow ways when we think we have grasped the situation rather than to broaden our thinking.

FUZZY. Part of the power of pattern-matching is that we can so often generalize the lessons from one situation to another similar one. In situations where the appearance is very similar but the underlying principles are different, again our pattern matching effectiveness leads to mistakes: we overgenerallize from our experience.

SPRAWLING. When a pattern-seeking process does not have a single clear path to follow, as often happens in very complex situations, it will tend to follow one path after another and keep switching back and forth rather than working toward an overall goal.

Experiential intelligence, Perkins concludes, is an elegant system for long-term moderate success. When situations are new to us or complex, we get help from our neural intelligence and we have also learned various tricks for getting around our weaknesses, and these are largely accounted for in reflective intelligence. Reflective intelligence represents realms where we think about our own thinking in order to avoid settling on hasty conclusions, to broaden our thinking beyond the initial scope we assumed, to use precision to distinguish similar looking but different things, and to stay on track when notice we are sprawling.

This explains why reflective intelligence is so important to us in tricky situations where we have inadequate experience and where experience misleads us. But it also helps explain, in Perkins’ view, why reflective intelligence is so important for us to learn to be better thinkers in general. Neural intelligence does not replace experiential intelligence, it tends to reinforce it.

When we don’t have experience, neural intelligence helps us grasp the situation, but when we do have experience, we tend to use our neural intelligence to reinforce what our experience already tells us. That’s one big reason why genius is not simply high IQ. That’s why reflective intelligence is so important, it is the tool we use to remind us of the weak points in our own thinking and help us compensate for them regardless of our experience and general intelligence. The abilities and traits we need in order to overcome our blind spots are learnable. A large and crucial aspect of intelligence is learnable.

Existing Approaches: How they Compare

There are various approaches to teaching reflective intelligence, and Perkins reviews the best known and the best studied among them such as Project Intelligence, Reuven Feuerstein’s Instrumental Enrichment, Edward de Bono’s CORT, and Matthew Lipman’s Philosophy for Children, and others, reviewing their approaches and their results and comparing and contrasting them in order to get a sense of what it takes to enhance reflective intelligence.

One of the things that distinguishes Perkins as a deep reflective thinker himself is that he anticipates, researches, and deals fairly with opposition to his arguments. The very idea of learnable intelligence has in the past come under attack from several angles such as past failures of various programs which tried to teach improved thinking, the implications of expertise and psychometric research data, the apparent weakness of general methods for problem solving, and the challenge of transfer between learning domains. Perkins addresses each of these concerns in turn, resulting in a very persuasive case for the very real improvability of intelligence through changes in education.

The Future of Learnable Intelligence

Toward the end of the book, Perkins reveals the ingenuity of his vision through his discussion of several areas for the future evolution of reflective intelligence: areas which ended up being (remarkable for a book written in 1995) accurate predictions of areas that have since become central areas of interest for science and human improvement in general:

1. Intelligence can become distributed -- good thinking depends upon artifacts to offload the limitations of our attention and memory, and we can use our symbol systems and tools to help us keep track of things we could not track individually. This is a wonderful general description of how we are attempting to use computer networks to help us manage complexity (as opposed to some of the more superficial books in recent years which imply that networks somehow replace rather than enhance individual thinking).

2. Intelligence can embrace complexity -- through information visualization tools, effective use of classification, tagging, and finding things by meaning, consolidation, filtering, the mathematical tools for finding large scale patterns in complex phenomena, and by eliminating narrow information silos, we can use our intelligence to solve increasingly complex problems.

3. Intelligence can be dialectical -- this means raising the level of thinking from lower level more concrete concerns to higher order patterns by recognizing the properties specific to complex systems. Perkins offers Peter Senge’s “The Fifth Discipline” and Murray Gell-Mann’s “The Quark and the Jaguar” as exemplifying ways of understanding dialectical intelligence.

Perkins covers a massive amount of data about intelligence and problem solving, summarizes it effectively, and applies it to a practical, powerfully supported, and exceptionally understandable approach to improving human life by teaching ourselves to be more intelligent. Thinking well in general is an unnatural act but we can learn to do it. All that is left is for us to overcome the ideological and political barriers. This book would make a wonderful, gentle manifesto for that grand effort.

Thursday, May 20, 2010

Updated Book Review: David Shenk's "The Genius in All of Us"

Review of "The Genius in All of Us," by David Shenk, Doubleday, 2010.

An effective deconstruction of hereditary talent, and clues for a new model of exceptional ability

Link to the review on Amazon: http://www.amazon.com/review/R3DGLT1WYK6QRG/ref=cm_cr_rdp_perm

It is easy to like or dislike this book from a casual reading based on how you feel about the premise: that everyone has the potential for genius, and that heredity is not destiny in any sense. This sounds at first like a liberal political statement, but Shenk's treatment is far more nuanced than that characterization would imply.

In brief, Shenk's book is a very good deconstruction of hereditary talent, a competent but one-sided (or upon reflection I'll say very selectively focused) review of supporting research in several fields, and an interesting but abbreviated practical introduction to the interactionist (gene X environment) paradigm of development.

Just to be clear, this book is not about the psychometric definition of genius in terms of how far down the bell curve one is on Raven's Progressive Matrices or standardized tests of any sort. Nor is it about clever calculating tricks or precocious abilities, although it does do a very nice job putting those into a larger perspective. This book is more centrally about the expansive and inclusive sense of genius meaning people that accomplish something truly special and significant, and the potential that any given person may be able to get to that point. Somehow. And that's where the nuance is needed and appropriate.

Ok, I didn't like this book all that much when I first read it, and I at first gave it a mediocre 3 star rating on Amazon. I felt it did a great job deconstructing the concept of hereditary talent, but I strongly criticized it for leaving a gap where we need a better theory of where talent comes from and what it is, since obviously we don't all become true geniuses. Even among the folks who appear to have the seeds of genius in them from early on, most don't become genius adults in the broader sense.

In my original review I said this was a one-sided review of the evidence for the interactionist model. I do think it's a very selective review, but one-sided implies that he deliberately ignores contradictory evidence. He doesn't do that. He just doesn't talk about the evidence that led to the model Shenk says is obsolete, that genes are akin to blueprints. That is, the evidence that different variations of allele sometimes have strikingly specific effects in a seemingly "normal" range of environments. The case for the model of heredity that Shenk is deconstructing is not entirely ignored, but it is glossed over in order to make his case for the interactionist model. I think that is why hereditarians like Galton, Spearman, and Charles Murray get so apparently shorted in this book, Shenk focuses entirely on what they get wrong and glosses over the things they may get right.

I suspect that we do inherit "predispositions" in some form under a very wide range of conditions, even if the underlying mechanism is more complex than we previously assumed. Even if changes in environments do alter the expression of genes, something like inheritance of traits clearly does happen in a wide range of "normal" environments, and we can't just ignore that completely because of additional complexity and things that change at the extremes. That's why I say this is a very selective review. But no, it isn't really one-sided, the selectiveness is appropriate for a deconstruction, although it does mark this as a deconstruction rather than a scholarly review.

The more important problem is that the model of talent that arises from this book is not particularly easy to understand. The author is strongly against thinking of genes as predispositions, and rather offers the perspective that genes are akin to "settings." So it would be easy to conclude that the author is saying that we have the ability to make anyone a genius just by tweaking a few settings. He isn't. Or, if you read it as I did upon my first reading, you might hear the author saying that "anyone can be a genius, but talent is complicated process, we don't know what is happening at each step, and so we don't know how to help people get there, but we know it's possible." That's perhaps a little closer to the truth, but it didn't seem very helpful to me.

The reason I updated this review and why I'm now expressing more appreciation for David Shenk's accomplishment here is that while the "settings" model of genes doesn't quite convey the message, I did find upon close reading and careful reflection that the author captured a lot with his examples and case studies of individuals. The thing that is missing is some way of tying together how people manage to select and shape environments for themselves to accomplish great things, in spite of all the cultural, social, and physical constraints that tend to make environmental factors very hard to change for most of us. Shenk assiduously avoids attributing "predispositions" to genes, but then speculates that epigenetic factors may predispose us to things like musical ability. If non-genes can do this, why not genes? He just seems a little *too* intent on crushing hereditary talent in some places.

Geniuses don't just see things differently (although that is sometimes also going on), they don't just have unique abilities (although sometimes they do) geniuses are most distinct in that they manage to carve their own niche, exploiting their own uniqueness in a process where they are driven to mastery and are amazingly persistent, even where the goal seems way out of reach. This runs contrary to our popular wisdom that it makes sense to work toward small easily attained goals in most things. What we think of as really deep talent actually requires really deep faith in the long term process and the motivation to keep going. Shenk captures the significance of motivation, but I had to look very closely to see the patterns for it. It requires willingness to do things that others may find bizarre and to learn freely from what is available. The author illustrates this but seems to have a hard time really tying it all together, at least he did on my first reading. I've come to think of it in terms of niche construction, which to me really captures what exceptional people do that brings out and shapes their unique gene x environment combination in a targeted way. My reversal in the rating reflects my feeling that capturing this idea is more important than giving it a catchy name, which is really what the author is missing.

We don't know exactly how to take advantage of the dynamic nature of heredity and development, although the study of achievement and expertise reviewed by Shenk gives us many tantalizing clues to go on. And if knowing that the potential is there inspires the faith to keep going, then more and more of us will eventually learn to become better and better at using our minds, constructing our own niches from our own individuality, and the promise of "The Genius in All of Us" will eventually begin to be realized. There is a lot in this book that will repay careful reading and re-reading, as I discovered by doing exactly that.

Related Reading:

See also this classic manifesto of genetic interactionism: The Triple Helix: Gene, Organism, and Environment(Lewontin R (1998/2000) Triple Helix: Gene, Organism, Environment. Cambridge, MA, Harvard)

This superb earlier popular introduction to the emerging model Shenk offers: The Agile Gene: How Nature Turns on Nurture(Ridley, The Agile Gene)

This similar treatment of trait development in interactionist terms, but focused on personality: The Temperamental Thread: How Genes, Culture, Time and Luck make Us Who We Are(kagan, temperamental thread)

This alternative and original interactionist account of how personality develops: No Two Alike: Human Nature and Human Individuality (Judith rich harris no two alike)

This interesting challenge to some widely help assumptions about influence: Stranger in the Nest: Do Parents Really Shape Their Child's Personality, Intelligence, or Character?(Stranger in the Nest, D. Cohen)

This little known treasure by an old friend that offers its own unique challenges about human uniqueness and what it means: rebellion: physics to personal will (Brody, Rebellion)

This on the classic view from the perspective of behavior genetics: Genetics and Experience: The Interplay between Nature and Nurture (Individual Differences and Development)(Plomin, Genetics and Experience)

This on the fascinating broader biological implications of interactionism from a gene perspective, how the genes of organisms construct niches even beyond the organism itself: The Extended Organism: The Physiology of Animal-Built Structures(Turner, The Extended Organism)

And finally this wonderful broad account of biology and the role of heredity that appreciates the complexities of gene function in a demanding but uniquely engaging way: The Logic of Life(The logic of life, francois jacob)

Saturday, September 19, 2009

Book Review: NurtureShock by Po Bronson and Ashley Merryman

Amazon review link.

The recent easy availability of science news has been a mixed blessing. On the one hand, we get the breaking news every time someone publishes anything even remotely interesting. On the other hand, it is even harder to perceive the consensus in scientific fields. We get the feeling of things changing and new data coming in, but not a good sense of the overall patterns and how they affect existing theories, because the process of consensus building in science happens over decades, not weeks.

The scientific consensus on child development and parenting has been gradually but insistently shifting over the past decade or so. The overall picture can't easily be seen from individual news stories, so books like NurtureShock which give some insight into the big picture are very important. NurtureShock to me represents the second huge bombshell in child development theory applicable to the average person. The first was presented in Judith Rich Harris' The Nurture Assumption: Why Children Turn Out the Way They Do, Revised and Updated, which argued persuasively and shockingly that most differences in parenting made little difference to long term outcomes in their children's lives. Harris insisted that children instead were mostly raised by socialization in their peer groups. NurtureShock doesn't argue the Nurture Assumption viewpoint at all (in fact it is largely consistent with Harris in most respects), but it focuses instead on the areas where parents really might be able make a difference.

What is the emerging new consensus according to NutureShock? The term is intended to reflect the shock that new parents feel when the fountain of natural wisdom about caring for children that they expect to serve them just doesn't appear. Our instincts are to love and care for our children, not to automatically know the right things to do. So we often turn to child development research. What does that tell us?

NurtureShock tells us that some of our commonsense is actually right afterall, and that some of the popular assumptions made about children are wildly off the mark. In particular, two big "myths" are identified. First, children are not just small adults, and we can't just apply the same principles to them that we apply to adults. Second, there are no supertraits that confer only good things: emotional intelligence, intelligence, gratitude, happiness, self-esteem, honesty, fairness, and so on can all have their dark side as well as their positive side. Negative elements can and do co-exist with high levels of positive traits.

For example, measured analytic intelligence (IQ) appears stable in adults, but changes in fits and spurts during childhood. This has significant implications for the use of childhood standardized testing to predict later ability.

Praising adults usually helps encourage them (as long as they believe it is sincere), but in children the effect is more often paradoxical. Children praised too much or for the wrong things actually end up worse off as a result.

Similarly, the once-trendy and still popular obsession with self-esteem turned out to have very little evidential support, and bullies often turn out to have very high self-esteem. Emotional intelligence is also not the panacea that some of its proponents originally suggested. Criminals appear to have a higher, not lower, level of emotional intelligence than the general population, and they use that ability to manipulate others. Popular kids in school use their skills at empathy not so much to be sympathetic, but to play social games that improve their status.

Even insisting on honesty is a mixed bag. Kids seem to learn to lie as a natural part of developing their thinking and social skills, it is related to intelligence. But you can't just ignore it because it is part of development, or it will become a habitual pattern for dealing with difficult social situations. Parents have to learn when and how to encourage honesty. Kids faced with the constant threat of punishment lie more to protect themselves rather than less, and they learn better to evade getting caught. They are generally more motivated to be honest to please parents than to avoid punishment.

The best thing about this book is that it doesn't just promote or adhere to a standard socio-political agenda for child raising the way many books do. This isn't just rehashed progressive or conservative childrearing strategies, it reinforces some of the best elements of each of the different models. We see that threats and punishment are a particularly ineffective way of dealing with dishonesty, but setting limits and enforcing rules is crucial to helping teens know they are cared for.

We find that, perhaps unsurprisingly, teens are particularly prone to boredom and often act out as a result, and that there is not too much we can do about it. They don't respond to small or moderate rewards, but then respond in an exaggerated way to large rewards. This extreme-based decision making pattern in teens varies greatly between individuals but in many leads to the distinctive kind of risk-taking judgment that teens often exhibit. Unfortunately, there's not much in the way of advice here, just perhaps a little understanding.

Among the most important findings in this book are those dealing with thinking skills, especially metacognition and "executive function" skills. Both educational research and brain science seem to be reinforcing the importance of helping children learn the skills for teaching themselves, controlling their own attention and motivation, and evaluating their own performance. Some of these skills are general, but many are specific to particular subjects, so teaching thinking skills cannot be separated from teaching subject matter, as was sometimes mistakenly done in the past. This is a very difficult topic, not one amenable to many easy heuristics, but it is crucial to education.

This is a very important book, rich with research examples and also practical examples from the authors. It will make you think twice about some of your instincts and some of the things you've accepted from popular belief, and that will in turn help make you a more flexible and skilled parent or teacher.