How reliable are experts?

So Channel 4 has enjoyed itself ‘exposing’ disagreements among environmental experts on the use of GM and nuclear power. Nothing new or surprising about that.  On the whole we don’t expect them to agree — let alone be right.  Thus, according to David Aaronovitch (The Times, October 28 2010), The Economist in 1984 “asked 16 people to make a series of ten-year forecasts on various economic indicators.  Four were former finance ministers, four were economics students, four were corporate chairmen and four were London refuse collectors.  Their predictions, ten years on, were all pretty bad, but in first place, tied, were the chairmen and the dustmen. The finance ministers were last”.

The worry though is that the experts who speak with the loudest voices are the ones who seem to get the attention of policy makers.  This is borne out in another piece of research that Aaronovitch cites.  It’s a study conducted by Philip Tetlock, a psychology professor at the Haas School of Business in California.  “The Tetlock team collected more than 27,000 judgments delivered by nearly 300 experts over more than 30 years.  The conclusion was that their predictions were no more accurate than random guesses you or I might have made.  But there was more to it than that.  Some experts were better than others, and Tetlock looked at the characteristics of the worst and the best.  The wrongest experts were, in essence, reductionists . . . people who had a ‘core theoretical theme’ to their thinking and stuck to it. . . .[and] were more confident. . . But those who had doubts, were more tentative or embraced more possibilities did far better.” . . . Quoting a forth-coming book from Dan Gardner,  he concludes that “complex and cautious thinking trounced simple and confident”.

So a plea:  can we please substitute all this certainty for some caution and humility?

One thought on “How reliable are experts?

  1. Expert opinion is always worth using as a starting point, although Richard Dawkins regularly slams the argument from authority (without back up test evidence) in his conflicts with religious fundamentalists. Yet I like too particular comments (one sensible and one flippant) about experts.

    The great Harvard biologist E.O.Wilson wrote in “Consilience” that “I must always remember that I could be wrong”. My old boss in the late 1970’s had a wonderfully warped sense of humour and said an expert is a combination of an ex (i.e. a has-been) and a spurt (a drip under pressure).

    Frivolity aside, perhaps it is worth remembering the dangers of over-confidence in any area and that there are often exceptions to seeemingly clear cut rules. Anyone stating “I know I’m right” or even “You know you’re wrong” should set the alarm bells ringing. “I think this is true because …. but can you show me something new” is much more like it.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.