Randomness and the Clustering Illusion

by Stephen Mills on May 8, 2010

Dice

Human beings are not intuitively good at detecting randomness.  We see order where none exists.  We demand explanation where none is required.  The pattern seeking nature of our brains is what makes us different but sometimes it can lead us astray.  Experiments with split brain patients prove that when we don’t have an explanation for something, our brain will just make up an explanation that sounds plausible.

A Texan shoots at the side of a barn and then draws a target around the holes and claims he is a sharpshooter.  This is the humorous description of the Texas Sharpshooter Fallacy.  When you find find a clump or cluster of something in a sea of randomness and you draw your target around the cluster you are doing the same thing.  Clusters are normal and expected in the real world.  When a cancer cluster is found in a neighborhood, people start looking for explanations in power lines and chemical plants.  They are doing little more than drawing targets around holes in a Texas barn.  I am not saying there can be no environmental cause, just that most of the time there is not; it’s just random clustering.

There is a professor, of whom I can remember no biographical information, who likes to do a demonstration for his class.  Two pairs of students are selected to record a series of 100 coin flips in his absence.  One pair is actually flipping a coin and recording the actual sequence, and the other pair is making up a fake series of flips.  It is very easy for the professor to detect the fake series upon his return.  The fake series does not contain enough streaks of heads or tails.  The clusters are missing.

I flipped a quarter 300 times.  At one point I had a series of nine consecutive heads.  If you picked up a quarter and flipped it nine times, the likelihood of getting nine heads would be extremely unlikely.  But finding such a streak in 300 flips is not all that unusual.  To draw your target around those after the fact and insist on an explanation makes no sense.  Here is a typically random twenty flips that to most people looks clumpy.  Six of the first seven flips are heads and seven of the last ten are tails

H T H H H H H T T H H T T H H T T H T T T T H T H

Everyone knows NBA basketball players get into the zone and go on hot shooting streaks and at other times go cold and can’t sink a shot to save their lives.  However, some scientists studied the Philadelphia 76er’s and proved that their hot and cold shooting streaks were no greater than would be expected by chance.  Their shooting streaks might has well have been coin flips.

I just dealt myself a bridge hand.  Below is the hand I drew where S=spades, C=clubs, H=hearts, and D=diamonds.

KD, 10D, 5D, QC, 10C, 3C, 8H, 2H, AS, 6S, 5S, 3S, 2S

It looks unremarkable right?  Here’s the thing.  The odds of me drawing that exact hand are 1 in 635 billion.  You will never get that hand in a card game for the rest of your life.  You are as likely to get all 13 spades as you are to get that hand.

Here are three possible coin flip series of ten flips:

H T T H H H T H T T
H H H H H H H H H H
T T T T T T T T T T

All three series are equally likely.

All of this is counter-intuitive but it’s true.  What we often see as order or requiring explanation is nothing more than random patterns.

What do YOU think?  Leave a comment and join the conversation.

arrow-small

Get Free Updates to The Rat Race Trap by Email here or via a reader in the top left sidebar.  I would love to have you on board.

Related posts:

  1. Should You Trust Your Intuition? I’m a strong believer in the power of human intuition,...
  2. The Last Monkey Jumping Darren Bukket has lasting fame and fortune in the monkey...
  3. More Myths That Need Busting Even Europeans who ban genetically modified food are eating exactly...
  4. How To Make Decisions When You Can’t Decide This article describes several methods of making quick decisions when...
  5. Irrational Decisions – Anchoring and Arbitrary Coherence Professor Ariely describes some experiments which demonstrated something he calls...

{ 2 comments… read them below or add one }

John D. Buerger, CFP® May 9, 2010 at 9:51 am

Pattern seeking is one of the biggest reasons why smart people make stupid choices with their money (if not the biggest). Combined with Recency Bias (the tendency to take whatever patterns have been visible recently – real or imagined – and project that trend into the far flung future) explain why the average investor “enjoys” investment returns 7% below the market (Dalbar Study).

Life happens folks – and it comes with risk. Sometimes you just can’t explain it, but rather have to go with it … maybe even embrace it. Now there’s a novel idea: “embrace risk.”

Reply

Dustin Goerndt May 11, 2010 at 4:40 pm

Your very argument is a clustering of evidence to support your position on randomness and it is a real pattern. Patterns are keys to a level of sight. This is a reason our brain has evolved in this way and I would suggest a little more faith in life.

Our brains are computers and life is the code we seek to understand. In order to find the nuggets of information that serve us we must exploit the redundancy or patterns. With every understood pattern we begin to see the redundancy on a new level and intern gain new information that will show a new pattern that exploits a new level of redundancy.

I’m not quoting anyone here, but I’m sure if you spent some time with information theory you’d catch my drift.

I wouldn’t say pattern seeking is a fault. The problem is thinking you’ve hit pay dirt just because you’ve found one.

This might sound like a rat race, but really it’s not. We’re human not rats. It’s about learning the weather of life. Soon enough you’ll get it. You learn to see the rain coming, the snow, and then sometimes the sun. Now the rat race becomes a journey.

Flipping coins, shooting baskets, and drawing cards are very simplistic events in a otherwise potentially infinite world. Now that you’ve found your pattern it’s time to move on.. ;)

Reply

Leave a Comment

CommentLuv badge