Predicting the Improbable

Great new paper from Voxeu on people’s behavior around the unpredictable. In lotteries they tend to under-select recently drawn numbers, only to get for them in a big way if the number is drawn repeatedly. Reproduced here, with permission.


Predicting the improbable

Claus Bjørn Jørgensen Sigrid Suetens Jean-Robert Tyran
22 April 2011

Japan’s trio of tsunami, earthquake, and nuclear disaster has left the world stunned. As this column points out, even the experts were shocked. But while these events were highly unlikely, they were still possible. This column uses evidence from the Danish lottery to show that people tend to adjust their expectations of future events based on only small pockets of recent experience, often at their cost.

Important events are hard to predict – a fact that is particularly hard-felt when it comes to low probability events with dramatic consequences. Nuclear catastrophe, financial crisis and the like are things that even experts struggle to predict. The difficulty stems from a lack of understanding of the underlying factors and complex interactions among causes (probabilities are not independent but conditional on other events).

Experts are thus to some extent forced to base their predictions on inference from observing the past. A difficult issue is to know when a model should be revised given that an event that has been deemed to be highly improbable happens to occur. The issue is most relevant for policy recommendations. For example, what recommendations should experts provide for the regulation of nuclear power in the wake of the Fukushima disaster or for the regulation of banks in the light of the recent financial crisis?

While experts struggle to predict such events accurately, the average person is often simply baffled. They tend to misperceive randomness in a variety of ways, especially when it comes to rare events.

  • One common tendency is to see patterns in random data when there are none.

This can lead to a tendency to overreact to recent events, allowing their occurrence to change beliefs about future events in exaggerated ways. More specifically, many people tend to over-infer characteristics of the underlying probability distribution when observing a small number of random events. A literature pioneered by Tversky and Kahneman (1971) has identified the belief in the “law of small numbers” as the source of such over-inference.

[Read more…]

End Times Ahead, Meteorologically Speaking

Texas Governor Perry today trying to out-Onion The Onion:


WHEREAS, the state of Texas is in the midst of an exceptional drought, with some parts of the state receiving no significant rainfall for almost three months, matching rainfall deficit records dating back to the 1930s; and

WHEREAS, a combination of higher than normal temperatures, low precipitation and low relative humidity has caused an extreme fire danger over most of the State, sparking more than 8,000 wildfires which have cost several lives, engulfed more than 1.8 million acres of land and destroyed almost 400 homes, causing me to issue an ongoing disaster declaration since December of last year; and

WHEREAS, these dire conditions have caused agricultural crops to fail, lake and reservoir levels to fall and cattle and livestock to struggle under intense stress, imposing a tremendous financial and emotional toll on our land and our people; and

WHEREAS, throughout our history, both as a state and as individuals, Texans have been strengthened, assured and lifted up through prayer; it seems right and fitting that the people of Texas should join together in prayer to humbly seek an end to this devastating drought and these dangerous wildfires;

NOW, THEREFORE, I, RICK PERRY, Governor of Texas, under the authority vested in me by the Constitution and Statutes of the State of Texas, do hereby proclaim the three-day period from Friday, April 22, 2011, to Sunday, April 24, 2011, as Days of Prayer for Rain in the State of Texas. I urge Texans of all faiths and traditions to offer prayers on that day for the healing of our land, the rebuilding of our communities and the restoration of our normal and robust way of life.

IN TESTIMONY WHEREOF, I have hereunto signed my name and have officially caused the Seal of State to be affixed at my Office in the City of Austin, Texas, this the 21st day of April, 2011.

Governor of Texas

Field Notes: OIl, Weather, Drought, Microsoft, iPhone, etc.

  • First Drought, Now Heat Impacting Kansas Wheat Fields (Source)
  • Strange Spring: Explaining This Year’s Wild Weather(Source)
  • Oil Through the Ages | Maps (Source)
  • Era of ‘tough oil’ won’t deter drillers  (Source)
  • Revolutions and the price of bread: 1848 and now (Source)
  • Five-Year TIPS go negative again (Source)
  • Microsoft plans sweeping pay rises (Source)
  • iPhones Fuel Rise in Subway Theft(Source)
  • What Mortenson Got Wrong : The New Yorker (Source)
  • McDonald’s warns of higher food inflation | Reuters (Source)

Some Me Notes: Crisis, Gambling, Fiat Money, Goldman, etc.

A couple of my recent notes over at Bloomberg:

  • Flash: Goldman Sachs has clients! (Source)
  • The fiat money experiment has failed (Source)
  • The crisis in the flight to safety crisis (Source)
  • Online gambling: Where to from here? (Source)

Cormac McCarthy and Werner Herzog Talk

From last week’s Science Friday, Cormac McCarthy and Werner Herzog. Remarkable to have them together talking art and science.

Ueli Steck Solo Speed Climb of Eiger

Ueli Steck’s solo speed climb of the Eiger’s North Face in 2008. Breathtaking — especially at fullscreen HD.

Desperately Seeking Symmetry

New Radiolab podcast is out: Desperate seeking symmetry.

Field Notes: Lester Bangs, Porsche, Boston Marathon, China Crops, Bicycles, Poker, MBAs, etc.

  • Lester Bangs’ Basement, or Why nothing’s rare anymore (Source)
  • 1985 Porsche 911 shift pattern as Los Angeles (Source)
  • Jane Jacobs on Cities (Source)
  • Wind and the Boston Marathon (Source)
  • The case for industrial real estate (Source)
  • The secret ratings agency downgrade meeting (Source)
  • China Crops in Short Supply as Fewer Farms Spur Food Futures (Source)
  • Federal Poker Indictments: Revisiting Prohibition (Source)
  • Out of Disaster, a Burst of Enthusiasm for Bicycling (Source)
  • High Wellbeing Eludes the Masses in Most Countries Worldwide (Source)
  • Motorcyclist Fatalities Continue Decline (Source)
  • Business education: The race to the bottom  (Source)
  • Business Educators Struggle to Put Students to Work (Source)

Being Wrong and Getting Stuck

One of my favorite talks at this year’s TED: Kathryn Shulz on errors & being wrong. In the spirit of full disclosure. Kathryn is a friend, but the talk is super and a must-watch. (And her book on being wrong is excellent.)

Progression of Boston Marathon Records

The progression of Boston Marathon record times since the course change in 1957. I added a trendline to put the incredible — and incredibly wind-assisted — 2:03:02 time posted today in perspective. [-]