Forecasting
Experts Guess. Superforecasters Know Better.
Tetlock proved forecasting is a trainable, measurable skill, yet officials deciding wars, pandemics, and economies are almost never scored on accuracy.
More accurate than intelligence analysts with classified access
The evidence
The Trillion-Dollar Guess
In 2002, US intelligence told policymakers Iraq had active weapons of mass destruction programs. A bipartisan commission later found the government was wrong in almost all of its pre-war judgments (Iraq Intelligence Commission, 2005). The resulting war cost an estimated $2 trillion and over 4,500 American lives (AP, 2023).
Experts, Barely Better Than Chance
Philip Tetlock tracked 284 political and economic experts across roughly 82,000 forecasts made over 20 years. The average expert performed only marginally better than random guessing (Tetlock, Expert Political Judgment, 2005).
Superforecasters Beat the CIA
When US intelligence funder IARPA ran a four-year forecasting tournament, the sharpest 2 percent of volunteers, dubbed superforecasters, outperformed professional intelligence analysts with access to classified information by more than 30 percent (Good Judgment Project).
Aggregation Beats Any Single Genius
Pooling and extremizing forecasts from small teams outperformed the raw crowd average by roughly 10 percent in Good Judgment Project trials. Metaculus has logged a community Brier score of approximately 0.11 across thousands of resolved questions.
We Disagree Most on What Matters Most
In the largest existential risk forecasting tournament ever run, domain experts estimated a median 6 percent chance of human extinction by 2100, while superforecasters estimated roughly 1 percent, a sixfold gap on the highest-stakes question humanity faces (Forecasting Research Institute, 2023).
A Field Still Waiting for Investment
Tetlock's research began in 1984. Roughly four decades later, Open Philanthropy only launched a dedicated Forecasting grantmaking program in 2024, and total philanthropic funding for the entire field remains small next to the scale of decisions it could improve.
What we can do
Phase 1: fund the infrastructure, platforms like Metaculus and organizations like Good Judgment and the Forecasting Research Institute need resources to run tournaments across defense, health, and economic policy. Phase 2: mandate it, agencies and legislatures making high-stakes calls should log calibrated, scored probability estimates before acting. Phase 3: make accuracy public, building permanent track records for pundits, officials, and institutions.
Sources: Philip Tetlock (Expert Political Judgment, Superforecasting), Good Judgment Project, IARPA ACE Tournament, Iraq Intelligence Commission (2005), Metaculus, Forecasting Research Institute, Open Philanthropy
Community Solutions
Anyone can propose a solution. The community votes on the most impactful ideas.
Loading solutions...
Companies Working On This
Submit a CompanyApproved companies and organizations addressing this problem. Anyone can submit one for review.
No companies listed yet for this problem. Submit one.
