Navigating Transformative AI
The Most Powerful Technology, The Least Oversight
The scientists building it warn it could kill us all, yet AI safety research remains a rounding error next to AI's investment boom.
Median AI researcher's estimate of extinction-level risk from advanced AI
Live counter
Estimated so far this year.
The evidence
Even Its Builders Are Scared
In a 2023 survey of 2,778 machine learning researchers, the median respondent put a 5% probability on advanced AI causing human extinction or similarly severe outcomes (AI Impacts, 2023). Over 500 AI scientists and executives, including the CEOs of OpenAI, Google DeepMind, and Anthropic, signed a statement calling this risk a global priority (Center for AI Safety, 2023).
The Compute Curve Is Vertical
The computing power used to train the largest AI models has grown 4 to 5 times every year since 2010, with frontier model training compute now doubling roughly every 5 to 6 months (Epoch AI, 2024).
Billions For Speed, Pennies For Safety
Global corporate investment in AI hit $252.3 billion in 2024 (Stanford HAI AI Index Report, 2025). By contrast, Open Philanthropy, the field's largest single funder of AI safety work, has directed roughly $336 million to the cause since 2017.
No Lab Passes The Safety Test
The Future of Life Institute's Winter 2025 AI Safety Index graded eight leading AI developers on existential safety planning. Not one company scored better than a D on existential safety specifically, for the second report running (Future of Life Institute, 2025).
The Harm Is Already Here
Documented AI-related incidents, including deepfake fraud and safety failures, rose to 233 in 2024, a 56.4% jump from 149 in 2023 (AI Incident Database, cited in Stanford HAI AI Index Report, 2025).
Governments Are Sprinting To Catch Up
US federal agencies introduced 59 AI-related regulations in 2024, more than double 2023's count, while US states passed 131 AI-related laws, up from 49 the year before (Stanford HAI AI Index Report, 2025).
What we can do
Phase 1 is funding technical AI safety and alignment research at a scale that matches the pace of capability investment, not a rounding error of it. Phase 2 is binding international governance, including compute monitoring, mandatory third-party audits, and pre-deployment testing for frontier systems. Phase 3 is requiring every lab racing toward AGI to clear an independent existential-safety bar before deployment, not just a public relations one.
Sources: AI Impacts, Center for AI Safety, Epoch AI, Stanford HAI AI Index Report, Open Philanthropy, Future of Life Institute, AI Incident Database
Community Solutions
Anyone can propose a solution. The community votes on the most impactful ideas.
Companies Working On This
Submit a CompanyApproved companies and organizations addressing this problem. Anyone can submit one for review.
No companies listed yet for this problem. Submit one.
