Why AGI Won't Fix Our Broken Institutions

The author argues that increasing intelligence through AI will not solve major societal problems because the true bottlenecks are regulatory and institutional rather than cognitive. In sectors like medicine, progress is hindered by the high cost of clinical trials and misaligned economic incentives that AI alone cannot fix. Ultimately, the piece warns that over-reliance on the promise of AGI distracts us from the necessary work of structural and social reform.

Key Points

• Real-world progress in fields like medicine and housing is bottlenecked by regulation, governance, and political will rather than a lack of raw intelligence.
• In biotechnology, the 'grind' of clinical trials and the difficulty of accessing human data are the primary constraints that AI cannot easily bypass.
• The current patent system encourages 'target herding,' where companies use AI to optimize known molecules rather than taking the risk to discover new biology.
• Technological diffusion into the economy is significantly slower than the growth of underlying capabilities due to institutional friction and human inertia.
• The San Francisco tech culture has become a monoculture that treats AGI optimism as a status symbol, leading to a blind spot regarding practical, structural barriers.

Sentiment

Skeptical of the 'AGI-pilled' narrative, with a focus on practical, social, and physical constraints that intelligence alone cannot solve.

In Agreement

• Human nature is a fundamental blocker that persists regardless of technological advancement.
• The limit of progress is often communication and I/O rather than raw intelligence.
• Systems are inherently limited by the communication structures of the organizations that create them, as per Conway's Law.
• Physical embodiment and the execution of plans in the real world represent a massive bottleneck that AI has not yet solved.
• AI currently lacks genuine ingenuity, which is distinct from intelligence.

Opposed

• The embodiment bottleneck could potentially be solved by using humans as physical 'avatars' for AI-guided work.
• The human 'bottleneck' should be viewed as a necessary feature for safety and control rather than a problem to be bypassed.