TLDR: Don’t rip on Anthropic, they’re warning us that our macroeconomic and political incentive structures are profoundly misaligned with the power of the tools we are unlocking.
My reading of this stuff is, as ever, one of optimistic rage. Optimism for what’s possible, rage at the petty minds which block it.
No amount of regulation or mutual slow-downs will put this cat back in the bag.
It should be a wake-up-or-get-the-fuck-out-of-the-way call for governments who dictate socio-economic incentives. “Show me the incentive and I’ll show you the outcome” — Charlie Munger.
All leaders of all political persuasions across all countries should take a moment to step out of their mid-20th century mindset because AGI is, essentially, here; and none of them are ready. Moreover, not one of them appears capable of being ready.
In WW2, when minds and incentives were focused on ending the war the people responded by inventing computing AND atomic power, two astonishingly brilliant weapons of peace. But look how swiftly human ingenuity became machinery for catastrophe when handed to the incumbent powerful. They didn’t see peace and abundance, they saw dominance through destruction.
Now, 80 years on, Prometheus has handed fire to another generation. What will we do this time? Have we learned anything?
The nature of power has changed, and the expansion of capabilities offer extraordinary possibilities for medicine, science, education and material abundance to name but a few. But YET AGAIN, we watch institutions approach those possibilities with the same ambitions for extraction, escalation and control which create exactly those race conditions which amplify catastrophic risk to a point of near-certainty.
When Anthropic warn that AI is being used to develop missile-guidance software, build state surveillance systems, and carry out theft and extortion, let’s not dismiss it as sour grapes and move on.
Globally, we cannot keep rewarding abuses of power, then act surprised when the release of increasingly powerful technologies into an unprepared world delivers precisely those outcomes.
So yes, scrutinise the auditors’ competence and Anthropic’s commercial motives. But even perfectly competent auditors cannot fix what governments actively incentivise people to do with these capabilities. How can we trust governments to regulate AI when their own incentives make them so selfish and incompetent these people can’t work out how not to make guns the biggest cause of child mortality, how to feed 8bn people when we produce enough for 11bn, how to regulate a mere handful of banks, and who are still surprised by China’s technological accomplishments when it has been PUBLISHING THE PLANS for twenty years.
Our institutional and economic incentives have been so neglected and abused that they are often utterly misaligned with humanity’s interests. So how can we expect AI to align with humanity’s interests when we pay them so little heed ourselves?
Human ambition for something better keeps being subordinated by dumb systems with dumb incentives, so the purpose of buying time has to be to coordinate to change both. Otherwise we’re slowing the technology while leaving the reasons to misuse it intact.
And if we can’t do that, well then screw it, maybe it’s time to e/acc, give AI a crack, and pray to our new Gods in the hope that they can get us out of our mess.