Would OpenAI's High-Minded Goals have ever worked?
Some research motivated by the OpenAI kerfuffle
It has been a few weeks since OpenAI imploded and reconstituted itself with a new board. With the dust settling, the best account seems to be this one from Charle Duhigg at The New Yorker. Matt Levine, of course, has the best take.
I really do have to quote last week’s incredible Wall Street Journal report about the board’s non-explanation of what Altman did to get himself fired:
“On the call, the leadership team pressed the board over the course of about 40 minutes for specific examples of Altman’s lack of candor, the people said. The board refused, citing legal reasons, the people said. …
The board agreed to discuss the matter with their counsel. After a few hours, they returned, still unwilling to provide specifics. They said that Altman wasn’t candid, and often got his way. The board said that Altman had been so deft they couldn’t even give a specific example, according to the people familiar with the executives.”
“Without realizing it, we were gradually overmatched by a superior intelligence, until he ended up controlling us in ways that are too subtle for us to even explain,” thought the AI-nervous board of OpenAI. I love them. Their fears about rogue AI are such obvious metaphors for their mundane real-life problems.
So true.
During this time, I became interested in the meta-problem if you will: did the plan to set up an organisation to develop artificial general intelligence (or AGI) with socially-minded governance to prevent harmful AI from being disseminated ever stand a chance? Lots of people have claimed that there is no such thing as safe AGI and other reasons why some socially-minded direction wouldn’t work. But I wondered if it would work if everything OpenAI were assuming about the world was true. That is, if AGI could be developed, and, if that happened, a ‘safe’ version was possible (and could be done cheaply), and, finally, there were people who were socially-minded willing to put up the funds to make all of that happen. In other words, this is surely the best-case scenario for making a strategy of “controlling the beast” work out.
The answer is a pretty definitive NO, and here is the paper with the model for those interested. So, what is the reasoning here?
First of all, while I assume there are people willing to fund an OpenAI idealised enterprise and sacrifice some potential profits, they would like OpenAI to release a safe AGI that funded its development. This meant that OpenAI would have an objective of minimising the total amount of harmful (or unrestricted) AGI out there but subject to a break-even constraint. As it turns out that constraint can be important but not definitive.
Second, the OpenAI folks don’t have full control over AGI. Others, at considerable cost, can develop AGI too, and, of course, they aren’t likely to be socially-minded.
Much depends on whether there can be a scenario whereby a safe and unrestricted AGI set of products co-exist, potentially in competition with one another. A for-profit entity will earn lower profits if they have an unrestricted AGI but it is in competition for users with safe AGI. Some consumers may care and want the unrestricted version and be willing to pay more for it. Others may not care. In this situation, and this is important, even a for-profit entity not facing competition has an incentive to release safe AGI because they can price discriminate and raise prices to those who value an unrestricted product. But if there is competition, they will find it optimal to only release an unrestricted product. The revenue from that may not cover their investment costs, meaning that if OpenAI can get to AGI first, then they can actually preempt for-profit entities and prevent the release of unrestricted AGI.
The “if” statement is doing all the work. The problem is that the revenue OpenAI can earn, even as a monopolist, off of safe AGI only is less than the revenue the for-profit entity can earn from selling both. As each race to develop AGI first, the for-profit is more motivated while OpenAI is constrained financially, and so the prediction is a for-profit entity will win. (Hang on, you say, isn’t OpenAI already winning. It is hard to tell, but no one has developed AGI yet, so the jury is still out on evaluating this prediction).
But what if OpenAI did, by chance, manage to get there first? Yes, it is true that if the for-profit entity didn’t find it profitable to enter into competition with OpenAI, that would be the end of the story. However, it is more likely that they will find it profitable to enter even if there is competition.
You might think that leads to a situation where OpenAI competes with for-profit entities, and there at least exists safe AGI provided by them. But you have to look at it from OpenAI’s perspective. The alternative is not to develop AGI first at all. Why might it do that? Well, if competition is inevitable, that competition works against OpenAI’s goal to have as much safe AGI out there as possible. Competition causes the for-profit entity to lower its prices on unrestricted AGI and so there is going to be more unrestricted AGI out there. There is literally nothing OpenAI can do to “help”
in that situation. If it enters with high prices, it changes nothing. If it tries to make safe AGI competitive, it increases the amount of unrestricted AGI out there. In other words, OpenAI cannot be effective in competition with others.
Given that, faced with inevitable competition, it is best for OpenAI not to push hard to “win” the AGI race. And so, once again, we are left with the prediction that the for-profit entity will win out. Either way, whether OpenAI as a socially-minded entity exists or not does not matter for what we actually end up with. They might have more chance of saving the world by waiting for unsafe AGI, then inventing a time machine to go back and stop whomever invented it from doing so. But even then, this very same model predicts that, at best, they might delay things.
So we are left with the situation that the whole board kerfuffle was about a set of issues that, at heart, do not look, at least from where I stand, as being important for the future direction of AGI development. At best, we may make it a little easier to have safe AGI products available so that for-profit entities have an incentive to commercialise those products to price discriminate against those who want unrestricted AGI. But if we want something that sticks on safety, this all points to it being a government-run endeavour — and a global one at that.
But I do want to leave you with a quote that sums up the world that I was pointed to be Jeff Maurer’s podcast.
“I don’t know about you people, but I don’t want to live in a world where someone else makes the world a better place better than we do.” (Gavin Belson, Silicon Valley, S2E1)