Is market power in AI inevitable?

In which I (well, Claude) channels Malcolm Gladwell

I have been a bit busy of late so I didn’t have the opportunity to write about a new paper of mine, “Market Power in Artificial Intelligence.” It’s a technical survey paper so it is a bit challenging to write about. But here goes …

In my journey to understand the emerging power dynamics in the world of artificial intelligence, I've discovered that the battle for AI supremacy is being waged across three distinct but interconnected markets: the market for training data, the market for input data, and the market for AI predictions themselves.

Think of training data as the textbooks from which AI systems learn. Just as a student from a well-funded school district might have access to a rich library of learning resources, an incumbent firm with vast troves of proprietary data can train its AIs to a high level of sophistication. Startups, on the other hand, are often left scrambling for second-hand books and outdated materials.

But here's where things get interesting. Imagine if that resource-strapped student had a secret weapon – a study group of engaged, active learners who constantly shared their insights and experiences. Suddenly, our underdog student can rapidly catch up and even surpass their privileged peers. In the world of AI, this is the "data feedback loop" – the process by which an AI system in the wild generates fresh data that can be fed back into its training pipeline, allowing it to learn and improve over time.

Of course, getting that AI system into the wild in the first place is no easy task. That's where the market for input data comes in. If training data is the textbook, input data is the steady stream of homework problems that keep an AI sharp and ready to tackle real-world challenges. And once again, the incumbent firms have a leg up. They're like the students with wealthy, well-connected parents who can secure them prestigious internships and real-world opportunities. Startups, meanwhile, are often left fighting for table scraps.

But what about the market for AI predictions themselves? Surely, if a startup can build a smarter AI system, it can outcompete the incumbents, right? Not so fast. Imagine our scrappy underdog student aces every test and graduates at the top of their class – but finds that all the top jobs are reserved for graduates of prestigious universities. In the world of AI, the platforms and services through which predictions are delivered can create a similar barrier to entry. An incumbent's AI may be inferior, but if it's deeply integrated into a platform with strong network effects, dislodging it can be a daunting challenge.

So, is the game rigged against AI startups? Not necessarily. Just as progressive education policies can level the playing field for disadvantaged students, carefully crafted regulations around data sharing and portability could give startups a fighting chance. But we must be cautious. Heavy-handed interventions could backfire, stifling the very innovation and competition we seek to promote.

In the end, the companies that will thrive in the AI era will be those that can deftly navigate these complex market dynamics. They'll need to be nimble, strategic, and relentlessly focused on delivering real value to users. The incumbents may have the early lead, but in the turbulent world of AI, fortunes can shift in the blink of an eye. The race is on, and the stakes couldn't be higher. As an economist, I'm thrilled to have a front-row seat to this unfolding drama – and I can't wait to see where it takes us next.

………

OK, that’s not me writing. Nor is it Malcolm Gladwell. However, economist Ben Golub had a fun idea to translate formal economics papers using Claude and I thought I’d give my paper the same treatment. It is surprisingly great, although it is a little flowery compared to the usual grumpy style of my writing. The image above is from DALL-E. Claude wasn’t willing to attempt one.