What will AI do to (p)research?

AI makes doing and communicating research much easier. Will there be any point to it?

This post today is about how academic research might be impacted by AI. The thesis is: when research is so easily produced and (where physically possible) generates knowledge on demand, the current research model — work now and use later — will collapse. But before getting to that, I ran a little personal experiment on the research system to test the potential efficacy of this hypothesis. It is not a research experiment. It used the current system as intended. It just did so with the help of AI.

Let me explain. Unlike many of my colleagues, I just like to work on papers, even if they are not likely to be on the most important of topics. Many academics, at least in economics, focus on “significant” work that will land them a publication in a top-5 journal. I do this too but I just don’t only do that. I get interested in an issue of the day and try to work out a model to understand it. It may turn out that model is pretty simple and straightforward although it could be useful to the small set of people who might eventually care.

I get ideas for this sort of stuff several times a day. When I was blogging those ideas might just become a post. An example of this is a post I wrote on the economics of time travel back in 2008 (here it is, although the link is wonky). I wrote:

Let’s get serious, for a moment, about the possibilities of time travel. When it comes down to it, the primary reason for someone to spend money inventing a time machine is to go back into the past, possessing future knowledge, and to exploit that knowledge either by inventing something (e.g., MS-DOS before Bill Gates) or trading on financial markets (e.g., airline stocks before late 2001). That is, on the face of it, a sound commercial rationale especially if time travel is Terminator rather than Back to the Future style.

The question is: if you think about the full equilibrium, will the investment in a time machine pay off? You see, you go back in the past and make a fortune. The problem is, even if you can patent or keep secret your time machine, someone in the future is going to duplicate your innovation. It only takes time. Then, they will target the maximum wealth generating opportunity in the past. If that was for you inventing something in particular, it will likely be the same for them. They will just go back in time and pre-empt your investment. The end result will be that you will find yourself going back in time but with the past already altered. All cost and no return. You would then have to go into the future to work out what you need to do but, and you can see where I am going here, that would have to be pretty much to the end of time at which point, the knowledge you need may not be so transparent.

A savvy (potentially evil) would be expropriator of future knowledge cannot possibly recoup all of the R&D costs on inventing a time machine. Hence, it won’t be done.

So if you are wondering why we haven’t seen time travellers from the future, you don’t need some elaborate theory of physics whose main purpose is to make science fiction really fiction. All you need is some economics: there is no incentive to invent time travel (unless you want to stick to historical theme parked tourism that is). That said, didn’t someone invent MS-DOS before Bill Gates?

It is a fun idea and I put it on my list of things to write a formal model and proof about if I ever got the urge¹ or fancied trying to win an Ig-Nobel prize.

Last December, along came GPT-o1-pro, and we got talking about stuff, including this old idea about time travel and whether it would be economically sensible. That conversation led to some speculation regarding what else time travel might impact, and I raised the possibility that “going back in time and inventing stuff” would be like “going back in time and trading on information that you now knew was true.” That caused me to wonder whether the efficient markets hypothesis — that is, that all information would get incorporated in asset market prices, so unless you had some private information, there would be no arbitrage opportunities — would withstand time travel. I conjectured that it would be because of the same pre-emption argument that I postulated in 2008.

That conversation went on for half an hour, and then I asked GPT-o1-pro to set up a model and provide proof of this conjecture. Five minutes later, it did that, and I checked the proof, and it was essentially correct with a few clarificatory things. Armed with that I asked GPT-o1-pro to write a draft of a short paper including my motivation, intuition and a main proposition. That draft was quite good. It required some editing for my own voice and some other minor things, but really was essentially done in about an hour. I am fast in producing research, but not that fast.

It was then that I wondered how far this could go. I viewed it as a paper. So why not submit it and see how it withstood the scrutiny of peers? I send it off to Economics Letters; a journal that publishes short papers with interesting but small contributions. The paper had a clear disclaimer.

That disclaimer reflected how I had used AI as a sounding board and I guess really as a research assistant. But the work was my responsibility.

After one round of review, the paper was accepted for publication. Here is the eventual publication link, and here is the link to the paper itself. The reviewer enjoyed the paper, and found it correct but wondered if a time traveller who arrived naked, Terminator-style, would have assets in the past to trade on their information. I added some citations to the literature on the efficient markets hypothesis with wealth constraints but basically said, “come on, you know that’s fiction right?” and the referee accepted that response.

Actually, before resubmitting, I asked GPT-o1-pro to review the paper and see that I had addressed the referee's concerns (I had) and whether there were any other issues. It noted that I had been loose with thinking about discount factors in the paper. I had not specified at what point discounting was taking place because my time traveller had made decisions in the future and then travelled back in time. AI wondered if that meant the discount factor was negative. This was an interesting point, and the AI was correct, so I added the clarification below. (The referee had not picked this up).

So I did it. I showed that you can dramatically compress the research process with AI and get all the way to a simple peer-reviewed publication as a result. There was no pretence that this was the most serious of research topics, but it also fell within the bounds of things that would count as research. (By the way, here is the obligatory AI podcast on the paper).

Will this destroy the system?

What are we to make of this? In the past, one might claim I had too much time on my hands but since I didn’t use much time in doing this, it is all good.

Instead, let’s consider what this means if a large chunk of research — in this case, economics research, but why stop there — can be produced essentially on demand. To understand this, consider what we are doing with respect to research now. The procedure is this:

1. Some academic ponders what might be an interesting problem or piece of knowledge that someone might find useful some day.

2. They spend a considerable amount of time researching that topic and finally, if successful, produce a paper.

3. The paper is peer-reviewed, and people decide whether it is worthy enough to be certified as being a piece of knowledge that might be useful if someone wants that knowledge in the future.

4. If so, the paper is added to the corpus of scientific knowledge.

5. In the future, someone has a problem and can then potentially find within that corpus someone who has spent time working on that knowledge and getting it certified.

6. Productivity ensues.

It is very much a work ahead of (potential) use model. And it won’t surprise you that in terms of the amount of work to actual use ratio, there is a ton of inefficiency there. Most knowledge is never put to use. It is really an option value. That doesn’t mean the system isn’t worthwhile as a whole, it is just that there is clear redundancy. We call it research, but I think a better name might be presearch because we are speculating on whether the knowledge is useful or not. This happens because research is far more expensive than search.

Now suppose that you take away the whole “it takes time to do good research” presumption as might be done with AI. Why do any presearch? Instead, why not wait until you have a use that requires some knowledge, then “ask AI” to tell you the answer? In other words, why not research on demand — that is, find a use and then do the work?²

What AI has done is laid bare that our underpinnings for the current approach to scientific knowledge production are a weak foundation because there is a large underlying redundancy. If you can answer questions with little effort, there is no need to get ahead of the game. Just work it out when you need it. Indeed, with a powerful enough AI, the current stock of knowledge residing in our libraries will be harder to look up than just “reinvent the wheel” and generate the research as it arises. Research could be cheaper than search.

What now?

This all starts to look like we are the coyote over the ledge but before gravity has set in. And I have to admit that I have no good response to that.

For me, my incentives haven’t changed so I can now use AI to help me research more efficiently and produce work I think might be useful in the future. But I also do this in the knowledge that surely that won’t be the case. Anyhow who might want what I am selling today sometime tomorrow won’t be bothered buying it tomorrow. And I’m not alone. But I am also more than willing to not look down because right now, it’s all good for me.

However, what will be interesting to watch is how all of this unfolds and shows up. Not all presearch will be worth abandoning. There is plenty to do. But I suspect the direction of scientific knowledge and progress may shift markedly.

Is it really worth thinking about what happens if we invent time travel or just worry about it at the time? Or is this research what that already is? Makes ya think.

1. Someone who came across the paper I wrote (but after it was already accepted for publication) sent me a link to a similar speculation that I wasn’t aware of from Marc Reinganum back in 1986 that was published in the Journal of Portfolio Management (not on the efficient markets hypothesis but on time travel).

2. Regular readers will notice that this is not unlike the Prediction Machines discussion of how AI could move our retail system from shop then ship to ship then shop.