Test Week: Scale and Speed
If we want to use widespread testing to mitigate and suppress Covid-19, we need cheap tests to allow population-wide frequent testing and fast test results so that we can act quickly.
Welcome to Plugging the Gap (my email newsletter mainly about Covid-19 and its economics). My goal is for several posts a week explaining economic research and the economic approach to understanding the pandemic. (In case you don’t know me, I’m an economist and professor at the University of Toronto. I have written lots of books including most recently on Covid-19. You can follow me on twitter (@joshgans) or subscribe to this email newsletter here).
This week is ‘testing week’ at this newsletter. Yesterday’s post discussed how important it is to match the test to the decision. Today, I look beyond individual tests to testing strategy — in particular, for the purpose of mitigation that will reduce the level of viral spread in the population. To achieve this requires having tests that are cheap enough to be conducted at scale and that can be done quickly and frequently enough to ensure infectious people are isolated quickly. Such tests are being developed but are currently being held up because they need to pass regulatory approval for diagnostic testing.
There was exciting news late last week that the FDA had approved the SalivaDirect PCR test that had been developed at Yale University and tested by the NBA inside their DisneyWorld bubble alongside standard PCR tests. The significance of this was that it was a saliva-based test that avoided certain expensive tests that were part of PCR testing procedures. If this wasn’t enough, to bring costs down further, Yale was opening up this testing methodology to be performed by any approved lab. In the end, the base price for this test will be $10 per sample as opposed to $50-$150 for standard PCR tests. So whereas tests may have been rationed to those deemed to be at high risk of being infected, these news tests may make widespread screening possible.
All of this is significant because the promise of testing for mitigation is that we test large segments of the population regularly and isolate those who test positive. This has the potential to dramatically slow and even suppress Covid-19 spread. In other words, it can achieve what a vaccine can achieve. It is the solution to the pandemic information gap. This is something emphasised in a report released Monday and led by former UK prime minister, Tony Blair, calling for mass testing in the UK. That report is a good reference for more details on how to achieve mass testing that I have been surveying this week on this newsletter.
Cost, however, is only one part of achieving the scale necessary for successful mitigation. The other facet is speed — how quickly results can be provided after someone has been tested. You can see why that matters from this diagram:
A PCR test is more sensitive than candidate rapid tests. However, the results take longer to arrive — 24 hours to a week or more. The problem is that the value of those tests for use in mitigation is dramatically reduced and any advantage you obtain from sensitivity is wiped out by being unable to act in time. This is the reason Bill Gates regarded most tests carried out in the US as “garbage.”
Thus, we need to get both the costs down and the speed up for testing as soon as possible. But to do that we will end up sacrificing sensitivity — that is, how many false negatives there are. For mitigation, that is potentially an issue because false negatives mean you are letting people go about their businesses who might be infected. However, as I will show if you think about these things in terms of a testing strategy rather than just a test, your risk assessment changes.
Getting cost down
There are many pain points for a PCR test. The sample has to be collected which is labour intensive. The samples have to be transported safely to a lab. They need to be stored. Then they need to be tested using skilled professionals and a machine. Finally, the results can be transmitted back to other medical professionals to deliver the news and have positive cases reported to public health authorities.
The SalivaDirect approach takes out some of these steps and that is how it manages to reduce the costs of each test. But the tests still need to be processed by a lab. That is costly and also slows things down.
The Atlantic reported on other non-PCR methods of delivering low-cost tests. These antigen tests look for chemicals that are present in SARS-CoV2 rather than RNA remnants. The key cost saving is that they can be conducted at the point of care and so can avoid the lab.
Two of the most anticipated such tests are already on the market. Manufactured by two companies, Quidel and Becton, Dickinson and Company, they look for an antigen called “nucleocapsid,” which is plentiful in the SARS-CoV-2 virus. The companies say they will be making a combined 14 million tests a month by the end of September; for comparison, the U.S. completed 23 million total tests in July.
That might look promising but it doesn’t stack up in cost terms with SalivaDirect.
And while these tests will be useful, they have their own supply-chain drawbacks. Both companies’ tests can be interpreted only with a proprietary reader, and while many clinics and offices already have these readers on hand, neither company is prepared to mass-produce them at the same scale as the tests. (Quidel now makes 2,000 of its readers a month, but is aiming to scale to 7,000 a month by September, a spokesperson told us.) Because both tests look for nucleocapsid, which exists only inside the coronavirus, they need a way to sever the virus’s outer membrane. This requires more reagents. For many technicians, these drawbacks aren’t worth the benefits. “Most people who are real lab experts are steering away from all that stuff because they can’t justify it,” Greene, the Kaiser lab director, said.
This means that these tests only cost half as much as PCR tests. So there are advantages but the cost side is limited.
To get costs down further, an increasingly popular process is to use pooled or group testing. This is where the samples of a group are combined and the PCR or other test applied to the entire group. If there is a positive result, then each member of the group is tested individually. Thus, if you had 25 people in a group, you would end up with just one test to cover them most of the time and occasionally another 25 tests for confirmation. Add some additional information and some mathematics and you can cut that down even further.
Group testing has been suggested as a good option for schools to identify outbreaks quickly and take action. Sometimes there is a suggestion to use them in households although I have my doubts that they help much there. If one person in a household is infected, they all are more likely to be infected. Thus, it seems that a group test does not give much additional information. For maximal information value, you actually want your group being tested to be people whose risks are uncorrelated rather than correlated. That said, for a school, group testing has a convenience factor that outweighs any loss in information value.
The advantage of getting costs down is that they allow you to test people more frequently. By doing this, you potentially test more people at the time they are infectious and this is a key part of any mitigation testing strategy. This helpful simulation (with R = 1.5) shows how more frequent testing can help mitigate the virus even if test results come back 5 days later.
In particular, it can allow you to move from no testing to weekly screenings which reduces the outbreak levels where large numbers (> 20%) of a population become infected by 5 fold and by 8 fold for daily testing.
Getting faster results
Even if you can get the dollar cost of a test down, there is another critical cost associated with testing: time. It takes time for results to come back to people and for an isolation action to be taken. The above simulation assumed that those results took 5 days to arrive which is likely what will happen if there was widespread testing. If, instead, you could reduce that time down to 1 day, this is the result:
This is a dramatic improvement in the viral spread. You can compare the 1 day and 5-day outcomes regarding the probability that 20% or more of the population become infected:
Even with monthly testing, the results are dramatic. With weekly or daily testing, a one-day turnaround wipes the virus pretty much out. This is why Harvard’s Michael Mina has called for investment in rapid testing and co-founded an organisation to promote it. This would play a critical role in, for instance, allowing students to return to campus safely.
To achieve speedy test results all of the bottlenecks — labs, machines and reagents — have to be removed from the testing process. Again, from The Atlantic.
Such tests exist—and have existed since April—and they are made by e25 Bio, a 12-person company in Cambridge. An e25 test is a paper strip, a few inches long and less than an inch wide. It needs only some spit, a saline solution, and a small cup—and it can deliver a result in 15 minutes. Like a pregnancy test, the strip has a faint line across its lower third. If you expose the strip to a sample and it fills in with color, then the test is positive. It does not require a machine, a reagent, or a doctor to work.
Its unusual quality is that it does not look for the same antigen as other tests. Instead of identifying nucleocapsid, the e25 test is keyed to something on the outside of the virus. It reacts to the presence of the coronavirus’s distinctive spike protein, the structure on the virus’s “skin” that allows it to hook onto and enter human cells. “I think we’re the only company in North America that has developed a spike antigen test,” Bobby Brooke Herrera, e25’s co-founder and chief executive, told us.
This has several advantages. It means, first, that the e25 test does not have to rupture the virus, which is why it doesn’t need reagents. And it means, second, that the e25 test is actually looking for something more relevant than the virus’s genetic material. The spike protein is the coronavirus’s most important structure—it plays a large part in determining the virus’s infectiousness, and it’s what both antibodies and many vaccine prototypes target—and its presence is a good proxy for the health of the virus generally. “We’ve developed our test to detect live viruses, or, in other words, spike protein,” Herrera said.
In other words, this test is even easier than a pregnancy test and is more like a temperature test. Moreover, it targets specific things that make a person infectious. Eric Topol has a handy graphic that summarises the differences between this test and PCR tests.
It is almost too good to be true.
The sensitivity of Testing Strategy
What is the sticking point in getting rapid, low-cost tests? It won’t surprise you that the answer is regulatory. Using traditional, diagnostic-based scoring of test sensitivity, the more rapid the test, the less sensitive it is for infection. As we have already seen, however, for mitigation we want to consider sensitivity for infectiousness and the e25 test may well be more direct than a PCR test for that purpose. But there is more. If tests can be provided cheaply, frequently and with rapid results, what we need to consider in evaluating their use in mitigation is how sensitive the entire strategy for testing is rather than the test itself.
Recall that we use tests to help give us information to make a better decision. Up until now, the decisions I have focussed on have been actions such as treatment, clearance or isolation. But when we have a mix of tests with different efficacy and we have the ability to undertake frequent testing, an individual test gives us information as to how to interpret subsequent tests. This can be especially useful if tests are imperfect because an imperfect test yields different information depending on our prior or pre-test probability that someone is infectious.
This issue was already discussed on Monday’s post when it was shown how the prevalence of Covid-19 in the population can impact on the interpretation of test results with respect to the probability that someone with a positive rather than a negative test was infected. When prevalence is low, a test that has less specificity (i.e., higher false positives) can mean that a positive result still means someone is more likely to be non-infected with Covid-19. For the tests we have been discussing — PCR and Antigen — specificity isn’t the issue but sensitivity. In this case, when there is low prevalence, someone who tests positive is still equally likely to be infected or not. (E.g., with a sensitivity of 70%, and specificity of 98%, when there is 3% prevalence, the probability that a positive person is infected is 50%). In this case, only 1% of the population who is positive is missed. When there is higher prevalence, there are fewer false positives and so a positive test result means that a person is very likely infected. (E.g., with 25% prevalence, the probability is 94%). In this case, 7% of the population who happen to be positive (almost a third of the infected), falsely return a negative result. [This result is proven formally by economists Jeff Ely, Andrea Galeotti and Jakub Steiner in a recent paper].
The point here is that we are less worried about low sensitivity when our pre-test probability that someone is infected is low and we are more worried about low sensitivity when it is higher. For diagnostic tests that are usually taken precisely when someone is suspected to be infected with Covid-19, it is appropriate to worry about sensitivity.
When it comes to mitigation, our pre-test probability is the level of prevalence and it is low. But if just 1% of the population is infected and we use a test that nominally has a low sensitivity (say 50%), then we will miss very few infected people in finding the needles in the haystack. But in this case, low sensitivity means that someone who tests positive only has a 20% chance of being infected. That is a problem for treatment and, indeed, for an isolation decision. But that is not what would happen if you had a positive result. Instead, you would receive a test with much higher sensitivity.
Herein lies the usefulness of the pre-screen. Rather than administering a PCR test (with 90% sensitivity) with a pre-test probability of 1%, we are now administering one with a pre-test probability of 20%. In this case, test positive and you have a 96% chance of being infected. Even with this, one in ten people who are positive will return a negative test. But the base is very low. A testing strategy that offers a low sensitivity (50%) test followed by a high sensitivity (90%) one has an effective sensitivity of 92%. In other words, we have achieved high sensitivity at a fraction of the cost and time it would take to conduct PCR tests as screens and we have higher sensitivity to boot!
These calculations are actually quite conservative. If we are thinking about mitigation, it is unlikely a rapid test with have such low sensitivity when testing for infectiousness. Suppose it is more like 75% sensitivity then the sensitivity of the testing strategy is 94%. By contrast, screening everyone with a PCR test only at 1% prevalence is a testing strategy with 31% sensitivity.
Moreover, this all presumes just a single round of testing. Adopt frequent testing and this becomes even more powerful. The amount of leverage in controlling the virus from marrying a low cost, rapid test with a testing strategy is quite extraordinary.
Mitigation requires a Strategy
In conclusion, it is important that when considering using testing for mitigation that you think in terms of a strategy or regime and you evaluate the efficacy of the portfolio of tests you might take. The problem to date is that many who approve such testing have been not thinking strategically and have consequently emphasised testing outcomes that are too costly and too slow for the purposes of mitigation. Breaking out of that thinking is the way out of this pandemic.
In tomorrow’s newsletter, I explore this more fully and add to the mix, human behaviour. It won’t surprise you that people could potentially spoil the potential party that comes from a rapid testing strategy. But there are things we can do to reduce the risk of that happening.