Pooled Testing Gets Smarter During the Pandemic
More than a year into the Covid-19 pandemic, efficient testing for the coronavirus remains relevant as variants spread and vaccinations have been slow to roll out in many parts of the world. That is why some academic groups and companies have been using a combination of math and artificial intelligence to improve pooled testing, which began as a proposal to screen the U.S. military for syphilis during World War II, and has since been used for blood donations and to conserve sometimes scarce testing supplies in HIV surveillance.
Pooled testing for Covid-19 enables such efficiency by taking the diluted samples from nasal swabs of two or more people and screening all the samples together using a single test kit. If the pool comes back negative, then every sample included in the pool can be assumed to be negative. If the pool comes back positive, the lab must usually go back and retest each sample individually to figure out who is infected.
At first glance, pooled testing seems like a no-brainer during a pandemic. Getting more tests done with fewer supplies could prove handy — for instance, at times like last January, when more than half of labs surveyed in the United States still reported testing supply shortages. Pooled testing could also make mass testing faster — China has already used it to screen millions of people during smaller Covid-19 outbreaks. But pooled testing’s efficiency drops off significantly as positivity rates rise and there are more contaminated pools.
One way around that may be to use what some researchers call smart pooled testing, which uses mathematically sophisticated techniques — sometimes augmented by artificial intelligence — to boost the efficiency of pooled testing. Many research groups around the world have published papers about how such smart pooling can identify those likely to be infected to reduce the number of positive pools and potentially even sidestep the need for retesting altogether. But most labs still don’t use pooled testing, let alone smart pooled testing.
The story is different in Israel, where several labs began using smart pooled testing based on both mathematical and AI techniques last winter. The mathematical technique was developed by Israeli researchers just several weeks after the World Health Organization declared Covid-19 a pandemic in March 2020. By spreading individual samples across multiple pools to create unique combinations, the researchers showed they could identify positive samples by simply comparing the pattern of the positive pools.
Turning that academic exercise into something that labs would adopt was another matter. “We already had proof-of-concept data that this is useful,” says Tomer Hertz, a computational immunologist at Ben-Gurion University in Israel. “But to get to a point where a lab is actually going to run what we’re doing took about nine months.”
One commercial lab operated by the biotech company Ilex Medical has since been using this combinatorial pooling approach to reduce the need for individual retesting. Two other labs operated by Clalit Health Services, Israel’s largest state-mandated health maintenance organization, are also using it together with an AI pre-screening technique that helps to prevent high-risk samples from contaminating the pools. Altogether, six robots programmed to implement the pooling strategy are helping them process up to 7,000 tests each day in Israel and more than 400,000 tests had been performed by mid-April.
Such operations could yield useful lessons for many countries — including the U.S., where some labs have used standard pooling, and Colombia, where a homegrown smart pooling effort is looking to take hold — in dealing with both Covid-19 and future pandemics.
Most labs haven’t tried standard pooled testing because of the limiting factors that can reduce pooling’s effectiveness. For example, large pool sizes can dilute the amount of virus to the point that it is undetectable. Pooled testing also becomes less efficient as the percentage of infected people in a population increases because more positive samples typically lead to more retesting. The high positivity rates across much of the country have been one reason why major American testing companies such as Quest Diagnostics have limited pooled testing.
“Every lab needs to do its own validation study for pooling because it really depends on the prevalence rate of Covid-19 in that specific region,” says Baha Abdalhamid, a physician and assistant director of the Nebraska Public Health Laboratory. In April 2020, Abdalhamid and colleagues at the University of Nebraska published the results of a proof-of-concept study in the American Journal of Clinical Pathology that showed how even a standard pooled testing approach could be cost-effective at Covid-19 positivity rates of 10 percent or less.
Some places have made standard pooled testing work. Last year, administrators at Saratoga Hospital in New York used rapid Covid-19 testing to screen everyone who was admitted to the hospital regardless of their health condition. But screening every incoming patient strained the hospital’s testing supplies at a time of nationwide shortages, so the hospital began pooling two or three samples at a time in April of last year, eventually expanding to pools of five. The hospital also relied on emergency room physicians to determine which incoming patients were more or less likely to have the disease, which helped to create pools with samples likely to test negative.
“It was very successful and it allowed us to rapidly test everyone being admitted to the hospital,” says David Mastrianni, a hematology specialist and oncologist at Saratoga Hospital. “We never would have been able to do it without the pooling.”
This worked while the Covid-19 positivity rate remained low among incoming patients. But when positivity rates began rising in the fall of 2020, Saratoga Hospital’s strategy fell apart; too many pools ended up with positive cases because physicians had relaxed their criteria for putting patient samples into pools over the summer. The hospital tightened up criteria once again and started preliminary screening with an even faster, though less accurate, form of rapid testing to help sort samples into low-risk pools for the main testing effort. Today, despite another recent uptick in the positivity rate in the community, the hospital is seeing fewer admissions, so they have dropped pre-screening and are now testing their low-risk samples in pools of two.
In comparison, the smart pooled testing strategies developed in Israel can boost efficiency in several ways. For instance, the Israeli researchers who successfully deployed smart pooling — organized under a startup called Poold Diagnostics — showed that their combinatorial approach was able to identify four people infected with Covid-19 out of a total of 384 samples, according to results published in Science Advances last summer. They did this by distributing each sample into six different pools to create 48 pools of 48 samples each. The cost of screening 384 people individually would be about $20,000 with standard testing at $50 per test; the pooled strategy cut that to approximately $2,500.
But the rate of positive Covid-19 in the broader population was just around one percent for the study, which is likely one reason why the results were so successful. The approach would still work around the “break-even point” of a positivity rate around 10 percent, says Noam Shental, a computational biologist at the Open University of Israel and cofounder of Poold Diagnostics. Any higher than that, though, and there would be too many contaminated pools for it to be cost effective.
This is where AI can seemingly squeeze out even more efficiency. Poold Diagnostics teamed up with the company Neura, which has developed an AI model to help predict and monitor the spread of Covid-19 cases. Neura uses an AI technique called machine learning to train a model on large amounts of behavioral and epidemiological data related to Covid-19 so that it can then automatically identify hidden patterns.
The data analyzed by Neura’s AI includes dozens of indicators relevant to Covid-19, such as recent travel from communities with high levels of Covid-19 and adherence to social distancing guidelines. The data, provided by Israel’s universal healthcare system, are anonymized.
The model was first created in March 2020 “and has been updating since then,” says Amit Hammer, Neura’s CEO. “And this model works at the country level and the county level, the city level, and even at the neighborhood level.”
For the smart pooled testing, Neura’s AI analyzes the anonymized data for new samples and assigns a risk score reflecting the probability that it will be positive or negative. A risk score of zero means a sample is highly likely to be negative, whereas a sample with a risk score of 100 is highly likely to be positive, Hammer explains.
The risk scores help labs understand which samples should undergo individual testing rather than pooled testing. And that keeps the pooled testing efficient even in circumstances when the positivity rate among the incoming samples may be relatively high.
 |
For all of Undark’s coverage of the global Covid-19 pandemic, please visit our extensive coronavirus archive. |
When the positivity rate in Israel was around 8.6 percent in August of last year, Neura’s risk scoring approach was able to help create pools with positivity rates of two percent or less during preliminary trial runs. More recently, Neura’s approach has helped labs to screen up to 50,000 samples per day to optimize both individual testing and pooled testing. This AI screening can keep pools at about two percent positivity despite higher community positivity rates of 20 or 25 percent, Hammer says.
But Hammer cautions that, in order to maintain this efficiency, the AI models need to be updated constantly and quickly as Covid-19 prevalence changes in the populations being tested.
“The key is to have good predictor variables, like type of symptoms or exposure to other infected individuals,” says Christopher Bilder, a statistician at the University of Nebraska-Lincoln, who has studied how to optimize pooled testing but was not involved in the efforts in Israel.
In theory, any method that perfectly predicted who would test positive for Covid-19 could completely replace testing. But AI models don’t work well enough to do that, especially given the potential impact of false positives or false negatives on life-or-death health decisions. Even the best AI models must strike a balance involving the inherent tradeoff between producing either more false positives or more false negatives.
“At the beginning of the pandemic, I observed there were a lot of projects aiming at using AI for Covid-19 screening with claims of it being faster and easier than using standard testing,” says Maria Camila Escobar, a biomedical engineer at the University of the Andes in Colombia. She described the idea of an AI-only approach as “irresponsible.”
By contrast, using AI in combination with pooled testing provides a fallback in case the AI predictions are inaccurate. At worst, inaccurate AI predictions may lead to mixing more positive results in with largely negative pools, which would force labs to spend more time and resources retesting people. “Yeah, you lose a couple of tests, but you don’t lose the lives of people that you’re telling to go outside, and they actually have Covid, and your model failed,” Escobar says.
Using samples and anonymized data collected by testing centers in Bogota, Colombia’s capital city, Escobar and colleagues showed how machine learning could enable efficient smart pooling with simulated Covid-19 positivity rates of up to 25 percent, as detailed in a paper the group posted last summer, which has not yet been peer reviewed. The researchers also conducted a separate pilot study with the Covida project, a university-associated testing effort that actively screens for Covid-19 cases in Bogota. That pilot study helped save more than 2,000 test kits using pool sizes of just two samples each.
Although the work is preliminary, Covida has already received half a million dollars in funding from the Rockefeller Foundation to deploy it more broadly in Colombia. “Considering the fact that it seemed like an innovative approach to increase testing capacity and efficiency, these early results made it particularly interesting,” says Greg Kuzmak, a manager with the Health Initiative at the Rockefeller Foundation. “Because perhaps there’s some catalytic capital we could provide that would allow this to expand and scale across the city of Bogota.”
With the Rockefeller Foundation’s backing, the University of the Andes team is working with Bogota’s health department to roll out smart pooling in every official testing center in the coming weeks. By the end of this year, the team hopes to have scaled up smart pooling across the entire city, which is also responsible for much of the Covid-19 lab testing in Colombia.
The University of the Andes team initially explored more mathematically complicated pooling schemes like the Israeli group’s approach. But local labs balked at the prospect of having to rearrange their workflow, especially in the absence of equipment necessary for handling more complex testing procedures — an issue that could hinder adoption of more complex pooled testing strategies in many places around the world.
Another challenge is having access to the health and other data that AI models need for their predictions. While Poold Diagnostics has ready access to such data through its partnership with Israel’s health care system, the University of the Andes team encountered labs in Colombia that only had the relevant data stored as scanned PDF files, which made it difficult to extract and analyze the necessary information. That has delayed the smart pooling rollout until the city of Bogota completes a new digital health system that will allow testing facilities to swiftly upload the relevant information to a central online database.
As year two of the pandemic continues, Poold Diagnostics and Neura are both seeking partners and regulatory approval to expand in the U.S. and Europe, while the University of the Andes team has discussed supporting pooled testing in countries such as Gambia. But the future of smart pooling will also depend on how easily labs can adopt it without complicating existing operations.
“I don’t know if machine learning would have helped us, or could help us in the future, with our pooled strategy,” says Mastrianni at Saratoga Hospital in New York. “The mix of science, logistics, supply lines, and politics changes pretty fast and sometimes seems random.”
The simplest pooling strategies still clearly have their uses, says Moran Yassour, a computational biologist at the Hebrew University of Jerusalem. As a computer scientist, she acknowledges the allure of playing with “fancier models of pooling.” But from a practical standpoint, she says, overworked labs want consistent procedures to interpret and implement.
Without using AI or smart pooling, Yassour and her colleagues screened almost 134,000 samples using just under 18,000 pools at Hadassah Medical Center in Jerusalem over a five-month period. This used just 24 percent of the tests that would have normally been required, as detailed in a paper published recently in Science Translational Medicine.
This simple strategy created pools based on whatever samples came into the lab together at the same time to take advantage of how samples were often collected all together from people in places where clusters of Covid-19 cases had occurred. That meant positive samples often ended up together in the same few pools, rather than showing up across many pools.
Such an approach held up while positivity rates among the Israeli samples fluctuated between less than one percent and six percent. Other situations involving higher positivity rates may benefit from the smart pooling schemes. But at the very least, there seems to be a growing body of evidence suggesting more labs could benefit from dipping their toes into pooling, Yassour says.
“We’re trying to spread the word of how a very simplistic pooling scheme can go a very long way,” she adds.