Tests of tumor tissue don't sample the entire tumor. And tumor cells aren't all the same, so a sample doesn't give a complete picture of tumor genetics.
Blood tests sample free-floating cells that break loose from tumors. That's a useful technology if a tumor is hard to sample directly, but again it provides an incomplete snapshot of the cancer's genetic mutations. "They're looking for a needle in a haystack," Blau says.
Blood samples drawn at different times can produce different results, because different cells may be in the blood. And tumors evolve over time, so some of the difference could reflect that as well.
"The mutations you are looking for that might guide you to a particular drug can present in most or only a tiny fraction of cells," Blau said. So either test can fail to detect a clinically important mutation.
These blood tests often include a list of cancer drugs for doctors to consider, based on the mutations detected. The recommendations also varied, because the tests often found different mutations. Only 25 percent of the time (in nine of 36 cases) did these two tests recommend the same drug among the eight patients in the study.
These observations build on similar results of a somewhat larger study, published in August in the journal Oncotarget. In a sample of 28 patients, researchers found consistent results only about 17 percent of the time.
"It all boils down to a question of tumor biology," says lead author of that study, Young Kwang Chae, an oncologist and a co-director of the Developmental Therapeutics Program of the Division of Hematology and Oncology at Northwestern University's Feinberg School of Medicine.
"You can never say one test is the gold standard, or that one is better than the other," he said. Because they are looking at different samples, it's not surprising their results vary.
Many of the results from these tests are hard to interpret in the first place. In many instances, the presence of a particular mutation doesn't tell a doctor exactly what form of therapy would work best.
But Chae says when he finds an "actionable genetic alteration" from either test, he uses that to guide a patient's therapy.
"Both tests are right," argues Dr. Rick Lanman, chief medical officer at Guardant Health, which produces the blood-based Guardant360 test. If either test detects a particular mutation, there is high confidence that the particular mutation exists. And doctors can confidently base therapy on that positive detection, he says.
Doctors could get more complete results if they ran both tests on everybody, "but that's not cost effective," Lanman says.
His counterpart at Foundation Medicine, which makes the FoundationOne test, isn't convinced that both tests are equally reliable. Dr. Vincent Miller says the blood tests can miss a lot more than tests run on tumor tissue.
He points to a recent study of pancreatic cancer. Gene variants very frequently found in pancreatic cancer were detected in 87 percent of tumor samples. But blood tests only found the mutations about 25 percent of the time.
Miller says his company has a blood test as well as the test for solid tumors, and they're in the process of running a direct comparison, using the same genetic information. That could help clarify how much trust to put in blood tests, he says. "The technology may have gotten a little ahead of the clinical practice and the science," he says.
Doctors may be lulled into thinking that these tests are providing definitive results, but they're not. And that's the overarching message for Blau at the University of Washington.
"You really have to be thoughtful about how you apply these to clinical decision making," he says. "If you don't understand these limitations, if you just treat the reports at face value, that could be leading to instances where oncologists use drugs that are unlikely to be effective."
Copyright 2016 NPR. To see more, visit http://www.npr.org/.