Since October, when The Wall Street Journal ran a story questioning the validity of its blood tests, healthcare startup Theranos has promised that an independent analysis of its testing services would be forthcoming. 

Now, an independent analysis has arrived. But it can't possibly be what Theranos expected, and it is sure to create more questions than it answers for the embattled company, which has been valued at about $10 billion.

Theranos CEO ELizabeth Holmes has said repeatedly that it would make data about its tests available for research. The results of that work are still to come. But on Monday, The Journal of Clinical Investigation published a study conducted by researchers at the Icahn School of Medicine at Mount Sinai, to see if Theranos' tests could be used to monitor the health of individuals. They did the research without consulting Theranos or informing the company of it, in an attempt to replicate real-world conditions as closely as possible.

The study found that results from Theranos differed from those of Quest Diagnostics and LabCorp, the other two labs in the study. "The Theranos fingerprick collection system yields higher sample rejection rates, and their testing services return results that mostly agree with other services with the exception of lipid panels," write the authors. But they said the variation in the lipid panel testing could have implications for patients' health.

In a letter to The Journal of Clinical Investigation, Theranos says the study is "flawed and inaccurate" and that it is "disappointed that any journal would accept this study for publication."

In the letter, Theranos says that collecting blood from a vein before collecting it from a fingertip is against its collection procedures, and could affect the blood being collected from the fingertip. It also says that the patients who generated outlying measurements were not more fully evaluated to see if indeed their bloodwork was abnormal. And Theranos says it never received an email or any other requests from the investigators about their work.

Multiple blood tests

The researchers asked 60 healthy volunteers to be tested twice in the same day at two different Theranos testing centers, and at Quest and LabCorp. The Theranos tests used the company's finger-prick methodology, in which just a few drops of blood are taken from a fingertip, even when multiple tests need to be run. The samples for Quest and LabCorp used traditional blood draws from the volunteers' arms.

(In most cases, Theranos is no longer using finger-pricks for its bloodwork, instead using venous blood draws from the arm.)

Most of the time, the three labs agreed on the results. But 12.2 percent of the measurements from Theranos were outside the normal range for healthy adults. That compares with 7.5 percent for Quest and 8.3 percent for LabCorp. The authors did not expect to find such differences, either from Theranos or the other two labs, referring to "an unexpected degree of variability within and among testing services."

In 2.2 percent of cases, Theranos came up with no results at all for the blood samples, compared to 0.2 percent at LabCorp and zero at Quest. The variance in testing from Theranos is not without consequences, write the authors.

This increase in abnormal test results can have negative consequences for medicine in the form of extra testing, additional patient visits to clinics/hospitals, and added doctor services, all of which result in additional costs and burdens to patients or to the healthcare system and are potentially harmful.

The researchers found that Theranos was likely to be more accurate in testing for white blood cells, but wrote that "additional precision was unlikely to alter clinical decisions."

Theranos' tests were more likely to differ from others on cholesterol values, which the researchers said could lead doctors to "either inappropriately initiate or fail to appropriately initiate statin therapy."

The research also turned up other results that may not have particular relevance to Theranos, but may eventually shed light on the overall variation in blood tests. People who participated in the tests had blood taken, either by fingerprick or from a vein in their arm, four times in the same day. Whether a test came from the first blood draw or the last seems to have affected the results. One set of tests--serum phosphorus levels, white blood cell counts, and counts of leukocyte subsets--were higher in the later collections. In the early collections, counts of total cholesterol, hemoglobin, and bilirubin, among others were higher.

 

Published on: Mar 29, 2016