Innumeracy Series: 1. No measurement is ever accurate
Recently my family had a brush with COVID. I seemed to have picked the infection and spread it to my son and my wife. My daughter was unaffected. We had a couple of stressful days and then things were OK. Last week, we completed our quarantine period as well. COVID had mild impact on me, medium impact on my wife and relatively strong impact on my son. Now to the interesting aspect — I tested negative on Rapid Antigen Test but tested positive on RT-PCR test — got isolated and we had our family get tested the next day. My son came back positive and reports of my wife and daughter returned negative. However, my wife developed strong fever, suffered from vomiting, listlessness and loss of energy — in all likelihood she was infected with the virus as well. My daughter did not show any symptoms and came through OK.
I tried to make sense of why only certain members of the family contracted this highly virulent virus when all of us live under the same roof and co-mingle like any close knit family does. Also, when I tried to trace where I may have contracted the virus — some of my friends who I interacted with 3–4 days before my positive test were infected whereas few others did not. There was no discernible pattern.
This made me wonder if the test results could be wrong. As it turns out, they could very well be.
I have come to discover and understand that no screening test in the world is accurate (bold statement, I know), let alone, rt-pcr. In fact, no measurement ever is accurate hundred percent. Depending on the threshold (of tolerance band) for accuracy, even the best measurement devices will return wrong observations every once in a while. This may not come as a surprise to those familiar with Heisenberg’s uncertainty principle (the act of measurement itself affects the state of observing parameter) — there is a clear opportunity for analogy.
More to the point here — medical screening test results can be inaccurate (or even downright wrong) few or many times. Does that mean they are completely unreliable and unusable — no, but it means that test results must to be interpreted in the context of patient’s symptoms and history.
Basic mathematics makes a play behind understanding accuracy of test results. For a given test methodology, key determinants in understanding test results are: sensitivity, specificity, positive and negative predictive values. These, in turn, can be (and are) calibrated by carefully choosing the population on whom the reference tests are conducted (more precisely, prevalence of disease/sickness in test population). Test sensitivity and specificity indicate the effectiveness of the specific test methodology itself whereas test positive and negative test results reflect the likelihood of the infection/sickness in the person tested, given — this is a lot of English and it gets even more confusing once you go through how these four values are determined.
But, is there a need to understand all this — consider this example from the book — “Risk Savvy: How to make good decisions” by Gerd Gigerenzer:
“In a certain population/demography, the probability that a woman has breast cancer is 1 percent (prevalence). If a woman has breast cancer, the probability that she tests positive is 90 percent (sensitivity). If a woman does not have breast cancer, the probability that she nevertheless tests positive is 9 percent (false alarm rate). A woman tests positive. She wants to know whether that means that she has breast cancer for sure or what the chances are. What is the answer?
The best answer is one in ten. That is, out of ten women who test positive in screening, one has cancer. The other nine women receive false alarms.”
Thankfully in the case of COVID-19, for the large part, a false positive result does not have significant downside aside from increased anxiety on the part of the patient and her/his family. Whereas false negative results do have significant downside — increased spread of infection, load on medical infrastructure, lockdowns/restrictions and their impact on livelihood of affected population etc.
However, in case of a disease like breast cancer, a false positive result can have significant implications for the affected person — the anxiety and stress alone can be devastating but if one ends up proceeding treatment procedures like chemotherapy, there could be real physical harm to the body and standard of living as well.
As far as innumeracy goes, I clearly ended up selecting a relatively complex and obscure topic. The wider implication of this misconception may seem relatively limited — it may seem to have some exaggerated relevance in today’s world due to the COVID-19 pandemic. However it’s importance should not be understated because for those who are returning positive results on any (medical condition) screening tests, it is easy to misinterpret test results leading to potentially enormous anxiety in the best case and unnecessary health procedures leading to completely avoidable health complications, in the worst.
From what I understood, even the medical practitioners are not necessarily trained and retrained periodically (continuous education program) on matters such as this and its implications are easy to extrapolate for all. As much as we may blame as-it-is over-stretched health care professionals, I believe the onus of learning, understanding and managing such tests and procedures is on each one of us.
PS:
- Internet is full of information related to test efficacy — I found this article relatively simple to understand: https://www.frontiersin.org/articles/10.3389/fpubh.2017.00307/full
- Over this year (2022), I intend to cover more such items on innumeracy
