There are any number of reasons why people don’t get medical care or don’t follow up on a referral to a specialist. They might not think they have a serious problem. They might lack time off work, reliable transportation, or health insurance. And those are problems AI alone can’t solve.
Source: Mullin, E. (2018). The first AI approved to diagnose disease is tackling blindness in rural areas.
There’s a good point to be made here; an algorithm may be 100% accurate in diagnosing a condition but the system can still fail for many reasons, one of which may be the all too human characteristic of ignoring medical advice. Of course, there are many good reasons for why we may not be able to follow the advice, which is mentioned in the article. However, the point is that, even if an algorithm gets it absolutely right, it may still not be the solution to the problem.
Note: I mentioned this story a few posts ago. It’s going to be interesting to follow it and see how the system fares in the uncertainty of real-world situations.
Based on these results, FDA authorized the system for use by health care providers to detect more than mild DR and diabetic macular edema, making it, the first FDA authorized autonomous AI diagnostic system in any field of medicine, with the potential to help prevent vision loss in thousands of people with diabetes annually.
Source: Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices | npj Digital Medicine
The first AI-based diagnostic system to be approved for use in the wild. It begins.
This is the point at which the risk from medical AI becomes much greater. Our inability to explain exactly how AI systems reach certain decisions is well-documented. And, as we’ve seen with self-driving car crashes, when humans take our hands off the wheel, there’s always a chance that a computer will make a fatal error in judgment.
Source: Vincent, J. (2018). DeepMind’s AI can detect over 50 eye diseases as accurately as a doctor.
This is just lazy. “When humans take their hands off the wheel…”? OK, then who is responsible for all the death and suffering that happens when humans have their hands on the wheel? Thinking about this for 3 seconds should make it clear that human beings are responsible for almost all human deaths. Getting to the point where we take our hands off the wheel (and off the scalpel, and off the prescription charts, and off the stock exchange) could be the safest thing we will ever do.
Also, DeepMind has moved on from only being able to diagnose diabetic retinopathy, to accurately identifying 50 different conditions of the eye. Tomorrow, it’ll be more.
Doctors are human. And humans make mistakes. And while scientific advancements have dramatically improved our ability to detect and treat illness, they have also engendered a perception of precision, exactness and infallibility. When patient expectations collide with human error, malpractice lawsuits are born. And it’s a very expensive problem.
Source: Defensive Diagnostics: the legal implications of AI in radiology
There are few things to note in this article. The first, and most obvious, was that we have a much higher standard for AI-based expert systems (i.e. algorithmic diagnosis and prediction) than we do for human experts. Our expectations for algorithmic clinical decision-making are far more exacting than those we have for physicians. It seems strange that we accept the fallibility of human beings but expect nothing less than perfection from AI-based systems. 
Medical errors are more frequent than anyone cares to admit. In radiology, the retrospective error rate is approximately 30% across all specialities, with real-time error rates in daily practice averaging between 3% and 5%.
The second takeaway was that one of the most significant areas of influence for AI in clinical settings may not be in the primary diagnosis but rather the follow up analysis that highlights potential mistakes that the clinician may have made. These applications of AI for secondary diagnostic review will be cheap and won’t add any additional workload to healthcare professionals. They will simply review the clinician’s conclusion and flag those cases that may benefit from additional testing. Of course, this will probably be driven by patient litigation.
 Incidentally, the same principle seems to be true for self-driving cars; we expect nothing but a perfect safety record for autonomous vehicles but are quite happy with the status quo for human drivers (1.2 million traffic-related deaths in a single year). Where is the moral panic around the mass slaughter of human beings by human drivers? If an algorithm is only slightly safer than a human being behind the wheel of a car it would result in thousands fewer deaths per year. And yet it feels like we’re going to delay the introduction of autonomous cars until they meet some perfect standard. To me at least, that seems morally wrong.
In 2012, doctors at Memorial Sloan Kettering Cancer Center partnered with IBM to train Watson to diagnose and treat patients. But according to IBM documents dated from last summer, the supercomputer has frequently given bad advice, like when it suggested a cancer patient with severe bleeding be given a drug that could cause the bleeding to worsen.
Source: IBM’s Watson gave unsafe recommendations for treating cancer
I don’t suggest that these findings aren’t problematic, or that isn’t essential for expert systems like Watson to improve in their diagnosis and prediction but I do feel that it’s a bit like blaming a medical student for not being perfect. Yes, Watson is still learning and won’t always get it right and yes, it can improve. But is it reasonable to hold these systems to standards of performance that we would never expect of human doctors (see the data on the human causes of medical error that lead to patient deaths).
But while the cold perfection of A.I. makes for a more perfect medical diagnosis, it can’t replace a human. After all, we love and embrace physicians who approach their work with empathy for patients and a warm understanding of what their fellow human beings are going through.
Source: Why AI Doesn’t Threaten Doctors
I don’t think that this is right. Very few patients would choose to see a doctor or physio because they’re looking for a little bit of empathy. The primary objective is to address a health problem that they’re experiencing and the fact that they would prefer a clinician with empathy is secondary. The author goes on to say that, “No amount of empathy will offset the pitfalls of human error…” and I think that this makes my point.
Of course, patients may be more likely to forgive errors if they’re committed by clinicians who cares for them. But, if given a preference, I imagine they would like a clinician to help them with the problem that brought them to the service in the first place, regardless of empathy. Also, I’m not convinced that the majority of health care providers are all that empathic towards their patients, especially those who are underpaid and over-worked in barely functioning health systems (i.e. the majority of health systems in the world).
The word “diagnosis,” he reminded me, comes from the Greek for “knowing apart.” Machine-learning algorithms will only become better at such knowing apart—at partitioning, at distinguishing moles from melanomas. But knowing, in all its dimensions, transcends those task-focussed algorithms. In the realm of medicine, perhaps the ultimate rewards come from knowing together.
Source: A.I. Versus M.D. What happens when diagnosis is automated?
This New Yorker article by Siddhartha Mukherjee explores the implications for practice and diagnostic reasoning in a time when software is increasingly implicated in clinical decision-making. While the article is more than a year old (a long time in AI and machine learning research), it still stands up as an excellent, insightful overview of the state of AI-based systems in the domain of clinical care. It’s a long read but well worth it.