WCPT poster: Introduction to machine learning in healthcare

It’s a bit content-heavy and not as graphic-y as I’d like but c’est la vie.

I’m quite proud of what I think is a novel innovation in poster design; the addition of the tl;dr column before the findings. In other words, if you only have 30 seconds to look at the poster then that’s the bit you want to focus on. Related to this, I’ve also moved the Background, Methods and Conclusion sections to the bottom and made them smaller so as to emphasise the Findings, which are placed first.

Here is the tl;dr version. Or, my poster in 8 tweets:

  • Aim: The aim of the study was to identify the ways in which machine learning algorithms are being used across the health sector that may impact physiotherapy practice.
  • Image recognition: Millions of patient scans can be analysed in seconds, and diagnoses made by non-specialists via mobile phones, with lower rates of error than humans are capable of.
  • Video analysis: Constant video surveillance of patients will alert providers of those at risk of falling, as well as make early diagnoses of movement-related disorders.
  • Natural language processing: Unstructured, freeform clinical notes will be converted into structured data that can be analysed, leading to increased accuracy in data capture and diagnosis.
  • Robotics: Autonomous robots will assist with physical tasks like patient transportation and possibly even take over manual therapy tasks from clinicians.
  • Expert systems: Knowing things about conditions will become less important than knowing when to trust outputs from clinical decision support systems.
  • Prediction: Clinicians should learn how to integrate the predictions of machine learning algorithms with human values in order to make better clinical decisions in partnership with AI-based systems.
  • Conclusion: The challenge we face is to bring together computers and humans in ways that enhance human well-being, augment human ability and expand human capacity.
My full-size poster on machine learning in healthcare for the 2019 WCPT conference in Geneva.

Reference list (download this list as a Word document)

  1. Yang, C. C., & Veltri, P. (2015). Intelligent healthcare informatics in big data era. Artificial Intelligence in Medicine, 65(2), 75–77. https://doi.org/10.1016/j.artmed.2015.08.002
  2. Qayyum, A., Anwar, S. M., Awais, M., & Majid, M. (2017). Medical image retrieval using deep convolutional neural network. Neurocomputing, 266, 8–20. https://doi.org/10.1016/j.neucom.2017.05.025
  3. Li, Z., Zhang, X., Müller, H., & Zhang, S. (2018). Large-scale retrieval for medical image analytics: A comprehensive review. Medical Image Analysis, 43, 66–84. https://doi.org/10.1016/j.media.2017.09.007
  4. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118. https://doi.org/10.1038/nature21056
  5. Pratt, H., Coenen, F., Broadbent, D. M., Harding, S. P., & Zheng, Y. (2016). Convolutional Neural Networks for Diabetic Retinopathy. Procedia Computer Science, 90, 200–205. https://doi.org/10.1016/j.procs.2016.07.014
  6. Ramzan, M., Shafique, A., Kashif, M., & Umer, M. (2017). Gait Identification using Neural Network. International Journal of Advanced Computer Science and Applications, 8(9). https://doi.org/10.14569/IJACSA.2017.080909
  7. Kidziński, Ł., Delp, S., & Schwartz, M. (2019). Automatic real-time gait event detection in children using deep neural networks. PLOS ONE, 14(1), e0211466. https://doi.org/10.1371/journal.pone.0211466
  8. Horst, F., Lapuschkin, S., Samek, W., Müller, K.-R., & Schöllhorn, W. I. (2019). Explaining the Unique Nature of Individual Gait Patterns with Deep Learning. Scientific Reports, 9(1), 2391. https://doi.org/10.1038/s41598-019-38748-8
  9. Cai, T., Giannopoulos, A. A., Yu, S., Kelil, T., Ripley, B., Kumamaru, K. K., … Mitsouras, D. (2016). Natural Language Processing Technologies in Radiology Research and Clinical Applications. RadioGraphics, 36(1), 176–191. https://doi.org/10.1148/rg.2016150080
  10. Jackson, R. G., Patel, R., Jayatilleke, N., Kolliakou, A., Ball, M., Gorrell, G., … Stewart, R. (2017). Natural language processing to extract symptoms of severe mental illness from clinical text: The Clinical Record Interactive Search Comprehensive Data Extraction (CRIS-CODE) project. BMJ Open, 7(1), e012012. https://doi.org/10.1136/bmjopen-2016-012012
  11. Kreimeyer, K., Foster, M., Pandey, A., Arya, N., Halford, G., Jones, S. F., … Botsis, T. (2017). Natural language processing systems for capturing and standardizing unstructured clinical information: A systematic review. Journal of Biomedical Informatics, 73, 14–29. https://doi.org/10.1016/j.jbi.2017.07.012
  12. Montenegro, J. L. Z., Da Costa, C. A., & Righi, R. da R. (2019). Survey of Conversational Agents in Health. Expert Systems with Applications. https://doi.org/10.1016/j.eswa.2019.03.054
  13. Carrell, D. S., Schoen, R. E., Leffler, D. A., Morris, M., Rose, S., Baer, A., … Mehrotra, A. (2017). Challenges in adapting existing clinical natural language processing systems to multiple, diverse health care settings. Journal of the American Medical Informatics Association, 24(5), 986–991. https://doi.org/10.1093/jamia/ocx039
  14. Oña, E. D., Cano-de la Cuerda, R., Sánchez-Herrera, P., Balaguer, C., & Jardón, A. (2018). A Review of Robotics in Neurorehabilitation: Towards an Automated Process for Upper Limb. Journal of Healthcare Engineering, 2018, 1–19. https://doi.org/10.1155/2018/9758939
  15. Krebs, H. I., & Volpe, B. T. (2015). Robotics: A Rehabilitation Modality. Current Physical Medicine and Rehabilitation Reports, 3(4), 243–247. https://doi.org/10.1007/s40141-015-0101-6
  16. Leng, M., Liu, P., Zhang, P., Hu, M., Zhou, H., Li, G., … Chen, L. (2019). Pet robot intervention for people with dementia: A systematic review and meta-analysis of randomized controlled trials. Psychiatry Research, 271, 516–525. https://doi.org/10.1016/j.psychres.2018.12.032
  17. Jennifer Piatt, P., Shinichi Nagata, M. S., Selma Šabanović, P., Wan-Ling Cheng, M. S., Casey Bennett, P., Hee Rin Lee, M. S., & David Hakken, P. (2017). Companionship with a robot? Therapists’ perspectives on socially assistive robots as therapeutic interventions in community mental health for older adults. American Journal of Recreation Therapy, 15(4), 29–39. https://doi.org/10.5055/ajrt.2016.0117
  18. Troccaz, J., Dagnino, G., & Yang, G.-Z. (2019). Frontiers of Medical Robotics: From Concept to Systems to Clinical Translation. Annual Review of Biomedical Engineering, 21(1). https://doi.org/10.1146/annurev-bioeng-060418-052502
  19. Riek, L. D. (2017). Healthcare Robotics. ArXiv:1704.03931 [Cs]. Retrieved from http://arxiv.org/abs/1704.03931
  20. Kappassov, Z., Corrales, J.-A., & Perdereau, V. (2015). Tactile sensing in dexterous robot hands — Review. Robotics and Autonomous Systems, 74, 195–220. https://doi.org/10.1016/j.robot.2015.07.015
  21. Choi, C., Schwarting, W., DelPreto, J., & Rus, D. (2018). Learning Object Grasping for Soft Robot Hands. IEEE Robotics and Automation Letters, 3(3), 2370–2377. https://doi.org/10.1109/LRA.2018.2810544
  22. Shortliffe, E., & Sepulveda, M. (2018). Clinical Decision Support in the Era of Artificial Intelligence. Journal of the American Medical Association.
  23. Attema, T., Mancini, E., Spini, G., Abspoel, M., de Gier, J., Fehr, S., … Sloot, P. M. A. (n.d.). A new approach to privacy-preserving clinical decision support systems. 15.
  24. Castaneda, C., Nalley, K., Mannion, C., Bhattacharyya, P., Blake, P., Pecora, A., … Suh, K. S. (2015). Clinical decision support systems for improving diagnostic accuracy and achieving precision medicine. Journal of Clinical Bioinformatics, 5(1). https://doi.org/10.1186/s13336-015-0019-3
  25. Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018). Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data. JAMA Internal Medicine, 178(11), 1544. https://doi.org/10.1001/jamainternmed.2018.3763
  26. Kliegr, T., Bahník, Š., & Fürnkranz, J. (2018). A review of possible effects of cognitive biases on interpretation of rule-based machine learning models. ArXiv:1804.02969 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1804.02969
  27. Weng, S. F., Reps, J., Kai, J., Garibaldi, J. M., & Qureshi, N. (2017). Can machine-learning improve cardiovascular risk prediction using routine clinical data? PLOS ONE, 12(4), e0174944. https://doi.org/10.1371/journal.pone.0174944
  28. Suresh, H., Hunt, N., Johnson, A., Celi, L. A., Szolovits, P., & Ghassemi, M. (2017). Clinical Intervention Prediction and Understanding using Deep Networks. ArXiv:1705.08498 [Cs]. Retrieved from http://arxiv.org/abs/1705.08498
  29. Vayena, E., Blasimme, A., & Cohen, I. G. (2018). Machine learning in medicine: Addressing ethical challenges. PLOS Medicine, 15(11), e1002689. https://doi.org/10.1371/journal.pmed.1002689
  30. Verghese, A., Shah, N. H., & Harrington, R. A. (2018). What This Computer Needs Is a Physician: Humanism and Artificial Intelligence. JAMA, 319(1), 19. https://doi.org/10.1001/jama.2017.19198

Who is planning for the future of physiotherapy?

In the middle ages, cities could spend more than 100 years building a cathedral while at the same time believing that the apocalypse was imminent. They must’ve had a remarkable conviction that commissioning these projects would guarantee them eternal salvation. Compare this to the way we think about planning and design today where, for example, we don’t think more than 3 years into the future simply because that would fall outside of this organisational election cycle. Sometimes it feels like the bulk of the work that a politician does today is to secure the funding that will get them re-elected tomorrow. Where do we see real-world examples of long-term planning that will help guide our decision-making in the present?

A few days ago I spent some time preparing feedback on a draft of the HPCSA minimum requirements for physiotherapy training in South Africa and one of the things that struck me was how much of it was just more-of-the-same. This document is going to inform physiotherapy education and practice for at least the next decade and there was no mention of advances at the cutting edge of medical science and the massive impact that emerging technologies are going to have on clinical practice. Genetic engineering, nanotechnology, artificial intelligence and robotics are starting to drive significant changes in healthcare and it seems that, as a profession, we’re largely oblivious to what’s coming. It’s dawned on me that we have no real plan for the future of physiotherapy (the closest I’ve seen is Dave Nicholls new book, called ironically, The End of Physiotherapy).

What would a good plan look like? In the interests of time, I’m just going to take the high-level suggestions from this article on how the US could improve their planning for AI development and make a short comment on each (I’ve expanded on some of these ideas in my OpenPhysio article on the same topic).

  • Invest more: Fund research into practice innovations that take into account the social, economic, ethical and clinical implications of emerging technologies. Breakthroughs in how we can best utilise emerging technologies as core aspects of physiotherapy practice will come through funded research programmes in universities, especially in the early stages of innovation. We need to take the long-term view that, even if robotics, for example, isn’t having a big impact on physiotherapy today, one day we’ll see things like percussion and massage simply go away. We will also need to fund research on what aspects of the care we provide are really valued by patients (and what they, and funders, will pay for).
  • Prepare for job losses: From the article: “While [emerging technologies] can drive economic growth, it may also accelerate the eradication of some occupations, transform the nature of work in other jobs, and exacerbate economic inequality.” For example, self-driving cars are going to massively drive down the injuries that occur as a result of MVAs. Orthopaedic-related physiotherapy work is, therefore, going to dry up as the patient pool gets smaller. Preventative, personalised medicine will likewise result in dramatic reductions in the incidence of chronic conditions of lifestyle. The “education” component of practice will be outsourced to apps. Even if physiotherapy jobs are not entirely lost, they will certainly be transformed unless we start thinking of how our practice can evolve.
  • Nurture talent: We will need to ensure that we retain and recapture interest in the profession. I’m not sure about other countries but in South Africa, we have a relatively high attrition rate in physiotherapy after a few years of clinical work. The employment prospects and long-term career options, especially in the public health system, are quite poor and many talented physiotherapists leave because they’re bored or frustrated. I recently saw a post on LinkedIn where one of our most promising graduates from 5 years ago is now a property developer. After 4 years of intense study and commitment, and 3 years of clinical practice, he just decided that physiotherapy isn’t where he sees his long-term future. He and many others who have left health care practice represent a deep loss for the profession.
  • Prioritise education: At the undergraduate level we should re-evaluate the curriculum and ensure that it is fit for purpose in the 21st century. How much of our current programmes are concerned with the impact of robotics, nanotechnology, genetic engineering and artificial intelligence? We will need to create space for in-depth development within physiotherapy but also ensure development across disciplines (the so-called T-shaped graduate). Continuing professional development will become increasingly important as more aspects of professional work change and over time, are eradicated. Those who cannot (or will not) continue learning are unlikely to have meaningful long-term careers.
  • Guide regulation: At the moment, progress in emerging technologies is being driven by startups who are funded with venture-capital and whose primary goal is rapid growth to fuel increasing valuations. This ecosystem doesn’t encourage entrepreneurs to limit risks and instead pushes them to “move fast and break things”, which isn’t exactly aligned with the medical imperative to “first do no harm”. Health professionals will need to ensure that technologies that are introduced into clinical practice are first and foremost serving the interests of patients, rather than driving up the value of medical technology startups. If we are not actively involved in regulating these technologies, we are likely to find our practice subject to them.
  • Understand the technology: In order to engage with any of the previous items in the list, we will first need to understand the technologies involved. For example, if you don’t know how the methods of data gathering and analysis can lead to biased algorithmic decision-making, will you be able to argue for why your patient’s health insurance funder shouldn’t make decisions about what interventions you need to provide? We need to ensure that we are not only specialists in clinical practice, but also specialists in how technology will influence clinical practice.

Each of the items in the list above is only very briefly covered here, and each could be the foundation for PhD-level programmes of research. If you’re interested in the future of the profession (and by that I mean you’re someone who wonders what health professional practice will look like in 100 years), I’d love to hear your thoughts. Do you know of anyone who has started building our cathedrals?

The evolution of Atlas from Boston Dynamics

This overview of the changes in capabilities of the Atlas humanoid robot from Boston Dynamics is both fascinating and bit unsettling. In 5 years Atlas has gone from struggling to stand on one leg, to walking on uneven surfaces, to running on uneven surfaces, to doing backflips and now, in October 2018, to bounding up a staggered series of wooden platforms. It’s worth noting that very few human beings would be able to accomplish this last feat.

According to Boston Dynamics, Atlas’ software uses all parts of the body to generate the necessary force to propel the robot up the platforms. The most impressive part of the last demo is the fact that “...Atlas uses computer vision and visible markers on the platforms to decide when and how to shift it‘s weight. So, it’s not just executing a program, it’s making it up as it goes along.” In other words, Atlas is making real-time decisions about how to move, based on what it sees in front of it. No-one has told it what to do.

The profound implication of this is that these things are only ever going to get better, and the rate of change is going to increase. Now that they’ve solved “balance”, “walking”, “running”, and “jumping”, what will Boston Dynamics turn to next? Once Atlas has achieved parity with human performance it’s only a matter of time before it’s superhuman in every physical ability we care about.

Rodney Brooks | Robotics & AI – Their Present & Future

Rodney Brooks was one of the leading developers of AI coding tools throughout the 80s and early 90s at MIT, where he spent a decade running one of the two largest and most prominent AI centres in the world. There are few who can match the breadth, depth, and duration of Rodney’s purview on the tech industry and this makes for a fascinating conversation.

In this podcast, Brooks diverges from fashionable narratives on the risk of super AI risk; the extent to which jobs will be imperiled by automation (he’s more worried about a labor shortage than a job shortage); and the timeline of the rise of self-driving cars (this being intersection of his two domains of foundational expertise: robotics and AI).

See also

Physiotherapy in 2050: Ethical and clinical implications

This post describes a project that I began earlier this week with my 3rd year undergraduate students as part of their Professional Ethics module. The project represents a convergence of a few ideas that have been bouncing around in my head for a couple of years and are now coming together as a result of a proposal that I’m putting together for a book chapter for the Critical Physiotherapy Network. I’m undecided at this point if I’ll develop it into a full research proposal, as I’m currently feeling more inclined to just have fun with it rather than turn it into something that will feel more like work.

The project is premised on the idea that health and medicine – embedded within a broader social construct – will be significantly impacted by rapidly accelerating changes in technology. The question we are looking to explore in the project is: What are the moral, ethical, legal, and clinical implications for physiotherapy practice when the boundaries of medical and health science are significantly shifted as a result of technological advances?

The students will work in small groups that are allocated an area of medicine and health where we are seeing significant change as a result of the integration of advanced technology. Each week in class I will present an idea that is relevant to our Professional Ethics module (for example, the concept of human rights) and then each group will explore that concept within the framework of their topic. So, some might look at how gene therapy could influence how we think about our rights, while others might ask what it even means to be human. I’m not 100% how this is going to play out and will most likely adapt the project as we progress, taking into account student feedback and the challenges we encounter. I can foresee some groups having trouble with certain ethical constructs simply because it may not be applicable to their topic.

Exoskeletons are playing an increasingly important role in neurological rehabilitation.
Exoskeletons playing an increasingly important role in neurological rehabilitation.
The following list and questions aim to stimulate the discussion and to give some idea of what we are looking at (this list is not exhaustive and I’m still playing around with ideas – suggestions are welcome):

  1. Artificial intelligence and algorithmic ethical decision-making. Can computers be ethical? How is ethical reasoning incorporated into machines? How will ethical algorithms impact health, for example, when computers make decisions about organ transplant recipients? Can ethics programmed into machines?
  2. Nanotechnology. As our ability to manipulate our world at the atomic level advances, what changes can we expect to see for physiotherapists and physiotherapy practice? How far can we go with integrating technology into our bodies before we stop being “human”?
  3. Gene therapy. What happens when genetic disorders that provide specialisation areas for physiotherapists are eradicated through gene therapy? What happens when we can “fix” the genetic problems that lead to complications that physiotherapists have traditionally had a significant role in. For example, what will we do when cystic fibrosis is cured? What happens when we have a vaccine for HIV? Or when ALS is little more than an inconvenience?
  4. Robotics. What happens when patients who undergo amputations are fitted with prosthetics that link to the nervous system? When exoskeletons for paralysed patients are common? How much of robotic systems will students need to know about? Will exoskeletons be the new wheelchairs?
  5. Aging. What happens when the aging population no longer ages? How will physiotherapy change as the human lifespan is extended? There is an entire field of physiotherapy devoted to the management of the aging population; what will happen to that? How will palliative care change?
  6. Augmented reality. When we can overlay digital information onto our visual field, what possibilities exist for effective patient management? For education? What happens when that information is integrated with location-based data, so that patient-specific information is presented to us when we are near that patient?
  7. Virtual reality. What will it mean for training when we can build entire hospitals and patient interactions in the virtual world? When we can introduce students to the ICU in their first year? This could be especially useful when we have challenges with finding enough placements for students who need to do clinical rotations.
  8. 3D printing. What happens when we can print any equipment that we need, that is made exactly to the patient’s specifications? How will this affect the cost of equipment distribution to patients? Can 3D printed crutches be recycled? Reused by other patients? What new kinds of equipment can be invented when we are not constrained by the production lines of the companies who traditionally make the tools we use?
  9. Brain-computer interfaces. When patients are able to control computers (and by extension, everything linked to the computer) simply by thinking about it, what does that mean for their roles in the world? What does it mean when someone with a C7 complete spinal cord injury can still be a productive member of society? What does it mean for community re-integration? How will “rehabilitation” change if computer science is a requirement to even understand the tools our patients use?
  10. Quantified self. As we begin to use sensors close to our bodies (inside our phones, watches, etc.) and soon – inside our bodies – we will have access to an unprecedented amount of personal (very personal) data about ourselves. We will be able to use that data to inform decision making about our health and well-being, which will change the patient-therapist relationship. This will most likely have the effect of modifying the power differential between patients and clinicians. How will we deal with that? Are we training students to know what to do with that patient information? To understand how these sensors work?
  11. Processing power. While this is actually something that is linked to every other item in the list, it might warrant it’s own topic purely because everything else depends on the continuous improvements in processing power and parallel reduction in cost.
  12. The internet. I’m not sure about this. While the architecture of the internet itself is unlikely to change much in the next few decades (disregarding the idea that the internet as we know it might be supplanted with something better), who has access to it and how we use it will most certainly change.

An artist's depiction of a nanobot that is smaller than blood cells.
Nanobot smaller than blood cells.
I should state that we will be working under certain assumptions:

  • That the technology will not be uniformly integrated into society and health systems i.e. that wealth disparity or income inequality will directly affect implementation of certain therapies. This will,obviously have ethical and moral implications.
  • That the technology will not be freely available i.e. that corporations will license certain genetic therapies and withhold their use on those who cannot pay the license.
  • That technological progression will continue over time i.e. that regulations will not prevent, for example, further research into stem cell therapy.
  • …we may have to make additional assumptions as we move forward but this is all I can think of now

We’ll probably find that there will be significant overlap in the above topics, since some are specific technologies that will have an influence on other areas. For example, gene therapy and nanotechnology may have an impact on aging; artificial intelligence will impact many areas, as will robotics and computing power. The idea isn’t that these topics are discrete and separate, but that they provide a focus point for discussion and exploration, with the understanding that overlap is inevitable. In fact, overlap is preferable, since it will help us explore relationships between the different areas and to find connections that we maybe were not previously aware of.

Giving patients bad news in virtual spaces where we can control the interaction.
Giving patients bad news in virtual spaces where we can control the interaction.
The activities that the students engage in during this project are informed by the following ideas, which overlap with each other:

  • Authentic learning is a framework for designing learning tasks that lead to deeper engagement by students. Authentic tasks should be complex, collaborative, ill-defined, and completed over long periods.
  • Inquiry-based learning suggests that students should identify challenging questions that are aimed at addressing gaps in their understanding of complex problems. The research that they conduct is a process they go through in order to achieve outcomes, rather than being an end in itself.
  • Project-based learning is the idea that we can use full projects – based in the real world – to discuss and explore the disciplinary content, while simultaneously developing important skills that are necessary for learning in the 21st century.

I should be clear that I’m not really sure what the outcome of this project will be. I obviously have objectives for my students’ learning that relate to the Professional Ethics module but in terms of what we cover, how we cover it, what the final “product” is…these are all still quite fluid. I suppose that, ideally, I would like for us as a group (myself and the students) to explore the various concepts together and to come up with a set of suggestions that might help to guide physiotherapy education (or at least, physiotherapy education as practiced by me) over the next 5-10 years.

Augmented reality has significant potential for education.
Augmented reality has significant potential for education.
So much of physiotherapy practice – and therefore, physiotherapy education – is premised on the idea that what has been important over the last 50 years will continue to be important for the next 50. However, as technology progresses and we see incredible advances in the integration of technology into medicine and health systems, we need to ask if the next 50 years are going to look anything like the last 50. In fact, it almost seems as if the most important skill we can teach our students is how to adapt to a constantly changing world. If this is true, then we may need to radically change what we prioritise in the curriculum, as well as how we teach students to learn. When every fact is instantly available, when algorithms influence clinical decision-making, when amputees are fitted with robotic prosthetics controlled directly via brain-computer interfaces…where does that leave the physiotherapist? This project is a first step (for me) towards at least beginning to think about these kinds of questions.