Tomorrow I’ll be presenting a short seminar at the University of Cape Town on a book chapter that was published earlier this year, called Shaping our algorithms before they shape us. Here are the slides I’ll be using, which I think are a useful summary of the chapter itself.
I’ve just had a chapter published in an edited collection entitled: Artificial Intelligence and Inclusive Education: Speculative Futures and Emerging Practices. The book is edited by Jeremy Knox, Yuchen Wang and Michael Gallagher and is available here.
Here’s the citation: Rowe M. (2019) Shaping Our Algorithms Before They Shape Us. In: Knox J., Wang Y., Gallagher M. (eds) Artificial Intelligence and Inclusive Education. Perspectives on Rethinking and Reforming Education. Springer, Singapore. https://doi.org/10.1007/978-981-13-8161-4_9.
And here’s my abstract:
A common refrain among teachers is that they cannot be replaced by intelligent machines because of the essential human element that lies at the centre of teaching and learning. While it is true that there are some aspects of the teacher-student relationship that may ultimately present insurmountable obstacles to the complete automation of teaching, there are important gaps in practice where artificial intelligence (AI) will inevitably find room to move. Machine learning is the branch of AI research that uses algorithms to find statistical correlations between variables that may or may not be known to the researchers. The implications of this are profound and are leading to significant progress being made in natural language processing, computer vision, navigation and planning. But machine learning is not all-powerful, and there are important technical limitations that will constrain the extent of its use and promotion in education, provided that teachers are aware of these limitations and are included in the process of shepherding the technology into practice. This has always been important but when a technology has the potential of AI we would do well to ensure that teachers are intentionally included in the design, development, implementation and evaluation of AI-based systems in education.
An interesting (and sane) conversation about the defeat of AlphaGo by AlphaGo Zero. It almost completely avoids the science-fiction-y media coverage that tends to emphasise the potential for artificial general intelligence and instead focuses on the following key points:
Go is a stupendously difficult board game for computers to play but it’s a game in which both players have total information and where the rules are relatively simple. This does not reflect the situation in any real-world decision-making scenario. Correspondingly, this is necessarily a very narrow definition of what an intelligent machine can do.
AlphaGo Zero represents an order of magnitude improvement in algorithmic modelling and power consumption. In other words, it does a lot more with a lot less.
Related to this, AlphaGo Zero started from scratch, with humans providing only the rules of the game. So Zero used reinforcement learning (rather than supervised learning) to figure out the same (and in some cases, better) moves than human beings have done over the last thousand years or so).
It’s an exciting achievement but shouldn’t be conflated with any significant step towards machine intelligence that transfers beyond highly constrained scenarios.
A long-standing goal of artificial intelligence is an algorithm that learns, tabula rasa, superhuman proficiency in challenging domains. Recently, AlphaGo became the first program to defeat a world champion in the game of Go. The tree search in AlphaGo evaluated positions and selected moves using deep neural networks. These neural networks were trained by supervised learning from human expert moves, and by reinforcement learning from self-play. Here we introduce an algorithm based solely on reinforcement learning, without human data, guidance or domain knowledge beyond game rules. AlphaGo becomes its own teacher: a neural network is trained to predict AlphaGo’s own move selections and also the winner of AlphaGo’s games. This neural network improves the strength of the tree search, resulting in higher quality move selection and stronger self-play in the next iteration. Starting tabula rasa, our new program AlphaGo Zero achieved superhuman performance, winning 100–0 against the previously published, champion-defeating AlphaGo.
Allowing the proliferation of algorithmic surveillance as a substitution for human engagement and judgment helps pave the road to an ugly future where students spend more time interacting algorithms than instructors or each other. This is not a sound way to help writers develop robust and flexible writing practices.
First of all, I don’t use Turnitin and I don’t see any good reason for doing so. Combating the “cheating economy” doesn’t depend on us catching the students; it depends on creating the conditions in which students believe that cheating offers little real value relative to the pedagogical goals they are striving for. In general, I agree with a lot that the author is saying.
So, with that caveat out of the way, I wanted to comment on a few other pieces in the article that I think make significant assumptions and limit the utility of the piece, especially with respect to how algorithms (and software agents in particular) may be useful in the context of education.
The use of the word “surveillance” in the quote above establishes the context for the rest of the paragraph. If the author had used “guidance” instead, the tone would be different. Same with “ugly”; remove that word and the meaning of the sentence is very different. It just makes it clear that the author has an agenda which clouds some of the other arguments about the use of algorithms in education.
For example, the claim that it’s a bad thing for students to interact with an algorithm instead of another person is empirical; it can be tested. But it’s presented here in a way that implies that human interaction is simply better. Case closed. But what if we learned that algorithmic guidance (via AI-based agents/tutors) actually lead to better student outcomes than learning with/from other people? Would we insist on human interaction because it would make us feel better? Why not test our claims by doing the research before making judgements?
The author uses a moral argument (at least, this was my take based on the language used) to position AI-based systems (specifically, algorithms) as being inherently immoral with respect to student learning. There’s a confusion between the corporate responsibility of a private company – like Turnitin – to make a profit, and the (possibly pedagogically sound) use of software agents to enhance some aspects of student learning.
Again, there’s some good advice around developing assignments and classroom conditions that make it less likely that students will want to cheat. This is undoubtedly a Good Thing. However, some of the claims about the utility of software agents are based on assumptions that aren’t necessarily supported by the evidence.
To open up the AI black box and facilitate trust, companies must develop AI systems that perform reliably — that is, make correct decisions — time after time. The machine-learning models on which the systems are based must also be transparent, explainable, and able to achieve repeatable results.
It still bothers me that we insist on explainability for AI systems while we’re quite happy for the decisions of clinicians to remain opaque, inaccurate, and unreliable. We need to move past the idea that there’s anything special about human intuition and that algorithms must satisfy a set of criteria that we would never dream of applying to ourselves.
Doctors are human. And humans make mistakes. And while scientific advancements have dramatically improved our ability to detect and treat illness, they have also engendered a perception of precision, exactness and infallibility. When patient expectations collide with human error, malpractice lawsuits are born. And it’s a very expensive problem.
There are few things to note in this article. The first, and most obvious, was that we have a much higher standard for AI-based expert systems (i.e. algorithmic diagnosis and prediction) than we do for human experts. Our expectations for algorithmic clinical decision-making are far more exacting than those we have for physicians. It seems strange that we accept the fallibility of human beings but expect nothing less than perfection from AI-based systems. 
Medical errors are more frequent than anyone cares to admit. In radiology, the retrospective error rate is approximately 30% across all specialities, with real-time error rates in daily practice averaging between 3% and 5%.
The second takeaway was that one of the most significant areas of influence for AI in clinical settings may not be in the primary diagnosis but rather the follow up analysis that highlights potential mistakes that the clinician may have made. These applications of AI for secondary diagnostic review will be cheap and won’t add any additional workload to healthcare professionals. They will simply review the clinician’s conclusion and flag those cases that may benefit from additional testing. Of course, this will probably be driven by patient litigation.
 Incidentally, the same principle seems to be true for self-driving cars; we expect nothing but a perfect safety record for autonomous vehicles but are quite happy with the status quo for human drivers (1.2 million traffic-related deathsin a single year). Where is the moral panic around the mass slaughter of human beings by human drivers? If an algorithm is only slightly safer than a human being behind the wheel of a car it would result in thousands fewer deaths per year. And yet it feels like we’re going to delay the introduction of autonomous cars until they meet some perfect standard. To me at least, that seems morally wrong.
We should stop using images of humanoid robots to represent an embodied form of artificial intelligence, especially when the AI being referenced is an algorithm, which in almost all cases in the mainstream media, it is. It’s confusing for readers because we’re nowhere near the kind of general intelligence that these pictures imply. For the foreseeable future, “AI” is a set of machine learning algorithms that “maximise a reward function” and is incapable of anything more than solving very specific problems with a lot of help.
AI isn’t magic, it’s just maths. I know that statistical methods aren’t as cool as the androids but if we really want people to get a better conceptual understanding of AI we’d be better off using images like this to illustrate the outputs of AI-based systems:
The word “diagnosis,” he reminded me, comes from the Greek for “knowing apart.” Machine-learning algorithms will only become better at such knowing apart—at partitioning, at distinguishing moles from melanomas. But knowing, in all its dimensions, transcends those task-focussed algorithms. In the realm of medicine, perhaps the ultimate rewards come from knowing together.
This New Yorker article by Siddhartha Mukherjee explores the implications for practice and diagnostic reasoning in a time when software is increasingly implicated in clinical decision-making. While the article is more than a year old (a long time in AI and machine learning research), it still stands up as an excellent, insightful overview of the state of AI-based systems in the domain of clinical care. It’s a long read but well worth it.
Two weeks ago I presented some of my thoughts on the implications of AI and machine learning in clinical practice and health professions education at the 2018 SAAHE conference. Here are the slides I used (20 slides for 20 seconds each) with a very brief description of each slide. This presentation is based on a paper I submitted to OpenPhysio, called: “Artificial intelligence in clinical practice: Implications for physiotherapy education“.
This post describes a project that I began earlier this week with my 3rd year undergraduate students as part of their Professional Ethics module. The project represents a convergence of a few ideas that have been bouncing around in my head for a couple of years and are now coming together as a result of a proposal that I’m putting together for a book chapter for the Critical Physiotherapy Network. I’m undecided at this point if I’ll develop it into a full research proposal, as I’m currently feeling more inclined to just have fun with it rather than turn it into something that will feel more like work.
The project is premised on the idea that health and medicine – embedded within a broader social construct – will be significantly impacted by rapidly accelerating changes in technology. The question we are looking to explore in the project is: What are the moral, ethical, legal, and clinical implications for physiotherapy practice when the boundaries of medical and health science are significantly shifted as a result of technological advances?
The students will work in small groups that are allocated an area of medicine and health where we are seeing significant change as a result of the integration of advanced technology. Each week in class I will present an idea that is relevant to our Professional Ethics module (for example, the concept of human rights) and then each group will explore that concept within the framework of their topic. So, some might look at how gene therapy could influence how we think about our rights, while others might ask what it even means to be human. I’m not 100% how this is going to play out and will most likely adapt the project as we progress, taking into account student feedback and the challenges we encounter. I can foresee some groups having trouble with certain ethical constructs simply because it may not be applicable to their topic.
The following list and questions aim to stimulate the discussion and to give some idea of what we are looking at (this list is not exhaustive and I’m still playing around with ideas – suggestions are welcome):
Artificial intelligence and algorithmic ethical decision-making. Can computers be ethical? How is ethical reasoning incorporated into machines? How will ethical algorithms impact health, for example, when computers make decisions about organ transplant recipients? Can ethics programmed into machines?
Nanotechnology. As our ability to manipulate our world at the atomic level advances, what changes can we expect to see for physiotherapists and physiotherapy practice? How far can we go with integrating technology into our bodies before we stop being “human”?
Gene therapy. What happens when genetic disorders that provide specialisation areas for physiotherapists are eradicated through gene therapy? What happens when we can “fix” the genetic problems that lead to complications that physiotherapists have traditionally had a significant role in. For example, what will we do when cystic fibrosis is cured? What happens when we have a vaccine for HIV? Or when ALS is little more than an inconvenience?
Robotics. What happens when patients who undergo amputations are fitted with prosthetics that link to the nervous system? When exoskeletons for paralysed patients are common? How much of robotic systems will students need to know about? Will exoskeletons be the new wheelchairs?
Aging. What happens when the aging population no longer ages? How will physiotherapy change as the human lifespan is extended? There is an entire field of physiotherapy devoted to the management of the aging population; what will happen to that? How will palliative care change?
Augmented reality. When we can overlay digital information onto our visual field, what possibilities exist for effective patient management? For education? What happens when that information is integrated with location-based data, so that patient-specific information is presented to us when we are near that patient?
Virtual reality. What will it mean for training when we can build entire hospitals and patient interactions in the virtual world? When we can introduce students to the ICU in their first year? This could be especially useful when we have challenges with finding enough placements for students who need to do clinical rotations.
3D printing. What happens when we can print any equipment that we need, that is made exactly to the patient’s specifications? How will this affect the cost of equipment distribution to patients? Can 3D printed crutches be recycled? Reused by other patients? What new kinds of equipment can be invented when we are not constrained by the production lines of the companies who traditionally make the tools we use?
Brain-computer interfaces. When patients are able to control computers (and by extension, everything linked to the computer) simply by thinking about it, what does that mean for their roles in the world? What does it mean when someone with a C7 complete spinal cord injury can still be a productive member of society? What does it mean for community re-integration? How will “rehabilitation” change if computer science is a requirement to even understand the tools our patients use?
Quantified self. As we begin to use sensors close to our bodies (inside our phones, watches, etc.) and soon – inside our bodies – we will have access to an unprecedented amount of personal (very personal) data about ourselves. We will be able to use that data to inform decision making about our health and well-being, which will change the patient-therapist relationship. This will most likely have the effect of modifying the power differential between patients and clinicians. How will we deal with that? Are we training students to know what to do with that patient information? To understand how these sensors work?
Processing power. While this is actually something that is linked to every other item in the list, it might warrant it’s own topic purely because everything else depends on the continuous improvements in processing power and parallel reduction in cost.
The internet. I’m not sure about this. While the architecture of the internet itself is unlikely to change much in the next few decades (disregarding the idea that the internet as we know it might be supplanted with something better), who has access to it and how we use it will most certainly change.
I should state that we will be working under certain assumptions:
That the technology will not be uniformly integrated into society and health systems i.e. that wealth disparity or income inequality will directly affect implementation of certain therapies. This will,obviously have ethical and moral implications.
That the technology will not be freely available i.e. that corporations will license certain genetic therapies and withhold their use on those who cannot pay the license.
That technological progression will continue over time i.e. that regulations will not prevent, for example, further research into stem cell therapy.
…we may have to make additional assumptions as we move forward but this is all I can think of now
We’ll probably find that there will be significant overlap in the above topics, since some are specific technologies that will have an influence on other areas. For example, gene therapy and nanotechnology may have an impact on aging; artificial intelligence will impact many areas, as will robotics and computing power. The idea isn’t that these topics are discrete and separate, but that they provide a focus point for discussion and exploration, with the understanding that overlap is inevitable. In fact, overlap is preferable, since it will help us explore relationships between the different areas and to find connections that we maybe were not previously aware of.
The activities that the students engage in during this project are informed by the following ideas, which overlap with each other:
Authentic learning is a framework for designing learning tasks that lead to deeper engagement by students. Authentic tasks should be complex, collaborative, ill-defined, and completed over long periods.
Inquiry-based learning suggests that students should identify challenging questions that are aimed at addressing gaps in their understanding of complex problems. The research that they conduct is a process they go through in order to achieve outcomes, rather than being an end in itself.
Project-based learning is the idea that we can use full projects – based in the real world – to discuss and explore the disciplinary content, while simultaneously developing important skills that are necessary for learning in the 21st century.
I should be clear that I’m not really sure what the outcome of this project will be. I obviously have objectives for my students’ learning that relate to the Professional Ethics module but in terms of what we cover, how we cover it, what the final “product” is…these are all still quite fluid. I suppose that, ideally, I would like for us as a group (myself and the students) to explore the various concepts together and to come up with a set of suggestions that might help to guide physiotherapy education (or at least, physiotherapy education as practiced by me) over the next 5-10 years.
So much of physiotherapy practice – and therefore, physiotherapy education – is premised on the idea that what has been important over the last 50 years will continue to be important for the next 50. However, as technology progresses and we see incredible advances in the integration of technology into medicine and health systems, we need to ask if the next 50 years are going to look anything like the last 50. In fact, it almost seems as if the most important skill we can teach our students is how to adapt to a constantly changing world. If this is true, then we may need to radically change what we prioritise in the curriculum, as well as how we teach students to learn. When every fact is instantly available, when algorithms influence clinical decision-making, when amputees are fitted with robotic prosthetics controlled directly via brain-computer interfaces…where does that leave the physiotherapist? This project is a first step (for me) towards at least beginning to think about these kinds of questions.