I’ve just had a chapter published in an edited collection entitled: Artificial Intelligence and Inclusive Education: Speculative Futures and Emerging Practices. The book is edited by Jeremy Knox, Yuchen Wang and Michael Gallagher and is available here.
Here’s the citation: Rowe M. (2019) Shaping Our Algorithms Before They Shape Us. In: Knox J., Wang Y., Gallagher M. (eds) Artificial Intelligence and Inclusive Education. Perspectives on Rethinking and Reforming Education. Springer, Singapore. https://doi.org/10.1007/978-981-13-8161-4_9.
And here’s my abstract:
A common refrain among teachers is that they cannot be replaced by intelligent machines because of the essential human element that lies at the centre of teaching and learning. While it is true that there are some aspects of the teacher-student relationship that may ultimately present insurmountable obstacles to the complete automation of teaching, there are important gaps in practice where artificial intelligence (AI) will inevitably find room to move. Machine learning is the branch of AI research that uses algorithms to find statistical correlations between variables that may or may not be known to the researchers. The implications of this are profound and are leading to significant progress being made in natural language processing, computer vision, navigation and planning. But machine learning is not all-powerful, and there are important technical limitations that will constrain the extent of its use and promotion in education, provided that teachers are aware of these limitations and are included in the process of shepherding the technology into practice. This has always been important but when a technology has the potential of AI we would do well to ensure that teachers are intentionally included in the design, development, implementation and evaluation of AI-based systems in education.
In this wide-ranging conversation, Vanessa and I discuss her 25 years in health professions education and research. We look at the changes that have taken place in the domain over the past 5-10 years and how this has impacted the opportunities available for South African health professions educators in the early stages of their careers. We talk about developing the confidence to approach people you may want to work with, from the days when you had to be physically present at a conference workshop, to explore novel ways to connect with colleagues in a networked world. We discuss Vanessa’s role in establishing the Southern African FAIMER Regional Institute (SAFRI), as well as the African Journal of Health Professions Education (AJHPE) and what we might consider when presented with opportunities to drive change in the profession.
Vanessa has a National Excellence in Teaching and Learning Award from the Council of Higher Education and the Higher Education Learning and Teaching Association of South Africa (HELTASA), and holds a Teaching at University (TAU) fellowship from the Council for Higher Education of South Africa. She is a Deputy Editor at the journal Medical Education, and Associate Editor of Advances in Health Sciences Education. Vanessa was Professor and Chair of Clinical Medicine at the University of Cape Town from 2008-2018in health and is currently Honorary Professor of Medicine at UCT. She works as an educational consultant to the Colleges of Medicine of South Africa.
It’s been about a year and a half since Ben and I started the In Beta community (see my first post in July 2017) and I wanted to reflect on what we’ve achieved in the past 18 months or so. Here are the major aspects of the project with some statistics and my thoughts on the process.
Website: We’re hosting our website on a server provided by the University of the Western Cape and use open source software (WordPress) to build the site, which means that the project costs Ben and I nothing except our time and energy. A few months ago I made a few big changes to the site, which hadn’t been updated since it launched, including a new theme and layout, new per-episode images, and an embedded media player for each episode. This is also going to be more important as the site becomes more central to our plans and needs to do more than simply distribute the audio for the podcasts.
We’ve had a fair amount of traffic since we launched the site in October 2017; far more than I expected. The numbers are obviously quite low relative to more popular sites, but consider that this is a project about physiotherapy education.
Most of our visitors came from the UK (where Ben lives) and the Netherlands (where Joost lives). I’m not sure if that’s a coincidence or if the two of them are just uncommonly popular. Incidentally, Joost has been a major supporter and promoter of the project through his connections with ENPHE and we hope that this collaboration continues to grow.
Podcasts: We’ve released 8 episodes including our first one in October 2017, so we publish about one episode every 1.5 months. We have another 3 episodes recorded but which we haven’t finished editing yet. The audio editing is, by far, the most time-consuming part of the process. We’re hoping to limit the hassle of this component by improving the quality of the initial recording, through 1) getting better at moderating the conversations and so having less to cut, and 2) making more of an effort to record better audio in the first place. Here are the 8 episodes we’ve published so far, along with the number of times each has been downloaded. These statistics exclude the first 50 or so downloads of the first episode, which was hosted on Soundcloud before we moved to our own distribution platform.
Here are the top 10 countries by number of downloads:
Projects: One of our original ideas was to use the website as a way to share examples of classroom exercises, assignments, and teaching practices that others would be able to use as a resource. The plan was to describe in a fair amount of detail the process for setting up a learning task that others could simply copy, maybe with a few minor tweaks. The project pages would include the specific learning outcomes that the lecturer hopes to achieve, comprehensive descriptions of the learning activities, links to freely available resources, and examples of student work. This aspect of In Beta hasn’t taken off as much as we would’ve liked but the potential is still there and will hopefully continue growing over time.
Google Docs: We started with Google Docs as a way to plan for our podcast recordings, using a templated outline that we’d invite guests to complete. The idea is that guests on the podcast will use the template to establish the context for the conversation, including the background, the problem they’re trying to address, and a reading list for interested participants. We then take some of that information and incorporate it into the show notes for the episode and leave the Google Doc online for further reading if anyone is interested. The process (and template) has remained more or less the same since we initially described it but I’m uncertain about whether or not we should include it going forward. It seems like a lot of PT to ask guests to complete and, without statistics for Docs, we can’t be sure if anyone is going there. On the other hand, it really does seem to be good preparation for us to have a deep dive into the topic.
Membership: We had about 100 people join the Google+ community but saw little engagement on the site. I think that this is understandable considering that most people have more than enough going on in their personal and professional lives to add yet another online destination to their lists. Most people are already on several social media platforms and it’s not reasonable to expect them to add Google+ just for this project. So we weren’t too upset to see that Google is planning to sunset the consumer version of Google+, so in some ways it’s a bit of a relief not to have to worry about managing the community in different places. We’re in the process of asking people to migrate to the project website and sign up for email notifications of announcements.
Conference collaborations: Ben and I worked with Joost to run two In Beta workshops at the IPSM (Portugal) and ENPHE conferences (Paris) in 2018. We based both sessions on the Unconference format and used them as experiments to think differently about how conference workshops could be useful for participants in the room, as well as those who were “outside” of it. While neither of the workshops went exactly how we planned, I think the fact that both of these sessions actually happened, in large part due to the work that Joost and Ben put in, was a success in itself. We’ve recorded our thoughts on this process and will publish that as an episode early in 2019. It’d be nice to have more of these sessions where we try to do something “in the world”.
Plans for 2019: Our rough ideas for the next 12 months include the following:
More frequent podcast episodes, which should be possible if we can reduce the amount of time it takes to edit each episode. It’d also be nice to get assistance with the audio editing, so if you’re interested in being involved and have an interest in that kind of thing, let us know.
Work on more collaborative projects with colleagues who are interested in alternative approaches to physiotherapy education. For example, it might be interesting to publish an edited “book” of short stories related to physiotherapy education. It could be written by students, educators and clinicians, and might cover a broad range of topics that explore physiotherapy education from a variety of perspectives.
Grow the community so that In Beta is more than a podcast. We started the project because we wanted to share interesting conversations in physiotherapy education and we think that there’s enormous scope for this idea to be developed. But we also know that we’re never going to have all the good ideas ourselves and so we need to involve more of the people doing the interesting work in classrooms and clinical spaces around the world.
Host a workshop for In Beta community members, possibly at a time when enough of us are gathered together in the same place. Maybe in Europe somewhere. Probably in May. Something like a seminar or colloquium on physiotherapy education. If this sounds like something you may like to be involved with, please let us know.
It’s easy to forget what you’ve achieved when you’re caught up in the process. I think that both Ben and I would probably like to have done a bit more on the project over the past 18 months but if I look at where we started (a conversation over coffee at a conference in 2016) then I’m pretty happy with what we’ve accomplished. And I’m excited for 2019.
Action 1: We are shutting down Google+ for consumers.
This review crystallized what we’ve known for a while: that while our engineering teams have put a lot of effort and dedication into building Google+ over the years, it has not achieved broad consumer or developer adoption, and has seen limited user interaction with apps. The consumer version of Google+ currently has low usage and engagement: 90 percent of Google+ user sessions are less than five seconds.
I don’t think it’s a surprise to anyone that Google+ wasn’t a big hit although I am surprised that they’ve taken the step to shut it down for consumers. And this is the problem with online communities in general; when the decision is made that they’re not cost-effective, they’re shut down regardless of the value they create for community members.
When Ben and I started In Beta last year we decided to use Google+ for our community announcements and have been pretty happy with what we’ve been able to achieve with it. The community has grown to almost 100 members and, while we don’t see much engagement or interaction, that’s not why we started using it. For us, it was to make announcements about planning for upcoming episodes and since we didn’t have a dedicated online space, it made sense to use something that already existed. Now that Google+ is being sunsetted we’ll need to figure out another place to set up the community.
The human work of tomorrow will not be based on competencies best-suited for machines, because creative work that is continuously changing cannot be replicated by machines or code. While machine learning may be powerful, connected human learning is novel, innovative, and inspired.
A good post on why learning how to learn is the only reasonable way to think about the future of work (and professional education). The upshot is that Communities of Practice are implicated in helping us adapt to working environments that are constantly changing, as will most likely continue to be the case.
However, I probably wouldn’t take the approach that it’s “us vs machines” because I don’t think that’s where we’re going to end up. I think it’s more likely that those who work closely with AI-based systems will outperform and replace those who don’t. In other words, we’re not competing with machines for our jobs; we’re competing with other people who use machines more effectively than we do.
Trying to be better than machines is not only difficult but our capitalist economy makes it pretty near impossible.
This is both true and a bit odd. No-one thinks they need to be able to do complex mathematics without calculators, and those who are better at using calculators can do more complex mathematics. Why is it such a big leap to realise that we don’t have to be better image classifiers than machines either? Let’s accept that diagnosis from CT will be performed by AI and focus on how that frees up physician time for other human- and patient-centred tasks. What will medical education look like when we’re teaching students that adapting while working with machines is the only way to stay relevant? I think that clinicians who graduate from medical schools who take this approach are more likely to be employed in the future.
You didn’t need to know about how to print on a printing press in order to read a printed book. Writing implements were readily available in various forms in order to record thoughts, as well as communicate with them. The use was simple requiring nothing more than penmanship. The rapid advancement of technology has changed this. Tech has evolved so quickly and so universally in our culture that there is now literacy required in order for people to effectively and efficiently use it.
Reading and writing as a literacy was hard enough for many of us, and now we are seeing that there is a whole new literacy that needs to be not only learned, but taught by us as well.
I wrote about the need to develop these new literacies in a recent article (under review) in OpenPhysio. From the article:
As clinicians become single nodes (and not even the most important nodes) within information networks, they will need data literacy to read, analyse, interpret and make use of vast data sets. As they find themselves having to work more collaboratively with AI-based systems, they will need the technological literacy that enables them to understand the vocabulary of computer science and engineering that enables them to communicate with machines. Failing that, we may find that clinicians will simply be messengers and technicians carrying out the instructions provided by algorithms.
It really does seem like we’re moving towards a society in which the successful use of technology is, at least to some extent, premised on your understanding of how it works. As educators, it is incumbent on us to 1) know how the technology works so that we can 2) help students use it effectively while at the same time avoid exploitation by for-profit companies.
See also: Aoun, J. (2017). Robot proof: Higher Education in the Age of Artificial Intelligence. MIT Press.
Allowing the proliferation of algorithmic surveillance as a substitution for human engagement and judgment helps pave the road to an ugly future where students spend more time interacting algorithms than instructors or each other. This is not a sound way to help writers develop robust and flexible writing practices.
First of all, I don’t use Turnitin and I don’t see any good reason for doing so. Combating the “cheating economy” doesn’t depend on us catching the students; it depends on creating the conditions in which students believe that cheating offers little real value relative to the pedagogical goals they are striving for. In general, I agree with a lot that the author is saying.
So, with that caveat out of the way, I wanted to comment on a few other pieces in the article that I think make significant assumptions and limit the utility of the piece, especially with respect to how algorithms (and software agents in particular) may be useful in the context of education.
The use of the word “surveillance” in the quote above establishes the context for the rest of the paragraph. If the author had used “guidance” instead, the tone would be different. Same with “ugly”; remove that word and the meaning of the sentence is very different. It just makes it clear that the author has an agenda which clouds some of the other arguments about the use of algorithms in education.
For example, the claim that it’s a bad thing for students to interact with an algorithm instead of another person is empirical; it can be tested. But it’s presented here in a way that implies that human interaction is simply better. Case closed. But what if we learned that algorithmic guidance (via AI-based agents/tutors) actually lead to better student outcomes than learning with/from other people? Would we insist on human interaction because it would make us feel better? Why not test our claims by doing the research before making judgements?
The author uses a moral argument (at least, this was my take based on the language used) to position AI-based systems (specifically, algorithms) as being inherently immoral with respect to student learning. There’s a confusion between the corporate responsibility of a private company – like Turnitin – to make a profit, and the (possibly pedagogically sound) use of software agents to enhance some aspects of student learning.
Again, there’s some good advice around developing assignments and classroom conditions that make it less likely that students will want to cheat. This is undoubtedly a Good Thing. However, some of the claims about the utility of software agents are based on assumptions that aren’t necessarily supported by the evidence.
We identify and discuss three different interpretations of the influence of raters’ emotions during assessments: (i) emotions lead to biased decision making; (ii) emotions contribute random noise to assessment, and (iii) emotions constitute legitimate sources of information that contribute to assessment decisions. We discuss these three interpretations in terms of areas for future research and implications for assessment.
Source: Gomez‐Garibello, C. and Young, M. (2018), Emotions and assessment: considerations for rater‐based judgements of entrustment. Med Educ, 52: 254-262. doi:10.1111/medu.13476
When are we going to stop thinking that assessment – of any kind – is objective? As soon as you’re making a decision (about what question to ask, the mode of response, the weighting of the item, etc.) you’re making a subjective choice about the signal you’re sending to students about what you value. If the student considers you to be a proxy of the profession/institution, then you’re subconsciously signalling the values of the profession/institution.
If you’re interested in the topic of subjectivity in assessment, you may be interested in two of our In Beta episodes:
The students had also been asked what grade they thought they would get, and it turned out that levels of trust in those students whose actual grades hit or exceeded that estimate were unaffected by transparency. But people whose expectations were violated – students who received lower scores than they expected – trusted the algorithm more when they got more of an explanation of how it worked. This was interesting for two reasons: it confirmed a human tendency to apply greater scrutiny to information when expectations are violated. And it showed that the distrust that might accompany negative or disappointing results can be alleviated if people believe that the underlying process is fair.
This article uses the example of algorithmic grading of student work to discuss issues of trust and transparency. One of the findings I thought was a useful takeaway in this context is that full transparency may not be the goal, but that we should rather aim for medium transparency and only in situations where students’ expectations are not met. For example, a student who’s grade was lower than expected might need to be told something about how it was calculated. But when they got too much information it eroded trust in the algorithm completely. When students got the grade they expected then no transparency was needed at all i.e. they didn’t care how the grade was calculated.
For developers of algorithms, the article also provides a short summary of what explainable AI might look like. For example, without exposing the underlying source code, which in many cases is proprietary and holds commercial value for the company, explainable AI might simply identify the relationships between inputs and outcomes, highlight possible biases, and provide guidance that may help to address potential problems in the algorithm.