10 recommendations for the ethical use of AI

In February the New York Times hosted the New Work Summit, a conference that explored the opportunities and risks associated with the emergence of artificial intelligence across all aspects of society. Attendees worked in groups to compile a list of recommendations for building and deploying ethical artificial intelligence, the results of which are listed below.

  1. Transparency: Companies should be transparent about the design, intention and use of their A.I. technology.
  2. Disclosure: Companies should clearly disclose to users what data is being collected and how it is being used.
  3. Privacy: Users should be able to easily opt out of data collection.
  4. Diversity: A.I. technology should be developed by inherently diverse teams.
  5. Bias: Companies should strive to avoid bias in A.I. by drawing on diverse data sets.
  6. Trust: Organizations should have internal processes to self-regulate the misuse of A.I. Have a chief ethics officer, ethics board, etc.
  7. Accountability: There should be a common set of standards by which companies are held accountable for the use and impact of their A.I. technology.
  8. Collective governance: Companies should work together to self-regulate the industry.
  9. Regulation: Companies should work with regulators to develop appropriate laws to govern the use of A.I.
  10. “Complementarity”: Treat A.I. as tool for humans to use, not a replacement for human work.

The list of recommendations seems reasonable enough on the surface, although I wonder how practical they are given the business models of the companies most active in developing AI-based systems. As long as Google, Microsoft, Facebook, etc. are generating the bulk of their revenue from advertising that’s powered by the data we give them, they have little incentive to be transparent, to disclose, to be regulated, etc. If we opt our data out of the AI training pool, the AI is more susceptible to bias and less useful/accurate, so having more data is usually better for algorithm development. And having internal processes to build trust? That seems odd.

However, even though it’s easy to find issues with all of these recommendations it doesn’t mean that they’re not useful. The more of these kinds of conversations we have, the more likely it is that we’ll figure out a way to have AI that positively influences society.

Comment: In competition, people get discouraged by competent robots

After each round, participants filled out a questionnaire rating the robot’s competence, their own competence and the robot’s likability. The researchers found that as the robot performed better, people rated its competence higher, its likability lower and their own competence lower.

Lefkowitz, M. (2019). In competition, people get discouraged by competent robots. Cornell Chronicle.

This is worth noting since it seems increasingly likely that we’ll soon be working, not only with more competent robots but also with more competent software. There are already concerns around how clinicians will respond to the recommendations of clinical decision-support systems, especially when those systems make suggestions that are at odds with the clinician’s intuition.

Paradoxically, the effect may be even worse with expert clinicians who may not always be able to explain their decision-making. Novices, who use more analytical frameworks (or even basic algorithms like, IF this, THEN that) may find it easier to modify their decisions because their reasoning is more “visible” (System 2). Experts, who rely more on subconscious pattern recognition (System 1), may be less able to identify where in their reasoning process they were victim to confounders like confirmation or availability bia, and so less likely to modify their decisions.

It seems really clear that we need to start thinking about how we’re going to prepare current and future clinicians for the arrival of intelligent agents in the clinical context. If we start disregarding the recommendations of clinical decision support systems, not because they produce errors in judgement but because we simply don’t like them, then there’s a strong case to be made that it is the human that we cannot trust.


Contrast this with automation bias, which is the tendency to give more credence to decisions made by machines because of a misplaced notion that algorithms are simply more trustworthy than people.

The Future of Artificial Intelligence Depends on Trust

To open up the AI black box and facilitate trust, companies must develop AI systems that perform reliably — that is, make correct decisions — time after time. The machine-learning models on which the systems are based must also be transparent, explainable, and able to achieve repeatable results.

Source: Rao, A. & Cameron, E. (2018). The Future of Artificial Intelligence Depends on Trust.

It still bothers me that we insist on explainability for AI systems while we’re quite happy for the decisions of clinicians to remain opaque, inaccurate, and unreliable. We need to move past the idea that there’s anything special about human intuition and that algorithms must satisfy a set of criteria that we would never dream of applying to ourselves.

Separating the Art of Medicine from Artificial Intelligence

Writing a radiology report is an extreme form of data compression — you are converting around 2 megabytes of data into a few bytes, in effect performing lossy compression with a huge compressive ratio.

Source: Separating the Art of Medicine from Artificial Intelligence

For me, there were a few useful takeaways from this article. The first is that data analysis and interpretation is a data compression problem.  The trick is to find a balance between throwing out information that isn’t useful and maintaining the relevant message during the processing. Consider the patient interview, where you take 15-20 minutes of audio data (about  10-15 MB using mp3 compression) and convert it to about a page of text (a few kilobytes at most). The subjective decisions we make about what information to discard and what to highlight have a real impact on our final conclusions and management plans.

Human radiologists are so bad interpreting chest X-rays and/or agreeing what findings they can see, that the ‘report’ that comes with the digital image is often either entirely wrong, partially wrong, or omits information.

This is not just a problem in radiology. I haven’t looked for any evidence of this but from personal experience I have little doubt that the inter and intra-rater reliability of physiotherapy assessment is similarly low. And even in cases where the diagnosis and interventions are the same, there would likely be a lot of variation in the description and formulation of the report. And this links to the last thing that I found thought-provoking:

…chest X-ray reports were never intended to be used for the development of radiology artificial intelligence. They were only ever supposed to be an opinion, an interpretation, a creative educated guess…A chest X-ray is neither the final diagnostic test nor the first, it is just one part of a suite of diagnostic steps in order to get to a clinical end-point.

We’re using unstructured medical data captured in a variety of contexts, to train AI-based systems but the data were never obtained, captured or stored in a system that was designed for that purpose. The implication is that the data we’re using to train medical AI simply isn’t fit for purpose. As long as we don’t collect the metadata (i.e. the contextual information “around” a condition), and continue using poorly labeled information and non-standardised language, we’re going to have problems with training machine learning algorithms. If we want AI-based systems to be anything more than basic triage then these are important problems to address.

We Need Transparency in Algorithms, But Too Much Can Backfire

The students had also been asked what grade they thought they would get, and it turned out that levels of trust in those students whose actual grades hit or exceeded that estimate were unaffected by transparency. But people whose expectations were violated – students who received lower scores than they expected – trusted the algorithm more when they got more of an explanation of how it worked. This was interesting for two reasons: it confirmed a human tendency to apply greater scrutiny to information when expectations are violated. And it showed that the distrust that might accompany negative or disappointing results can be alleviated if people believe that the underlying process is fair.

Source: We Need Transparency in Algorithms, But Too Much Can Backfire

This article uses the example of algorithmic grading of student work to discuss issues of trust and transparency. One of the findings I thought was a useful takeaway in this context is that full transparency may not be the goal, but that we should rather aim for medium transparency and only in situations where students’ expectations are not met. For example, a student who’s grade was lower than expected might need to be told something about how it was calculated. But when they got too much information it eroded trust in the algorithm completely. When students got the grade they expected then no transparency was needed at all i.e. they didn’t care how the grade was calculated.

For developers of algorithms, the article also provides a short summary of what explainable AI might look like. For example, without exposing the underlying source code, which in many cases is proprietary and holds commercial value for the company, explainable AI might simply identify the relationships between inputs and outcomes, highlight possible biases, and provide guidance that may help to address potential problems in the algorithm.

HELTASA conference, 2011 – day 3

Today was the last day of the HELTASA 2011 conference. It was a challenging and stimulating exchange of ideas that I really enjoyed. Thank you to everyone who was there and who I learned from.

 

Crossing (some) traditional borders
Prof Delia Marshall

There needs to be wider social, historical, ethical and environmental dimensions of science

Students need to graduate not only with domain expertise but with broader attributes that contribute towards the public good

Start with “modern” physics, with an emphasis on new ideas and concepts, rather than an equations

Resist scientism: science as “a” way of knowing about the world, not the only way

Draw in wider cultural dimensions and interests e.g. students who play instruments come in when there is a discussion on vibration / sound, etc.

“Border crossing” inot the sub-culture of science

Learning as a process of identity formation through accessing a disciplinary discourse

Looking at interactive engagement in classroom communities e.g. SCALE-UP classrooms using “lec-torials” → short lecturer inputs, working in groups, extensive and immediate feedback, learning happens in class, you can’t pass by borrowing notes

University needs to be a place for the “difficult dialogues”

Conceptualise academic literacy, not as skills, but as the social practices of discipline communities

If learning is social, then commitment from the whole department is vital. You can’t have a marginalised programme within the department

 

Len Steenkamp
Students appreciate honesty from teachers, especially when we say “I don’t know, let’s find out together”

Teachers need to be compassionate, every student has their own story
Teachers need to be humble
Teachers need to change, but not for it’s own sake, must be driven by a need

Engage in research because you want answers, not because you have to

Be generous with your time

21st century teaching tales
Liezel Nel

Electronic worksheets before attending class, must answer questions to familiarise students with content, class is used for discussion, not covering content. Worksheets also used for self-assessment (what tools?)

Students can practice using the tools in a non-assessed environment, tools introduced gradually

Uses reflective activities mid-semester and end-of-semester, much more useful than official course evaluations at end of year

Lecture recordings in audio and video, posted afterwards, useful for students with language difficulties

Students submit digital assignments, feedback in same format

Uses SMS for regular communication with students, establish a sense of caring and trust (community) “I felt a little bit special”

Glogs: online, interactive posters, students add interactive elements to their posters (glogster

Scholarship not just about publications

 

Digital storytelling and reflection in higher education: a case of pre-service student teachers and a University of Technology
Eunice Ivala, Daniela Gachago, Janet Condy, Agnes Chigona

There is a focus on passing exams, rather than on the learning process
No research on digital storytelling in higher education in South Africa, as well as limited evidence that reflective opportunities are effective

Digital story: short, 5 minute first person video-narrative, created by combining voice, still and moving images, and music or other audio

Project took place over 8 weeks, with the intention of reflecting-on-action on 7 roles of a teacher, a seed story was created to demonstrate to students, had to be 500 words

Students could choose a paper-based portfolio, or to use the digital story (half chose either one → students made their own choices)

Students had to be shown how to write their stories, learn how to find relevant images or music. Some students asked colleagues to sing for them and recorded their own music, and also took their own pictures. Also needed training in digital manipulation tools.

Used Strampel and Oliver (2007) to determine levels of reflection and stages of cognitive processing

Structuraction theory (Giddens, 1984): material resources influence social practices through their incorporation

“I’ve always known what the 7 roles were but I didn’t know what they meant and what they meant to me, but now, after incorporating it into my story, I kind of understand what they are about” (paraphrased quote from student)

The Structure of digital storytelling enabled the Agents (students), whereas before students were not enabled

“Paper-based reflections lose the personality along the way. You lose the effect of you wating to show somebody what this reflection really means. In a digital story you get the tone and atmosphere across with your own voice”

Students reflected at descriptive, dialogic and critical levels (not all students though, some only at a descriptive level)

“We can use these stories for our future employers…this is who I am, this is what I am about”

Question: why did some students not reflect at the higher cognitive levels?

Focus should be on the content of the story, not the technology because technology does nothing, except as implicated in the actions of human beings (Giddens and Pierson, 1982:82)

 

Improving teaching and learning in higher education through practitioner self-enquiry action research (action research for professional development)
Kathleen Pithouse-Morgan, Mark Schofield, Lesley Wood, Omar Esau, Joan Conolly (panel discussion)

An approach to action research in which the object of the study is the self

Trust is important for encouraging “nervous and novice” researchers (“The speed of trust” – Stephen Covey)

Integrity and honesty builds trust

Courage and generosity (“Courage to teach” – Palmer Parker)
See also, Jack Whitehead (actionresearch.net)

Recognise the unique and situated nature of the novice researcher

Learning through direct experience is more valuable than being told about something

Emphasis on “critical friendship” as part of validation

Be more understanding of the “lived experiences” of others

“People get smarter by having conversations with people who are smart”

Action research is a paradigm i.e. more than a research method

In South Africa we need critical, emancipatory paradigms that promote social change and uphold the values of the constitution

There is a lack of participatory, learner-centred pedagogies

Action research gives rise to dynamic, personal and life changing theories that operationalise the values of inclusion, people-centredness, democracy, social justice, compassion, respect. It is critical, evaluative, participatory and collaborative. It holds people to be accountable, self-evaluative and focuses on lifelong learning.

It is difficult to validate action research i.e. it must be trustworthy

Action research has the potential to minimise the hard borders between curriculum design and its delivery. The academic operates simultaneously as a researcher, designer, practitioner, and evaluator, while following an iterative and systematic process that leads to continual improvement in the curriculum, as well as teaching and learning practices.

Finding a balance between support and challenge