AI at Google: Our principles

  1. Be socially beneficial
  2. Avoid creating or reinforcing unfair bias
  3. Be built and tested for safety
  4. Be accountable to people
  5. Incorporate privacy design principles
  6. Uphold high standards of scientific excellence
  7. Be made available for uses that accord with these principles

Source: AI at Google: Our principles

This list isn’t a bad start if you’re looking for guidance when it comes to AI systems development, and it’s a pretty good substitute for what is currently lacking in the development of healthcare AI. For example, you could easily map these principles onto the Principle ethics (beneficence, non-maleficence, justice, autonomy), which many consider to be the cornerstone of professional ethical practice.

Note: You could argue that this is a self-serving list, published to support Google’s position as a company committed to doing the Right Thing (since “Don’t be evil” was removed from their code of conduct). However, Google’s recent decision not to renew a lucrative contract with the Pentagon says a lot about their willingness to at least try and uphold their position. Regardless, even taking the list at face value is a useful approach to thinking about how to develop AI-based systems.

Psychology’s top 20 principles for enhancing teaching and learning

Every once in a while an article is published that you know is Important and that you should take Note of, and in this post I’m going to summarise a paper that I think fits into that category. It’s a recent publication in Mind, Brain and Education that attempts to summarise and explain the Top 20 principles of teaching and learning, as determined by the last few decades of psychological research. The article is called Science Supports Education: The behavioural research base for Psychology’s top 20 principles for enhancing Teaching and Learning, and it’s by Lucariello, Nastasi, Anderman, Dwyer, Ormiston, and Skiba. See the bottom of this post for the abstract and citation information.

After a brief introduction and description of the Methods the article gets stuck into the principles, which I’ll list and describe below. For some reason, Principle 8 – on the development of student creativity – is not included in the paper and no explanation is given for the omission.


Principles 1-8: How do students learn?

1. Students’ beliefs or perceptions about intelligence and ability affect their cognitive functioning and learning: If students believe that intelligence has a fixed value, they are less likely to learn than if they believe that intelligence can be changed. Teachers should communicate to students that “…failure at a task is not due to lack of ability and that performance can be enhanced, particularly with added effort or through the use of different strategies.”

2. What students already know affects their learning: Students prior knowledge influences how they incorporate new ideas because what they already know interacts with the new material being learned. This is an especially important concept when considering students’ misconceptions and how those misconceptions impede new learning. Teachers could create tasks that give students an active role in confronting and then reducing their cognitive dissonance.

3. Students’ cognitive development and learning is not limited by general stages of development: Cognitive growth is uneven and not linked to stages. Therefore, teachers’ ideas around how, and what new material should be presented, are more effective when they can take into consideration the domain-relevant and contextual knowledge of their students.

4. Learning is based on context, so generalizing learning to new contexts is not spontaneous, but rather needs to be facilitated: In order for learning to be effective, it should generalise to new or different contexts and situations. However, student transfer of knowledge and skills is not spontaneous or automatic. Teachers could therefore teach concepts in multiple contexts so that students can recognise contextual similarities, and focus on the application of their knowledge to the real world.

5. Acquiring long-term knowledge and skill is largely dependent on practice: What people know is laid down in long-term memory and information must be processed before it can move from short-term to long-term memory. This processing is accomplished through different strategies, and practice is key. Teachers should consider a variety of frequent assessment tasks given at spaced intervals (distributive practice). In addition, interleaved practice (a schedule of repeated opportunities) to rehearse and transfer skills or content by practicing with tasks that are similar to the target task, or using several methods for the same task, is also recommended.

6. Clear, explanatory, and timely feedback to students is important for learning: Students should receive regular, specific, explanatory, and timely feedback on their work. Feedback is more effective when it includes specific information that is linked to current knowledge and performance to clear learning goals. Teachers should consider providing feedback on assessment tasks – particularly after incorrect responses – in order to improve classroom performance in the future.

7. Students’ self-regulation assists learning and self-regulatory skills can be taught: Self-regulatory skills include setting goals for learning; such as planning, and monitoring progress; and self-reflection, which consists of making judgements about performance and self-efficacy in reaching goals. Self-regulatory skills include the regulation of motivation, which consists of students’ knowledge, monitoring, and active management of their motivation or motivational processing. Teachers can teach these skills directly to learners, by modelling strategies or coaching on their effectiveness. Teachers can also provide opportunities for learners to set goals and manage their attainment and for self-appraisal. A reflective community also can be established by teachers.

8. Missing from this paper

Principles 9-12: What motivates students?

9. Students tend to enjoy learning and perform better when they are more intrinsically than extrinsically motivated: Learners who are intrinsically motivated engage in academic tasks for the pure enjoyment of such engagement, and are more likely to achieve at higher levels and to continue engaging with activities in the future. Intrinsic motivation is linked to effective learning because students persist longer at tasks, experience lower levels of anxiety and develop positive competence beliefs. Learners who are extrinsically motivated engage in tasks in order to receive a reward or avoid a punishment, and are at risk for a number of problematic long term outcomes. Teachers can facilitate intrinsic motivation by de-emphasising high-stakes assessment, by allowing students to engage in projects they are interested in, encouraging students to take academic risks and by ensuring that students have enough time to engage with tasks.

10: Students persist in the face of challenging tasks and process information more deeply when they adopt mastery goals rather than performance goals: When teachers emphasise test scores, ability differences, and competition, students are more likely to adopt performance goals. Moreover, when test scores and grades are presented publicly, students are encouraged to focus on performance goals. In contrast, when teachers emphasise effort, self-improvement, and taking on challenges, students are more likely to adopt mastery goals. At the same time, they are likely to use effective and more complex cognitive strategies, to persist at challenging tasks, to report being intrinsically motivated, and to report feeling efficacious. Mastery goals are therefore more likely to be adopted when grades and test scores are shared privately and not compared across individuals.

11. Teachers’ expectations about their students affect students’ opportunities to learn, their motivation, and their learning outcomes: In classroom settings, teachers’ expectations for students’ successes and failures influence student achievement and motivation. When educators hold high expectations for their students, they often rise to the occasion and achieve at high levels (provided that the necessary support structures are in place). In contrast, when teachers hold low expectations for student success, students may come to believe that they lack skills and abilities, and thus confirm the teachers’ expectations. It is important to understand that teachers may interact differently with students, and provide differential instruction, based on their expectations for each student’s success or failure, regardless of how accurate those expectations are.

12. Setting goals that are short term (proximal), specific, and moderately challenging enhances motivation more than establishing goals that are long term (distal), general, and overly challenging: Goal setting is the process by which an individual sets a standard of performance and is important for motivation because students with a goal and adequate self-efficacy are likely to engage in the activities that lead to achievement of that goal. Three properties of goal setting are important for motivation. First, short-term goals are more motivating than long-term goals because it is easier to assess progress toward short-term goals. Students tend to be less adept at thinking concretely with respect to the distant future. Second, specific goals are preferable to more general goals because it is easier to quantify and monitor specific goals. Third, moderately difficult goals are the most likely to motivate students because they will be perceived as challenging but also attainable.

Principles 13–15: Why are social context, interpersonal relationships, and emotional well-being important to student learning?

13. Learning is situated within multiple social contexts.

14. Interpersonal relationships and interpersonal communication are critical to both the teaching–learning process and the social–emotional development of students.

15. Emotional well-being influences educational performance, learning, and development.

These principles are interrelated and are represented in theory and research relevant to schools as systems that support psychological (social and emotional) well-being as well as cognitive development and academic learning. According to developmental–ecological theory, the child or learner is best viewed as embedded within multiple social contexts or ecosystems (e.g., school, family, neighbourhood, peer group), that influence learning:

  • Microsystem: student-teacher and student-student interactions influence learning
  • Ecosystem: microsystem interactions occur within a school where policies and norms (teaching and learning practices and organisational structure) influence learning
  • Macrosystem: ecosystems interact (e.g. school and families) within a society which reflects culture, values and norms

These interactions within and between systems influence students’ learning significantly, and are documented more extensively in the article (pg. 61-62).

Principles 16–17. How can the classroom best be managed?

16. Expectations for classroom conduct and social interaction are learned and can be taught using proven principles of behaviour and effective classroom instruction.

17. Effective classroom management is based on (1) setting and communicating high expectations, (2) consistently nurturing positive relationships, and (3) providing a high level of student support.

Classroom management is a fundamental, bedrock set of
procedures and skills that establish a climate for instruction and learning. Class and school rules must be positively stated, concrete, observable, posted, explicitly taught, frequently reviewed, and positively reinforced. This allows students to learn the social curriculum in each classroom and enables teachers to develop classroom climates that maximise student engagement and minimises conflict and disruption.

Classrooms that are structured to offer multiple opportunities for students to respond facilitate the development of quality teacher–student relationships, which in turn lead to fewer behavioural problems and increased academic performance. Students who are at risk for classroom disruption may need more attention to relationship-building in order to develop and maintain connections in the classroom.

Culturally responsive classroom management is an approach that aims to actively engage students by offering a curriculum that is relevant to their lives. Teachers demonstrate a willingness to learn about important aspects of their students’ lives and create a physical environment that is reflective of students’ cultural heritage. Culturally responsive teachers understand the ways in which schools reflect and perpetuate discriminatory practices of the larger society and are characterised as “warm demanders”; “strong yet compassionate, authoritative yet loving, firm yet respectful”.

Finally, a high ratio of positive statements / rewards to negative consequences, and nurturing an atmosphere of respect for all students and their heritage, builds trust in the classroom that can prevent behavioural conflict.

Principles 18–20: how to assess student progress?

18. Formative and summative assessments are both useful, but they require different approaches: Formative assessments are carried out during instruction and are aimed at improving learning in the classroom setting. Summative assessments measure learning at a given point in time, usually at the end of some period of instruction where they are used to provide a judgement about student learning. The goal of both formative and summative assessments is to produce valid, fair, useful, and reliable information for decision making. Teachers can also use their understanding of assessment information to decide whether they covered the material that they intended to cover, or to judge how effectively they met the objectives for student learning.

19. Students’ skill and knowledge should be assessed with processes that are grounded in psychological science and that have provided well-defined standards for quality and fairness: Valid and reliable assessments enable teachers to make inferences about what students are learning. To understand the validity of an assessment, there are four question that need to be considered:

  1. How much of what you intended to measure is actually being measured?
  2. How much of what you did not intend to measure actually ended up being measured?
  3. What consequences, either intended or unintended, occurred with the assessment?
  4. Do you have solid evidence to support your answers to the first three questions?

Validity is a judgement, over time and across a variety of situations, about what inferences can be drawn from the test data, and the consequences of using the test. Valid assessment entails specifying what an assessment is supposed to measure. Teachers can improve assessment quality by aligning teaching and testing. However, they should also:

  • Be mindful that valid tests in one context may not be valid for another
  • Ensure that high-stakes decisions be based on multiple measures, not on a single test
  • Examine outcomes for any discrepancies in performance among different cultural groups

20. Good use of assessment data depends on clear, appropriate, and fair interpretation: Effective teaching depends heavily on teachers being informed consumers of educational research, effective interpreters of data for classroom use, and good communicators to students and their families about assessment data and decisions that affect them. The interpretation of assessments involves addressing the following questions:

  1. What was the assessment intended to measure?
  2. On what are comparisons of the assessment data based? Are students being compared to one another? Or, are responses being directly compared to samples of acceptable and unacceptable responses?
  3. Are scores being classified using a standard or cut point, such as letter grades, or another indicator of satisfactory/unsatisfactory performance?How were these standards set?

Awareness of the strengths and limitations of any assessment is critical. Such awareness enables teachers to make others aware of important caveats, such as the imperfect reliability of scores and the importance of using multiple sources of evidence for high-stakes decisions.


And there you have it. Twenty principles (19 without the one on fostering student creativity) on how best to go about enhancing teaching and learning practices in the classroom. While I don’t think it’s feasible to try and incorporate all of these principles in every classroom session, it’s definitely worthwhile having these at the back of your mind when planning assessment tasks, assignments, lectures and activities in class. I also recommend reading the whole paper which provides additional insight and links to further reading that would be useful to dig into.

Abstract

Psychological science has much to contribute to preK-12 education because substantial psychological research exists on the processes of learning, teaching, motivation, classroom management, social interaction, communication, and assessment. This article details the psychological science that led to the identification, by the American Psychological Association’s Coalition for Psychology in Schools and Education, of the “Top 20 Principles from Psychology for PreK-12 Teaching and Learning.” Also noted are the major implications for educational practice that follow from the principles.

Citation: Lucariello, J. M., Nastasi, B. K., Anderman, E. M., Dwyer, C., Ormiston, H., & Skiba, R. (2016). Science Supports Education: The behavioural research base for Psychology’s top 20 principles for enhancing Teaching and Learning. Mind, Brain and Education, 10(1), 55–67.

I enjoyed reading (June)

reading mountainThe internet: Everything you ever need to know (John Naughton):

A funny thing happened to us on the way to the future. The internet went from being something exotic to being boring utility, like mains electricity or running water – and we never really noticed. So we wound up being totally dependent on a system about which we are terminally incurious. The internet has quietly infiltrated our lives, and yet we seem to be remarkably unreflective about it.

We really need to pay more attention to the internet and the impact it’s having on our lives. It’s not just another way to pay your bills, it’s a profoundly important . Related: Time to start taking the internet seriously (David Gelernter).

 

The coming war on general purpose computing (Cory Doctorow):

…it doesn’t take a science fiction writer to understand why regulators might be nervous about the user-modifiable firmware on self-driving cars, or limiting interoperability for aviation controllers, or the kind of thing you could do with bio-scale assemblers and sequencers…Regardless of whether you think these are real problems or merely hysterical fears, they are nevertheless the province of lobbies and interest groups that are far more influential than Hollywood and big content are on their best day, and every one of them will arrive at the same place — “can’t you just make us a general purpose computer that runs all the programs, except for the ones that scare and anger us? Can’t you just make us an Internet that transmits any message over any protocol between any two points, unless it upsets us?”

Here is a transcript of Cory’s presentation.

Bloom and bust (Steve Wheeler): Bloom’s taxonomy was my first introduction to the idea that teaching and learning could be better understood through the use of frameworks and theories. For that reason, it has a place in my heart but I’ve since come around to the fact that it is too simplistic and structured to really represent any form of meaningful learning.

And yet Bloom’s taxonomy raised some serious issues. How relevant is it in the digital age? Should we still be organising learning experiences as a gradient of ‘terminal learning objectives’ in an age where learning is changing, and where personal technologies and social media are increasingly significant? Learning is changing, because the boundaries between discrete learning activities are blurring.

Creating classrooms we need: 8 ways into Inquiry Learning

Teachers teach kids, not subjects.

I often fall into the trap of thinking about the subjects I teach, rather than the people I teach. This reminded me of the difference.

 

What is transhumanism, or, what does it mean to be human? (Sebastian Anthony):  I’m increasingly interested in how the integration of technology into our lives seems to be a first step towards integrating it into our bodies. What happens when I can jack directly into the internet? Or store all digital texts in my augmented brain? What does it mean for testing when our students are able to do exactly that?

The uneasiness that surround new, paradigm-shifting technologies isn’t new, and it has only been amplified by the exponential acceleration of technology that has occurred during our lifetime. If you were born 500 years ago, odds are that you wouldn’t experience a single societal-shifting technology in your lifetime — today, a 40 year old will have lived through the creation of the PC, the internet, the smartphone, and brain implants, to name just a few life-changing technologies. It is unsettling, to say the least, to have the rug repeatedly pulled out from under you, especially when it’s your livelihood at stake. Just think about how many industries and jobs have been obliterated or subsumed by the arrival of the digital computer, and it’s easy to see why we’re wary of transhumanist technologies that will change the very fabric of human civilization.

Design principles for clinical reasoning

graphic_design smallerClinical reasoning is hard to do, and even harder to facilitate in novice practitioners who lack the experience and patterns of thinking that enable them to establish conceptual relationships that are often non-trivial. Experienced clinicians have developed, over many years and many patients, a set of thinking patterns that influence the clinical decisions they make, and which they are often unaware of. The development of tacit knowledge and its application in the clinical context is largely done unconsciously, which is why experienced clinicians often feel like they “just know” what to do.

Developing clinical reasoning is included as part of clinical education, yet it is usually implicit. Students are expected to “do” clinical reasoning, yet we find it difficult to explain just what we mean by that. How do you model a way of thinking?

One of the starting points is to ask what we mean when we talk about clinical education. Traditionally, clinical education describes the teaching and learning experiences that happen in a clinical context, maybe a hospital, outpatient or clinic setting. However, if we redefine “clinical education” to mean activities that stimulate the patterns of thinking needed to think and behave in the real world, then “clinical education” is something that can happen anywhere, at any time.

My PhD was about exploring the possibilities for change that are made available through the integration of technology into clinical education. The main outcome of the project was the development of a set of draft design principles that emerged through a series of research projects that included students, clinicians and clinical educators. These principles can be used to design online and physical learning spaces that create opportunities for students to develop critical thinking as part of clinical reasoning. Each top-level principle is associated with a number of “facets” that further describe the application of the principles.

Here are the draft design principles (note that the supporting evidence and additional discussion are not included here):

1. Facilitate interaction through enhanced communication

  • Interaction can be between people and content
  • Communication is iterative and aims to improve understanding through structured dialogue that is conducted over time
  • Digital content is not inert, and can transform interactions by responding and changing over time
  • Content is a framework around which a process of interaction can take place – it is a means to an end, not an end in itself
  • When content is distributed over networks, the “learning environment” becomes all possible spaces where learning can happen
  • Interaction is possible in a range of contexts, and not exclusively during scheduled times

2. Require articulation

  • Articulation gives form and substance to abstract ideas, thereby exposing understanding
  • Articulation is about committing to a statement based on personal experience, that is supported by evidence
  • Articulation is public, making students accountable for what they believe
  • Articulation allows students’ thinking to be challenged or reinforced
  • Incomplete understanding is not a point of failure, but a normal part of moving towards understanding

3. Build relationships

  • Knowledge can be developed through the interaction between people, content and objects, through networks
  • Relationships can be built around collaborative activity where the responsibility for learning is shared
  • Facilitators are part of the process, and students are partners in teaching and learning
  • Facilitators are not gatekeepers – they are locksmiths
  • Create a safe space where “not knowing” is as important as “knowing”
  • Teaching and learning is a dynamic, symbiotic relationship between people
  • Building relationships takes into account both personal and professional development
  • Building relationships means balancing out power so that students also have a say in when and how learning happens

4. Embrace complexity

  • Develop learning spaces that are more, not less, complex
  • Change variables within the learning space, to replicate the dynamic context of the real world
  • Create problems that have poorly defined boundaries and which defy simple solutions

5. Encourage creativity

  • Students must identify gaps in their own understanding, and engage in a process of knowledge creation to fill those gaps
  • These products of learning are created through an iterative activity that includes interaction through discussion and feedback
  • Learning materials created should be shared with others throughout the process, to enable interaction around both process and product
  • Processes of content development should be structured according to the ability of the students

6. Stimulate reflection

  • Learning activities should have reflection built in
  • Completing the reflection should have a real consequence for the student
  • Reflection should be modelled for students
  • Reflections should be shared with others
  • Feedback on reflections should be provided as soon after the experience as possible
  • Students need to determine the value of reflection for themselves, it cannot be told to them

7. Acknowledge emotion

  • Create a safe, non­judgemental space for students to share their personal experiences and thoughts, as well as their emotional responses to those experiences
  • Facilitators should validate students’ emotional responses
  • These shared experiences can inform valuable teaching moments
  • Facilitators are encouraged to share personal values and their own emotional responses to clinical encounters, normalising and scaffolding the process
  • Sensitive topics should be covered in face­to­face sessions
  • Facilitators’ emotional responses to teaching and learning should be acknowledged, as well their emotional responses to the clinical context

8. Flexibility

  • The learning environment should be flexible enough to adapt to the changing needs of students, but structured enough to scaffold their progress
  • The components of the curriculum (i.e. the teaching strategies, assessment tasks and content) should be flexible and should change when necessary
  • Facilitators should be flexible, changing schedules and approaches to better serve students’ learning

9. Immersion

  • Tasks and activities should be “cognitively real”, enabling students to immerse themselves to the extent that they think and behave as they would be expected to in the real world
  • Tasks and activities should use the “tools” of the profession to expose students to the culture of the profession
  • Technology should be transparent, adding to, and not distracting from the immersive experience

We have implemented these draft design principles as part of a blended module that made significant use of technology to fundamentally change teaching and learning practices in our physiotherapy department. We’re currently seeing very positive changes in students’ learning behaviours, and clinical reasoning while on placements, although the real benefits of this approach will only really emerge in the next year or so. I will continue to update these principles as I continue my research.

Note: The thesis is still under examination, and these design principles are still very much in draft. They have not been tested in any context other than in our department and will be undergoing refinement as I continue doing postdoctoral work in this area.

Principles of good assessment

I attended an assessment and learning workshop today and while the presentations were informative, I just wanted to highlight the principles of good assessment taken from my faculty’s assessment policy.  Since I don’t have a background in education, guidelines like these are incredibly useful when creating assessments for students.

  1. Responsibility for assessment – the module co-ordinator is responsible for designing the assessment and mark allocation.
  2. Assessing against outcomes – performance should be measured against pre-determined expectations of achievement (learning outcomes).
  3. Assessment criteria – the expectations of the assessment, including the specific criteria of judgement, should be available to students to ensure transparency.
  4. Validity and appropriateness – the assessment methods and tasks should accurately match what is being assessed (knowledge, understanding, content, skills, behaviour, etc.)
  5. Authenticity of evidence – measures should be taken to ensure that the evidence produced by the student is attributable to the student.  With group work, the lecturer must verify that each student has made a fair contribution.
  6. Formative and summative assessment – assessment should judge students’ performance (summative), as well as provide feedback to enhance learning (formative), although not simultaneously.  Students should be aware of whether they are being assessed formatively or summatively at each assessment.
  7. Continuous assessment – should have a strong formative focus and be undertaken over the course of the module.

For me, just knowing about these guidelines has already made a significant difference in how I approach the assessment of students.  Clearly, it’s not enough to re-use old test papers and merely change the scenarios.  We need to make sure that we’re actually testing what we set out to test, as well as linking the assessment to the curriculum and learning outcomes.

One other point I want to mention is a comment made by one of the presenters, regarding the importance of testing students interpretation of the course content.  This is one way to make sure that students actually understand what they’re writing, rather than just regurgitating bullet points.