assessment conference education learning research students teaching

SAAHE conference, 2011 – day 3

Today was the last day of the SAAHE conference. Coming as it did immediately after a week of the SAFRI programme, I can’t say I’m not glad it’s over. It’s been an amazing experience though, mostly as a result of the wonderful health educators I’ve been fortunate enough to meet. Here are the last of the notes I took during the conference.

Simulation-based asessment: challenges and opportunities by Jack Boulet

Simulations used for summative and formative assessment, as well as curriculum assessment and patient safety

Need to know what health practitioners actually do, (i.e. procedurally), as opposed to what they know

Simulated (standardised) patients are good for some things but not others e.g. trauma

Performance measures:

  • Link measures to scenario events
  • Focus on observable behaviour
  • Incorporate multiple measures from different sources

Types of scores:

  • Explicit process
  • Implicit process
  • Explicit outcome

Checklist for assessing acute scare skills

  • Certain actions are more important than others
  • Sequence and timing are important

Checklists reward thoroughness

Training and quality assurance are important when it comes to assessment and ratings

Developing reliable and valid scoring systems is difficult

Important to identify and minimise errors of measurement

Peer review is essentially about getting a high number of opinions that over time will average out to be an accurate measure

Cases (simluations) are “vehicles” to measure skills

  • Who are the target examinees?
  • Specificity
  • Difficulty
  • Essential manoeuvres and questions?
  • Sampling from a domain (identify the domain)

Predictive validity” – Even with simulation, it’s difficult to establish predictive value → performance in the real world


  • Cost
  • Logistics
  • Setting standards
  • Interdisciplinary skills (e.g. measure doctor-patient interaction but not doctor-nurse interaction)
  • Integration

What is the societal cost of having providers with inadequate knowledge and skills?

It’s more interesting to measure how people lose skills / competence over time, than to measure how they acquire skills

Inferences concering competence are dependent on linking scores to performance criteria

How can we best use technology as part of current clinical / educational efforts?

  • Electronic portfolios
  • Online testing
  • Combined methods

Use of advanced technology to increase fidelity e.g. virtual reality, haptic systems

Students change behaviour when they know how they’re being assessed e.g. with checklists

Simulation studies using confederates (can this be done with students and “broken” equipment?)

Good teamwork is easier to recognise than it is to define

Making use of adult learning theories by David Taylor

Attended a workshop yesterday, which covered much of the same content

Behaviourism – consequences drive actions

Picking up the rules of a community”, “learning the rules of the game”, “what does it mean to be?

How do you learn “enough of the words” to make sense of the discipline?

Exploration of a model based on Kolb’s learning cycle:

Elaborate” new knowledge → consider all propositions and discard ones that are irrelevant, experts navigate this path quickly

  • Work out the most likely resources to refine possibilities
  • Actively participate in the activity
  • Refine the information into a hypothesis

Reflecting / organising:

  • Test – retest the hypothesis
  • Organise information into a “story” that makes sense to them
  • Teachers need to provide cognitive structures upon which students can build → scaffolding
  • Encourage reflection-in-action / reflect while doing


  • Students needs to articulate prior knowledge
  • Assessment is a form of feedback
  • Feedback can only be given when students have articulated / exposed their understanding
  • Teachers must be open to accept (and to act on) feedback from students

Reflect / consolidate:

  • Take on board the feedback
  • Reflection in the light of new knowledge and the learning process
  • Evaluate personal responsibility for learning
  • Teacher needs to provide opportunities for the learner to rehearse / apply new knowledge i.e. encourage reflection-on-action


  • Using a challenge to help students make a conceptual leap by identifying / proposing an alternative concept that they had not considered
  • Is dissonance a way to help students move through Vygotsky’s ZPD?
  • Mezirow – “learning is a disorienting dilemma”
  • Can be created by manipulating”
    • Resources: should be appropriate, sufficient and relevant
    • Motivation (Knowles):
      • Intrinsic: adults learn because they need to know, have a self-concept as a learner, have life experiences, readiness to learn, orientated towards learning
      • Extrinsic: programme / curriculum, community of practice
    • Stage of development (Perry): Duality (“right and wrong answers and the teacher knows what’s right”) → multiplicity (“comfortable that in any given situation, there’s more than one answer, and that context is what matters most” – comfort in dealing with uncertainty)
    • Style of learning (Entwistle, Biggs): strategic, deep, surface ← how do you measure which of these is happening?

CoP (Wenger):

  • We don’t live or work (learn) in a vacuum
  • Everyone is part of a community
  • We only learn in community (does this mean that it’s impossible to learn independently?)
  • We develop as part of that community

Perceptions and experiences on community engagement as part of learning Student sessions. Points below taken from a variety of student presentations

Train of hope (Phulophepa)

Service learning, research and volunteerism: providing support to about 87 organisations in the area, entirely student run → builds confidence and experience, receive certificates for work done which are valued by future employers

Move from community service to community development

External evaluation bring accountability to projects

Most common health problem encountered by medical students on community-based learning placements is drug abuse

Community dynamics:

  • lack of medication
  • non-compliance
  • traditional healers
  • religion and beliefs
  • social problems

Patients’ stories are often heartbreaking

The patient is more than the illness”

Poor of the poorest”

Medical doctors and sangomas (traditional healers) have areas of overlapping practices and principles

As long as patients are living in this world, they are appreciated as human beings”

Challenge students to move out of their comfort zones

Community-based projects count for a very small percentage of the coursework grade, yet it takes an enormous amount of personal commitment and time, and is supposedly valued by educators

Language is a significant problem for student-patient interaction

Students conflicted when we tell them to think outside the box, but then have expectations for them to “do it by the book”

Does the institution learn as much as the students from the community experiences? When students report back to the institution, what changes do they experience as / if the institution responds? Does the institution respond?