Today was the last day of the SAAHE conference. Coming as it did immediately after a week of the SAFRI programme, I can’t say I’m not glad it’s over. It’s been an amazing experience though, mostly as a result of the wonderful health educators I’ve been fortunate enough to meet. Here are the last of the notes I took during the conference.
Simulation-based asessment: challenges and opportunities by Jack Boulet
Simulations used for summative and formative assessment, as well as curriculum assessment and patient safety
Need to know what health practitioners actually do, (i.e. procedurally), as opposed to what they know
Simulated (standardised) patients are good for some things but not others e.g. trauma
Performance measures:
- Link measures to scenario events
- Focus on observable behaviour
- Incorporate multiple measures from different sources
Types of scores:
- Explicit process
- Implicit process
- Explicit outcome
Checklist for assessing acute scare skills
- Certain actions are more important than others
- Sequence and timing are important
Checklists reward thoroughness
Training and quality assurance are important when it comes to assessment and ratings
Developing reliable and valid scoring systems is difficult
Important to identify and minimise errors of measurement
Peer review is essentially about getting a high number of opinions that over time will average out to be an accurate measure
Cases (simluations) are “vehicles” to measure skills
- Who are the target examinees?
- Specificity
- Difficulty
- Essential manoeuvres and questions?
- Sampling from a domain (identify the domain)
“Predictive validity” – Even with simulation, it’s difficult to establish predictive value → performance in the real world
Challenges:
- Cost
- Logistics
- Setting standards
- Interdisciplinary skills (e.g. measure doctor-patient interaction but not doctor-nurse interaction)
- Integration
What is the societal cost of having providers with inadequate knowledge and skills?
It’s more interesting to measure how people lose skills / competence over time, than to measure how they acquire skills
Inferences concering competence are dependent on linking scores to performance criteria
How can we best use technology as part of current clinical / educational efforts?
- Electronic portfolios
- Online testing
- Combined methods
Use of advanced technology to increase fidelity e.g. virtual reality, haptic systems
Students change behaviour when they know how they’re being assessed e.g. with checklists
Simulation studies using confederates (can this be done with students and “broken” equipment?)
Good teamwork is easier to recognise than it is to define
Making use of adult learning theories by David Taylor
Attended a workshop yesterday, which covered much of the same content
Behaviourism – consequences drive actions
“Picking up the rules of a community”, “learning the rules of the game”, “what does it mean to be?
How do you learn “enough of the words” to make sense of the discipline?
Exploration of a model based on Kolb’s learning cycle:
“Elaborate” new knowledge → consider all propositions and discard ones that are irrelevant, experts navigate this path quickly
- Work out the most likely resources to refine possibilities
- Actively participate in the activity
- Refine the information into a hypothesis
Reflecting / organising:
- Test – retest the hypothesis
- Organise information into a “story” that makes sense to them
- Teachers need to provide cognitive structures upon which students can build → scaffolding
- Encourage reflection-in-action / reflect while doing
Feedback:
- Students needs to articulate prior knowledge
- Assessment is a form of feedback
- Feedback can only be given when students have articulated / exposed their understanding
- Teachers must be open to accept (and to act on) feedback from students
Reflect / consolidate:
- Take on board the feedback
- Reflection in the light of new knowledge and the learning process
- Evaluate personal responsibility for learning
- Teacher needs to provide opportunities for the learner to rehearse / apply new knowledge i.e. encourage reflection-on-action
Dissonance:
- Using a challenge to help students make a conceptual leap by identifying / proposing an alternative concept that they had not considered
- Is dissonance a way to help students move through Vygotsky’s ZPD?
- Mezirow – “learning is a disorienting dilemma”
- Can be created by manipulating”
-
- Resources: should be appropriate, sufficient and relevant
- Motivation (Knowles):
- Intrinsic: adults learn because they need to know, have a self-concept as a learner, have life experiences, readiness to learn, orientated towards learning
- Extrinsic: programme / curriculum, community of practice
- Stage of development (Perry): Duality (“right and wrong answers and the teacher knows what’s right”) → multiplicity (“comfortable that in any given situation, there’s more than one answer, and that context is what matters most” – comfort in dealing with uncertainty)
- Style of learning (Entwistle, Biggs): strategic, deep, surface ← how do you measure which of these is happening?
CoP (Wenger):
- We don’t live or work (learn) in a vacuum
- Everyone is part of a community
- We only learn in community (does this mean that it’s impossible to learn independently?)
- We develop as part of that community
Perceptions and experiences on community engagement as part of learning Student sessions. Points below taken from a variety of student presentations
Train of hope (Phulophepa)
Service learning, research and volunteerism: providing support to about 87 organisations in the area, entirely student run → builds confidence and experience, receive certificates for work done which are valued by future employers
Move from community service to community development
External evaluation bring accountability to projects
Most common health problem encountered by medical students on community-based learning placements is drug abuse
Community dynamics:
- lack of medication
- non-compliance
- traditional healers
- religion and beliefs
- social problems
Patients’ stories are often heartbreaking
“The patient is more than the illness”
“Poor of the poorest”
Medical doctors and sangomas (traditional healers) have areas of overlapping practices and principles
“As long as patients are living in this world, they are appreciated as human beings”
Challenge students to move out of their comfort zones
Community-based projects count for a very small percentage of the coursework grade, yet it takes an enormous amount of personal commitment and time, and is supposedly valued by educators
Language is a significant problem for student-patient interaction
Students conflicted when we tell them to think outside the box, but then have expectations for them to “do it by the book”
Does the institution learn as much as the students from the community experiences? When students report back to the institution, what changes do they experience as / if the institution responds? Does the institution respond?