Assessment challenges in UG medical education (GG Mokane)
Medical school in Botswana is spiral, integrated, community based and problem-based, but the rest of the university is didactic
Format, content, timing and feedback are important components of assessment
Assessment in this course has an emphasis on 3 types of MCQ’s
- Matching
- Single best answer
- True/false (multiple answer) – study was based on evaluating this specific format
How should these questions be used, and what instructions issued when they are?
Retrospective analysis of students performance in cumulative and non-cumulative formative assessment methods (AA Adebesin)
If students consistently score above 60%, they are exempt from the final summative exam (university rule). This had implicit problems in that students couldn’t graduate with distinction because they scored high enough to not write the summative exam.
Introduced a cumulative assessment process that carried formative assessment marks over from block to block
How do you objectively measure student progress and understanding?
A student portfolio: the golden key to reflective, experiential and evidence-based learning (G Muubuke)
Portfolios are useful evidence of learning and reflective processes
Logbooks are not good indicators of learning
Portfolio content included bio-data, radiological images, critical learning incident, clinical evaluation forms, logbook – with guiding questions to assist reflection
Portfolio assessed formatively and summatively
Found initially that students and teachers had only limited knowledge of portfolios, although training workshops helped in this regard
Stakeholders welcomed the introduction of the tool
Assessment whittled down to 2 items, rather than whole portfolio (1 item selected by student, the other by the teacher)
Students learn and develop by reflecting on experiences
Unfair to judge learning based only on exam marks
Students should see portfolio management as on ongoing practice, and not just a “task” to be completed
The purpose of the portfolio must be defined at the outset (i.e. what is the benefit to the student?), and it should be simple to complete, students should not see it as additional work
It should be aligned with institutional goals and learning activities
There’s a lot of effort and time involved in assessing portfolios, and rubrics may help to assist marking (adds standardisation)
Making assessment matter: does a novel model of the pre-assessment effects of summative assessment on learning also operate in clinical contexts? (F Cilliers)
There is little evidence of what the impact of assessment is on learning, as well as the mechanism of the impact
Validating a model by looking at the following 4 factors:
- Explanatory power
- Generalisability
- Integration
- Utility
Daily exposure to consequences leads to evenly distributed learning in clinical settings, but in theory modules, periodic assessment would lead to “binge learning”. However, the more relaxed nature of the clinical (evenly distributed) model might actually lead to the binge-type learning model of theory blocks.
Relaxed environments allow students to go and follow up on work after the situation, but stressful environments force students to memorise content that they forget immediately afterwards
High risk environments lead to surface cognitive processing strategies, as opposed to supportive and low risk environments leading to deeper cognitive processing
The model is useful for explaining behaviour, is generalisable, and is integrated. Not able to determine if it is useful yet
It’s about personal and academic consequences (and their imminence), not just the act of assessment. When block marks are given to students at the end of a block, that were relevant to a situation that occurred during the block, students are less likely to pay attention to the feedback (in whatever form it takes). Consequences should be immediate and not scary.
Assessors can have a powerful (and potentially negative) influence on learning
Students study more for stressful situations, but they remember less. They study less for relaxed environments, but are more likely to follow up on the situations and remember more