SAAHE – short oral presentations

Assessment challenges in UG medical education (GG Mokane)

Medical school in Botswana is spiral, integrated, community based and problem-based, but the rest of the university is didactic

Format, content, timing and feedback are important components of assessment

Assessment in this course has an emphasis on 3 types of MCQ’s

  1. Matching
  2. Single best answer
  3. True/false (multiple answer) – study was based on evaluating this specific format

How should these questions be used, and what instructions issued when they are?

Retrospective analysis of students performance in cumulative and non-cumulative formative assessment methods (AA Adebesin)

If students consistently score above 60%, they are exempt from the final summative exam (university rule). This had implicit problems in that students couldn’t graduate with distinction because they scored high enough to not write the summative exam.

Introduced a cumulative assessment process that carried formative assessment marks over from block to block

How do you objectively measure student progress and understanding?

A student portfolio: the golden key to reflective, experiential and evidence-based learning (G Muubuke)

Portfolios are useful evidence of learning and reflective processes

Logbooks are not good indicators of learning

Portfolio content included bio-data, radiological images, critical learning incident, clinical evaluation forms, logbook – with guiding questions to assist reflection

Portfolio assessed formatively and summatively

Found initially that students and teachers had only limited knowledge of portfolios, although training workshops helped in this regard

Stakeholders welcomed the introduction of the tool

Assessment whittled down to 2 items, rather than whole portfolio (1 item selected by student, the other by the teacher)

Students learn and develop by reflecting on experiences

Unfair to judge learning based only on exam marks

Students should see portfolio management as on ongoing practice, and not just a “task” to be completed

The purpose of the portfolio must be defined at the outset (i.e. what is the benefit to the student?), and it should be simple to complete, students should not see it as additional work

It should be aligned with institutional goals and learning activities

There’s a lot of effort and time involved in assessing portfolios, and rubrics may help to assist marking (adds standardisation)

Making assessment matter: does a novel model of the pre-assessment effects of summative assessment on learning also operate in clinical contexts? (F Cilliers)

There is little evidence of what the impact of assessment is on learning, as well as the mechanism of the impact

Validating a model by looking at the following 4 factors:

  • Explanatory power
  • Generalisability
  • Integration
  • Utility

Daily exposure to consequences leads to evenly distributed learning in clinical settings, but in theory modules, periodic assessment would lead to “binge learning”. However, the more relaxed nature of the clinical (evenly distributed) model might actually lead to the binge-type learning model of theory blocks.

Relaxed environments allow students to go and follow up on work after the situation, but stressful environments force students to memorise content that they forget immediately afterwards

High risk environments lead to surface cognitive processing strategies, as opposed to supportive and low risk environments leading to deeper cognitive processing

The model is useful for explaining behaviour, is generalisable, and is integrated. Not able to determine if it is useful yet

It’s about personal and academic consequences (and their imminence), not just the act of assessment. When block marks are given to students at the end of a block, that were relevant to a situation that occurred during the block, students are less likely to pay attention to the feedback (in whatever form it takes). Consequences should be immediate and not scary.

Assessors can have a powerful (and potentially negative) influence on learning

Students study more for stressful situations, but they remember less. They study less for relaxed environments, but are more likely to follow up on the situations and remember more