Categories
assessment curriculum learning physiotherapy students teaching

Workshop on Intended Learning Outcomes

I’ve already mentioned that my institution has increased the emphasis on the scholarship of teaching and learning over the past few years, by placing it on an equal footing with research. This has forced all academics to reconsider their roles within the university, which many have resisted. Luckily (for me) I work in a department where we see these changes as opportunities for growth, rather than obstacles to be baulked at. As part of our response to this challenge, we’ve been working at integrating a scholarship of teaching and learning into our everyday teaching practices, including (among many other things), regular workshops to aid with both staff development and curricular alignment.

Today we had a workshop on using learning outcomes to structure a module, with the intention that we would each take one of our modules and refine the outcomes before working on aligning teaching strategies, assessment and content. I didn’t take too many notes, as we were provided with handouts but here are the notes I did take. Nothing ground breaking if you’ve read anything about learning outcomes before but bear in mind that the emphasis of the workshop was on the application of concepts to our actual modules, as opposed to a purely learning activity.

  • It is important to expose learners’ current and prior knowledge / understanding before beginning a learning activity, and then try to build on that
  • We don’t “teach”, so much as we create an environment conducive to students’ learning
  • Learning outcomes can be useful guides / “maps” for both new and experienced staff members
  • Objectives (related to teacher) are different to outcomes (related to student)
  • Programme outcomes (might be found in a mission statement) must be linked to module outcomes
  • Outcomes must be specific
  • Use concept mapping to link outcomes, lectures and assessment
  • Students just out of high school may need to be taught how to learn
  • Get students to agree to take responsibility for their own learning, possibly in the first lecture, by making explicit what the roles / expectations are of everyone involved
  • Make a list of the things that bug you about students, and get students to agree to not do those things. Then ask them what you do that bugs them, and agree to try and avoid those too. Ask regularly, “what is working / not working for you right now?”

Benefits of learning outcomes:

  • Helps to structure the module content
  • Helps to guide assessment practices
  • Helps to direct teaching strategies
  • Provides a measure of accountability
  • Helps students to focus on what to study / learning outcomes are the “scope of the exam”

 

  • Learning activities are “supported” by assessment / assessment drives learning
  • Many students come into higher education thinking “Give me the stuff, and I’ll give it back to you”. How can we change that mindset?
  • Look at how SAQA’s level descriptors link to learning outcomes, and make them specific to module descriptors (1st year = level 5, 2nd year = level 6, 3rd year = level 7, 4th year = level 8)
  • Try to get students to see why the outcomes are important i.e. do they have personal meaning for the students, or are they just words
  • Review the lecture outcomes at the beginning and end of the lecture. Ask students if they felt that the outcomes were achieved
Categories
education health PhD teaching workshop

Developing cases for Problem-Based Learning

Workshop on the development of case-based studies

Facilitators: Dr. Ethel Stanley, Dr. Margaret Waterman

Part of my PhD will be to look at alternative approaches to clinical education, including uses cases in problem-based learning (PBL). My specific interest is in the use of emerging technology to design and teach with those cases in small groups. Unfortunately I was only able to attend the first half of the workshop, and didn’t get the opportunity to develop my own case.

Here are my notes from the workshop:

Biology is an important topic for everyone to understand, as it impacts on every major health-related decision that has to be made, so we used biological case studies as working examples

Students must be able to ask good questions in order to solve their own problems in preparation for the types of adult learning (androgogy, as opposed to pedagogy) behaviour we’d expect to see in practice. Memorising content isn’t a good strategy for learning how to solve problems like “Why is this patient walking in a way that is different from “normal”?”

A lecture is a good method to deliver content, but is a poor method for active learning around problem solving

Case-based learning (CBL) is a good way to explore realistically complex situations

Begin by introducing a problem with no expectation that the student can solve the problem. Use that as a springboard to explore their ability to develop good research questions

CBL requires the confidence from teachers to give up control, but giving up control is the only way to get students to actively construct their own learning experiences by asking questions, gathering information, testing hypotheses, and convince others of their findings

Structure for working through a basic case

  • Define the boundaries / outline of the case
  • What do you already know (group knowledge, as well as information that can be obtained from the case study) / what do you still need to know (this can be used as a basis for a short lecture) in order to answer the question
  • Choose the most important questions to explore
  • Get into small groups and discuss / share information, knowledge, assumptions
  • Go away and try to answer the questions that were generated
  • Come back and only then get the teachers objectives
  • Then go away again and refine the questions and information collected

Why use cases?

  • To initiate investigations
  • To use new technologies and resources to solve problems
  • Develop local and international / global perspectives
  • Emphasise the value of interdisciplinary and collaborative approaches
  • Structure student assessment through student products
  • Support diverse objectives within a shared workspace (would be interesting to investigate the possibility of using a wiki to develop and build on cases using this approach)

Used Gapminder to demonstrate alternative ways of visually representing data while working through a case study. See Hans Rosling (founder of Gapminder) on the Joy of Statistics, and his TED presentations.

The teacher can set the context of the class, and the depth to which students should explore questions, by using an appropriate framework / case. Can also decide which questions are prioritised, and which ones can be answered via different methods e.g. lecture, essay, assignment, etc.

Highlight the fact that, as the teacher, you don’t have all the answers and that you’re a co-learner in the classroom. Students should understand that the teacher isn’t a font of all knowledge on the subject, and that it’s acceptable and appropriate for the teacher to have to also do research on the topic

Categories
assignments conference education ethics health PhD physiotherapy research social media technology

Using social networks to develop reflective discourse in the context of clinical education

My SAFRI project for 2010 looked at the use of a social network as a platform to develop clinical and ethical reasoning skills through reflective discussion between undergraduate physiotherapy students. Part of the assignment was to prepare a poster for presentation at the SAAHE conference in Potchefstroom later this year, which I’ve included below.

I decided to use a “Facebook style” layout to illustrate the idea that research is about participating in a discussion, something that a social network user interface is particularly well-suited to. I also like to try and change perceptions around academic discourse and do things that are a little bit different. I hate the general idea that “academic” equals “boring” and think that this is such an exciting space to work in.

 

I also included a handout with additional information (including references) that I thought the audience might find interesting, but which couldn’t fit onto the poster.

One of the major challenges I experienced during this project was that I didn’t realise how much time it’d take to complete. I’d thought that the bulk of my time would be used on building and maintaining the social network and facilitating discussion within in, but the assignment design (see handout) took a lot more effort than I expected. I had to make sure that it was aligned with the module learning objectives, as well as the university graduate attributes.

In terms of moving this project forward, I think that it might be possible to use a social network as a focus for other activities that might contribute towards a more blended approach to learning and clinical education. For example:

  • Moving online discussions into physical spaces, either in the classroom or clinical environment
  • Sharing and highlighting student and staff work
  • Sharing social and personal experiences that indicate personal development, or provide platforms for supportive engagement
  • Extensions of classroom assignments
  • Connecting and collaborating with students and staff from other physiotherapy departments, both local and international
  • Helping students to acquire skills to help them navigate an increasingly digital world

I think that one of the most difficult challenges to overcome as I move forward with this project is going to be getting students and staff to embrace the idea that the academic and social spaces aren’t necessarily separate options. Informal learning often happens within social contexts, but universities are about timetables and schedules. How do you convince a staff member that logging into a social network at 21:00 on a Saturday evening might be a valuable use of their time?

If we can soften the boundary between “social” and “academic”, I think that there’s a lot of potential to engage in the type of informal discussion I see during clinical supervision, and which students have reported to really enjoy. I think that the social, cognitive and teacher presences from the Community of Inquiry model may help me to navigate this space.

If you can think of any other ways that social networks might have a role to play in facilitating the clinical education of healthcare professional students, please feel free to comment.

Categories
learning PhD research teaching

CHAT workshop for Emerging ICT in HE research

Last week I attended a short workshop on Cultural Historical Activity Theory (given by Joanne Hardman) as part of an NRF-funded research project on the use of emerging ICTs in Higher Education. Here are my notes from the session (this was all very new to me and I probably got a few things wrong. Feel free to point out any problems with what I’ve written):
CHAT has no theoretical framework, and is partly a method and partly theoretical
There are 3 different approaches to AT, going through Vygotsky, Leont’ev, Engstrom
Rote learning is appropriate in some instances for e.g. learning terminology. “Good” or “bad” is dependent on the objective of the teaching and learning activity. Rote learning is not appropriate to develop conceptual understanding.
Will the introduction of innovative technology in the classroom lead to changes in T&L practice?
How do people learn? Learning = cognitive change
Learning new technical skills doesn’t lead to cognitive change
Piaget = learning happens when we add to what we already know
Lev Vygotsky (1896 – 1934)
“Authentic learning situation” = real life examples that students are familiar with using their own lived experiences
“It is not the consciousness of men that determines their being but, on the the contrary, their social being that determines their consciousness” – Marx (1859). At the time, learning was about the individual.
Learning and conceptual knowledge comes from interaction with others
If students are performing poorly, we may need to look at our social interactions with them.
Lower cognitive functions are things you’re born with i.e. we share these with animals e.g. perception, memory
Higher cognitive functions are unique to humans i.e. they need to be taught, you can’t learn them on your own e.g. selective attention, logical memory
Vogystky’s theory is an instructional theory.
“Structured process of mediation by a culturally more advanced peer” → you have to be taught, you can’t learn it on your own i.e. it must be mediated
What do people do with technology to effect change? This must happen through mediation. Instruction in and of itself doesn’t automatically lead to learning. So even if technological change is leading to innovative practice, without mediation, learning doesn’t actually happen.
Mediation requires an other.
Imitation is the first action towards developing mediation
General genetic law = every function in a child’s development appears twice. First on the social level, and later on the individual level. First between people (interpsychological), and then inside the child (intrapsychological). This applies equally to voluntary attention, logical memory, and the formulation of concepts. All the higher functions originate as actual relations between human individuals.
If students are not learning in your class, it’s a function of their social self, not their cognitive self.
Mind in society, not mind and society (“Mind in society” – Vygotsky)
The general genetic law → ZPD (the extent of what someone is currently able to do or think), the potential to learn with assistance (“the space that opens up in social interaction that leads to cognitive change”). ZPD = potential to learn with guided assistance.
In the classroom, consider staggering activities / tasks that open up different ZPD’s i.e. expect different things from / provide different inputs to students who are at different levels (requires sensitivity to where they’re at)
ZPD = the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance or in collaboration with more capable peers…the actual developmental level characterises mental development retrospectively, while the ZPD characterises mental development prospectively (Vygotsky, 1978)
ZPD = responsive pedagogy
“Instruction is only useful when it moves ahead of development”
Conceptualising working in the ZPD (Langer & Applebee, 1986):
  • Ownership – of the activity
  • Appropriateness – to the students current knowledge
  • Structure – embodying a “natural” sequence of thought and action
  • Collaboration – between teacher and student
  • Internalisation – via gradual withdrawal of the scaffolding and transfer of control
Mediation in the classroom, through:
  • modeling
  • guiding
  • questioning
  • helping
  • structuring
  • recruiting attention
  • prompting
Mediation is about providing tools to help someone get to the answer. It’s not about giving the answers.
Alexei Leont’ev
Vygotsky focused on how signs and symbols (i.e. semiotics → language and writing) led to cognitive change. This has problems in an illiterate society. This had issues in Stalin’s Russia, as many of the population were illiterate. So Leont’ev focused on activity to mediate cognitive development.
“Without a theoretical conception of the social world one cannot analyse activity”
Leont’ev differentiated between collective activity and individual action e.g. hunting is a collective activity made up of individual actions. You need to study the activity, and not the actions, as the actions provide only a very narrow view of the whole.
Individual students are part of many activity systems.
Leont’ev’s activity theory (second generation):
  • Operations – fossilised and automatic e.g. typing
  • Actions – individual
  • Activity – communal, motivator, need-state i.e. requires a motive (“what is the need-state that drives the activity?”)

Engestrom

The structure of human activity (1987), triangle = a heuristic diagram for representing a method, → reason that this is a method, rather than a theory (the theory is Vygotsky)
  • Division of labor – different roles e.g. we operationalise behaviour by assigning roles. This is also about power
  • Community – shares an object e.g. the object of this workshop is “understanding of AT”
  • Rules – are horizontal and vertical (like power). Instructional: “put up your hand if you want to ask a question”. Disciplinary: “no talking when I’m talking”
  • Object – the thing we have in common e.g. our understanding of AT. Activity systems are defined by the object
  • Subject – the thing / group that is being studied / analysed in an activity system (we are the subjects in this workshop)
  • Tools
If you want to study change in an activity system, you have to first identify the contradictions / tensions / conflicts / breakdowns:
  • Primary – within each node of a system
  • Secondary – between nodes in one system
  • Tertiary – between activity systems, when role-players outside the system impacts on those within the system e.g. how will students be affected by external activity systems during our study?
  • Quaternary – exists between nodes in activity systems
No activity system operates in isolation.
There is no defined analytical framework for studying an activity.
Change laboratory is the method for expansive learning (Engestrom). The following process could be used in our case studies. Very similar to Action Research, but with a strong underlying theory, which AR sometimes lacks.
  • What is the primary contradiction need-state for our question? Questioning.
  • Secondary contradictions double bind (historical analysis, actual empirical analysis). Why is the need there?
  • Model the new solution
  • Examine the new model
  • Implement the new model (tertiary contradiction resistance e.g. pushback from others)
  • Reflect on the process (quaternary contradictions re-alignment with neighbours)
  • Consolidate the new practice
Providing anonymity is one way to empower students to participate, especially shy, marginalised and disempowered learners.
Change laboratory: an intervention method that involves sustained efforts to analyse and transform social practice. It comprises a series of workgroup meetings to reflect on current practices and envision future activities. Essential aspects of change laboratories are:
  • Using videotaped practices as a “mirror” for assessing current activity
  • Generating ideas and tools e.g. charts, that help to assess past, present and future activity
  • Modeling present practices by using activity-system analysis
Categories
assessment education learning PhD

Constructive alignment workshop

Constructive alignment workshop – Dr. James Garraway

I attended a workshop this morning looking at constructive alignment, with the view to relating it to the work I’ll be doing on my PhD next year. The second of my objectives is to do an analysis of our undergraduate curriculum and then do a Delphi study evaluating certain components of it. After our planning meeting a few weeks ago, we’ve decided to begin working on our curriculum now, in preparation for our HPCSA audit next year.The process is going to be really valuable for us, as we move towards implementing our teaching and learning policy within the department, as well as for me as I try to get a better understanding of how we actually go about graduating physiotherapists.

Here are my notes from the morning.

Intended outcomes ↔ content / learning activity↔ assessment (make sure that they all “look the same”)

Constructive alignment and submission of new programmes on the HEQF:

  • Develop higher level cognitive skills from graduates
  • Explain how competences developed in the programme are aligned with the NQF levels (looking at systematic, coherent and critical understanding of the discipline
  • Map new knowledge onto the discipline
  • Explain the teaching methods, mode of delivery and materials development for the achievement of the stated outcomes of the qualification
  • How does the T&L strategy promote the achievement of the expected learning outcomes?
  • How does the assessment strategy promote the achievement of the learning outcomes?

Activity

What do you understand by “constructive”? Builds on the term “scaffolding”, i.e. knowledge is “built” by establishing a foundation of basic understanding and then gradually introducing new concepts / ideas → ZPD

What do you understand by “alignment”? All the components of a curriculum are aligned with each i.e. beginning (outcomes) looks like the middle (learning activity), looks like the end (assessment)

What do you understand by “constructive alignment”?

Main steps in the alignment process:

  1. Define the intended outcomes (should be described using verbs that emphasise the higher learning activities → avoid “list”, etc.
  2. Choose teaching/learning activities that assist/encourage students to achieve the objectives
  3. Engage students in learning activities through the teaching process
  4. Assess students’ learning outcomes using methods that enable students to demonstrate the intended learning and evaluating how well they match what was intended
  5. Arrive at a grade (summative) or give feedback (formative)

What happens when new outcomes are derived from a “loose” approach to teaching and learning i.e. discussion, etc. The lecturer can’t foresee the outcomes before the course starts, and what impact will that have on assessment?

People learn within a set of rules / environment that is often determined by the discipline, so it can be difficult to transfer “learning / knowledge / ways of knowing” between disciplines and subjects

Constructivism isn’t necessarily about questioning established knowledge, but to engage with it (“play with it”)

Structure of Observed Learning Outcomes (SOLO) taxonomy

  1. Prestructural (unconnected knowledge)
  2. Unistructural (simple, obvious connections, one relevant aspects)
  3. Multistructural (many connections are made, considered independently)
  4. Relational level (part and the whole are understood, links and integrates several parts into a coherent whole)
  5. Extended / abstract (going beyond – generalises beyond the information given)

Level 1 teachers: make assumptions about what students are i.e. blames the student

Level 2 teachers: make assumptions about what teachers do i.e. blames the teacher

Both of the above perspectives lead to passive students

Level 3 teachers are concerned with what a student does to achieve the outcomes of the course

What is understanding?

Humans are not good at memorising random information (only 7 +/- 2 pieces of random information). But we’re very good at building new information on top of old information i.e. associating new knowledge with old knowledge → ZPD. Knowledge is “constructed”, not transmitted

Learning is a result of what the student does/thinks, not what the teacher does

How do we get students to learn what we want them to?

Teachers intention → student’s activity → exam (it’s the assessment that drives the learning activity, not the teachers intention)

Good teaching gets more students to use higher cognitive processes, that “better” students use spontaneously

Categories
twitter feed

Twitter Weekly Updates for 2010-09-13

  • More on online learning & the visually impaired. Useful links 4 anyone working with learners who have visual impairments http://ow.ly/1qQGws #
  • Gilly Salmon’s 5 stage model http://ow.ly/1qQGsw #
  • Social Learning in the Positivist Paradigm http://ow.ly/2D2kT #
  • Presentation: A few minutes with John Cleese on creativity http://ow.ly/1qQDg5 #
  • Multitasking Lowers Academic Performance http://ow.ly/1qQDfL #
  • Dear Baby Boomers, Gen Xers, Gen Yers … Can We Please Move On? http://ow.ly/1qQDfa #
  • Documents and Data… http://ow.ly/1qQy7b #
  • As clinicians we tend 2 focus on results that are easy to measure e.g. ROM, & ignore ones that are hard e.g. learning, hope, quality of life #
  • Strange how some people’s first intuition re. open learning practices is that their colleagues will “steal from them”? #
  • Presentation on blended learning in clinical education for SASP went well, good discussion afterwards, some resistance from academics #
  • Reading Social Networks and Practice Knowlege (WCPT abstract) on Scribd http://scr.bi/cxhwWm #readcast #
  • Published Social Networks and Practice Knowlege (WCPT abstract) on Scribd http://scr.bi/cxhwWm #readcast #
  • RT @francesbell: 3 ALT Learning Technologisits of the Year 😉 http://flic.kr/p/8zx9ip #
  • @cristinacost Your colleagues…sure it’s them 🙂 in reply to cristinacost #
  • Reflections on Blogging | Virtual Canuck http://bit.ly/awblvh #
  • Is the Lecture Dead? http://ow.ly/1qQ0oP #
  • Can MOOCs make learning scale? Dont assume that learning comes from the teacher http://ow.ly/1qQ0nY #
  • IBM Helps Tennis Fans “See Through Walls” with Augmented Reality http://ow.ly/1qQ0hf #
  • ResearchGATE Offers Social Networking for Scholars and Scientists http://ow.ly/1qQ0gf #
  • RT @SalfordPGRs: Huge congratulations to Cristinacost on ALTC Learning Technologis award!! #
  • Was away the whole of last week planning for next year, making 2 big curricular changes, combining some theory subjects, and moving to OSCEs #
  • Just finished a week of assisting with clinical exams for #Stellenbosch good learning experience, one learns so much from colleagues #
Categories
conference education health research

SAAHE – short oral presentations

Assessment challenges in UG medical education (GG Mokane)

Medical school in Botswana is spiral, integrated, community based and problem-based, but the rest of the university is didactic

Format, content, timing and feedback are important components of assessment

Assessment in this course has an emphasis on 3 types of MCQ’s

  1. Matching
  2. Single best answer
  3. True/false (multiple answer) – study was based on evaluating this specific format

How should these questions be used, and what instructions issued when they are?

Retrospective analysis of students performance in cumulative and non-cumulative formative assessment methods (AA Adebesin)

If students consistently score above 60%, they are exempt from the final summative exam (university rule). This had implicit problems in that students couldn’t graduate with distinction because they scored high enough to not write the summative exam.

Introduced a cumulative assessment process that carried formative assessment marks over from block to block

How do you objectively measure student progress and understanding?

A student portfolio: the golden key to reflective, experiential and evidence-based learning (G Muubuke)

Portfolios are useful evidence of learning and reflective processes

Logbooks are not good indicators of learning

Portfolio content included bio-data, radiological images, critical learning incident, clinical evaluation forms, logbook – with guiding questions to assist reflection

Portfolio assessed formatively and summatively

Found initially that students and teachers had only limited knowledge of portfolios, although training workshops helped in this regard

Stakeholders welcomed the introduction of the tool

Assessment whittled down to 2 items, rather than whole portfolio (1 item selected by student, the other by the teacher)

Students learn and develop by reflecting on experiences

Unfair to judge learning based only on exam marks

Students should see portfolio management as on ongoing practice, and not just a “task” to be completed

The purpose of the portfolio must be defined at the outset (i.e. what is the benefit to the student?), and it should be simple to complete, students should not see it as additional work

It should be aligned with institutional goals and learning activities

There’s a lot of effort and time involved in assessing portfolios, and rubrics may help to assist marking (adds standardisation)

Making assessment matter: does a novel model of the pre-assessment effects of summative assessment on learning also operate in clinical contexts? (F Cilliers)

There is little evidence of what the impact of assessment is on learning, as well as the mechanism of the impact

Validating a model by looking at the following 4 factors:

  • Explanatory power
  • Generalisability
  • Integration
  • Utility

Daily exposure to consequences leads to evenly distributed learning in clinical settings, but in theory modules, periodic assessment would lead to “binge learning”. However, the more relaxed nature of the clinical (evenly distributed) model might actually lead to the binge-type learning model of theory blocks.

Relaxed environments allow students to go and follow up on work after the situation, but stressful environments force students to memorise content that they forget immediately afterwards

High risk environments lead to surface cognitive processing strategies, as opposed to supportive and low risk environments leading to deeper cognitive processing

The model is useful for explaining behaviour, is generalisable, and is integrated. Not able to determine if it is useful yet

It’s about personal and academic consequences (and their imminence), not just the act of assessment. When block marks are given to students at the end of a block, that were relevant to a situation that occurred during the block, students are less likely to pay attention to the feedback (in whatever form it takes). Consequences should be immediate and not scary.

Assessors can have a powerful (and potentially negative) influence on learning

Students study more for stressful situations, but they remember less. They study less for relaxed environments, but are more likely to follow up on the situations and remember more

Categories
conference education health research

SAAHE keynote – What generic skills do students bring with them?

This is the presentation from Professor Debbie Murdoch-Eaton.

What impacts do we have on students life ambitions?

Workers with general / transferable skills are better placed to succeed in a global knowledge economy. The skills need not be specific to the discipline

The attributes are not only about economic drivers i.e. getting a job…they are also about enabling people to be more successful in communities / life

We do need to develop specific skills that are course specific, but also more generic skills like communication and interpersonal skills, higher order reasoning, critical thinking, ability to use tech, etc.

Why do you go to university, if not to train the mind?

There needs to be a social agenda within the institution in terms of implementing generic skills (or graduate attributes, depending on who you’re reading)

Are clinical skills generic? If so, which ones? Maybe; enterprise, management, leadership, probity, altruism

Transferability is a key skill, helps to contextualise practice

Students are coming into UK HE with lower technical and numeracy skills, written presentation, and selecting and utilising information, but increased practice in IT skills, stress management

No change in student’s ability to manage their own learning (working with others, seeking and giving feedback, teamwork, taking responsibility for own learning), presentation skills (verbal communication skills, essay writing), time management and self-organisation

Does any of this matter? Might be course dependent. It has implications for course / curriculum design and student support

Huge gap between UK and SA student generic skills (e.g. email use, managing own learning). Demonstrated how longitudinal studies of student generic skills on entering HE can be used to make decisions about curriculum design for those students.

Is the curriculum additive (additional to the discipline) or transformative?

Generic skills can be developed by being remedial (by identifying students with deficits in skills) or associated (for all students, running alongside normal curriculum but additional and separate), or part of discipline content and process, engagement / participatory

Integrating lifelong learning into the curriculum should develop explicit generic skills that incorporate structured opportunities to practice those skills

Skills can be inferred from behaviour, which is subjective and we need to be aware that the observer (teacher) has a vested interest into what skills are being inferred

“Those who are least able are also least able to self-assess accurately”

Generic skills must be embedded into the discipline’s teaching methods rather than being bolted on

“We cannot teach science that is as yet undiscovered, nor can we forecast it’s future implications”

We have to educate doctors who are capable of adaptation and change

Categories
education learning students teaching

The role of feedback in medical education

This reflection on the role of feedback in education is based on a mailing list discussion as part of the SAFRI programme, as well as on a few of the assigned readings. I thank the organisers of the session, as well as all the participants in the conversation.

Guidelines for effective feedback

The assigned readings seem to focus on providing readers with a list of guidelines for providing effective feedback and while this list is not exhaustive, it serves as a starting point. Feedback is better when:
  • It is promptly given
  • It is specific to the task being assessed, and to the objectives that were set (of course, this assumes that the student has set objectives for the task)
  • It is performance-based i.e. the feedback is along the lines of what you did, as opposed to who you are. One participant made the point that while it is performance-based, it is done to improve performance, and not to evaluate it
  • It does not focus on too many things
  • It helps the student plan for future learning
  • Feedback should seek to encourage self-assessment, self-reflection and self-awareness
  • Should / Could be better received if the supervisor / clinician allows the student to comment first e.g. What did they think went well? What did they think could be improved? (one participant said that “…a lot of feedback that I have to give to students is actually on written assignments [which]…means that we cannot start with the student’s agenda. There is no interactive discussion, no possibility to listen, respond to non-verbal cues, observation, etc.”
  • Positive feedback should be given first (builds self-esteem and encourages better performance)
  • This positive feedback should not be followed with a qualifier e.g. “You did well, but…”
  • “Listen and ask, don’t tell and provide solutions”
I’m glad someone pointed out that this (long) list is wonderful in ideal situations but is impractical in many real-world situation, which is where we find ourselves most often e.g. large student:teacher ratios, overloaded faculty and limited time to spend with students. On further reflection, it seems clear that this issue is not specific to feedback, and affects almost everything we want to do as teachers. The problem with teaching is that the way we want to do it (i.e. small groups with focused attention) doesn’t scale very well and we need to come up with a fundamentally different approach to teaching. My own view is that the internet and various associated technologies can enhance communication in ways that do scale, and that therein lies part of the solution.

Complexity in feedback

My own initial thought on a list of guidelines was that it was a wonderful “how to” for providing effective feedback to students. However, as I progressed through the conversation, I began to have my doubts, starting with this one. I worry that having a list of guidelines may deceive us into thinking that if we follow the list then we’ve given good feedback. This is like a student thinking that following a list of questions is a good way to conduct an assessment. Giving feedback seems to be a dynamic process, affected by context (e.g. social / cultural background, type of placement / task) and an understanding of the person/ality receiving it. Someone suggested that the complex process of giving / receiving feedback was also about the identity and character of the receiver and that “…in order to protect the integrity of their beliefs and knowledge, [they] will reject corrective feedback and find ways to devalue it”. Some studies have identified the importance of eliciting thoughts and feelings before giving feedback, which might go some way to alleviating this.
One of the participants insightfully related this back to the MBTI session in March, where it was clear that “knowledge of personalities is useful when giving and receiving feedback. What we know about ourselves and others is important feedback management”. In relation to this, another participant raised the point that “…the issues of culture, gender, religion, belief, age are very important in feedback.  Also, feedback for average students is different than feedback for failing students [as well as for] students who think they’re doing well [but] are actually failing”. I think it’s important to note that feedback is dynamic, contextual and complex.

Inappropriate feedback

I found a common theme in the conversation that went along the lines of “feedback drives learning”. This may be a matter of semantics but I’d like to challenge the idea that it does drive learning and suggest that it can drive learning. This may seem pedantic but I think it’s important, because when we say (or imply) that it does, we’re operating under the assumption that all feedback is equal, which it clearly isn’t. This was pointed out by several participants, who suggested that feedback can be inappropriate “…because of how, when or where it is presented”. Some teachers seem to be guilty of using feedback to highlight their own skills and knowledge, while at the same time making it clear that the student lacks these things (or in one horrifying example, actively humiliating them).
When using my own experiences to make a point, I often include examples of my mistakes. I think that as role models, we need to model our failures (and to elaborate on how we moved past them) as well as our successes. Students seem to have an idea that we’re infallible, which unfortunately makes them believe they must be too. If we can highlight that we’re also subject to errors of judgement and prone to forgetfulness (a big one for me), we show them that we’re human and go a long way to establishing trust.

Feedback as a skill

There was a suggestion that giving “feedback is a skill that has to be learnt” and that we should emphasise its importance. This was taken further with the idea that receiving feedback is also a skill that needs to be learnt (e.g. listening, reflecting, analysing), and that we need to spend a lot more time preparing students to receive feedback effectively and with the right attitude / mindset. One participant spoke of students who receive feedback defensively, negating it whether it was appropriate or not. I liked the idea raised that feedback should not only emphasise knowledge, but also more generic skills like effective communication, conflict resolution, etc.

Feedback in teaching and learning

It was pointed out that feedback is not only about a student-teacher interaction, it can also be between peers or colleagues. In fact, feedback is “…for helping all who are interested in self development and actualization of goals, not only students”. This suggests to me that feedback between peers could be an important component of peer teaching, an area of that I’m increasingly interested in.
Feedback should also be given to students who are performing well. This will help to dispel the notion that feedback = criticism. It was pointed out that as teachers, we often have a tendency to focus on the student’s weakness (I know that I’m guilty of this), possibly as a remnant of our own experiences of being students, when this is how we received it. One of the dangers of focusing on the negative only, is inducing a lack of confidence on the students part, where they become incapable of identifying their strengths. Thus, feedback should be used to highlight strengths as well as weaknesses, in order to promote learning.

Feedback and evaluation

Before beginning, see the third point in the guidelines. All too often (as was pointed out by some participants), feedback = marks, and there is no action required after receiving the mark (or if there is, the response is along the lines of “What must I do to get a higher mark”). I wonder if we’re not the problem. We make the assumption (and model behavior showing) that feedback and marks are related, whereas they don’t need to be. Marks are quantitative, while feedback is qualitative. Marks are summative, feedback is formative. Apples are apples and oranges are oranges. When one person is evaluating another, there’s no real objectivity, and so quantitative measures of competence don’t seem to me to be a good fit.
We should distinguish between formative and summative assessment, and their relative relationships to feedback. In formative assessment, feedback is essential as its nature is to facilitate learning. In summative assessment, feedback is irrelevant because the nature of the examination is to evaluate, which we’ve seen is not the role of feedback.
I think we need to get away from this idea that feedback and marks are necessarily related. Of course they can be, but it doesn’t mean they must be. We have to disconnect feedback, which is about learning, and marking, which is about evaluation (and not a very good form of evaluation at that). If students are “marks driven” it’s because we’ve put marks at the centre of our curricula. How do we de-emphasise marks…well, by not marking (it’s radical, I know).

Feedback and reflection

In order for feedback to be effective and of any value, we must first identify the relationship between the task, the feedback, reflection on the task and feedback, and finally, acting on the feedback. Without first making sure that the receiver understands this, the feedback is of no value. We know that reflection drives deep learning, understanding and professional development, yet we leave little space in the curriculum for structured reflection (and forget about teaching students how to reflect). It was great to read the comment, “we do not learn from experience, but rather from reflecting on experience, and feedback must facilitate this reflection for the student”. I would argue that unless the feedback results in a behavioral change, it is ineffective. Of course, if the student cannot focus on anything but the mark, we clearly haven’t established for them the relationship between the task, the feedback, the reflection and the action.

“No feedback” as feedback?

There was a question of whether “no feedback could also be feedback”, and I agree with one response which stated that it was “…the most negative and most useless form of ‘feedback’ in that it borders on pure and simple indifference”, as well as being “regressive and inhibiting”. I’m reminded of a line from “A man for all seasons]]”, in which Thomas More states that …”the maxim of the law is ‘Silence gives consent'”. When I was a student and was not explicitly told to improve (and to do this), then it meant there was nothing to improve, which I found disturbing, knowing what kind of student I was. I worry that if we give no feedback, we risk the student believing that their performance is exemplary (which it may very well be, but then tell them that), or worse, that they don’t know what to believe, leading to anxiety and confusion.

Feedback as a form of academic literacy

I’ve also been thinking about feedback as a form of literacy, which was touched on by one participant, who suggested that “…conversation is a form of feedback, a sort of negotiation of understanding in which mutual feedback becomes recursive leading to shared understanding.  In this way, feedback is not one person who ‘knows’ helping another person who doesn’t ‘know’, rather it is mutual. By asking myself questions before and during the feedback, I am learning at the same time the other is learning”. I would take this in a more general direction and suggest that feedback is useless unless the student is at least familiar with the culture of the tribe, which includes the language and conventions we use.
We can give as much feedback as we like, but if we’re speaking a language the student doesn’t understand (I don’t mean English, etc.), we’re wasting everyone’s time. And in case you think that by final year, our students understand our “tribe”, think about this simple example: The first year student is starting out in the culture of higher education, the second year student is starting out in the culture of physiotherapy, the third year student is starting out in the culture clinical practice, and the final year student is starting out in the culture of being a physiotherapist. We need different languages and approaches for each of these different cultures / literacies / “tribes”.

Final thoughts

  • For most of the discussion, feedback was treated as a “thing”, rather than a process or interaction. My own view is that feedback is neither “given” nor “received”, but is a process that people participate in
  • How many of us ask our students for feedback on our feedback?
  • How many of use ask our students for feedback on our teaching? One participant had this to say “Should each of us offer the students who receive our teaching the opportunity to give feedback on our teaching as part of their learning? This may be a little threatening, but on the occasions that I have done so in the past it has mostly been encouraging and affirming. Occasionally uncomfortable.”
  • If we do ask for it, how many of us reflect on it and make an associated change in our teaching practice?
Categories
education research social media

Social networks as platforms to establish communities of practice

I finally managed to submit my application to participate in the FAIMER programme for 2010.  I’m not sure when the applications will be processed, but I’ll be sure to announce it here.  See below for the Introduction to my project proposal:

“A traditional curriculum is based on an accumulated body of knowledge that has been acquired, collated and verified over a long period of time, with experts in a particular field being determined by comparing their assertions to those of the established canon. This method of acquiring knowledge isn’t possible in a society where content is becoming available faster than the ability of any one individual to process.

While the emergence of the internet and a fully networked society have ushered in a period of convenient access to vast amounts of content, this has often been misinterpreted as access to knowledge. Even though  the distribution of massive amounts of data is certainly welcome, it misses the point that the power of the internet is not in being a content repository, but as a platform to facilitate communication through social networks and communities of practice.

The internet has created a realistic opportunity to share and exchange learning experiences, not only beyond the walls of the classroom, but across oceans and continents. As a result of a densely connected society, our acceptance of conventional wisdom is being challenged, as the concept of knowledge is increasingly being seen as a negotiated outcome of social learning experiences that are tightly integrated within the network.

These ideas do not only force us to reconsider the traditional meanings of “curriculum”, “education”, and “teacher”. They also challenge us to find innovative ways of guiding students through a curriculum where memorising the prescribed content is less important than their ability to make meaning through knowledge sharing within their professional community.”

Sources

  • Fitzgerald, R., & Steele, J. (2008). Digital learning communities – Investigating the application of social software to support networked learning. Australian Learning and Teaching Council, PO Box 2375, Strawberry Hills, NSW 2012, Australia.

  • Fox, R., Yeung, L., Law, N., Yuen, A., Yeung, A., Kong, H., et al. (2006). Sustaining and transferring curriculum and pedagogical innovation through establishing communities of practice. Proceedings of the 23rd annual Ascilite conference (pp. 251-255). University of Sydney.

  • Siemenns, G. & Tittenberger, P. (2009). Handbook of emerging technologies for Learning. Learning Technologies Centre, University of Mantiboba. Available online at http://ltc.umanitoba.ca/wikis/etl/index.php/Handbook_of_Emerging_Technologies_for_Learning