I’m giving a workshop later today. The idea is that we’ll get all of the facilitators who’ll be working on the module we’re designing (and which I’m evaluating for my PhD) and help them get a grip on the approach to facilitation that we’d like them to use. The objective of the workshop is to help them get an understanding of the conceptual basis for facilitation in this module. We’re going to use Laurillard’s “Conversational Framework” as a structure to guide how the facilitators should try and engage with their groups, both in the classroom and in the clinical context. The following notes have been taken from Laurillard’s “Rethinking University Teaching“.

Learning needs to be situated within a context and we can’t separate the knowledge to be learned from the context in which it has to be applied. Conceptual knowledge is not an abstract, intangible thing. It is a tool that can be used as part of an authentic learning activity. There is a unity between the problem, context and solution when the problem is experienced, that is absent when an answer is merely given.

Teaching is essentially an activity that tries to help students change the way the see the world by interpreting the insights of others.

• “Everyday learning” = a result of our experiences in the world i.e. we develop an implicit awareness of gravity by falling
• “Academic learning” = a result of our reflections on others’ descriptions of the world i.e. we develop an understanding of a theory of gravity by reading about experiments conducted by other people

Academic learning is different to everyday learning in the sense that it is the student learning through interpreting the symbols (i.e. language, images, diagrams) of someone else’s view of the world

The knowledge that students bring with them will impact on how they integrate the new knowledge that they learn. Remember the ZPD and how the MKO guides the student to higher cognitive levels by building on what they already know.

It makes no sense to correct a faulty procedure without also correcting the faulty conceptualisation that supported it (knowledge is situated in action, and action manifests knowledge). Correcting fundamental misconceptions automatically corrects all of the faulty procedures associated with it. Correcting the procedure corrects only one way of doing it incorrectly. This is one problem with merely demonstrating a technique. The student is forced to conceive a rationale for the technique, which may be incorrect. By taking them through an experience of solving a problem, the rationale for the technique is implicitly tied to its performance.

Before we can challenge the students’ fundamental misconceptions, we need to know what those misconceptions are. Again, this links back to the ZPD. Without knowing where the student is, we cannot help them get to where they want to be.

Researching the learning process (which is essentially what a facilitator is…a dynamic researcher into student learning) should include an observation of student performance on a task e.g. worked problems or written explanations, with a retrospective interview of the student looking back at the task and describing how they experienced it. The interviewer uses the task to provide cues to the student.

The learning process includes 5 interdependent aspects:

• Apprehending structure. Students often fail to apprehend the structure of a discourse (e.g. a body of text), and there is often meaning that is implicit in structure (e.g. headings, paragraphs, etc.). When students take a surface approach to studying a text they lose the structure of the arguments and end with a series of statements that are not related to each other. When they take a deep approach they preserve the structure was well as the original meaning.
• Integrating parts. Students must learn how to interpret the discipline-specific representations if they are to make sense of them. The way that information is presented can lend itself to deep or surface approaches, as well as create potential “distractors” for the student. The idea is not to ensure that data representation is “easy” for the student to interpret but rather to prepare the student to handle the different representations. Complex scenarios provide opportunities to determine students’ ability to interpret the representations. For example, consider how students are confused when different clinicians advocate different management approaches for the same patient. The student who only comprehends the superficial structure of the interaction is stuck because they cannot perceive that interpretations can be different.
• Acting on the world. Learning is an activity (classroom-based problem-solving), an imitation of practice (practical sessions in the classroom), or actual practice (seeing patients). The student must engage with the world (i.e. solving problems in the classroom, or treating patients) by performing an action that is based on their understanding of how the world works.
• Using feedback. As we learn about the world by acting on it, we receive direct feedback and adjust the action in relation to the feedback. The feedback must be perceived as useful to the student (i.e. it must be meaningful). It must be given immediately (or soon) after the students’ action in order for the student to relate the feedback to the action. Helpful feedback also provides the student with specific information on how to adapt their performance.
• Reflecting on goals. Reflection is about establishing conceptual links between the action, feedback, and integration of the two as they relate to the achievement of a goal (e.g. solving a problem). Students often interpret goals as being something required by the teacher and go through the steps necessary to reproduce an outcome, with little intention of understanding the task or the goal (i.e. the tasks are a series of hoops that they have to jump through). The same task is therefore perceived differently by the students and teacher, and therefore operationalised in different ways. For many students, what it means to achieve the objective / goal is different to what the teacher is trying to do.

Using the above steps, we can see how learning something deeply is complex and difficult to facilitate. In short, the facilitator should try to conduct an interactive dialogue that supports the learning process. The following points describe the components of a teacher-student dialogue that promotes deep learning of a topic.

Apprehending structure

• Students role: look for structure, discern topic goal (if the goal isn’t explicitly identified, the student lacks the structure to guide their thinking), relate goal to structure of discourse
• Facilitators role: explain phenomena, clarify structure, negotiate topic goal, ask about internal relations (explain phenomena, make predictions, compare analogous situations)

Interpreting forms of representation

• Students role: model events / systems in terms of forms of representation, interpret forms of representation to model systems / events
• Facilitators role: set mapping tasks between forms of representation and systems / events, relate forms of representation to students’ view

Acting on descriptions

• Students role: derive implications, solve problems, and test hypotheses to produce descriptions
• Facilitators role: elicit descriptions, compare descriptions, highlight inconsistencies

Using feedback

• Students role: link teachers redescription to relation between action and goal, to produce new action on description (student gives a description of something, teacher responds with a different viewpoint that demonstrates inconsistency, student must therefore reframe / describe it again)
• Facilitators role: provide redescription, elicit new description, support linking process

Reflecting on goal-action-feedback cycle

• Students role: engage with goal, relate to actions and feedback (this is why the goal of the dialogue must be explicit, to allow students to reflect its relationship to the action / description and feedback)
• Facilitators role: prompt reflection, support reflection on goal-action-feedback cycle

There should be a continuing, iterative dialogue between teacher and student, that reveals both parties conceptions and differences between the conceptions, which then determines the focus for continuing dialogue. However, it’s not just the process of conducting the dialogue that matters but HOW it is conducted e.g there must be an opportunity for the student to interpret forms of representation other than language.

A teaching strategy should be:

• Discursive - the teachers and students conceptions should be continually accessible to each other; teacher and student must agree on the learning goals for the topic; the teacher must provide an environment for the discussion, within which the student can generate and receive feedback on descriptions appropriate to the topic goal; the teachers description must be meaningful to the student
• Adaptive - the relationship between the teacher’s and student’s conceptions must serve as the focus for the continuing dialogue; it is the student’s responsibility to use the feedback from their work on the task and relate it to their conception
• Interactive - the teacher must provide an environment in which the student can act on, generate and receive intrinsic feedback on actions appropriate to the task goal; the student must act to achieve the task goal; the teacher must provide meaningful feedback on their actions that relates to the nature of the task goal
• Reflective - the teacher must support the process in which students link the feedback on their actions to the topic goal for every level of description within the topic structure; the student must reflect on the task goal, their action on it, and the intrinsic feedback they receive, and link this to their description of their conception to the topic goal

Share

“I think we should make sure that, in addition to the ‘answers’, we should identify the main concepts we want students to understand. Remember that we’re using our paper patient (i.e. the case) as a framework for students to learn about concepts. Then, they apply those concepts in the real world to patients. They reflect on those real-world interactions and identify dissonance between their experienced reality (the patient contact) and their abstract conceptions of reality (how they originally conceived of the patient contact). After the patient contact, they feed back to their small groups and facilitators, who together help students create new relationships between concepts. So, in short, the clinical concepts are learned initially through the paper patient, tested in the real world with an actual patient, discussed online (maybe) and then brought back to the classroom for further reflection and refinement. The next week they are exposed to new concepts that build on their previous experiences, and then they get to test those abstractions in the real world again.”

I’m trying to take an intentional approach to using Laurillard’s conception of academic learning that I’m exploring in “Rethinking University Teaching”

Share

I was quite happy with this until I had a conversation with a colleague, who asked how I planned on determining whether or not the new teaching strategy had actually worked. This threw me a little bit. I thought that I had it figured out…do small research projects to develop understanding of the students and the teaching / learning environment, use those results to inform the development of an intervention, implement the intervention and evaluate the process. Simple, right?

Then why haven’t I been able to shake the feeling that something was missing? I thought that I’d use a combination of outputs or “products of learning” (e.g. student reflective diaries, concept mapping assignments, semi-structured interviews, test results, focus groups, etc.) to evaluate my process and make a recommendation about whether others should consider taking a blended approach to clinical education. I’ve since begun to wonder if that method goes far enough in making a contribution to the field, and if there isn’t something more that I should be doing (my supervisor is convinced that I’ve got enough without having to change my plan at this late stage, and she may be right).

However, when I finally got around to reading Laurillard’s “Rethinking University Teaching”, I was quite taken with her suggested approach. It’s been quite an eye opener, not only in terms of articulating some of the problems that I see in clinical practice with our students, but also helping me to realize the difference between designing teaching activities (which is what I’ve been concentrating on), and evaluating learning (which I’ve ignored because this is hard to do). I also realized that, contrary to a good scientific approach, I didn’t have a working hypothesis, and was essentially just going to describe something without any idea of what would happen. Incidentally, there’s nothing wrong with descriptive research to evaluate a process, but if I can’t also describe the change in learning, isn’t that limiting the study?

I’m now wondering if, in addition to what I’d already planned, I need to conduct interviews with students using the phenomenological approach suggested by Laurillard i.e. the Conversational Framework. I don’t yet have a great understanding of it but I’m starting to see how merely aligning a curriculum can’t in itself make any assertions about changes in student learning. I need to be able to say that a blended approach does / does not appear to fundamentally change how students’ construct meaning and in order to do so I’m thinking of doing the following:

• Interview 2nd year and 3rd students at the very beginning of the module (January, 2012), before they’ve been introduced to case-based learning. My hypothesis is that they’ll display quite superficial mental constructs in terms of their clinical problem-solving ability as neither group has had much experience with patient contact
• Interview both groups again in 6 months and evaluate whether or not there constructs have changed. At this point, the 2nd years will have been through 6 months of a blended approach, while the 3rd years will have had one full term of clinical contact with patients. My hypothesis is that the 2nd years will be better able to reason their way through problems, even though the 3rd years will have had more time on clinical rotation

I hope that this will allow me to make a stronger statement about the impact of a blended approach to teaching and learning in clinical education, and to be able to demonstrate that it fundamentally changes students constructs from superficial to deep understanding. I’m just not sure if the Conversational Framework is the most appropriate model to evaluate students’ problem-solving ability, as it was initially designed to evaluate multimedia tools.

Share

Photo taken while on a short walk during the retreat.

A few weeks ago I spent 3 days at Mont Fleur near Stellenbosch, on a teaching and learning retreat. Next year we’re going to be restructuring 2 of our modules as part of a curriculum review, and I’ll be studying the process as part of my PhD. That part of the project will also form a case study for an NRF-funded, inter-institutional study on the use of emerging technologies in South African higher education.

I used the workshop as an opportunity to develop some of the ideas for how the module will change (more on that in another post), and these are the notes I took during the workshop. Most of what I was writing was specific to the module I was working with, so these notes are the more generic ones that might be useful for others.

————————

Content determines what we teach, but not how we teach. But it should be the outcomes that determine the content?

“Planning” for learning

Teaching is intended to make learning possible / there is an intended relationship between teaching and learning

Learning = a recombination of old and new material in order to create personal meaning. Students bring their own experience from the world that we can use to create a scaffold upon which to add new knowledge

We teach what we usually believe is important for them to know

What (and how) we teach is often constrained by external factors:

• Amount of content
• Time in which to cover the content (this is not the same as “creating personal meaning”)

We think of content as a series of discrete chunks of an unspecified whole, without much thought given to the relative importance of each topic as it relates to other topics, or about the nature of the relationships between topics

How do we make choices between what to include and exclude?

• Focus on knowledge structuring
• What are the key concepts that are at the heart of the module?
• What are the relationships between the concepts?
• This marks a shift from dis-embedded facts to inter-related concepts
• This is how we organise knowledge in the discipline

Task: map the knowledge structure of your module

“Organising knowledge” in the classroom is problematic because knowledge isn’t organised in our brains in the same way that we organise it for students / on a piece of paper. We assign content to discrete categories to make it easier for students to understand / add it to their pre-existing scaffolds, but that’s not how it exists in minds.

Scientific method (our students do a basic physics course in which this method is emphasised, yet they don’t transfer this knowledge to patient assessment):

1. Observe something
2. Construct an hypothesis
3. Test the hypothesis
4. Is the outcome new knowledge / expected?

Task: create a teaching activity (try to do something different) that is aligned with a major concept in the module, and also includes graduate attributes and learning outcomes. Can I do the poetry concept? What about gaming? Learners are in control of the environment, mastering the task is a symbol of valued status within the group, a game is a demarcated learning activity with set tasks that the learner has to master in order to proceed, feedback is built in, games can be time and resource constrained

The activity should include the following points:

• Align assessment with outcomes and teaching and learning activities (SOLO taxonomy – Structured Observation of Learning Outcomes)
• Select a range of assessment tools
• Justify the choice of these tools
• Explain and defend marks and weightings
• Meet the criteria for reliability and validity
• Create appropriate rubrics

Assessment must be aligned with learning outcomes and modular content. It provides students with opportunities to show that they can do what is expected of them. Assessment currently highlights what students don’t know, rather than emphasising what they can do, and looking for ways to build on that strength to fill in the gaps.

Learning is about what the student does, not what the teacher does.

How do you create observable outcomes?

The activity / doing of the activity is important

As a teacher:

• What type of feedback do you give?
• When do you give it?
• What happens to it?
• Does it lead to improved learning?

Graduate attributes ↔ Learning outcomes ↔ Assessment criteria ↔ T&L activities ↔ Assessment tasks ↔ Assessment strategy

Assessment defines what students regard as important, how they spend their time and how they come to see themselves as individuals (Brown, 2001; in Irons, 2008: 11)

Self-assessment is potentially useful, although it should be low-stakes

Use a range of well-designed assessment tasks to address all of the Intended Learning Outcomes (ILOs) for your module. This will help to provide evidence to teachers of the students competence / understanding

In general quantitative assessment uses marks while qualitative assessment uses rubrics

Checklist for a rubric:

• Do the categories reflect the major learning objectives?
• Are there distinct levels which are assigned names and mark values?
• Are the descriptions clear? Are they on a continuum and allow for student growth?
• Is the language clear and easy for students to understand?
• Is it easy for the teacher to use?
• Can the rubric be used to evaluate the work? Can it be used for assessing needs? Can students easily identify growth areas needed?

Evaluation:

• What were you evaluating and why?
• When was the evaluation conducted?
• What was positive / negative about the evaluation?
• What changes did you make as a result of the feedback you received?

Evaluation is an objective process in which data is collected, collated and analysed to produce information or judgements on which decisions for practice change can be based

Course evaluation can be:

• Teacher focused – for improvement of teaching practice
• Learner focused – determine whether the course outcomes were achieved

Evaluation be conducted at any time, depending on the purpose:

• At the beginning to establish prior knowledge (diagnostic)
• In the middle to check understanding (formative) e.g. think-pair-share, clickers, minute paper, blogs, reflective writing
• At the end to determine the effectiveness of the course / to determine whether outcomes have been achieved (summative) e.g. questionnaires, interviews, debriefing sessions, tests

Obtaining information:

• Feedback from students
• Peer review of teaching
• Self-evaluation

References

• Knight (n.d.). A briefing on key concepts: Formative and summative, criterion and norm-referenced assessment
• Morgan (2008). The Course Improvement Flowchart: A description of a tool and process for the evaluation of university teaching

Share

This is quite a long post. Basically I’ve been trying to situate my current research into a larger curriculum development project and this post is just a reflection of my progress so far. It’s probably going to have big gaps and be unclear in sections. I’m OK with that.

Earlier this week our department had a short workshop on developing the cases that we’re going to use next year in one of our modules. We’re going to try and use cases to develop a set of skills and attitudes that are lacking in our students. These include challenges with (text in brackets are stereotypical student perspectives):

• Problem solving and clinical reasoning (Tell me what the answer is so that I can memorise it)
• Critical analysis (Everything I read has the same value)
• Empathy (The patient is an object I use to develop technical skills)
• Communication (The use of appropriate professional terminology isn’t important)
• Groupwork (Assessment is a zero sum game…if you score more than me it bumps me down the ranking in the class, therefore I don’t help you)
• Knowing vs Understanding (It’s more important for me to know the answer than to understand the problem)
• Integration of knowledge into practice (What I learn in class is separate to what I do with patients)
• Integration of knowledge from different domains (I can’t examine a patient with a respiratory problem because I’m on an orthopaedic rotation)
• Poor understanding of the use of technology to facilitate learning (social networks are for socialising, not learning)

I know it might seem like a bit much to think that merely moving to case-based learning is going to address all of the above, but we think it’ll help to develop these areas in which the students are struggling. The results of my ongoing PhD research project will be helping in the development of this module in the following ways:

• The survey I began with in 2009 has given us an idea of digital literacy skills of this population, as well as some of the ways in which they learn.
• The systematic review has helped us to understand some of the benefits and challenges of a blended approach to clinical education.
• The Delphi study (currently in the second round) has already identified many of the difficulties that our clinicians and clinical supervisors experience in terms of developing the professional and personal attributes of capable and competent students. This study will attempt to highlight teaching strategies that could help to develop the above mentioned problems.
• I’ve also just finished developing and testing the data capture sheet that I’ll be using for a document analysis of the curriculum in order to determine alignment.
• Later next year I’ll be conducting an evaluation of the new module, using a variety of methods.

All of the above information is being fed into the curriculum development process that we’re using to shift our teaching strategy from a top-down, didactic approach to a blended approach to teaching and learning. Development of the cases is one of the first major steps we’re taking as part of this curriculum development process. I’ll try to summarise how the cases are being developed and how they’ll be used in the module. This module is called “Applied Physiotherapy” and it’s basically where students learn about the physiotherapy management of common conditions.

In the past, these conditions were divided into systems and taught within those categories e.g. all orthopaedic conditions were covered together. The problem is that this effectively silo’s the information and students see little crossover. In fact, reality is very rarely so conveniently categorised. Patients with orthopaedic conditions may develop respiratory complications as a result of prolonged bed rest. Patients with TB often also present with peripheral neuropathy, as a result of the association of TB with HIV. So, the purpose of the cases is also to integrate different conditions to help students understand the complexity of real-world cases.

In the first term we’ll use 2 very simple cases that each run for 3 weeks. The reason that the cases are simple is that we’re also going to be introducing many new ideas that the students may have little experience with and which are important for participation in the cases e.g. computer workshops for the online environment, concept mapping, group dynamics, presentation skills, etc. The cases will increase in complexity over time as the students feel more comfortable with the process.

Each case will have an overview that highlights the main concepts, learning outcomes, teaching activities, assessment tasks and evaluation components that the case encompasses. The case will be broken up into parts, the number of which will depend on the duration and complexity of the case. After the presentation of each part, the students (in their small groups) will go through this process:

• What do I know that will help me to solve this problem?
• What do I think I know that I’m uncertain of?
• What don’t I know that I need to learn more about?

These questions should help the students develop a coherent understanding of the knowledge they already have that they can build on, as well as the gaps in understanding that they need to fill before they can move on with the case. Each part will involve students allocating tasks that need to be completed before the next session and role allocation is done by each group prior to the introduction of the case. During this process, facilitators will be present within the groups in order to make sure that students have not left out important concepts e.g. precautions and contraindications of conditions.

At the next session, each member of the small groups present to each other within the small groups. The purpose of this is to consolidate what has been learned, clarify important concepts and identify how they’re going to move forward. At the end of each week each small group presents to the larger group. This gives them the opportunity to evaluate their own work in relation to the work of others, make sure that all of the major concepts are included in their case notes, as well as opportunities to learn and practice presentation skills. Students will also be expected to evaluate other groups’ work.

There will be a significant online component to the cases in the form of a social network built on WordPress and Buddypress. We will begin by providing students with appropriate sources that they can consult at each stage of the process. Over time we’ll help them develop skills in the critical analysis of sources so that they begin to identify credibility and authority and choose their own sources. They will also use the social network for collaborative groupwork, communication, and the sharing of resources.

Finally, here are some of the tasks we’re going to include as part of the cases, as well as the outcomes they’re going to measure (I’ve left out citations because this has been a long post and I’m tired, but all of these are backed by research):

• Concept mapping – determine students’ understanding of the relationships between complex concepts
• Poetry analysis – development of personal and professional values e.g. compassion, empathy
• Reflective blogging – development of self-awareness, critical evaluation of their own understanding, behaviours and professional practices
• Peer evaluation – critical analysis of own and others’ work
• Case notes – development of documentation skills
• Presentations – ability to choose important ideas and convey them concisely using appropriate language

This is about where we are at the moment. During the next few months we’ll refine these ideas, as well as the cases, and begin with implementation next year. During my evaluation of the module, I’ll be using the results of the student tasks listed above, as well as interviews and focus groups with students and staff. We’ll review the process in June and make changes based on the results of my, and 2 other, research projects that will be running. We want the curriculum to be responsive to student needs and so we need to build in the flexibility that this requires.

After reading through this post, I think that what I’m saying is that this forms a basic outline of how we’re defining “blended learning” for this particular module. If you’ve managed to make it this far and can see any gaping holes, I’d love to hear your suggestions on how we can improve our approach.

Share