Teaching and learning workshop at Mont Fleur

Photo taken while on a short walk during the retreat.

A few weeks ago I spent 3 days at Mont Fleur near Stellenbosch, on a teaching and learning retreat. Next year we’re going to be restructuring 2 of our modules as part of a curriculum review, and I’ll be studying the process as part of my PhD. That part of the project will also form a case study for an NRF-funded, inter-institutional study on the use of emerging technologies in South African higher education.

I used the workshop as an opportunity to develop some of the ideas for how the module will change (more on that in another post), and these are the notes I took during the workshop. Most of what I was writing was specific to the module I was working with, so these notes are the more generic ones that might be useful for others.

————————

Content determines what we teach, but not how we teach. But it should be the outcomes that determine the content?

“Planning” for learning

Teaching is intended to make learning possible / there is an intended relationship between teaching and learning

Learning = a recombination of old and new material in order to create personal meaning. Students bring their own experience from the world that we can use to create a scaffold upon which to add new knowledge

We teach what we usually believe is important for them to know

What (and how) we teach is often constrained by external factors:

  • Amount of content
  • Time in which to cover the content (this is not the same as “creating personal meaning”)

We think of content as a series of discrete chunks of an unspecified whole, without much thought given to the relative importance of each topic as it relates to other topics, or about the nature of the relationships between topics

How do we make choices between what to include and exclude?

  • Focus on knowledge structuring
  • What are the key concepts that are at the heart of the module?
  • What are the relationships between the concepts?
  • This marks a shift from dis-embedded facts to inter-related concepts
  • This is how we organise knowledge in the discipline

Task: map the knowledge structure of your module

“Organising knowledge” in the classroom is problematic because knowledge isn’t organised in our brains in the same way that we organise it for students / on a piece of paper. We assign content to discrete categories to make it easier for students to understand / add it to their pre-existing scaffolds, but that’s not how it exists in minds.

Scientific method (our students do a basic physics course in which this method is emphasised, yet they don’t transfer this knowledge to patient assessment):

  1. Observe something
  2. Construct an hypothesis
  3. Test the hypothesis
  4. Is the outcome new knowledge / expected?

Task: create a teaching activity (try to do something different) that is aligned with a major concept in the module, and also includes graduate attributes and learning outcomes. Can I do the poetry concept? What about gaming? Learners are in control of the environment, mastering the task is a symbol of valued status within the group, a game is a demarcated learning activity with set tasks that the learner has to master in order to proceed, feedback is built in, games can be time and resource constrained

The activity should include the following points:

  • Align assessment with outcomes and teaching and learning activities (SOLO taxonomy – Structured Observation of Learning Outcomes)
  • Select a range of assessment tools
  • Justify the choice of these tools
  • Explain and defend marks and weightings
  • Meet the criteria for reliability and validity
  • Create appropriate rubrics

Assessment must be aligned with learning outcomes and modular content. It provides students with opportunities to show that they can do what is expected of them. Assessment currently highlights what students don’t know, rather than emphasising what they can do, and looking for ways to build on that strength to fill in the gaps.

Learning is about what the student does, not what the teacher does.

How do you create observable outcomes?

The activity / doing of the activity is important

As a teacher:

  • What type of feedback do you give?
  • When do you give it?
  • What happens to it?
  • Does it lead to improved learning?

Graduate attributes ↔ Learning outcomes ↔ Assessment criteria ↔ T&L activities ↔ Assessment tasks ↔ Assessment strategy

Assessment defines what students regard as important, how they spend their time and how they come to see themselves as individuals (Brown, 2001; in Irons, 2008: 11)

Self-assessment is potentially useful, although it should be low-stakes

Use a range of well-designed assessment tasks to address all of the Intended Learning Outcomes (ILOs) for your module. This will help to provide evidence to teachers of the students competence / understanding

In general quantitative assessment uses marks while qualitative assessment uses rubrics

Checklist for a rubric:

  • Do the categories reflect the major learning objectives?
  • Are there distinct levels which are assigned names and mark values?
  • Are the descriptions clear? Are they on a continuum and allow for student growth?
  • Is the language clear and easy for students to understand?
  • Is it easy for the teacher to use?
  • Can the rubric be used to evaluate the work? Can it be used for assessing needs? Can students easily identify growth areas needed?

Evaluation:

  • What were you evaluating and why?
  • When was the evaluation conducted?
  • What was positive / negative about the evaluation?
  • What changes did you make as a result of the feedback you received?

Evaluation is an objective process in which data is collected, collated and analysed to produce information or judgements on which decisions for practice change can be based

Course evaluation can be:

  • Teacher focused – for improvement of teaching practice
  • Learner focused – determine whether the course outcomes were achieved

Evaluation be conducted at any time, depending on the purpose:

  • At the beginning to establish prior knowledge (diagnostic)
  • In the middle to check understanding (formative) e.g. think-pair-share, clickers, minute paper, blogs, reflective writing
  • At the end to determine the effectiveness of the course / to determine whether outcomes have been achieved (summative) e.g. questionnaires, interviews, debriefing sessions, tests

Obtaining information:

  • Feedback from students
  • Peer review of teaching
  • Self-evaluation

References

  • Knight (n.d.). A briefing on key concepts: Formative and summative, criterion and norm-referenced assessment
  • Morgan (2008). The Course Improvement Flowchart: A description of a tool and process for the evaluation of university teaching

Developing case studies for holistic clinical education

This is quite a long post. Basically I’ve been trying to situate my current research into a larger curriculum development project and this post is just a reflection of my progress so far. It’s probably going to have big gaps and be unclear in sections. I’m OK with that.

Earlier this week our department had a short workshop on developing the cases that we’re going to use next year in one of our modules. We’re going to try and use cases to develop a set of skills and attitudes that are lacking in our students. These include challenges with (text in brackets are stereotypical student perspectives):

  • Problem solving and clinical reasoning (Tell me what the answer is so that I can memorise it)
  • Critical analysis (Everything I read has the same value)
  • Empathy (The patient is an object I use to develop technical skills)
  • Communication (The use of appropriate professional terminology isn’t important)
  • Groupwork (Assessment is a zero sum game…if you score more than me it bumps me down the ranking in the class, therefore I don’t help you)
  • Knowing vs Understanding (It’s more important for me to know the answer than to understand the problem)
  • Integration of knowledge into practice (What I learn in class is separate to what I do with patients)
  • Integration of knowledge from different domains (I can’t examine a patient with a respiratory problem because I’m on an orthopaedic rotation)
  • Poor understanding of the use of technology to facilitate learning (social networks are for socialising, not learning)

I know it might seem like a bit much to think that merely moving to case-based learning is going to address all of the above, but we think it’ll help to develop these areas in which the students are struggling. The results of my ongoing PhD research project will be helping in the development of this module in the following ways:

  • The survey I began with in 2009 has given us an idea of digital literacy skills of this population, as well as some of the ways in which they learn.
  • The systematic review has helped us to understand some of the benefits and challenges of a blended approach to clinical education.
  • The Delphi study (currently in the second round) has already identified many of the difficulties that our clinicians and clinical supervisors experience in terms of developing the professional and personal attributes of capable and competent students. This study will attempt to highlight teaching strategies that could help to develop the above mentioned problems.
  • I’ve also just finished developing and testing the data capture sheet that I’ll be using for a document analysis of the curriculum in order to determine alignment.
  • Later next year I’ll be conducting an evaluation of the new module, using a variety of methods.

All of the above information is being fed into the curriculum development process that we’re using to shift our teaching strategy from a top-down, didactic approach to a blended approach to teaching and learning. Development of the cases is one of the first major steps we’re taking as part of this curriculum development process. I’ll try to summarise how the cases are being developed and how they’ll be used in the module. This module is called “Applied Physiotherapy” and it’s basically where students learn about the physiotherapy management of common conditions.

In the past, these conditions were divided into systems and taught within those categories e.g. all orthopaedic conditions were covered together. The problem is that this effectively silo’s the information and students see little crossover. In fact, reality is very rarely so conveniently categorised. Patients with orthopaedic conditions may develop respiratory complications as a result of prolonged bed rest. Patients with TB often also present with peripheral neuropathy, as a result of the association of TB with HIV. So, the purpose of the cases is also to integrate different conditions to help students understand the complexity of real-world cases.

In the first term we’ll use 2 very simple cases that each run for 3 weeks. The reason that the cases are simple is that we’re also going to be introducing many new ideas that the students may have little experience with and which are important for participation in the cases e.g. computer workshops for the online environment, concept mapping, group dynamics, presentation skills, etc. The cases will increase in complexity over time as the students feel more comfortable with the process.

Each case will have an overview that highlights the main concepts, learning outcomes, teaching activities, assessment tasks and evaluation components that the case encompasses. The case will be broken up into parts, the number of which will depend on the duration and complexity of the case. After the presentation of each part, the students (in their small groups) will go through this process:

  • What do I know that will help me to solve this problem?
  • What do I think I know that I’m uncertain of?
  • What don’t I know that I need to learn more about?

These questions should help the students develop a coherent understanding of the knowledge they already have that they can build on, as well as the gaps in understanding that they need to fill before they can move on with the case. Each part will involve students allocating tasks that need to be completed before the next session and role allocation is done by each group prior to the introduction of the case. During this process, facilitators will be present within the groups in order to make sure that students have not left out important concepts e.g. precautions and contraindications of conditions.

At the next session, each member of the small groups present to each other within the small groups. The purpose of this is to consolidate what has been learned, clarify important concepts and identify how they’re going to move forward. At the end of each week each small group presents to the larger group. This gives them the opportunity to evaluate their own work in relation to the work of others, make sure that all of the major concepts are included in their case notes, as well as opportunities to learn and practice presentation skills. Students will also be expected to evaluate other groups’ work.

There will be a significant online component to the cases in the form of a social network built on WordPress and Buddypress. We will begin by providing students with appropriate sources that they can consult at each stage of the process. Over time we’ll help them develop skills in the critical analysis of sources so that they begin to identify credibility and authority and choose their own sources. They will also use the social network for collaborative groupwork, communication, and the sharing of resources.

Finally, here are some of the tasks we’re going to include as part of the cases, as well as the outcomes they’re going to measure (I’ve left out citations because this has been a long post and I’m tired, but all of these are backed by research):

  • Concept mapping – determine students’ understanding of the relationships between complex concepts
  • Poetry analysis – development of personal and professional values e.g. compassion, empathy
  • Reflective blogging – development of self-awareness, critical evaluation of their own understanding, behaviours and professional practices
  • Peer evaluation – critical analysis of own and others’ work
  • Case notes – development of documentation skills
  • Presentations – ability to choose important ideas and convey them concisely using appropriate language

This is about where we are at the moment. During the next few months we’ll refine these ideas, as well as the cases, and begin with implementation next year. During my evaluation of the module, I’ll be using the results of the student tasks listed above, as well as interviews and focus groups with students and staff. We’ll review the process in June and make changes based on the results of my, and 2 other, research projects that will be running. We want the curriculum to be responsive to student needs and so we need to build in the flexibility that this requires.

After reading through this post, I think that what I’m saying is that this forms a basic outline of how we’re defining “blended learning” for this particular module. If you’ve managed to make it this far and can see any gaping holes, I’d love to hear your suggestions on how we can improve our approach.

Results of my Delphi first round

I’ve recently finished the analysis of the first round of the Delphi study that I’m conducting as part of my PhD. The aim of the study is to determine the personal and professional attributes that determine patient outcomes, as well as the challenges faced in clinical education. These results will serve to inform the development of the next round, in which clinical educators will suggest teaching strategies that could be used to develop these attributes, and overcome the challenges.

Participants from the first round had a wide range of clinical, supervision and teaching experience, as well as varied domain expertise. Several themes were identified, which are summarised below.

In terms of the knowledge and skills required of competent and capable therapists, respondents highlighted the following:

  • They must have a wide range of technical and interpersonal skills, as well as a good knowledge base, and be prepared to continually develop in this area.
  • Professionalism, clinical reasoning, critical analysis and understanding were all identified as being important, but responses contained little else to further explain what these concepts mean to them.

In terms of the personal and professional attributes and attitudes that impact on patient care and outcomes, respondents reported:

  • A diverse range of personal values that they believe have relevance in terms of patient care
  • These values were often expressed in terms of a relationship, either between teachers and students, or between students and patients
  • Emotional awareness (of self and others) was highlighted

In terms of the challenges that students face throughout their training:

  • Fear and anxiety, possibly as a result of poor confidence and a lack of knowledge and skills, leading to insecurity, confusion and uncertainty
  • Lack of self-awareness as it relates to their capacity to make effective clinical decisions and reason their way through problems
  • A disconnect between merely “providing a service” and “serving”
  • They lack positive and supportive clinical learning environments, have poor role models and often aren’t given the time necessary to reflect on their experiences
  • The clinical setting is complex and dynamic, a fact that students struggle with, especially when it comes to dealing with complexity and uncertainty inherent in clinical practice
  • Students often “silo” knowledge and skills, and struggle to transfer between different contexts
  • Students struggle with the “hidden culture” of the professional i.e. the language, values and norms that clinicians take for granted

These results are not significantly different from the literature in terms of the professional and personal attributes that healthcare professionals deem to be important for patient outcomes.

The second round of the Delphi is currently underway and will focus on the teaching  strategies that could potentially be used to develop the attitudes and attributes highlighted in the first round.

Twitter Weekly Updates for 2011-11-21

  • Papert: “…the practice of segregating children by age into “grades” will be seen as…old-fashioned, and inhumane” http://t.co/pvXVRayG #
  • Great way to learn physics http://t.co/oNRel2Qm #
  • Scientists invent lightest material on Earth. What now? http://t.co/i1BF632n via @zite #
  • The Top 10+1 apps in the Mendeley-PLoS Binary Battle! http://t.co/oVT6cva8 via @zite #
  • Dave Cormier: Explaining Rhizomatic Learning to my five year old. http://t.co/R7Pjrdez via @zite #
  • Microsoft’s table-sized tablet Surfaces for pre-order http://t.co/UDCeAq7D via @zite? Cool health-related concept image at the end #
  • @mendeley_com I love the ipad app but hate that I can’t annotate / highlight text. Any plans for that functionality in the lite version? #
  • How odd that #Mendeley isn’t @mendeley. Made an assumption earlier today with a tweet (embarrassed face) #
  • The really basic skill today is the skill of learning http://t.co/yzfBZHcx #
  • @mendeley I love the ipad app but hate that I can’t annotate / highlight text. Any plans for that functionality in the lite version? #
  • ECAR National Study of Undergraduate Students and Information Technology, 2011 Report | EDUCAUSE http://t.co/luaja5Pl #
  • Just published the 2nd round of my survey on clinical education. If you teach healthcare students, please respond at http://t.co/mIm3l9H8 #
  • @whataboutrob Could probably make that work 🙂 #

Graphically representing a curriculum

Schematic map of the Milky Way

I’ve been a bit quiet on the blog lately, owing to the fact that I’ve been putting a lot of time into the next phase of my PhD. This post is in part an attempt to summarise and try to make sense of what’s going on there, as well as to assuage my feeling of guilt at not having posted for a while.

In terms of my research progress I’m currently running a Delphi study among clinicians and clinical educators, as well as a document analysis of the curriculum. The Delphi is trying to identify the personal and professional attributes that clinicians believe are important in terms of positively impacting patient outcomes, the relevant teaching activities that could be used to develop and assess these attributes, and any appropriate technologies that might facilitate the above teaching and learning activities.

I’m busy with the second round of the Delphi study (I’ll post the main results of the first shortly) and will begin analysing the curriculum documentation soon. The combination of these two projects will (hopefully) give me enough data to determine how we need to change the curriculum in order to better develop the attributes we’ve identified.

As part of that process I’m starting to look at curriculum mapping. What I’m struggling with at the moment is to figure out how best to represent what I’m learning as far as what the curriculum looks currently like, and how we need to change it. These are the difficulties I’ve come up with:

  • The learning process isn’t linear, which cuts out a narrative representation
  • A curriculum is organised by many things e.g. outcomes, content, teaching approach, assessment tasks, time, space, etc. How do you emphasise all of these (and their relationships) while keeping some measure of sanity?
  • There are many interrelated concepts i.e. multiple connections, nested connections, linear and non-linear components, etc. all of which makes a mindmap difficult to work with (mindmaps are usually hierarchical, and a curriculum presented as a hierarchy would be necessarily simplistic)
  • A Gantt chart might be useful to show how activities or projects progress over time, but it doesn’t have much scope for depth
  • Tabular representation doesn’t allow you to expand / collapse sections, or add detailed notes. It also allows only very simple, one-to-one connections e.g. content over time but not time, content and outcomes.
  • At the moment I seem to have settled on CmapTools for concept mapping. It’s not the ideal solution but it seems to be the one that enables most of what I need (see list below)

As much as I’ve read around curriculum mapping I haven’t yet found a solution that helps me to address everything that I think I need. I know that I probably won’t be able to find a tool that enables all of the following, but this is what I’d like to be able to do:

  • Create relationships between concepts e.g. outcomes, teaching activity, assessment task, etc.
  • Emphasise the nature of the relationships
  • Annotate concepts and relationships
  • Expand and collapse sections i.e. see the big picture (e.g. national exit level outcomes) as well as drill-down into the details (e.g. lesson plans)
  • I should be able to show a process over time i.e. workflow should be built in
  • I’d like the ability to input more data over time, and delete outdated content
  • I’d like to be able to detect redundancy, inconsistency and omissions (of content, tasks, outcomes, etc.)
  • It’d be great if it was collaborative
  • Must be able to review vertical (subjects between years) and horizontal (between subjects in the same year) alignment, as well as the sequencing of activities
  • Define a shared vocabulary for use in our department (we often use different terms for the same thing, creating confusion)

I’ve also been also looking into other domains for ideas that will help me to get a better understanding of graphical modelling to represent complex information. One example is Unified Modelling Language (UML), a general purpose modelling language that is used to represent the various facets of objects and systems in computer science. It is used to “…specify, visualize, modify, construct and document the artefacts of…a system”. It also offers a standard way to visualise the different elements of that system e.g. activities, actors, processes, components, etc. I’m still holding out for a modelling tool from another domain (besides education) that might serve my purposes.

During the above-mentioned process, I also had fun looking at a curriculum as a computer platform. A computer platform includes:

  • The operating system (OS), which is basically a set of instructions for what to do in certain situations, including task scheduling and resource allocation. I think that this is a useful way to think about the structure of a curriculum i.e. what should happen, when it should happen, who is responsible for it, etc.
  • Architecture (hardware) that includes the CPU, data bus, chipsets, graphics cards, motherboard, sound card. CPU is concerned with how programmes access memory. The physical structures that enable the manifestation of the curriculum.
  • Frameworks are collections of software libraries that contain generic functionality that can be modified by within certain constraints. Frameworks allow developers to spend time working on useful features rather than having to write code for low level functionality. Within the curriculum there are modules that share generic features e.g. problem solving. A way of assessing whether or not a student can solve problems is a generic “framework” that can be modified slightly to be used in other modules. Why should every lecturer have to re-create the same libraries of tools in order to assess the same thing in a different context?
  • Programming languages that use a standardised set of vocabulary and grammar to create a set of instructions that the OS will understand.
  • The user interface (UI) that allows a user to interact with the computer and its peripherals. This is the most visible part of the platform, and often the part that draws the most attention. This is the part of the curriculum that everyone can see. The handouts, the lecture, the assessment tasks i.e. this is what the students and lecturers  use to interact with the curriculum. Is is also the part that people will love or hate. No matter how “good” the underlying structure is, the student engages with the UI and most people in higher education haven’t caught onto the idea that “pretty is a feature“.

Schematic transit maps and Venn diagrams might also be useful in terms of thinking about curriculum mapping in a different way. I’m inclined to think that a combination of all of the above will be an interesting experiment.

I guess the biggest issue I’m having is trying to figure out a way to show how we can go from what we have to what we want, from a very high to very low level. It’s harder than I thought it’d be…

TEDxStellenbosch: designing spaces

A few months ago I attended TEDxStellenbosch at Spier wine farm near Stellenbosch. It was one of the better TEDx events I’ve been to during past few years and I enjoyed it immensely. During the day I re-tweeted comments from other participants, mainly as a record of speakers and the comments that resonated with me. As with most TEDx events, it was such a whirlwind that a lot of what happened was gone before I’d had an opportunity to reflect on it. During the event, attendees were asked to doodle their solutions to problems on large whiteboards placed throughout the venue (see pics). Anyway, I just wanted to mention it here.

 

Twitter Weekly Updates for 2011-10-24

  • Daily Papert http://t.co/IzTvBxZk. What is the role of the teacher in society? #
  • Nudity, Pets, Babies, and Other Adventures in Synchronous Online Learning http://t.co/pRyPVvzU #
  • If you are a clinician who supervises or teaches healthcare students, please consider completing my survey at http://t.co/x1MXf3AJ #
  • The hierarchical structure of an ePortfolio http://t.co/65gIpn5Y. If your e-portfolio is structured hierarchically, you’re doing it wrong #
  • @mpascoe if they don’t perceive that the class has value, then it doesn’t, at least not for them. Forced attendance won’t change that #
  • Don’t offer students grades in return for attendance in your classes. Just be interesting #
  • @suhaifa hey Su, it was a great day to be out and about. Glad that you and @jacquesmillard could make it #

Blended learning in clinical education (AMEE presentation)

This is the presentation that I gave at the AMEE conference earlier today. It’s the results of a systematic literature review I did as part of my PhD, where I looked at the use of blended learning in clinical education. The abstract doesn’t give much information owing to the fact that I had to be very brief with my submission. The presentation is (a little) more detailed.

Here’s the abstract:

Here’s the presentation (better to view at Prezi.com, space is limited here):

Posted to Diigo 07/15/2011

    • For some time now the ‘slow food’ movement has questioned the value of ‘fast food’ and called for a return to more authentic modes of cooking and eating
    • For some time now the ‘slow food’ movement has questioned the value of ‘fast food’ and called for a return to more authentic modes of cooking and eating
    • an appreciation for the value of taking more time and care to make something
    • appreciation for the value of taking more time and care to make something
    • For some time now the ‘slow food’ movement has questioned the value of ‘fast food’ and called for a return to more authentic modes of cooking and eating
    • One of the traps which thesis writers fall into is over thinking everything, which can be solved by Fast. But recently I’ve started to think about Slow and how it might apply to academic work
    • you must absorb information and engage with other people’s ideas. In a way, doing a thesis is like a long, slow conversation with these ideas and things
    • A thesis is of you, but it has many other parents: scholars, research participants, archives test tubes to name a few. Consciously thinking about this sense of writing ‘taking control’ of you can be helpful
    • developing your relationship with the literatures who accompany your thesis takes time. While I can and do encourage you to ‘read like a mongrel’ (fast and furious), Fast reading is really a way of finding out which pieces and authors are worth investing time in
    • Deep understanding of literature needs repeated reading and thinking. as well as writing. In other words, a Slow conversation with the ideas
    • if applied correctly, a bit of Slow will ensure that your thesis has more flavour than most.
    • What if losing control is an essential part of writing a thesis? Realising you have lost control forces you to slow down. When you stop talking so much, you can listen better. Maybe then your thesis will tell you what it needs

Twitter Weekly Updates for 2011-06-20