Tag: outcomes

Categories
altphysio curriculum

altPhysio | Describing a novel curriculum

I’ve spent the last 2 weeks or so trying to get my head around what a new curriculum might look like in practical terms, largely to the detriment of everything else that I’m supposed to be doing. It seems to be a harder problem than I anticipated (or maybe I’m just missing something). One of the main issues I’m struggling with is how to describe the new curriculum if we abandon the idea of discrete collections of facts (i.e. modules) that encourage students to compartmentalise their learning. Consider the following:

  • If you break the concepts down into very small, granular pieces (making it easy to link concepts and to generally describe the curriculum), it’s hard to create a structure that allows for independent pathways through the concepts. You’re more inclined to provide a structured, linear path through what (the lecturer decides) should be known. While this approach does provide security for students navigating unfamiliar territory and also makes it easier to administrate the curriculum, it encourages students to split their thinking too. This is a “neuro” test, this is a “resp patient”, this is a “paeds block”, etc. This is pretty much how our current curricula are represented i.e. by grouping related concepts into collections called modules / papers.
  • If you go the other direction and organise the curriculum into large, complex, relatively undefined projects that allow students the space to create their own learning pathways, you lose the ability to accurately describe what concepts are presented in those relatively undefined projects. While this approach has a greater chance of leading to self-directed and lifelong learning, critical and creative thinking, and comfort with ambiguity, it also means that the curriculum becomes harder to describe. It’s not like in the first approach where you can be certain that “informed” consent” is included in the “Ethics” module. What project explicitly includes the concept of consent? It’s hard to tell from the name of the project alone.

A module (or paper) describes – in it’s name – what related concepts can reasonably be expected to arise in that module. You know what ideas are going to come up when you’re thinking about the Neuro module. A project – in it’s name – might describe at some high level what to expect but it cannot describe with any accuracy everything that can reasonably be expected to emerge in the project. The one thing that gives me some solace is knowing that in actual fact, we struggle to describe our own curriculum anyway, even with our very granular, silo’d structure. For example, the concept of CVA comes up in so many places and in so many contexts that it’s really hard to say “where” it exists in the curriculum.

The only solution I can think of is to take the approach that, once designed – with all major concepts included in the projects – you trust the curriculum to run it’s course and don’t spent too much time worrying where a certain concept is embedded. You know that consent is included somewhere in some project and when someone asks where exactly it is, you simply search for it?

Note: I’m assuming that the content of the curriculum would stay the same. In other words I’m still trying to teach the same “stuff”,  I’m just trying to find a way to package it differently. Dave Nicholls has suggested a different approach where we begin with a question around what the curriculum is for. What is it for? This would naturally lead us to think about the curriculum in a very different – and far more interesting – way. I need to spend some time with this idea.

On a side note, I came across this paper while looking into the challenges of describing alternative curriculum approaches: Moore, A. & House, P. (1973). The open access curriculum—an approach to individualization and student involvement. Science Education, 57(2):215-218. Ironically the paper is not available with an open access license so I couldn’t get the full text. The paper – from 1973 – articulates what I think are some pretty important approaches that a new physiotherapy curriculum should include:

  1. Multiple entry points to each large body of content, usually beginning at the exploratory level and proceeding toward in-depth facts (I really love this idea and will be looking into it in some depth for the course)
  2. Guidelines for student study that facilitate a self-commitment to fully personalised projects (Obviously I’m a fan of the project approach to learning, so this resonates with me as well)
  3. Students assuming direct responsibility for a significant part of their own education
  4. Differentiated teacher roles
  5. Both the teachers and the students helping to define and implement the meaning of the concept of open access
  6. Making the assumption that all students will succeed
Categories
education students

Assessment in an outcomes based curriculum

I attended a seminar / short course on campus yesterday, presented by Prof. Chrissie Boughey from Rhodes University. She spoke about the role of assessment in curriculum development and the link between teaching and assessing. Here are the notes I took.

Assessment is the most important factor in improving learning because we get back what we test. Therefore assessment is acknowledged as a driver of the quality of learning.

Currently, most assessment tasks encourage the reproduction of content, whereas we should rather be looking for the production of new knowledge (the analyse, evaluate and create parts of Bloom’s top level cognitive processes).

Practical exercise: Pick a course / module / subject you currently teach (Professional Ethics for Physiotherapists), think about how you assess it (Assignment, Test, Self-study, Guided reflection, Written exam) and finally, what you think you’re assessing (Critical thinking / Analysis around ethical dilemmas in healthcare, Application of theory to clinical practice). I went on to identify the following problems with assessment in the current module:

  • I have difficulty assigning a quantitative grade to what is generally a qualitative concept
  • There is little scope in the current assessment structure for a creative approach

This led to a discussion about formal university structures that determine things like, how subjects will be assessed, as well as the regimes of teaching and learning (“we do it this way because this is the way it’s always been done”). Do they remove your autonomy? It made me wonder what our university official assessment policy is.

Construct validity: Are we using assessment to asses something other than what we say we’re assessing? If so, what are we actually assessing?

There was also a question about whether or not we could / should asses only what’s been formally covered in class. How do you / should you asses knowledge that is self-taught? We could for example, measure the process of learning, rather than the product. I made a point that in certain areas of what I teach, I no longer assign a grade to an individual peice of work and rather give a mark for the progress that the student has made, based on feedback and group discussion in that area.

Outcomes based assessment / criterion referenced assessment

  1. Uses the principle of ALIGNMENT (aligning learning outcomes, passing criteria, assessment)
  2. Is assessing what students should be able to do
  3. “Design down” is possible when you have standardised exit level outcomes (we do, prescribed by the HPCSA)
  4. The actual criteria are able to be observed and are not a guess at a mental process, “this is what I need to see in order to know that the student can do it”
  5. Choosing the assessment tasks answers the question “How will I provide opportunities for students to demonstrate what I need to see?” When this is the starting point, it knocks everything else out of alignment
  6. You need space for students / teachers to engage with the course content and to negotiate meaning or understanding of the course requirements, “Where can they demonstrate competence?”

Criteria are negotiable and form the basis of assessment. They should be public, which makes educators accountable.

When designing outcomes, the process should be fluid and dynamic.

Had an interesting conversation about the priviliged place of writing in assessment. What about other expressions of competence? Since speech is the primary form of communication (we learn to speak before we learn to write), we find it easier to convey ideas through conversation, as it includes other cues that we use to construct meaning. Writing is a more difficult form because we lack visual (and other) cues. Drafting is one way that constructing meaning through writing could be made easier. The other point I thought was interesting was that academic writing is communal (drafting, editors, reviewers all provide a feedback mechanism that isn’t as fluid as speech, but is helpful nonetheless), but we often don’t allow students to write communally.

Outcomes based assessment focusses on providing students with multiple opportunities to practice what they need to do, and the provision of feedback on that practice (formative). Eventually, students must demonstrate achievement (summative).

We should only assign marks when we evaluate performace against the course outcomes.

Finally, in thinking about the written exam as a form of assessment, we identified these characteristics:

  • It is isolated and individual
  • There is a time constraint
  • There is pressure to pass or fail

None of these characteristics are present in general physiotherapy practice. We can always ask a colleage / go to the literature for assistance. There is no constraint to have the patient fully rehabilitated by any set time, and there are no pass or fail criteria.

If assessment is a method we use to determine competence to perform a given task, and the way we asses isn’t related to the task physio students will one day perform, are we assessing them appropriately?

Note: the practical outcomes of this session will include the following:

  • Changing the final assessment of the Ethics module from a written exam to a portfolio presentation
  • Rewriting the learning outcomes of the module descriptors at this year’s planning meeting
  • Evaluating the criteria I use to mark my assignments to better reflect the module outcomes
Categories
Uncategorized

Principles of good assessment

I attended an assessment and learning workshop today and while the presentations were informative, I just wanted to highlight the principles of good assessment taken from my faculty’s assessment policy.  Since I don’t have a background in education, guidelines like these are incredibly useful when creating assessments for students.

  1. Responsibility for assessment – the module co-ordinator is responsible for designing the assessment and mark allocation.
  2. Assessing against outcomes – performance should be measured against pre-determined expectations of achievement (learning outcomes).
  3. Assessment criteria – the expectations of the assessment, including the specific criteria of judgement, should be available to students to ensure transparency.
  4. Validity and appropriateness – the assessment methods and tasks should accurately match what is being assessed (knowledge, understanding, content, skills, behaviour, etc.)
  5. Authenticity of evidence – measures should be taken to ensure that the evidence produced by the student is attributable to the student.  With group work, the lecturer must verify that each student has made a fair contribution.
  6. Formative and summative assessment – assessment should judge students’ performance (summative), as well as provide feedback to enhance learning (formative), although not simultaneously.  Students should be aware of whether they are being assessed formatively or summatively at each assessment.
  7. Continuous assessment – should have a strong formative focus and be undertaken over the course of the module.

For me, just knowing about these guidelines has already made a significant difference in how I approach the assessment of students.  Clearly, it’s not enough to re-use old test papers and merely change the scenarios.  We need to make sure that we’re actually testing what we set out to test, as well as linking the assessment to the curriculum and learning outcomes.

One other point I want to mention is a comment made by one of the presenters, regarding the importance of testing students interpretation of the course content.  This is one way to make sure that students actually understand what they’re writing, rather than just regurgitating bullet points.

Categories
Uncategorized

Teaching and learning workshop outcomes

Two weeks ago I attended a teaching and learning workshop on campus that was pretty interesting. I just received an email from the coordinator highlighting the following key points that were raised:

  1. Our students…experience many social problems and this could be regarded as a barrier to their learning.
  2. As lecturers we have to make active use of the support structures on campus when we are constructing our curricula, which means we have to involve our librarians and the individuals in the Centre for Student Support Services and other support structures on campus in an active
    process of collaboration. This could mean that we involve these individuals in meetings, making explicit what we require from them and they will in turn make their expectations and needs explicit.
  3. Lecturers mentioned that we should consider what we as teachers in higher education could do to improve the teaching and learning on campus rather than focus on the deficits that students have.
  4. We need to coordinate the academic and support structures on campus so that we can provide a holistic higher education experience to our students.
  5. The academic programme, support structures and social activities should add value to students’ experiences so that when they graduate they are confident, competent and independent
    thinkers.
  6. We admit students with different language backgrounds and different mother tongues and we should look at ways of using this as a resource.

For me, the main benefit of attending the workshop was finding out just how many resources are available to the students. Whether or not they’ll make use of them is another story 🙂