Using the web to empower agents of change: Presentation at The Network – TUFH conference

These are the slides from the presentation I gave at The Network: Towards Unity for Health conference in Fortaleza, Brazil (2014).

The talk looked at how we’re trying to prepare health professional students for an increasingly complex health system, but we’re still using teaching methods that originated centuries ago. I ask questions about how we can change teaching practices to take into account the characteristics we expect of our graduates. I discussed the importance of taking a critical stance towards the implementation of technological solutions, and to be careful of making assumptions about the use of technology to solve all problems.

Simon Barrie presentation on Graduate Attributes

“Curriculum renewal to achieve graduate learning outcomes: The challenge of assessment”
Prof Simon Barrie, Director of T&L, University of Sydney

Last week I had the opportunity to attend a presentation on graduate attributes and curriculum renewal by Prof Simon Barrie. The major point I took away from it was that we need to be thinking about how to change teaching and assessment practices to make sure that we’re graduating the kinds of students we say we want to. Here are the notes I took.


Assessment is often a challenge when it comes to curriculum renewal. The things that are important (e.g. critical thinking) are hard to measure. Which is why we often don’t even try.

Curriculum is more powerful than simply looking at T&L, although bringing in T&L is an essential aspect of curriculum development. Is curriculum renewal just “busy bureaucracy”? It may begin with noble aims but it can degenerate into managerial traps. Curriculum renewal and graduate attributes (GA) should be seen as part of a transformative opportunity.

GA are complex “things” and need to be engaged with in complex ways

GA should be focused on checking that higher education is fulfilling it’s social role. UNESCO World Declaration on Higher Education: “higher education has given ample proof of it’s viability over the centuries and of its ability to change and induce change and progress in society”.

GA should be a starting point for a conversation about higher education. If they exist simply as a list of outcomes, then they haven’t achieved their purpose.

How is an institution’s mission embodied in the learning experiences of students and teaching experiences of teachers?

What is the “good” of university?

  • Personal benefit – work and living a rich and rewarding life
  • Public benefit – economy and prosperity, social good
  • The mix of intended “goods” can influence our descriptions of the sorts of graduates that universities should be producing and how they should be taught and assessed. But, the process of higher education is a “good” in itself. The act of learning can itself be a social good e.g. when students engage in collaborative projects that benefit the community.

Universities need to teach people how to think and to question the world we live in.

If you only talk to people like you about GA, you develop a very narrow perspective about what they are. Speaking to more varied people, you are exposed to multiple set of perspectives, which makes curriculum renewal much more powerful. We bring our own assumptions to the conversation. Don’t trust your assumptions. Engage with different stakeholders. Don’t have the discussion around outcomes, have it around the purpose and meaning of higher education.

A framework for thinking about GA: it is complex and not “one size fits all”. Not all GA are at the same “level”, there are different types of “understand”, which means different types of assessment and teaching methods.

  • Precursor: approach it as a remedial function, “if only we got the right students”
  • Complementary: everybody needs “complementary” skills that are useful but not integral to domain-specific knowledge
  • Translation: applied knowledge in an intentional way, should be able to use knowledge, translating classroom knowledge into real world application, changing the way we think about the discipline
  • Enabling: need to be able to work in conditions of uncertainty, the world is unknowable, how to navigate uncertainty, develop a way of being in the world, about openness, going beyond the discipline to enable new ways of learning (difficult to pin down and difficult to teach, and assess, hard to measure)

The above ways of “understanding” are all radically different, yet many are put on the same level and taught and assessed in the same way. Policies and implementation needs to acknowledge that GA are different.

Gibbons: knowledge brought into the world and made real

The way we talk about knowledge can make it more or less powerful. Having a certain stance or attitude towards knowledge will affect how you teach and assess.

What is the link, if any, between the discipline specific lists and institutional / national higher education lists?

The National GAP – Graduate Attribute Project

What are the assessment tasks in a range of disciplines that generate convincing evidence of the achievement of graduate learning outcomes? What are the assurance processes trusted by disciplines in relation to those assessment tasks and judgments? Assessing and assuring graduate learning outcomes (AAGLO project). Here are the summary findings of the project.

Assessment for learning and not assessment of learning.

Coherent development and assessment of programme-level graduate learning outcomes requires an institutional and discipline statement of outcomes. Foundation skills? Translation attributes? Enabling attributes and dispositions? Traditional or contemporary conceptions of knowledge?

Assessment not only drives learning but also drives teaching.

  • Communication skills – Privileged
  • Information literacy – Privileged
  • Research and inquiry – Privileged
  • Ethical social professional understandings – Ignored (present in the lists, but not assessed)
  • Personal intellectual autonomy – Ignored (present in the lists, but not assessed)

Features of effective assessment practices:

  • Assessment for learning
  • Interconnected, multi-component, connected to other assessment, staged, not isolated
  • Authentic (about the real world), relevant (personally to the student), roles of students and assessors
  • Standards-based with effective communication of criteria, assessment for GA can’t be norm-referenced, must be standards-based
  • Involve multiple decision makers – including students
  • Programme level coherence, not just an isolated assessment but exists in relation to the programme

The above only works as evidence to support learning if it is coupled with quality assurance

  • Quality of task
  • Quality of judgment (calibration prior to assessment, and consensus afterwards)
  • Confidence

There is a need for programme-level assessment. Assessment is usually focused at a module level. There’s no need to assess on a module level if your programme level is effective. You can then do things like have assessments that cross modules and are carried through different year levels.

How does a university curriculum, teaching and learning effectively measure the achievement of learning outcomes? In order to achieve certain types of outcomes, we need to give them certain types of learning experiences.

Peter Knights “wicked competencies”: you can’t fake wickedness – it’s got to be the real thing, messy, challenging and consequential problems.

The outcomes can’t be used to differentiate programmes, so use teaching and learning methods and experiences to differentiate.

Stop teaching content. Use content as a framework to teach other things e.g. critical thinking, communication, social responsibility

5 lessons:

  1. Set the right (wicked) goals collaboratively
  2. Make a signature pedagogy for complex GA part of the 5 year plan
  3. Develop policies and procedures to encourage and reward staff
  4. Identify and provide sources of data that support curriculum renewal, rather than shut down conversations about curriculum
  5. Provide resources and change strategies to support curriculum renewal conversations

Teaching GA is “not someone else’s problem”, it needs to be integrated into discipline-specific teaching.

Be aware that this conversation is very much focused on “university” or “academic” learning, and ignores the many different ways of being and thinking that exist outside the university. How is Higher Education connecting with the outside world? Is there a conversation between us and everyone else?

We try to shape students into a mold of what we imagine they should be. We don’t really acknowledge their unique characteristics and embrace their potential contribution to the learning relationship?

We (academics) are also often removed from where we want our students to be. Think about critical thinking, inquiry-based learning, collaboration, embracing multiple perspectives. Is that how we learn? Our organisational culture drives us away from the GA we say we want our students to have.


Structure of a curriculum

We had the first of our curriculum review meetings earlier this week, with the intention of evaluating the alignment of our module outcomes (using our module descriptors) and the course content (using the course outline). The plan is that we will spend time in small groups reviewing components of the curriculum, and at each meeting, look at one part in the context of the whole thing. Here’s the structure of the curriculum as we understand it:

We’ve already started to identify inconsistencies between the 2 documents within modules. Here are a few issues that were raised:

  • The module descriptor is sometimes too vague to be of any use in determining whether the content is aligned to it
  • The course outlines often lack structure and would be difficult to use in evaluating the module content
  • We’re not sure if the module objectives are aligned with the HEQF level outcomes
  • There’s no evidence of graduate attribute integration in most of the module descriptors

Once we’ve finished looking at the module outcomes and course outlines to highlight the gaps, we’ll address those issues before moving onto another component, probably the link between what we teach and how we teach it.

I was struck at how complex the process is. It’s easy to draw diagrams outlining a method but when you actually get down to it, these are hard problems to solve.

Twitter Weekly Updates for 2010-07-26

SAAHE keynote – What generic skills do students bring with them?

This is the presentation from Professor Debbie Murdoch-Eaton.

What impacts do we have on students life ambitions?

Workers with general / transferable skills are better placed to succeed in a global knowledge economy. The skills need not be specific to the discipline

The attributes are not only about economic drivers i.e. getting a job…they are also about enabling people to be more successful in communities / life

We do need to develop specific skills that are course specific, but also more generic skills like communication and interpersonal skills, higher order reasoning, critical thinking, ability to use tech, etc.

Why do you go to university, if not to train the mind?

There needs to be a social agenda within the institution in terms of implementing generic skills (or graduate attributes, depending on who you’re reading)

Are clinical skills generic? If so, which ones? Maybe; enterprise, management, leadership, probity, altruism

Transferability is a key skill, helps to contextualise practice

Students are coming into UK HE with lower technical and numeracy skills, written presentation, and selecting and utilising information, but increased practice in IT skills, stress management

No change in student’s ability to manage their own learning (working with others, seeking and giving feedback, teamwork, taking responsibility for own learning), presentation skills (verbal communication skills, essay writing), time management and self-organisation

Does any of this matter? Might be course dependent. It has implications for course / curriculum design and student support

Huge gap between UK and SA student generic skills (e.g. email use, managing own learning). Demonstrated how longitudinal studies of student generic skills on entering HE can be used to make decisions about curriculum design for those students.

Is the curriculum additive (additional to the discipline) or transformative?

Generic skills can be developed by being remedial (by identifying students with deficits in skills) or associated (for all students, running alongside normal curriculum but additional and separate), or part of discipline content and process, engagement / participatory

Integrating lifelong learning into the curriculum should develop explicit generic skills that incorporate structured opportunities to practice those skills

Skills can be inferred from behaviour, which is subjective and we need to be aware that the observer (teacher) has a vested interest into what skills are being inferred

“Those who are least able are also least able to self-assess accurately”

Generic skills must be embedded into the discipline’s teaching methods rather than being bolted on

“We cannot teach science that is as yet undiscovered, nor can we forecast it’s future implications”

We have to educate doctors who are capable of adaptation and change