“Curriculum renewal to achieve graduate learning outcomes: The challenge of assessment”
Prof Simon Barrie, Director of T&L, University of Sydney
Last week I had the opportunity to attend a presentation on graduate attributes and curriculum renewal by Prof Simon Barrie. The major point I took away from it was that we need to be thinking about how to change teaching and assessment practices to make sure that we’re graduating the kinds of students we say we want to. Here are the notes I took.
Assessment is often a challenge when it comes to curriculum renewal. The things that are important (e.g. critical thinking) are hard to measure. Which is why we often don’t even try.
Curriculum is more powerful than simply looking at T&L, although bringing in T&L is an essential aspect of curriculum development. Is curriculum renewal just “busy bureaucracy”? It may begin with noble aims but it can degenerate into managerial traps. Curriculum renewal and graduate attributes (GA) should be seen as part of a transformative opportunity.
GA are complex “things” and need to be engaged with in complex ways
GA should be focused on checking that higher education is fulfilling it’s social role. UNESCO World Declaration on Higher Education: “higher education has given ample proof of it’s viability over the centuries and of its ability to change and induce change and progress in society”.
GA should be a starting point for a conversation about higher education. If they exist simply as a list of outcomes, then they haven’t achieved their purpose.
How is an institution’s mission embodied in the learning experiences of students and teaching experiences of teachers?
What is the “good” of university?
- Personal benefit – work and living a rich and rewarding life
- Public benefit – economy and prosperity, social good
- The mix of intended “goods” can influence our descriptions of the sorts of graduates that universities should be producing and how they should be taught and assessed. But, the process of higher education is a “good” in itself. The act of learning can itself be a social good e.g. when students engage in collaborative projects that benefit the community.
Universities need to teach people how to think and to question the world we live in.
If you only talk to people like you about GA, you develop a very narrow perspective about what they are. Speaking to more varied people, you are exposed to multiple set of perspectives, which makes curriculum renewal much more powerful. We bring our own assumptions to the conversation. Don’t trust your assumptions. Engage with different stakeholders. Don’t have the discussion around outcomes, have it around the purpose and meaning of higher education.
A framework for thinking about GA: it is complex and not “one size fits all”. Not all GA are at the same “level”, there are different types of “understand”, which means different types of assessment and teaching methods.
- Precursor: approach it as a remedial function, “if only we got the right students”
- Complementary: everybody needs “complementary” skills that are useful but not integral to domain-specific knowledge
- Translation: applied knowledge in an intentional way, should be able to use knowledge, translating classroom knowledge into real world application, changing the way we think about the discipline
- Enabling: need to be able to work in conditions of uncertainty, the world is unknowable, how to navigate uncertainty, develop a way of being in the world, about openness, going beyond the discipline to enable new ways of learning (difficult to pin down and difficult to teach, and assess, hard to measure)
The above ways of “understanding” are all radically different, yet many are put on the same level and taught and assessed in the same way. Policies and implementation needs to acknowledge that GA are different.
Gibbons: knowledge brought into the world and made real
The way we talk about knowledge can make it more or less powerful. Having a certain stance or attitude towards knowledge will affect how you teach and assess.
What is the link, if any, between the discipline specific lists and institutional / national higher education lists?
What are the assessment tasks in a range of disciplines that generate convincing evidence of the achievement of graduate learning outcomes? What are the assurance processes trusted by disciplines in relation to those assessment tasks and judgments? Assessing and assuring graduate learning outcomes (AAGLO project). Here are the summary findings of the project.
Assessment for learning and not assessment of learning.
Coherent development and assessment of programme-level graduate learning outcomes requires an institutional and discipline statement of outcomes. Foundation skills? Translation attributes? Enabling attributes and dispositions? Traditional or contemporary conceptions of knowledge?
Assessment not only drives learning but also drives teaching.
- Communication skills – Privileged
- Information literacy – Privileged
- Research and inquiry – Privileged
- Ethical social professional understandings – Ignored (present in the lists, but not assessed)
- Personal intellectual autonomy – Ignored (present in the lists, but not assessed)
Features of effective assessment practices:
- Assessment for learning
- Interconnected, multi-component, connected to other assessment, staged, not isolated
- Authentic (about the real world), relevant (personally to the student), roles of students and assessors
- Standards-based with effective communication of criteria, assessment for GA can’t be norm-referenced, must be standards-based
- Involve multiple decision makers – including students
- Programme level coherence, not just an isolated assessment but exists in relation to the programme
The above only works as evidence to support learning if it is coupled with quality assurance
- Quality of task
- Quality of judgment (calibration prior to assessment, and consensus afterwards)
There is a need for programme-level assessment. Assessment is usually focused at a module level. There’s no need to assess on a module level if your programme level is effective. You can then do things like have assessments that cross modules and are carried through different year levels.
How does a university curriculum, teaching and learning effectively measure the achievement of learning outcomes? In order to achieve certain types of outcomes, we need to give them certain types of learning experiences.
Peter Knights “wicked competencies”: you can’t fake wickedness – it’s got to be the real thing, messy, challenging and consequential problems.
The outcomes can’t be used to differentiate programmes, so use teaching and learning methods and experiences to differentiate.
Stop teaching content. Use content as a framework to teach other things e.g. critical thinking, communication, social responsibility
- Set the right (wicked) goals collaboratively
- Make a signature pedagogy for complex GA part of the 5 year plan
- Develop policies and procedures to encourage and reward staff
- Identify and provide sources of data that support curriculum renewal, rather than shut down conversations about curriculum
- Provide resources and change strategies to support curriculum renewal conversations
Teaching GA is “not someone else’s problem”, it needs to be integrated into discipline-specific teaching.
Be aware that this conversation is very much focused on “university” or “academic” learning, and ignores the many different ways of being and thinking that exist outside the university. How is Higher Education connecting with the outside world? Is there a conversation between us and everyone else?
We try to shape students into a mold of what we imagine they should be. We don’t really acknowledge their unique characteristics and embrace their potential contribution to the learning relationship?
We (academics) are also often removed from where we want our students to be. Think about critical thinking, inquiry-based learning, collaboration, embracing multiple perspectives. Is that how we learn? Our organisational culture drives us away from the GA we say we want our students to have.
- Barrie, S. (2004). A research-based approach to generic graduate attributes policy. Higher Education Research and Development, 23(3).
- Barrie, S. Rethinking generic graduate attributes.
- Barrie, S. Understanding what we mean by the generic attributes of graduates.