Category: Assessment
-
In Beta podcast: Assessment and learning
http://inbetaphysio.com/2023/06/29/31-assessment-and-learning/ In this conversation, Ben and I had discuss the assessment process, linking it to broader themes of learning, curriculum design, and student experience. We talk about the centralisation of assessment and explore the tensions between institutional control and the autonomy of teachers. We discuss student satisfaction and the influence of risk aversion in educational…
-
When an unstoppable force meets an immovable object
TL;DR (generated by Claude, lightly edited by me). The rise of abundant expertise in the form of generative AI questions the university monopoly on expertise provision and validation. Leadership in the creative deployment of AI for learning, teaching, and assessment will require a change in mindset and a shift towards a new paradigm, which universities…
-
Don’t use assessment design to solve the ChatGPT problem…
Note: These are my first thoughts on this idea, so I may not be articulating this well. Feel free to tell me how wrong I am in the comments 🙂 Don’t use assessment design to solve the ChatGPT problem. Use ChatGPT to solve the assessment design problem. I’m seeing a lot of chatter around the…
-
Assessment idea: Debate ChatGPT
I know a lot of people are concerned about students’ use of ChatGPT, especially when it comes to essays. The question I get asked most often is, “How do I make sure that students don’t cheat?” I always try to reframe the question to something like, “How do I make sure students are learning?” When…
-
(Teacher+AI) + (Student+AI) = better outcomes
I think that teachers who use AI well, combined with students who use AI well, will outperform those who don’t use AI at all. And I don’t mean “outperform” on the assessment tasks that universities typically use e.g. MCQs, essays, and Q&A-type formats. I’m not sure how much longer university assessment tasks (in their current…
-
Updating assessment policies in response to generative AI is a waste of time
I had a conversation with a colleague yesterday who had a student submission with a report that was 100% confident of AI-generated content. This is not only going to become prevalent…it’s going to become normal. Because we’re quickly getting to a point where it will be very difficult to create anything that isn’t in some…
-
Do language models threaten the economic viability of universities?
Recent developments in generative AI – not least the notorious ChatGPT – offer opportunities to stimulate long-overdue reforms to assessment practices. But these reforms need careful thought and principled implementation within an already hard-pressed sector. Carless, D. (2023). How ChatGPT can help disrupt assessment overload. Times Higher Education Campus Note that the article I link…
-
Assessment and data assemblages: Changing the world simply by ‘being’ in it
We use student achievement in tests and exams to try and make accurate predictions about their future performance but we know that this practice is neither valid nor reliable. The test environment doesn’t look like the real world environment (so it’s not valid), and the metrics we use to measure test outcomes aren’t reliable because…
-
Let students negotiate the deadlines that suit them
Most of my assignments include a peer review component where students send their initial draft to two peers who review it and give feedback on how the assignment can be improved. I provide guidance on what effective feedback looks like, with an emphasis on helping peers submit their best work; it’s about building each other…
-
The exam was fair
in AssessmentThe exam was fair because I treat all of our students the same. It was an open book assessment. Every student had the same amount of time to complete the task. They all had internet access. They received the same clear instructions. They all had the same list of resources to use. They could all…
-
Recalibrating expectations?
Last week we had a discussion about teaching practical physiotherapy techniques remotely and one of our participants asked (in the text chat) if anyone had any plans to teach fewer techniques. Unfortunately we didn’t get to the question because the conversation moved on quickly to explore other lines of inquiry, which is a pity because…
-
Resource: Online learning in a hurry
Dave Cormier has provided an excellent series of short (5 minute) videos called Online learning in a hurry, for teachers who are now expected to move their teaching and assessment online. What I really like about this collection is that it’s not a list of [insert service here] for remote teaching and learning. For example,…
-
Resource: Essays or projects instead of proctored exams
This may be an opportune time to explore essay exams (or projects) that don’t require proctoring. These would be open-book and open web, and time limits could be imposed by controlling start and end times. Since online provides flexibility, students could be allowed to submit their exams within a 24-hour period. This would be a…
-
Emotions and assessment: considerations for rater‐based judgements of entrustment
We identify and discuss three different interpretations of the influence of raters’ emotions during assessments: (i) emotions lead to biased decision making; (ii) emotions contribute random noise to assessment, and (iii) emotions constitute legitimate sources of information that contribute to assessment decisions. We discuss these three interpretations in terms of areas for future research and…
-
SAAHE podcast 03 – Standard setting, with Scarpa Schoeman
Would it be accurate to say the almost universal pass mark of 50% is completely arbitrary and almost indefensible?I would say it’s completely arbitrary and totally indefensible. Scarpa Schoeman In this conversation, Scarpa and I talk about his experience of standard setting in medical education. Scarpa has published a variety of peer-reviewed articles and presented…
-
SAAHE podcast 02 – Mapping exit-level assessment, with Christina Tan
In this conversation with Christina Tan, we discuss her research around clinical competence and assessment. This is the second in our podcast series on research in health professions education. If you have any suggestions for future conversations, please let us know in the comments. If you’d like to read more about Christina’s work, here is…
-
Using online multimedia to teach practical skills
During 2016 I supervised an undergraduate research group in my department and we looked at the possibility of using multimedia – including video, images and text – to teach students practical skills. Traditionally, we teach these skills by having the lecturer demonstrate the technique on a model while the class watches. Students then break into small…
-
Public posting of marks
My university has a policy where the marks for each assessment task are posted – anonymously – on the departmental notice board. I think it goes back to a time when students were not automatically notified by email and individual notifications of grades would have been too time consuming. Now that our students get their…
-
Interrogating the mistakes
We tend to focus our attention on the things that students got right. This seems perfectly appropriate at first glance because we want to celebrate what they know. Their grades are reported in such a way as to highlight the number of questions answered correctly. The cut score (pass mark) is set based on what we…
-
How my students do case studies in clinical practice
Our students do small case studies as part of their clinical practice rotations. The basic idea is that they need to identify a problem with their own practice; something that they want to improve. They describe the problem in the context of a case study which gives them a framework to approach the problem like a research…