Note: These are my first thoughts on this idea, so I may not be articulating this well. Feel free to tell me how wrong I am in the comments 🙂
Don’t use assessment design to solve the ChatGPT problem. Use ChatGPT to solve the assessment design problem.
I’m seeing a lot of chatter around the need to change assessment tasks to make it harder for students to cheat using ChatGPT. I think this is the wrong approach, because it positions ChatGPT (and all language models) as the problem.
I prefer to think of ‘assessment design’ as the problem, for which ChatGPT is a potential solution. For example, many assessment tasks are not valid or reliable; they don’t assess what we say they assess (not valid), and we get different results depending on who the assessor is (not reliable).
The reframing of the problem helps to explore a more open-ended problem space. What I mean is that the space of all possible problems with assessment design is vast; there are many more ways to get assessment wrong than there are to get it right. If we think of ChatGPT as the problem, we’re limited in the potential solutions that are possible to find.
But, if we think of assessment design as the problem, it opens up many possible ways for language models to help us converge on better design.
I think the careful use of language models in assessment design will help us move closer to authentic assessment, where we address problems with assessment that predate language models.
Comments
2 responses to “Don’t use assessment design to solve the ChatGPT problem…”
Indeed!
I think we are assessing wrong, traditional assessments mainly focus on recal with some attempt of problem solving questions. I think we should think about what should be assessed? Also how much do we really need formal assessments?