Michael Rowe

Trying to figure out better ways to help people learn.

Introduction

“Our results show that i) ChatGPT is capable of generating more detailed feedback that fluently and coherently summarizes students’ performance than human instructors; ii) ChatGPT achieved high agreement with the instructor when assessing the topic of students’ assignments; and iii) ChatGPT could provide feedback on the process of students completing the task, which benefits students developing learning skills.”

Dai, et al. (n.d.). Can Large Language Models Provide Feedback to Students?

The provision of timely, constructive feedback is a pillar of higher education, yet its consistent delivery can place significant strain on educators. Custom Generative Pre-trained Transformers (GPTs) offer a potential solution, with the possibility of providing streamlined feedback to students, that aim to enhance student learning, and optimise educators’ time.

The case for custom GPTs

  • Optimising expertise: Automating elements of feedback with custom GPTs frees staff time, allowing for a deeper focus on other areas of academic work.
  • Ensuring equitable feedback: By embedding a robust theoretical framework for feedback (such as Hattie and Timperley – more detail below) into a custom GPT, institutions can ensure consistent, high-quality feedback for every student, minimising inconsistencies that may arise within larger teaching teams.
  • 24/7 support and iterative improvement: Custom GPTs provide instant feedback on initial drafts, empowering students to self-assess and make revisions.  This builds critical revision habits and increases the quality of work before final submission.
  • A shift towards student agency: Real-time feedback can boost confidence and engagement. Students are motivated to take ownership of their writing process and become more likely to seek additional support.
  • Developing feedback literacy: Students need to be actively involved in the feedback process to develop the ability to understand, process, and use feedback effectively. This contrasts with the traditional teacher transmission model, where teachers tell students what they did wrong. See Carless, D. (2020). From teacher transmission of information to student feedback literacy: Activating the learner role in feedback processes.

Constructing custom GPTs

From the Jisc National Centre for AI (the exact process may differ from this summary):

  1. Focus: Select tasks for GPT-supported feedback where automation has high return (e.g., grammar, structure, citation style).
  2. Framework: Choose a feedback framework that aligns with your pedagogical goals. For example, Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112.
  3. Dataset curation: Assemble student writing samples with exemplary human feedback grounded in your chosen framework. This dataset becomes the GPT’s foundation.
  4. Model fine-tuning: Customise your GPT through training on this curated dataset so that it learns to emulate the patterns of your chosen feedback approach.
  5. It is not clear that ‘fine-tuning’ is correct in this context (even though some sources describe it as such), as the custom GPT uses the GPT API to interact with the GPT foundation model. In other words, the custom GPT does not ‘fine-tune’ the foundation model. See here for more context.
  6. Evaluation: Collaborate with students and lecturers in an iterative process of testing, refining, and evaluating the custom GPT. Ensure that the output is valuable and aligns with students’ learning needs, and that the output is aligned with institutional regulations.

Potential benefits

  • Scalability: A single, well-developed GPT can provide feedback to an arbitrary number of students concurrently, addressing capacity constraints.
  • Data-informed tutoring: GPTs can track student needs over time, offering data-driven insights to inform and enhance future instructional delivery.
  • Date-informed institutional decision-making: If something like a custom GPT is rolled out at at scale, it might provide important cross-institutional data related to student writing, which forms a significant part of the learning process. This might in turn guide curriculum planning. For example, a decision to implement a programme of writing across the curriculum. See, Zinsser, W. (1988). Writing to learn: How to write and think clearly about any subject at all. Harper & Row.

AI advancements present an opportunity to transform student feedback. By strategically designing and implementing custom GPTs, we can create a more personalised, efficient, and ultimately more impactful learning experience for all students.

Additional notes

  • Custom GPTs are powerful tools but require careful implementation. Transparency with students about the GPT’s role and its limitations is crucial. Human educators remain indispensable, especially for nuanced or high-stakes assessments.
  • If this proof-of-concept is shown to be feasible, there’s no reason to think it can’t be adapted to provide peer-review-type feedback to academic authors writing for publication. See for example, Claude, you are an expert peer-reviewer.

Discover more from Michael Rowe

Subscribe to get the latest posts to your email.