Example: Using ChatGPT to generate ideas for authentic and innovative assessments, based on workplace requirements

I copied the content of an advert for a physiotherapy post in the NHS. The exact content of the post doesn’t matter, as the process below will work for most jobs in most sectors.

The idea is that you take a posting for a real job in your professional domain, and then ask the language model to generate some ideas for teaching and assessment that would prepare students for those roles. I was wondering if the process would help me generate novel ideas for authentic assessment tasks that more closely align with the expectations for practice.

I was impressed with the range of alternative assessment tasks that ChatGPT suggested. In my opinion, these tasks (and suggested topics) align well with the requirements of the job advert I used as the prompt.

See below for the interaction with ChatGPT. I tried using Claude, but in this instance, ChatGPT was better (i.e. more creative, in this context).

---

MR: I’m going to paste in some text from an advert for a job in the NHS. I’d like you to suggest a framework for what the training programme needs to include, and 3 assessment tasks that would help me determine if someone is able to do the job being advertised here. Do you understand?

ChatGPT: Yes, I understand. Please go ahead and paste the text from the advert for the job in the NHS, and I will suggest a suitable training programme and three assessment tasks to evaluate candidates’ qualifications and abilities for the advertised position. Once you provide the text, I’ll be able to proceed with the recommendations.

MR: Pasted the full-text of the job description, including main duties and responsibilities, purpose, key relationships, person specification, and so on. It was about 8 pages of content.

ChatGPT: Suggested Training Programme

1. Advanced Musculoskeletal Physiotherapy Course: This course would cover the latest evidence-based practices, advanced assessment techniques, and treatment modalities for musculoskeletal conditions. It would also include topics on clinical reasoning, leadership skills, and effective communication.
2. Clinical Governance and Quality Improvement Training: This training would focus on understanding clinical governance principles, implementing quality improvement measures, and participating in audit and research activities to enhance patient care and service delivery.
3. Leadership and Management Development: A program focused on developing leadership and management skills, including team management, conflict resolution, and effective delegation, to prepare candidates for the responsibilities of being part of the senior physiotherapy team.
4. Advanced Imaging Interpretation Course: This course would train candidates in interpreting and understanding various imaging modalities such as X-rays, MRI scans, and ultrasound, allowing them to make more informed clinical decisions.

MR comment: This seems reasonable, but not unexpected and not especially helpful at such a high-level.

ChatGPT: Assessment Tasks

1. Clinical Case Study: Candidates will be presented with a complex musculoskeletal case and will be required to assess, diagnose, and propose a treatment plan based on their clinical reasoning and evidence-based practices. This task will assess their ability to work autonomously and manage a highly specialized caseload.
2. Leadership Scenario: Candidates will be given a hypothetical leadership scenario where they need to manage and support a team of physiotherapists in a challenging clinical situation. They will be assessed on their leadership qualities, ability to provide guidance, and handle staff appraisal and performance management.
3. Written Presentation: Candidates will be asked to prepare a written presentation on a chosen musculoskeletal topic, such as the latest advancements in treating a specific condition or the implementation of evidence-based practices. This task will evaluate their research skills, ability to present complex information, and knowledge of national guidelines.

MR: Please give me 3 examples of the leadership scenario you suggested for one of the assessment tasks.

ChatGPT: Here are three examples of leadership scenarios for the leadership assessment task:

Leadership Scenario 1: Managing Staff Conflict

You are the Advanced Physiotherapy Practitioner (APP) in charge of the Musculoskeletal Outpatient Physiotherapy Team. Two of your team members have been having conflicts that are affecting the team’s morale and performance. Describe how you would approach this situation as a leader. Outline the steps you would take to address the conflict, facilitate open communication between the team members, and find a resolution that promotes a positive working environment.

Leadership Scenario 2: Implementing a New Service Protocol

As an APP at Lewisham and Greenwich NHS Trust, you have identified an opportunity to improve the efficiency and quality of the Musculoskeletal Community Assessment, Triage, and Treat Service (MCATS) based on feedback from patients and staff. You believe that implementing a new service protocol could address these concerns. Outline your approach to introducing this protocol to your team. Include the steps you would take to gain buy-in from your colleagues, address potential challenges, and ensure the successful adoption of the new protocol.

Leadership Scenario 3: Leading a Continuous Improvement Initiative

In your role as an APP, you are responsible for leading a continuous improvement initiative to enhance patient outcomes and streamline service delivery within the Musculoskeletal Outpatient Physiotherapy Team. Propose a specific improvement project that you would lead, detailing its objectives, target outcomes, and key performance indicators. Describe how you would engage your team in this project, delegate responsibilities, and use clinical governance principles to monitor progress and measure success.

MR: Would these scenarios be essay-based? I’d prefer to use a more innovative approach to assess the candidates’ ability to demonstrate their leadership skills. Can you suggest the same general themes, but give me a few other options for the format candidates can use.

ChatGPT: Of course! To assess candidates’ leadership skills in a more innovative and interactive way, you can consider alternative formats for the leadership scenarios. Here are the same general themes with different assessment formats:

Leadership Scenario 1: Managing Staff Conflict

Alternative Format Options:

1. Role-Play Exercise: Conduct a role-play session where the candidate plays the role of the Advanced Physiotherapy Practitioner, and you or another assessor portrays a team member with a conflict. The candidate must demonstrate their conflict resolution and communication skills in real-time.
2. Group Discussion Simulation: Organize a group discussion with multiple candidates. Assign each candidate a unique perspective related to the conflict scenario. The candidates must collectively work to find a resolution and demonstrate their ability to lead and collaborate in resolving conflicts.

Leadership Scenario 2: Implementing a New Service Protocol

Alternative Format Options:

1. Action Plan Presentation: Ask the candidates to prepare and deliver a brief presentation outlining their action plan for implementing the new service protocol. They can use visual aids and engage the assessors as stakeholders, addressing potential concerns and receiving feedback in real-time.
2. Team-based Simulation: Divide the candidates into small teams and provide them with different challenges related to the implementation of the new protocol. Each team must collaboratively develop a comprehensive strategy and present their approach to the entire group, showcasing their teamwork and leadership skills.

Leadership Scenario 3: Leading a Continuous Improvement Initiative

Alternative Format Options:

1. Innovative Project Proposal: Request candidates to create an innovative project proposal that outlines their improvement initiative. The proposal can be in the form of a video pitch, infographic, or interactive presentation, allowing candidates to showcase their creativity and ability to present ideas effectively.
2. Simulation with Stakeholder Feedback: Organize a simulation where candidates interact with various stakeholders (played by assessors or actors) throughout the improvement initiative’s implementation. The candidates must respond to feedback and adapt their approach based on stakeholder input.

---