The CONSORT guidelines for systematic reviews of RCTs

When I was at the WCPT conference last year I came across the CONSORT guidelines for the publication of systematic reviews of RCTS, which I’d never heard of before. I made a note to look it up and finally got around to doing it. I thought would be quite helpful in planning and carrying out these kinds of research projects, so I’m sharing a few notes here.

CONSORT stands for Consolidated Standards of Reporting Trials and is an “evidence-based, minimum set of recommendations for reporting randomized trials”. In addition to the CONSORT statement, there is a checklist that can be used for evaluating the quality of reports of clinical trials. It is, in essence, a description of how to conduct and report on systematic reviews. If you’re interested in conducting systematic reviews of any trials, then this is definitely something to pay attention to.

consort-flow-diagram

Additional resources for the CONSORT guidelines

Abstract: Student Success and Engagement project

Our faculty has implemented a 3 year research project looking at improving Student Success and Engagement in the faculty. The project is being coordinated across several departments in the faculty and is the first time that we are collaborating on this scale. I will be using this blog as a public progress report of the project, in order to highlight our processes and challenges, as well as to report on draft findings. Here is the abstract of the project proposal.

Achieving promising throughput rates and improving retention remains a challenge for most higher education institutions. Student success in South African higher education has been unsatisfactory and universities have not been effective in developing strategies to enhance students’ learning experiences. Low throughput and poor retention rates have been identified as challenges in the Faculty of Community and Health Sciences at UWC. While success rates of students in the faculty are reported as students who complete their qualification in the shortest possible time, many students require an additional year to graduate. It is important to develop strategies that exploit students’ capacity to engage in their learning, as this may create a space that is conducive to student success. Therefore, the aim of this project is to identify and implement strategies to improve student success in the CHS faculty at UWC through an exploration of student and lecturer engagement. This project will explore student engagement in relation to the domains of assessment, academic literacies and tutoring.

Design-based research has been selected the overarching method as it is informed by the teacher’s desire to improve learning, based on sound theoretical principles. All of the undergraduate students (N=2595) and lecturers in the CHS faculty will be invited to participate in this study. Phase 1 includes the implementation of the South African Survey for Student Engagement (SASSE) and the Lecturer Survey for Student Engagement (LSSE). We will also conduct in-depth interviews and focus group discussions among key informants, who are likely to have insight into the challenges experienced in the areas of assessment, literacy and tutoring, and will be identified through purposive sampling. In addition, document analyses of UWC Assessment policy, Teaching and Learning policy and the Charter of Graduate Attributes will be conducted.

During phase 2, a systematic review will be conducted in order to ascertain which interventions have been demonstrated to increase student engagement in higher education. This data will be combined with the insights gained from Phase 1, and used to inform a series of workshops and seminars in the faculty, aimed at developing and refining principles to enhance student engagement. In addition, course evaluations and other documents will be reviewed, and data related to the domains of assessment, literacies and tutoring will be extracted and compared to the recommended guidelines and principles derived from the systematic review. These principles will then be used to inform interventions that are then implemented in the CHS faculty.

Following implementation of the interventions, Phase 3 will consist of focus group discussions with lecturers and the students who were involved in the project, especially those in areas of assessment, literacy and tutoring. A second South African Survey of Student Engagement (SASSE) and Lecturer Survey of Student Engagement (LSSE) will be conducted at the end of 2016 in order to determine if there has been a change in student engagement. By the end of Phase 3 of the project, a range of interventions within the domains of assessment, literacies and tutoring would have been implemented and evaluated. Ethics clearance will be sought from the University of the Western Cape Senate Research Committee, as well as permission from the Registrar and the various Heads of Department in the Faculty.

Developing case studies for holistic clinical education

This is quite a long post. Basically I’ve been trying to situate my current research into a larger curriculum development project and this post is just a reflection of my progress so far. It’s probably going to have big gaps and be unclear in sections. I’m OK with that.

Earlier this week our department had a short workshop on developing the cases that we’re going to use next year in one of our modules. We’re going to try and use cases to develop a set of skills and attitudes that are lacking in our students. These include challenges with (text in brackets are stereotypical student perspectives):

  • Problem solving and clinical reasoning (Tell me what the answer is so that I can memorise it)
  • Critical analysis (Everything I read has the same value)
  • Empathy (The patient is an object I use to develop technical skills)
  • Communication (The use of appropriate professional terminology isn’t important)
  • Groupwork (Assessment is a zero sum game…if you score more than me it bumps me down the ranking in the class, therefore I don’t help you)
  • Knowing vs Understanding (It’s more important for me to know the answer than to understand the problem)
  • Integration of knowledge into practice (What I learn in class is separate to what I do with patients)
  • Integration of knowledge from different domains (I can’t examine a patient with a respiratory problem because I’m on an orthopaedic rotation)
  • Poor understanding of the use of technology to facilitate learning (social networks are for socialising, not learning)

I know it might seem like a bit much to think that merely moving to case-based learning is going to address all of the above, but we think it’ll help to develop these areas in which the students are struggling. The results of my ongoing PhD research project will be helping in the development of this module in the following ways:

  • The survey I began with in 2009 has given us an idea of digital literacy skills of this population, as well as some of the ways in which they learn.
  • The systematic review has helped us to understand some of the benefits and challenges of a blended approach to clinical education.
  • The Delphi study (currently in the second round) has already identified many of the difficulties that our clinicians and clinical supervisors experience in terms of developing the professional and personal attributes of capable and competent students. This study will attempt to highlight teaching strategies that could help to develop the above mentioned problems.
  • I’ve also just finished developing and testing the data capture sheet that I’ll be using for a document analysis of the curriculum in order to determine alignment.
  • Later next year I’ll be conducting an evaluation of the new module, using a variety of methods.

All of the above information is being fed into the curriculum development process that we’re using to shift our teaching strategy from a top-down, didactic approach to a blended approach to teaching and learning. Development of the cases is one of the first major steps we’re taking as part of this curriculum development process. I’ll try to summarise how the cases are being developed and how they’ll be used in the module. This module is called “Applied Physiotherapy” and it’s basically where students learn about the physiotherapy management of common conditions.

In the past, these conditions were divided into systems and taught within those categories e.g. all orthopaedic conditions were covered together. The problem is that this effectively silo’s the information and students see little crossover. In fact, reality is very rarely so conveniently categorised. Patients with orthopaedic conditions may develop respiratory complications as a result of prolonged bed rest. Patients with TB often also present with peripheral neuropathy, as a result of the association of TB with HIV. So, the purpose of the cases is also to integrate different conditions to help students understand the complexity of real-world cases.

In the first term we’ll use 2 very simple cases that each run for 3 weeks. The reason that the cases are simple is that we’re also going to be introducing many new ideas that the students may have little experience with and which are important for participation in the cases e.g. computer workshops for the online environment, concept mapping, group dynamics, presentation skills, etc. The cases will increase in complexity over time as the students feel more comfortable with the process.

Each case will have an overview that highlights the main concepts, learning outcomes, teaching activities, assessment tasks and evaluation components that the case encompasses. The case will be broken up into parts, the number of which will depend on the duration and complexity of the case. After the presentation of each part, the students (in their small groups) will go through this process:

  • What do I know that will help me to solve this problem?
  • What do I think I know that I’m uncertain of?
  • What don’t I know that I need to learn more about?

These questions should help the students develop a coherent understanding of the knowledge they already have that they can build on, as well as the gaps in understanding that they need to fill before they can move on with the case. Each part will involve students allocating tasks that need to be completed before the next session and role allocation is done by each group prior to the introduction of the case. During this process, facilitators will be present within the groups in order to make sure that students have not left out important concepts e.g. precautions and contraindications of conditions.

At the next session, each member of the small groups present to each other within the small groups. The purpose of this is to consolidate what has been learned, clarify important concepts and identify how they’re going to move forward. At the end of each week each small group presents to the larger group. This gives them the opportunity to evaluate their own work in relation to the work of others, make sure that all of the major concepts are included in their case notes, as well as opportunities to learn and practice presentation skills. Students will also be expected to evaluate other groups’ work.

There will be a significant online component to the cases in the form of a social network built on WordPress and Buddypress. We will begin by providing students with appropriate sources that they can consult at each stage of the process. Over time we’ll help them develop skills in the critical analysis of sources so that they begin to identify credibility and authority and choose their own sources. They will also use the social network for collaborative groupwork, communication, and the sharing of resources.

Finally, here are some of the tasks we’re going to include as part of the cases, as well as the outcomes they’re going to measure (I’ve left out citations because this has been a long post and I’m tired, but all of these are backed by research):

  • Concept mapping – determine students’ understanding of the relationships between complex concepts
  • Poetry analysis – development of personal and professional values e.g. compassion, empathy
  • Reflective blogging – development of self-awareness, critical evaluation of their own understanding, behaviours and professional practices
  • Peer evaluation – critical analysis of own and others’ work
  • Case notes – development of documentation skills
  • Presentations – ability to choose important ideas and convey them concisely using appropriate language

This is about where we are at the moment. During the next few months we’ll refine these ideas, as well as the cases, and begin with implementation next year. During my evaluation of the module, I’ll be using the results of the student tasks listed above, as well as interviews and focus groups with students and staff. We’ll review the process in June and make changes based on the results of my, and 2 other, research projects that will be running. We want the curriculum to be responsive to student needs and so we need to build in the flexibility that this requires.

After reading through this post, I think that what I’m saying is that this forms a basic outline of how we’re defining “blended learning” for this particular module. If you’ve managed to make it this far and can see any gaping holes, I’d love to hear your suggestions on how we can improve our approach.

Blended learning in clinical education (AMEE presentation)

This is the presentation that I gave at the AMEE conference earlier today. It’s the results of a systematic literature review I did as part of my PhD, where I looked at the use of blended learning in clinical education. The abstract doesn’t give much information owing to the fact that I had to be very brief with my submission. The presentation is (a little) more detailed.

Here’s the abstract:

Here’s the presentation (better to view at Prezi.com, space is limited here):

Twitter Weekly Updates for 2011-08-29

  • @AMEE_Online this is great, how do we go about claiming the year’s membership? #
  • RT @Jane_Mooney: Great game-based learning resources for educators from @judithway http://t.co/f1wyv1P #
  • Just registered for #amee2011 after spending 19 hours in transit. The world is smaller than it used to be but it could be smaller still #
  • @jane_mooney I’ll look out for u & your poster. If u want 2 chat I’d love to hook up. My PhD is on blended learning in clinical education #
  • @paulderoos Good luck with the free accommodation for #amee2011 I’d help you out if I could 🙂 #
  • Gearing up for #amee2011 where I’ll b presenting a systematic review on blended learning in clinical education. Let me know if u’ll b there #
  • @amcunningham Official AMEE & Medical Teacher twitter accounts are using #amee2011 #
  • @amcunningham no doubt there are good sessions, it’s just all a bit overwhelming right now. Trying to make some sense of the programme #
  • @amcunningham nothing official about hashtags, just assumed it’d be the full date, will use whatever the convention is 🙂 #
  • @amcunningham I’ve been looking at the presentation sessions for that period & nothing has grabbed me yet. Maybe I’ll come to your workshop #
  • @amcunningham Yes, I’ll be at #amee2011 starting to get excited about it now. We’ll definitely hook up 🙂 #
  • @amcunningham See you’re facilitating a workshop on social media at #amee2011 You know what level the session is aimed at? #
  • Announcing the Zotero 3.0 Beta Release http://t.co/EmDnU32 #

Twitter Weekly Updates for 2011-05-16

  • RT @amcunningham: An analysis of clinical reasoning through a recent and comprehensive approach: the dual-process theory http://is.gd/WrvHwI #
  • The use of tense in Lit review. http://bit.ly/ma0MBm. I also prefer the present tense to situate the conversation in a current context #
  • 13 Photographs That Changed the World. http://bit.ly/iK9LFP #
  • “Dropbox Lied to Users about Data Security, Complaint to FTC Alleges” » http://bit.ly/kfkEBD #
  • Let Them Surf http://ow.ly/1sX2Lt. Eliminate cheating by redefining what it means “to cheat” #
  • How I Talk About Searching, Discovery and Research in Courses http://ow.ly/1sX2Gx. Good tips for novice & experienced researchers #
  • $10 Million Tricorder X-Prize http://ow.ly/1sX2vG. A tricorder is a (currently) fictional mobile device for medical diagnosis #
  • @the_archive We’re pushing for systematic reviews in most postgrad work in our dept. May go some way to increase status? #
  • Why Social Media Tools Have a Place in the Classroom http://ow.ly/1sX2il #
  • @the_archive Just finished a systematic review & agree that they have value. Not about removing them, just being more rigorous #
  • The Problem with Literature Reviews http://bit.ly/lLaQh5. The context is in regard to looking to the past, as opposed to the future #
  • Guidelines on addressing negative comments in social media http://bit.ly/jCPR8u #
  • Managing your library with tags and filters http://bit.ly/jhuBoQ. Helped address many issues I had with my MSc research #
  • Doctor In Your Pocket, WebMD Comes to Android http://rww.to/kRPwry #
  • Interactive 3D Human Body Search Engine Debuts http://rww.to/laGspb. Interesting. Not sure if there’s enough detail for clinical education #
  • Yale Collections Now Free Online http://rww.to/k4Z9ln. More commons resources for academics and students #
  • Organise research on Mendeley with tags and other latent information http://bit.ly/iUzdpN #
  • RT @primarytrainee: Curriculum should be experienced not delivered #earlyyears #
  • Learning outcomes mean starting at the end and working backwards i.e. figure out where you want to be, then how you’re going to get there #

Writing support group

I was talking to a colleague on Friday about how difficult it is to commit to spending a few hours to writing, and that even when you do, there’re always emails, phone calls, etc. that interrupt the process. I’d been thinking about this for about a week, after a friend of mine told me about the Pomodoro technique for more productive time management. Now that work pressure is easing up a bit (all exams have been prepared, very few lectures remain, and we only have one more group to move through their clinical placements), we can start spending more time on our own research activities in the department.

A few of us have started collecting data and drafting initial outlines of our first phase articles (we’re going for PhD through publication), and need to spend dedicated time to get finished. In addition, I have 2 other articles unrelated to my PhD that I need to get out by the end of the year. That’s 4 articles in total that I’d like to have submitted in the next few months…what was I thinking?

So anyway, we’ve decided to start a writing group within our department. We’ll meet once a week for a few hours, out of our offices and disconnected from the net, with instructions from admin not to disturb us. I’m hoping that this will give me enough momentum to get it done, as well as create a bit of a support group. It’s easy to find an excuse when it’s just you, but when your group is asking how you’re getting on, it’s much harder to procrastinate.

We’ll see how it goes.

Twitter Weekly Updates for 2010-07-26

Summary of PhD progress

I’m writing this after having read Christina’s post on her thoughts on the PhD process, and following a few of her links to other PhD students who are blogging their own progress. As I’m going through a little slump at the moment, I thought it might be useful to write a short post on where I’m at right now, to review what I’ve done so far.

A few weeks ago I spent 3 days on a writing workshop with colleagues in my department who are also registered for their PhD’s (there are 4 of us), where I worked on my systematic review (see the proposal). I managed to trim the original 103 articles that I gathered during my first, second and third search rounds, to about 60. Then I went through those 60 with a more critical eye, removing what wasn’t appropriate. Finally I narrowed the list down to 20 articles that we eventually conducted independent critical reviews on, and came to consensus with my supervisor, where we finally agreed on 7 articles that matched my inclusion criteria. The article is now ready to be written up, although I’m uncertain of the format. The outcome of the systematic review will be a peer-reviewed publication that identifies some of the ways in which blended learning has been applied in clinical education, and which will inform the development of my own module (one of the later objectives).

My fourth year research group has just finished capturing the data they gathered from a survey we drew up together, where they looked at the role of social networks to facilitate reflective learning. This survey forms part of my first objective, as well as the first component of my SAFRI project (which will later include focus group interviews with staff members, and an additional survey of the students). Immediately after conducting the survey, I have also held workshops with 2 classes so far, to facilitate the process of working within the network, and will be completing workshops with the last 2 classes in the next few weeks. Tomorrow the group will submit an outline of the first few sections of a draft article, and I’ll be presenting some tentative results at the SAAHE conference next week (see the abstract).

I’ve also recently finished a first draft of an article based on a small, wiki-based project I ran in our department last year (you can still comment on it). Strictly speaking it’s unrelated to my PhD as it doesn’t fit into the proposal, but is still work in a related field. Finally, I gave a presentation on PLE’s to the Centre for Teaching and Learning at Stellenbosch University. Again, PLE’s are not explicitly addressed in my PhD proposal, but as I’m leaning more and more towards that concept as having great potential in reflective learning, I think it might ultimately end up playing an important part in the project.

Now that I look back at my progress over the past 6 months, maybe a short break is in order…?