What does scholarship sound like?

Creative work is scholarly work

The Specialist Committee recognises the importance of both formal academic research and creative outputs for the research cultures in many departments, as well as for individual researchers; it thus aims to give equal value to theoretical/empirical research (i.e. historical, theoretical, analytic, sociological, economic, etc. studies from an arts perspective) and creative work (i.e. in cases where the output is the result of a demonstrable process of investigation through the processes of making art.); the latter category of outputs is treated as fully equivalent to other types of research output, but in all cases credit is only given to those outputs which demonstrate quality and have a potential for impact and longevity.

The South African National Research Foundation has recently shared guidelines for the recognition of creative scholarly outputs, which serves to broaden the concept of what kind of work can be regarded – and importantly, recognised – as “scholarly”. The guidelines suggest that the creative work could include (among others):

  • Non-conventional academic activities related to creative work and performance: Catalogues, programmes, and other supporting documentation describing the results of arts research in combination with the works themselves;
  • In Drama and theatre: scripts or other texts for performances and the direction of and design (lighting, sound, sets, costumes, properties, etc.) for live presentations as well as for films, videos and other types of media presentation; this also applies to any other non-textual public output (e.g. puppetry, animated films, etc.), provided they can be shown to have entered the public domain;

I’m going to talk about podcasts as scholarly outputs because I’m currently involved in three podcast projects; In Beta (conversations about physiotherapy education), SAAHE health professions educators (conversations about educational research in the health professions), and a new project to document the history of the physiotherapy department at the University of the Western Cape.

These podcasts take up a lot of time; time that I’m not spending writing the articles that are the primary form of intellectual capital in academia and I wondered, in the light of the new guidelines from the NRF, if a podcast could be considered to be a scholarly output. There are other reasons for why we may want to consider recognising podcasts as scholarly outputs:

  1. They increase access for academics who are doing interesting work but who, for legitimate reasons, may not be willing to write an academic paper.
  2. They increase diversity in the academic domain because they can be (should be?) published in the language of preference of the hosts.
  3. They reduce the dominance of the PDF for knowledge distribution, which could only be a good thing.
  4. Conversations among academics is a legitimate form of knowledge creation, as new ideas emerge from the interactions between people (like, for example, in a focus group discussion).
  5. Podcasts – if they are well-produced – are likely to have a wider audience than academic papers.
  6. Audio gives an audience another layer of interesting-ness when compared to reading a PDF.
  7. Academic podcasts may make scholarship less boring (although, to be honest, we’re talking about academics, so I’m not convinced with this one).

What do we mean by “scholarship”?

Most people think of scholarly work as the research article (and probably the conference presentation) but there’s no reason that the article/PDF should remain the primary form of recognised scholarly output. It also requires that anyone wanting to contribute to a scholarly conversation must learn the following:

  • “Academic writing” – the specific grammar and syntax we expect from our writers.
  • Article structure – usually, the IMRAD format (Introduction, Methods, Results and Discussion).
  • Journals – where to submit, who is most likely to publish, what journals cater for which audiences.
  • Research process – I’m a big fan of the scientific method but sometimes it’s enough for a new idea to be shared without it first having to be shown to be “true”.

Instead of expecting people to first learn the traditions and formal structures that we’ve accepted as the baseline reality for sharing scholarly work, what if we just asked what scholarship is? Instead of defining “scholarship” as “research paper/conference presentation”, what if we started with what scholarship is considered to be and then see what maps onto that? From Wikipedia:

The scholarly method or scholarship is the body of principles and practices used by scholars to make their claims about the subject as valid and trustworthy as possible and to make them known to the scholarly public… Scholarship…is creative, can be documented, can be replicated or elaborated, and is peer-reviewed.

So there’s nothing about publishing PDFs in journals as part of this definition of scholarship. What about the practice of doing scholarly work? I’m going to use Boyer’s model of scholarship, not because it’s the best but because it is relatively common and not very controversial. Boyer includes four categories of scholarly work (note that this is not a series of progressions that one has to move through in order to reach the last category…each category is a form of scholarship on its own):

  • Scholarship of discovery: what is usually considered to be basic research or the search for new knowledge.
  • Scholarship of integration: where we aim to give meaning to isolated facts that consider them in context; it aims to ask what the findings of discovery mean.
  • Scholarship of application: the use of new knowledge to solve problems that we care about.
  • Scholarship of teaching: the examination of how teaching new knowledge can both educate motivate those in the discipline; it is bout sharing what is learned.

Here are each of Boyer’s categories with reference to podcasts:

  • Discovery (advancing knowledge): Can we argue that knowledge can be advanced through conversation? Is there something Gestalt in a conversation where a new whole can be an emergent property of the constituent parts? How is a podcast conversation any different to a focus group discussion where the cohort is a sample with specific characteristics of interest?
  • Integration (synthesis of knowledge): Can the editing and production of a podcast, using the conversation as the raw data, be integrated with other knowledge in order to add new levels of explanation and critique? This could either be in the audio file or as show notes. Could podcast guests be from different disciplines, exploring a topic from different perspectives?
  • Application/engagement (applied knowledge): Can we use emergent knowledge from the podcast to do something new in the world? Can we take what is learned from the initial conversation, which may have been modified and integrated with other forms of knowledge (in multiple formats e.g. text, images, video), and apply it to a problem that we care about?
  • Teaching (openly shared knowledge): Can we, after listening to a podcast and applied what we learned, share what was done, as well as the result, with others in order that the process (methods) and outcomes (results) can be evaluated by our peers?

This may not be a definitive conclusion to the question of whether podcasts could be regarded as scholarly work but at the very least, it suggests that it’s something we could consider. If you accept that a podcast might be regarded as scholarly we can then ask how we might go about formally recognising it as such.

Workflow to distribute scholarly work

I’m going to use an academic, peer-reviewed, traditional journal (or at least, the principle of one) to explore a workflow that we can use to get a sense of how a podcast could be formally recognised as scholarly work. We first need to note that a journal has two primary functions:

  1. Accreditation, which is usually a result of the journals peer review process, and their brand/history/legacy. The New England Journal of Medicine is a recognised “accreditor” of scholarly work, not because there is anything special about the journal but simply because it is the New England Journal of Medicine. Their reputation is enough for us to trust them when they say that the ideas presented in a piece of work have been tested through peer review and has not been found wanting.
  2. Distribution, which in the past meant printing those ideas on paper and literally shipping them around the world. Today, this distribution function has changed to Discoverability; the journal does what it can to make sure your article can be found by search engines, and if you’re the New England Journal of Medicine you don’t need to do much because Google will do your quality signalling for you by surfacing your articles above others. Theefore, ournals host content and try to increase the chances that we can find it, and the distribution function has largely been taken over by us (because we share articles on behalf of the journals).

By separating out the functions of a journal we can see that it’s possible for a journal to accredit work that it does not necessarily have to host itself. We could have a journal that is asked to accredit a piece of work i.e. signal to readers (or in our case, listeners) that the work has passed some set of criteria that we use to describe it as “scholarly”.

What might this workflow look like? Since I’m trying to show how podcasts could be accredited within the constraints of the existing system of journal publications, I’m going to stick to a traditional process as closely as possible, even though I think that this makes the process unnecessarily complicated, especially when you think about what needs to happen following the peer review. Here is what I think the accreditation process could look like:

  1. Create a podcast episode (this is basically a FGD) on a topic of interest where guests discuss a question or a problem that their community of peers recognises as valid. This could be done by a call to the community for topics of interest.
  2. Edit the podcast, including additional resources and comments as show notes. The podcast creators could even include further comments and analysis, either before, during or after the initial recorded conversation. The audio includes the raw data (the recorded conversation), real-time analysis and critique by participants, discussion of potential applications of the emergent knowledge, and conclusion (maybe via post-recording reflection and analysis).
  3. Publish the episode on any podcast-hosting platform. The episode is now in the public domain.
  4. Submit a link to the episode to a journal, which embeds the podcast episode as a post (“article”) along with a short description of what it includes (like an abstract), a description of the process of creation (like the methods), the outcome of the conversation (like a conclusion), and a list of additional reading (like a reference list).
  5. The journal begins the process of accrediting the podcast by allocating peer reviewers, whose reviews are published alongside the embedded podcast in the journal.
  6. Reviewers review the “methods”, “conclusions”, “references” and knowledge claims of the podcast guests, add comments to the post, and highlight the limitations of the episode. The show notes include a description of the process, participants, additional readings, DOI, etc. This could be where the process ends; the journal has used peer review to assign a measure of “quality” to the episode and does not attempt to make a judgement on “value” (which is what journals do when they reject submissions). It is left to the listener to decide if the podcast has value for them.
  7. The following points are included for completeness as they follow a traditional iterative process following peer review. I don’t think these steps are necessary but are only included to map the workflow onto a process that most authors will be familiar with:
    1. The podcast creators make some changes to the audio file, perhaps by including new analysis and comments in the episode, or maybe by adding new information to the textual component of the episode (i.e. the show notes).
    2. The new episode is released. This re-publication of the episode would need to be classified as an entirely different version since the original episode would have been downloaded and shared to networks. An updated version would, therefore, need a new URL, a new page on the podcast hosting service, etc.

In the example workflow above, the journal never hosts the audio file and does not “publish” the podcast. It includes an embedded version of the episode, the show notes (which include the problem under discussion, the participants and their bios, an analysis of the conversation, and a list of references), as well as the full peer reviews. Readers/listeners then decide on the “importance” of the episode and whether or not to assign value to it. In other words, the readers/listeners decide what work is valuable, rather than the peer reviewers or the journal.

In summary, I’ve tried to describe why podcasts are potentially a useful format for creating and sharing the production of new knowledge, presented a framework for determining if a podcast could be considered to be scholarly, and described the workflow and some practical implications of an accreditation process using a traditional journal.

Results of the first round of my Delphi study

I’m away on a 3 day writing retreat where I’m trying to put together a full draft of the Delphi study that I’m busy wrapping up. I thought I’d take a break from writing  and do something different (from writing, I mean). I took the full text of the open-ended responses from the first round of my study and created this Wordle…because I can.

The questions were related to the attributes that clinicians and clinical supervisors thought healthcare students should have.

Extreme writing

In February 2010 I came across this blog post discussing the possibility of writing a draft literature review in a short period of time i.e. 1 hour, which I found intriguing. I left a comment on the post saying I was keen to try, and also bounced the idea around on Twitter a bit, but nothing came of it. I’ve kept it in the back of my mind though, and recently tried to put more structure into the idea. The following is a simple method for putting together a small, focused team to write an academic article very quickly, without sacrificing quality.
The idea is to produce a draft article in a short period of time (a “sprint” of a few hours) by collaborating in small teams of distributed authors, potentially even from different disciplines, who each bring different skills to the writing process (e.g. editorial, methodology, theoretical frameworks, etc). The primary author is the organiser / planner / person who pulls it all together.

The team is organised by the primary author according to the skills that are necessary for a particular piece of work. In addition, unless the work is only going to be a literature review, someone needs to bring the data to the sprint. Since the person with the data is the most likely person to have a good understanding of the study, this person will most likely be the primary author.

Contributors are all acknowledged as authors, unless their contribution only qualifies for an acknowledgement. For example, if certain team members are asked to conduct a critical review of the draft when it is finished, they may not qualify for author status. See below for a template of the tasks required to complete a draft. All team members will agree to their tasks prior to beginning the writing process, as well as understanding the requirements for authorship credit.

Writing tasks

  • Introduction (written near the end of the sprint, probably as each team member adds a short section based on the work they’ve done)
  • Literature review
    • Primary author to identify the main aim / theme / outline of this section
    • Secondary author/s to identify and distribute (e.g. via a shared folder on Mendeley) relevant articles prior to beginning the sprint
    • Secondary author/s to also build the Reference list during this process
  • Method (completed by study designer / primary author)
  • Results (completed by study designer / primary author)
  • Discussion (Secondary author/s identified based on expertise with linking ideas from Literature review and Results sections)
  • Conclusion (Primary author ties the work together, possibly based on points highlighted by others)
  • Reference list (bulk of this work to be done by Literature review author/s, but also sees contributions from other authors as they add their own citations)
  • Additional tasks that may not be eligible for authorship credit if no other tasks are completed by these team members:
    • Editorial (team member who will take responsibility for spelling / grammatical / formatting work, as well as ensuring a “unified voice” throughout the article)
    • Critical review (at least 2 reviewers, who may or may not be team members, who will take responsibility for critically reviewing the final draft)
Team members choose a date and period of time to dedicate to the process. The primary author determines the main tasks and preparatory work that the team must complete so that when the sprint arrives, members are prepared. This includes clearly defining the aim, objectives, etc of the article prior to beginning writing.

The idea is to use the time to write, not to search for literature, so all preparation should be done beforehand. Authors should use online collaborative tools like Google Docs and Skype where possible, in order to work together in real time.

Let me know if you think I’ve left anything out, or if you think it’s just a silly idea 🙂

Summary of PhD progress

I’m writing this after having read Christina’s post on her thoughts on the PhD process, and following a few of her links to other PhD students who are blogging their own progress. As I’m going through a little slump at the moment, I thought it might be useful to write a short post on where I’m at right now, to review what I’ve done so far.

A few weeks ago I spent 3 days on a writing workshop with colleagues in my department who are also registered for their PhD’s (there are 4 of us), where I worked on my systematic review (see the proposal). I managed to trim the original 103 articles that I gathered during my first, second and third search rounds, to about 60. Then I went through those 60 with a more critical eye, removing what wasn’t appropriate. Finally I narrowed the list down to 20 articles that we eventually conducted independent critical reviews on, and came to consensus with my supervisor, where we finally agreed on 7 articles that matched my inclusion criteria. The article is now ready to be written up, although I’m uncertain of the format. The outcome of the systematic review will be a peer-reviewed publication that identifies some of the ways in which blended learning has been applied in clinical education, and which will inform the development of my own module (one of the later objectives).

My fourth year research group has just finished capturing the data they gathered from a survey we drew up together, where they looked at the role of social networks to facilitate reflective learning. This survey forms part of my first objective, as well as the first component of my SAFRI project (which will later include focus group interviews with staff members, and an additional survey of the students). Immediately after conducting the survey, I have also held workshops with 2 classes so far, to facilitate the process of working within the network, and will be completing workshops with the last 2 classes in the next few weeks. Tomorrow the group will submit an outline of the first few sections of a draft article, and I’ll be presenting some tentative results at the SAAHE conference next week (see the abstract).

I’ve also recently finished a first draft of an article based on a small, wiki-based project I ran in our department last year (you can still comment on it). Strictly speaking it’s unrelated to my PhD as it doesn’t fit into the proposal, but is still work in a related field. Finally, I gave a presentation on PLE’s to the Centre for Teaching and Learning at Stellenbosch University. Again, PLE’s are not explicitly addressed in my PhD proposal, but as I’m leaning more and more towards that concept as having great potential in reflective learning, I think it might ultimately end up playing an important part in the project.

Now that I look back at my progress over the past 6 months, maybe a short break is in order…?

First article published

I just had my first research article published. It’s based mainly on the literature review I did for my Masters degree last year, with a few updates. It’s strange, but when I submitted it about 6 months ago, I thought it was a reasonable piece of work. Reading it now, I feel like taking it back and editing the hell out of it. Does anyone else look back at their earliest work and feel like hiding under a table?

I’m putting the abstract up here in case anyone is interested. The title of the article is “Information and communication technology in health: a review of the literature”.

Abstract

Information and communication technology has been shown to be increasingly important in the education andprofessional practice of healthcare workers. The World Health Organisation (WHO) discusses the benefits of using ICT in the Primary Healthcare setting in terms of better access to information, improved communication between colleagues, facilitating continuing professional development and providing learning tools for healthcareprofessionals, patients and the community as a whole. This review of the literature describes the role of information and communication technology (ICT) in the education and professional practice of healthcare workers and goes on to outline the challenges facing the widespread adoption of ICT. The conclusion is that ICT does indeed have a positive role to play in both the education and professional practice of healthcare workers, including physiotherapists, as long as it is implemented as an adjunct to established and proven practice, and not a replacement.

6 minute walk test and pulmonary hypertension

The 6 minute walk test is a common outcome measure of endurance used by physiotherapists and students.  I’m not sure of the full significance of this article, but it seems to indicate that the test may not be a useful indicator of outcome in patients with mild pulmonary hypertension.

Just thought I’d put this out there.  Here’s the link to the article:
http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(08)61725-0/fulltext