What does scholarship sound like?

Creative work is scholarly work

The Specialist Committee recognises the importance of both formal academic research and creative outputs for the research cultures in many departments, as well as for individual researchers; it thus aims to give equal value to theoretical/empirical research (i.e. historical, theoretical, analytic, sociological, economic, etc. studies from an arts perspective) and creative work (i.e. in cases where the output is the result of a demonstrable process of investigation through the processes of making art.); the latter category of outputs is treated as fully equivalent to other types of research output, but in all cases credit is only given to those outputs which demonstrate quality and have a potential for impact and longevity.

The South African National Research Foundation has recently shared guidelines for the recognition of creative scholarly outputs, which serves to broaden the concept of what kind of work can be regarded – and importantly, recognised – as “scholarly”. The guidelines suggest that the creative work could include (among others):

  • Non-conventional academic activities related to creative work and performance: Catalogues, programmes, and other supporting documentation describing the results of arts research in combination with the works themselves;
  • In Drama and theatre: scripts or other texts for performances and the direction of and design (lighting, sound, sets, costumes, properties, etc.) for live presentations as well as for films, videos and other types of media presentation; this also applies to any other non-textual public output (e.g. puppetry, animated films, etc.), provided they can be shown to have entered the public domain;

I’m going to talk about podcasts as scholarly outputs because I’m currently involved in three podcast projects; In Beta (conversations about physiotherapy education), SAAHE health professions educators (conversations about educational research in the health professions), and a new project to document the history of the physiotherapy department at the University of the Western Cape.

These podcasts take up a lot of time; time that I’m not spending writing the articles that are the primary form of intellectual capital in academia and I wondered, in the light of the new guidelines from the NRF, if a podcast could be considered to be a scholarly output. There are other reasons for why we may want to consider recognising podcasts as scholarly outputs:

  1. They increase access for academics who are doing interesting work but who, for legitimate reasons, may not be willing to write an academic paper.
  2. They increase diversity in the academic domain because they can be (should be?) published in the language of preference of the hosts.
  3. They reduce the dominance of the PDF for knowledge distribution, which could only be a good thing.
  4. Conversations among academics is a legitimate form of knowledge creation, as new ideas emerge from the interactions between people (like, for example, in a focus group discussion).
  5. Podcasts – if they are well-produced – are likely to have a wider audience than academic papers.
  6. Audio gives an audience another layer of interesting-ness when compared to reading a PDF.
  7. Academic podcasts may make scholarship less boring (although, to be honest, we’re talking about academics, so I’m not convinced with this one).

What do we mean by “scholarship”?

Most people think of scholarly work as the research article (and probably the conference presentation) but there’s no reason that the article/PDF should remain the primary form of recognised scholarly output. It also requires that anyone wanting to contribute to a scholarly conversation must learn the following:

  • “Academic writing” – the specific grammar and syntax we expect from our writers.
  • Article structure – usually, the IMRAD format (Introduction, Methods, Results and Discussion).
  • Journals – where to submit, who is most likely to publish, what journals cater for which audiences.
  • Research process – I’m a big fan of the scientific method but sometimes it’s enough for a new idea to be shared without it first having to be shown to be “true”.

Instead of expecting people to first learn the traditions and formal structures that we’ve accepted as the baseline reality for sharing scholarly work, what if we just asked what scholarship is? Instead of defining “scholarship” as “research paper/conference presentation”, what if we started with what scholarship is considered to be and then see what maps onto that? From Wikipedia:

The scholarly method or scholarship is the body of principles and practices used by scholars to make their claims about the subject as valid and trustworthy as possible and to make them known to the scholarly public… Scholarship…is creative, can be documented, can be replicated or elaborated, and is peer-reviewed.

So there’s nothing about publishing PDFs in journals as part of this definition of scholarship. What about the practice of doing scholarly work? I’m going to use Boyer’s model of scholarship, not because it’s the best but because it is relatively common and not very controversial. Boyer includes four categories of scholarly work (note that this is not a series of progressions that one has to move through in order to reach the last category…each category is a form of scholarship on its own):

  • Scholarship of discovery: what is usually considered to be basic research or the search for new knowledge.
  • Scholarship of integration: where we aim to give meaning to isolated facts that consider them in context; it aims to ask what the findings of discovery mean.
  • Scholarship of application: the use of new knowledge to solve problems that we care about.
  • Scholarship of teaching: the examination of how teaching new knowledge can both educate motivate those in the discipline; it is bout sharing what is learned.

Here are each of Boyer’s categories with reference to podcasts:

  • Discovery (advancing knowledge): Can we argue that knowledge can be advanced through conversation? Is there something Gestalt in a conversation where a new whole can be an emergent property of the constituent parts? How is a podcast conversation any different to a focus group discussion where the cohort is a sample with specific characteristics of interest?
  • Integration (synthesis of knowledge): Can the editing and production of a podcast, using the conversation as the raw data, be integrated with other knowledge in order to add new levels of explanation and critique? This could either be in the audio file or as show notes. Could podcast guests be from different disciplines, exploring a topic from different perspectives?
  • Application/engagement (applied knowledge): Can we use emergent knowledge from the podcast to do something new in the world? Can we take what is learned from the initial conversation, which may have been modified and integrated with other forms of knowledge (in multiple formats e.g. text, images, video), and apply it to a problem that we care about?
  • Teaching (openly shared knowledge): Can we, after listening to a podcast and applied what we learned, share what was done, as well as the result, with others in order that the process (methods) and outcomes (results) can be evaluated by our peers?

This may not be a definitive conclusion to the question of whether podcasts could be regarded as scholarly work but at the very least, it suggests that it’s something we could consider. If you accept that a podcast might be regarded as scholarly we can then ask how we might go about formally recognising it as such.

Workflow to distribute scholarly work

I’m going to use an academic, peer-reviewed, traditional journal (or at least, the principle of one) to explore a workflow that we can use to get a sense of how a podcast could be formally recognised as scholarly work. We first need to note that a journal has two primary functions:

  1. Accreditation, which is usually a result of the journals peer review process, and their brand/history/legacy. The New England Journal of Medicine is a recognised “accreditor” of scholarly work, not because there is anything special about the journal but simply because it is the New England Journal of Medicine. Their reputation is enough for us to trust them when they say that the ideas presented in a piece of work have been tested through peer review and has not been found wanting.
  2. Distribution, which in the past meant printing those ideas on paper and literally shipping them around the world. Today, this distribution function has changed to Discoverability; the journal does what it can to make sure your article can be found by search engines, and if you’re the New England Journal of Medicine you don’t need to do much because Google will do your quality signalling for you by surfacing your articles above others. Theefore, ournals host content and try to increase the chances that we can find it, and the distribution function has largely been taken over by us (because we share articles on behalf of the journals).

By separating out the functions of a journal we can see that it’s possible for a journal to accredit work that it does not necessarily have to host itself. We could have a journal that is asked to accredit a piece of work i.e. signal to readers (or in our case, listeners) that the work has passed some set of criteria that we use to describe it as “scholarly”.

What might this workflow look like? Since I’m trying to show how podcasts could be accredited within the constraints of the existing system of journal publications, I’m going to stick to a traditional process as closely as possible, even though I think that this makes the process unnecessarily complicated, especially when you think about what needs to happen following the peer review. Here is what I think the accreditation process could look like:

  1. Create a podcast episode (this is basically a FGD) on a topic of interest where guests discuss a question or a problem that their community of peers recognises as valid. This could be done by a call to the community for topics of interest.
  2. Edit the podcast, including additional resources and comments as show notes. The podcast creators could even include further comments and analysis, either before, during or after the initial recorded conversation. The audio includes the raw data (the recorded conversation), real-time analysis and critique by participants, discussion of potential applications of the emergent knowledge, and conclusion (maybe via post-recording reflection and analysis).
  3. Publish the episode on any podcast-hosting platform. The episode is now in the public domain.
  4. Submit a link to the episode to a journal, which embeds the podcast episode as a post (“article”) along with a short description of what it includes (like an abstract), a description of the process of creation (like the methods), the outcome of the conversation (like a conclusion), and a list of additional reading (like a reference list).
  5. The journal begins the process of accrediting the podcast by allocating peer reviewers, whose reviews are published alongside the embedded podcast in the journal.
  6. Reviewers review the “methods”, “conclusions”, “references” and knowledge claims of the podcast guests, add comments to the post, and highlight the limitations of the episode. The show notes include a description of the process, participants, additional readings, DOI, etc. This could be where the process ends; the journal has used peer review to assign a measure of “quality” to the episode and does not attempt to make a judgement on “value” (which is what journals do when they reject submissions). It is left to the listener to decide if the podcast has value for them.
  7. The following points are included for completeness as they follow a traditional iterative process following peer review. I don’t think these steps are necessary but are only included to map the workflow onto a process that most authors will be familiar with:
    1. The podcast creators make some changes to the audio file, perhaps by including new analysis and comments in the episode, or maybe by adding new information to the textual component of the episode (i.e. the show notes).
    2. The new episode is released. This re-publication of the episode would need to be classified as an entirely different version since the original episode would have been downloaded and shared to networks. An updated version would, therefore, need a new URL, a new page on the podcast hosting service, etc.

In the example workflow above, the journal never hosts the audio file and does not “publish” the podcast. It includes an embedded version of the episode, the show notes (which include the problem under discussion, the participants and their bios, an analysis of the conversation, and a list of references), as well as the full peer reviews. Readers/listeners then decide on the “importance” of the episode and whether or not to assign value to it. In other words, the readers/listeners decide what work is valuable, rather than the peer reviewers or the journal.

In summary, I’ve tried to describe why podcasts are potentially a useful format for creating and sharing the production of new knowledge, presented a framework for determining if a podcast could be considered to be scholarly, and described the workflow and some practical implications of an accreditation process using a traditional journal.

Emotions and assessment: considerations for rater‐based judgements of entrustment

We identify and discuss three different interpretations of the influence of raters’ emotions during assessments: (i) emotions lead to biased decision making; (ii) emotions contribute random noise to assessment, and (iii) emotions constitute legitimate sources of information that contribute to assessment decisions. We discuss these three interpretations in terms of areas for future research and implications for assessment.

Source: Gomez‐Garibello, C. and Young, M. (2018), Emotions and assessment: considerations for rater‐based judgements of entrustment. Med Educ, 52: 254-262. doi:10.1111/medu.13476

When are we going to stop thinking that assessment – of any kind – is objective? As soon as you’re making a decision (about what question to ask, the mode of response, the weighting of the item, etc.) you’re making a subjective choice about the signal you’re sending to students about what you value. If the student considers you to be a proxy of the profession/institution, then you’re subconsciously signalling the values of the profession/institution.

If you’re interested in the topic of subjectivity in assessment, you may be interested in two of our In Beta episodes:

Notes on podcast from Stephen Downes

I thought I’d make some notes while listening to this podcast interview from Stephen Downes., where he talks about personal learning environments, problems with e-learning and open vs. closed educational content.

————————————

Mentions Plearn as part of the opening discussion and bio.

What is a PLE? Compares LMS to PLE. LMS is based around the institution, and when the student leaves the system, they lose access to that learning. Same applies when changing institutions, or learning in different environments. PLE provides access to services and educational services from a personal space, rather than an institutional one.

Very new category of “learning system” right now, so there are no applications that exist that define a PLE. Rather, it’s a generic collection of tools and concepts.

Most resources are accessed on the fly, through the browser. Some people have small libraries that they keep locally, but only for backup purposes or content they need to access offline. Students will access lectures as audio and video streams if available. I disagree with the assumption that we’re all connected all the time and that there is no longer a need to download content to be kept locally.

There’s always going to be a mix of local and remote content that’s relevant for learning. A PLE should support whatever works best / whatever the learner needs in whatever context.

Discussed the Khan academy and the role of online video (YouTube) as an educational resource. Quality of the video production isn’t as important as the quality of the video content. The problem is that the video format is linear, which means that it consumes time, it isn’t searchable (it’s not random access). You can’t find the specific piece of information you’re looking for. Content can be more efficiently acquired through text and images.

Videos are also not social or interactive (although video conferences are). Skype conferencing mentioned. Contextual, flexible teaching and learning isn’t really possible when watching video.

Classrooms are not especially well designed for personal learning “1 size fits 30+”.

Is artificial intelligence a viable approach to education? “Going to be tricky”. Some components of the concept available in primitive recommender algorithms currently present in Amazon, iTunes, etc. But going to be a long time before true AI is going to be able to truly personalise the learning experience.

Software will continue to get smarter and understand more and more about what we want to do. It will be able to aggregate, filter, categorise content dynamically.

Discussion on online identity as a tangent to the above point i.e. that your point of entry into the network (i.e. the browser) would be the software that would aggregate, etc. the content you’re interested in. Downes created a tool that did something like this, but which was subsequently superseded by OpenID. Also a brief mention of OAuth.

Briefly talked about SCORM / IMS and the Common Cartridge format (i.e. learning objects). Useful for closed organisations’ learning requirements e.g. the military. Not useful for learning content that needs to be interactive and to engage with other environments / scenarios. Doesn’t do much for the social component and is unnecessarily complex in trying to create “units of knowledge”. The best model is the open web. Many companies trying to create common formats, but also lock consumers in.

Not an easy, decentralised way to create a “learning” management system. But the context there is in managing students or content, not learning. Nothing wrong with the LMS to manage students, but it’s not about learning. How do you give people the freedom to learn in a personal way?

Ends with some discussion on revenue, profit and commercial aspects of education.

Posted to Diigo 05/20/2010

    • A podcast is an audio file that can be downloaded from the internet to a portable listening device and/or a computer. Originally, the definition of a podcast required a file to be in XML (eXtensible Markup Language) format enclosed in a RSS (Really Simple Syndication) feed
    • Today, the definition of a “podcast” has been broadened to include any audio file that is placed on any online location that is accessible to others. There does not need to be a subscription to a regularly updated, topic-consistent program
    • University instructors may create a podcast series based on lectures for a course, ends at the completion of the semester. Students may create a single podcast that is part of a class collection of audio projects
    • The most common use of podcasts in higher education is creating audio archives of classroom lectures
    • most students report listening to lecture podcasts at home or on a computer, rather than in a mobile environment with a portable device (Brittain, Glowacki, Van Ittersum, & Johnson, 2006; Lane, 2006; Malan, 2007)
    • Podcasts do not need to contain the full information from a 60-to-90 minute lecture. San Juan College is experimenting with “microlectures,” a traditional lecture in which key concepts and themes are condensed down to a one to three minute segment (Shieh, 2009, p. 1)
    • Because microlectures are limited in the amount of content they can convey, students are required to complete their learning with additional readings and assignments
    • Another use of educational podcasting involves the delivery of supplemental course materials
    • Students report a higher satisfaction with a course that has audio as a supplement to print material versus only a print material supplement (Miller & Piller, 2005)
    • Access logs to the podcasts showed that many of the lectures were downloaded well after they were posted, which suggests that students do not often use podcasts for immediate review of recent lectures
    • White learned that the overwhelming majority of the lectures downloaded in the week before each of his exams were relevant to the corresponding exam
    • There is some concern among many instructors that when lecture podcasts are placed online, students will stop attending class. However, White found that students are not using the full-lecture podcasts as a substitute for attending lectures
    • This finding is confirmed by Bonge, Cizadlo, and Kalnbach, (2006), where 95% of their students self-reported that they did not attend class less often as a result of having the podcasts available.
    • In both of these studies, as with many others (e.g., Flanagan & Calandra, 2005; Windham, 2007), students reported great value in having the audio files for lectures available. The podcasts provided the ability to pause, rewind, and listen to difficult material several times
    • Some faculty are very innovative in their use of podcasts
    • Weekly Discussions of Course Content

      Miller (2006) at the University of Connecticut uses podcasts for a post-lecture discussion with students

    • Review Sessions for Quizzes and Tests
    • The pre-recorded audio review session ensures that students have a flexible, mobile learning opportunity to engage in content review structured by the faculty member
    • Alleviating Pre-Class Anxiety
    • Customized podcasts provided before a course begins can help alleviate some of the pre-class anxiety and allay student concerns about issues such as tips for time management, social aspects of the subject, and course assessment
    • Providing Answers to the Most Frequently-Asked Questions Noland White at Georgia College & State University has found a strategic use for podcasting that opens more class time for discussion instead of a question and answer session (Bluestein, 2006)
    • Reducing the Sense of Isolation in Online Learners
    • Students reported the podcasts effective in clarifying and enhancing their understanding of the subject, providing a reinforcement of the material recently learned, and supplying guidance on the direction in which to channel their study efforts
    • Exercises for Student-Created Podcasts
    • Many instructors have developed assignments that require students to produce and submit their own podcasts
    • students are able to create their own podcasts to record reflections, a summary of notes, or additional creative accomplishments
    • Summaries of Course Lectures
    • Higher education institutions are using podcasts outside of the classroom in a variety of different applications. Universities have found that digital audio offers new possibilities for lifelong learning outside the academic classroom (Pownell, 2004)
    • the positive impact of podcasts to enhance student learning is still debated and has not undergone extensive rigorous pedagogical research and review
    • it needs to be carefully integrated into the curriculum in a thoughtful way” (Blaisdell, 2006, p. 4)
    • Faculty need to clearly define and identify their objectives for using podcasts while instructing students how to make the most effective use of this technological tool

KDE 4.3 is awesome

It’s been a busy few weeks at the university, with mid-year assessment (in all it’s various forms) having to take precedence over everything else.  Now that it’s over and students are on holiday, I’ve finally gotten around to doing the things I’ve been putting off for a while…like installing the beta version of KDE 4.3 on Kubuntu Jaunty.

The 4.x series of the desktop is getting more and more impressive with every iteration, so much so that I felt I needed to put it on show a little.  I’ve been playing around with it for a few days now and while it’s still a little buggy, it’s stable enough for me.  In this post, I’m going to go through some of the applications I use most often, and give my own thoughts about why I’m loving this update.

Desktop showing the Lancelot menu
Desktop showing the Lancelot menu

Folder view with expanding folders
Folder view with expanding folders

Desktop.  The Folder view widgets on the desktop do a great job of keeping it clean and useful, and I love the way they expand on mouseover to make navigation really easy and intuitive.  The Lancelot menu is brilliant, keeping unused applications out of the way, but making it simple to find them when needed.

Dolphin file manager with Konsole view (bottom)
Dolphin file manager with Konsole view (bottom)

Dolphin file manager with Split pane, Information, Places and File tree views
Dolphin file manager with Split pane, Information, Places and File tree views

File management.  There was a lot of controversy when the KDE developers decided to
create Dolphin and replace Konqueror as the default file manager, but it was clearly the right move.  There are a couple of things that I love about Dolphin, including the Information side panel, split view mode, Terminal view and the integration of Nepomuk semantic search.

OpenOffice.org word processor
OpenOffice.org word processor

Work stuff.  I tried using KOffice2 even though it’s a platform release (because it looks so very cool), but there are a few issues that keep me from switching from OpenOffice.org, the main one being that it doesn’t support OpenDocument or MS Word files as well as OpenOffice does, and the fonts look terrible.

BasKet note taking application (showing default example)
BasKet note taking application (showing default example)

I’ve installed and am using BasKet notepads for my note taking application, which unfortunately is still a KDE 3.5 application.  There were some concerns about the project stalling when the lead developer decided that he couldn’t continue maintaining it, but it seems as if it’s been taken up by others and may yet have a future.  I hope so because it’s a great application, even in it’s current state.  A project to watch out for in this field is SemNotes, a semantic note taking application being built on Nepomuk (see here for screencast).

Okular universal file viewer (showing PDF with annotations)
Okular universal file viewer (showing PDF with annotations)

Okular is a universal document viewer, although I don’t use if for much other than PDFs.  The feature I like most is the ability to annotate documents, although the default colour scheme of the notes isn”t great.

Calendar in Kontact
Calendar in Kontact

I used to use Kontact for email for the longest time but then I switched to Thunderbird for a while, then Spicebird and finally back to Kontact.  In terms of functionality, nothing comes close to it right now.  I’d like to say that I use Akkregator for my feeds, but it’s missing something that I can’t quite put my finger on.  The interface also hasn’t changed much in the past few years and it seems very slow.

Firefox web browser
Firefox web browser

I have to admit that I’m using the 3.5 preview release of Firefox as the web browser, rather than Konqueror.  While Konqueror was awesome a few years ago, it hasn’t kept up with the changes on the web, and is really starting to show it’s age.  There’s a lot happening at Mozilla that Konqueror jsut can’t keep up with and unless there’s a radical change of pace in it’s development, I can’t imagine using it again.

Amarok media player
Amarok media player

Multimedia.  I’m always switching between different media players, but generally I’ve been keen on Songbird and Amarok for managing my whole library, and Audacious as a light-weight player for quickly playing single files.  Gwenview (the image viewer) has been given an overhaul and

Gwenview in Browse mode with Information side panel on view
Gwenview in Browse mode with Information side panel on view

does a brilliant job of managing image libraries.  Amarok is a bit buggy right now (although I am running the beta version of 2.1) and it’s still lacking some functionality that was present in 1 (the port to Qt4 means a lot of catching up has to be done), which is why I use Songbird on occasion.  But as with other KDE apps right now, it’s in a state of transition and every release is building on the solid platform that was laid down with 2.0.

Marble desktop globe showing satelite view
Marble desktop globe showing satelite view

Marble.  This is a great tool that’s something along the lines of Google Earth and Maps, but it’s open and a native KDE application.  I’ve included these screenshots showing a satelite view, as well as a

Marble desktop globe showing Open Streetmap view
Marble desktop globe showing Open Streetmap view

street view using Open Streetmap.  It’s already got Wikipedia and Flickr integration for additional information, as well as being able to overlay additional data, like temperature and precipitation maps.  It’s a young project that’s come really far and has the capability to be incorporated into other KDE apps, like using it together with geo-tagging photos in Digikam.

The one thing that I can’t find anywhere is a decent podcast catcher…something like Gpodder for Gnome, but native to KDE.  I know that Amarok has one but it’s not working for me and besides, it’s lacking the finishing touches that would win me over.  Little things like being able to read a summary of the podcast would be so useful but is currently impossible.

I’m also not a fan of Kpackagekit, as it’s still very much in development and doesn’t always work very well.  Generally the command line is quicker anyway, but there’s always Synaptic if a GUI is needed.

Anyway, that’s a brief overview of some of the apps that i use and while most of them are still in beta, there’s so much happening in KDE right now that this post will be outdated very shortly.  Sigh…

If you’re interested in following the developments in KDE, check out KDE.News

Twitter Weekly Updates for 2009-03-26

  • @JeffNugent Saw demo a few months ago, looks great but expensive (in South Africa). Looking for open source alternatives. Know of any? in reply to JeffNugent #
  • Pause/rewind podcast lectures = higher test scores. But technology just a tool, students must still work. http://bit.ly/OBbQW #
  • SA government turns it’s back on Zimbabwean refugees fleeing instability, starvation and collapsed health system http://bit.ly/AIQpF #
  • “Cholera just tip of Zimbabwe’s…crisis”, near total collapse of healthcare systems creates massive medical emergency http://bit.ly/10Vk7W #
  • Multimedia resources from Doctors without borders, great for teaching Human Rights and their relationship to health http://bit.ly/no8Qw #
  • Life-saving surgery by text message. Cellphones seem like old tech now, but it’s the only tech available in some places http://bit.ly/EWst0 #
  • @sbestbier Where’s the link to the page describing the widgets? in reply to sbestbier #
  • 3D globe with live (?) Twitter stream. Pretty cool http://bit.ly/10A4A #
  • 2D version here http://bit.ly/19wMqf. Striking how few tweets originate in Africa #

Powered by Twitter Tools.

How web 2.0 is changing medicine

The British Medical Journal published this article in December (2006), which may not seem like a long time ago in the traditional approach to academic publication but which in terms of the Internet is already old news. It asks, “Is a medical wikipedia the next step?”, a question I think is becoming more and more relevant as we see user-generated content proliferating in all spheres of our lives, but more and more frequently in the field of healthcare.

The author, Dean Giustini (librarian at the University of British Columbia Biomedical Branch), looks at the advantages of web 2.0 technologies or social software (e.g. RSS, blogs, wikis and podcasts) with particular reference to the creation of open content, improving access to information and the impact all of this has on medicine. We need to be asking ourselves how we can use these new technologies to better inform the way we teach, learn and communicate with our students and colleagues.

I think the final paragraph sums up my own opinion of the role of the Internet in influencing those of us who are creators and publishers of content:

“The web is a reflection of who we are as human beings – but it also reflects who we aspire to be. In that sense, Web 2.0 may be one of the most influential technologies in the history of publishing, as old proprietary notions of control and ownership fall away.”