SAAHE conference, 2010 – day 1

I had made notes for the presentations I attended during the first half of the day using the Scribefire plugin for Chromium, when it crashed and I lost all my notes 🙁 I knew that there wasn’t a save button in the plugin (I didn’t have an internet connection so couldn’t publish as a draft) and was relying on the Lazarus plugin to auto-save everything in the text box. Sadly, I only realised afterwards that Lazarus doesn’t work on pages that don’t have a URL. Oh well, we live and learn.

Here are some notes from the afternoon’s workshop.

How to best teach research skills to clinicians (Dr. Alison Bentley)

The problem is:

  • Few clinicians have higher degrees → “research is an imposition”, and there’s been a reduction in research intensive environments

  • Few qualified supervisors

  • Few experienced internal examiners

  • Students can move through entire departments and never discuss research

  • Research is perceived to be hard, a different field, and not relevant to clinical practice

Attitudes vs. skills: If they don’t want to do it, they won’t get the skills. If they don’t have the skills, they won’t go through the process

Research is important because knowledge isn’t a passive substance but is constantly created and reformulated

Students won’t be able to think for themselves if they can’t query the world around them

We need to generate more South African based, contextually relevant statistics and knowledge, to better situate our patient care

Important for clinicians to model the concept that they don’t know everything

Is there a relationship between clinical reasoning and research methods?

Split into groups to discuss clinical reasoning vs. research methods

What is the research process (for a researcher)?

Before identifying a problem, you need a certain level of experience, as well as a context in which to frame the problem. When the experience, knowledge and context are missing, it’s hard to even find a problem.

How do you find a problem?

  • You can search the literature, but this might be hard in cases where the volume of literature is massive

  • You can observe phenomena

How can you be sure that no-one else has already solved the problem? → literature review

Design the study, then run it

Collection, analysis and interpretation of data

Go back to the literature to situate your results in the broader context

Have you answered the question / solved the problem? Either way is acceptable.

On the other hand, clinicians begin with data collection, but they must first have the right background (skills, knowledge, context), then you interpret the data and come up with a differential, which has to be confirmed by gathering more data

A researcher begins with a problem that will determine data collection. A clinician will begin with data collection and end with identifying a problem (the assessment component…the treatment component might be more similar to the research approach).

Students don’t like the PBL approach because it requires thinking, and the high school system doesn’t set them up to think. They learn that they have to memorise content and get high marks, which isn’t the best way to learn. Then they get to higher education and too often continue in departments that don’t encourage them to think critically.

One of the problems may be that we’re not linking research to teaching. We also need to get clinicians to link research to practice.

Research skills:

  • Searching for literature

  • Some basic statistical knowledge

  • Using literature

  • Writing

  • Research process

  • Time management

  • Project management

  • Collaboration

  • Digital and information literacy

  • Referencing

  • Information management

  • Budgeting

  • Available resources / access

  • Research ethics

  • Critical thinking

  • Data collection skills

  • There were others, but we stopped counting…

Came to the conclusion that many of the above are really hard to teach, and if they are, it’s often not taught at an undergraduate level, which makes it hard to introduce to postgraduates.

One of the problems seems to be that these skills shouldn’t be taught as part of a separate subject, but should be integrated into all other modules

There is value in clinicians having research skills

“Objectivity” in educational research

Earlier today I had a productive meeting with one of my supervisors where, in addition to updating her on my progress, we discussed my reluctance to get “too involved” in my research projects. This is something I’ve touched on before and will probably continue returning to it during the course of my research.

I come from a quantitative background where many believe that the RCT is the gold standard of research, and where the focus on objectivity and distance is meant to reduce or avoid researcher bias. My supervisor, on the other hand, has a good understanding of participatory action research (PAR), in which researcher interaction is not only acceptable, it’s essential. While PAR is the methodological approach I’m tending towards, I still don’t really have my head wrapped around it (note: I’m not trying to compare RCTs and PAR, or to say anything substantial about  either one).

I keep wondering “how much” engagement is appropriate before I start to influence people, and I keep having to be reminded that it’s not the “correct amount” of interaction that I need to worry about, but the objectives I’m trying to achieve that should be my guiding light. I can (should?) influence the interactions with my colleagues and students, as long as I’m documenting not only the interaction, but my own progression and reflection. To be honest, this is new ground for me and it’s not a process I’m entirely comfortable with. It seems very…messy.

Personal attachment to research

Yesterday I had a meeting with my supervisor to discuss the assignments I’m going to run as part of the first objective of my PhD. Together with a systematic review and a survey, I was interested in using student and staff participation in a social network to derive additional data that would help me form a baseline understanding of their attitudes and skills around teaching and learning practice, as well as establish the level of digital and information literacy within the department.

After joining the SAFRI programme, I incorporated the social network idea into my SAFRI project, but unconsciously ended up with a different agenda. Instead of using the network to highlight potential problem areas and the challenges of teaching with technology, it morphed into me trying to demonstrate the effectiveness of using a social network to facilitate reflective practice. In hindsight, it’s clear that the 2 projects were at odds with one another, and the objectives were definitely not aligned.

When my supervisor pointed out that there was inconsistency in the 2 projects I really struggled to accept it. I was adamant that my methods were fine and she suggested that I hand over facilitation of the assignments within the network to other staff who didn’t have such a high personal stake in the success of the project, and I strongly disagreed. I found several reasons to explain why I had to be the person to run it, the strongest of which was that “…no-one else will try as hard as I will to make sure it works”. Which kind of made her point.

When I went away and thought about our conversation I reviewed my objectives for the 2 projects, and then it was clear that they really were 2 different projects. One was suggesting that this would be a useful tool to describe the current state of affairs, which I know will be less than ideal. The other was intent on proving that the network would be a positive tool, rather than describing what would happen if we just incorporated one into the department.

After the painful realisation that I’d let my personal desire for this project to succeed override my objectivity as a researcher, I agreed to let others lead the social network assignments, with guidance from me. This will greatly reduce the impact of researcher bias, as well as synchronise the objectives of the 2 projects. As it stands now, it will more accurately describe the state of the department in terms of attitudes and skills around teaching and learning, and the levels of digital and information literacy, which will give me valuable data that will inform the next objectives of my study.

This was a great learning experience for me, and a warning of the dangers of getting too close to one’s project. There are some situations where the researcher can be an integral part of the project, but this experience has shown me when it would be detrimental to the process.

Research development workshop: research methods

This section included a general discussion on methods, then provided a brief overview of the 2 main types.

Make sure to choose a design that’s appropriate for your project

Research tends to fall broadly into one approach or the other, and is often not entirely quantitative or qualitative

Make sure to avoid using the language of one approach when you are using the other. Example: talking about “proving” something when using a qualitative approach isn’t appropriate

Continuum from QuantitativeQualitative

  • Predetermined ↔ Unfolding
  • Tight design ↔ Emerging
  • Architectural / blueprint ↔ Open ended

Planning your research design

  • Often begins with an identified gap in the literature → initial research question
  • What do you want to find out?
  • What is the purpose of your study?
  • What are the questions you want to address?
  • What methods can you use to answer these questions (data collection / analysis)?

Identify the problem and then choose a method, rather than deciding from the outset what type of research you want to do

Overview of quantitative research: what types of questions can quantitative research answer?

  • Give an overview of information with regard to a population
  • Measuring the extent of something using numerical values
  • Identify trends over time
  • Measure attitudes / opinions of large groups e.g. political surveys
  • There is a tightly designed structure that comes before implementing the research to ensure that one is measuring what one intends to measure
  • There are clear variables
  • You would define concepts
  • Formulate measures or indicators for assessing outcomes

Often makes a claim of a causal relationship between 2 variables that requires:

  • Control for interfering variables
  • Sample and control groups
  • Period of time in which to run the intervention
  • Pre- and post-test
  • What statistical tests can you run to analyse data
  • What results would be significant
  • What can one claim based on the sample size
  • Can your results be generalised to a larger population?

Overview of qualitative research: what types of questions can qualitative research answer?

  • Aim to gain more understanding of people, processes, organisations and relationships
  • Naturalistic approach, research something within a natural context in it’s full complexity i.e. not trying to control for interfering variables
  • Aims for depth, rather than breadth → limits how many cases one can study i.e. not a large population
  • Sceptical about the concept of objectivity i.e. acknowledges that the researcher comes to the project with a background and their own values and doesn’t try to completely eliminate bias
  • Don’t claim that findings are generalisable
  • Tends to have a more flexible design, open-ended and iterative process
  • Data analysis → codes for analysis and themes are derived from the data
  • May apply initial theory to data analysis (deductive research)
  • May use grounded theory → themes emerge from the data and influences the conceptual framework (inductive research)
  • Often there is a combination of deductive and inductive research

Research development workshop: writing a proposal

These are some of the notes I made during the presentation on preparing a proposal, but include some points that relate to the general process of conducting research

There are 3 things a proposal should try to address:

  • What? – What contribution will this research make? What is it about? What do you want to study?
  • Why? – Why should be bother? Why is it significant? Why do you want to study this?
  • How? – How are you going to study this? What tools and techniques will you use? Who else will be involved? How long is it going to take?

There is no “one way” to write a proposal, rather conform to the norms of your department. How do you find your “voice”, and what is the voice of your department? Bear in mind that the proposal voice is more tentative and uncertain than the thesis voice, which has defiinite ideas to convey

Research = Inquiry

PhD research = Making a contribution to a field, linking it to what already exists

Your research question should be real i.e. the answer is not readily available

Hypothesis = “a bold guess” → I think the answer could be…

Research journey

  1. Ask a question
  2. Begin reading (the “river of words” – who else has asked this question, where do they live, what did they find i.e. what is already out there, what already exists
  3. Could lead to the question changing → you find your question has been answered but there are other, related questions that need answering
  4. Interact with others who are looking at similar questions
  5. Research design is central, may include a redesign following a pilot
  6. Be aware of gatekeepers and how to get around them
  7. Know when to stop gathering data, and when to start analysing it (if this is addressed in part in the proposal, it can give guidance to this process)
  8. Following the analysis, you may have to adapt or even to discard some ideas if the data doesn’t support it
  9. Be careful of despair when your ideas aren’t supported
  10. Following analysis comes synthesis (this is the hardest part where many candidates drop out) → putting the thesis together / linking all the ideas
  11. During the writing / rewriting process, you may have to return to the “river of words” to review significant contributions
  12. What is the contribution of the editor and supervisor of the final document?
  13. What do you have to say when all is said and done i.e. what is your contribution to the current understanding and knowledge base of your field?
  14. Be wary of those who presume to “know it all”. How will you defend the unique nature of your research?

Synthesis involves structuring data (following analysis) and linking it to literature. Analysis follows a formula, whereas synthesis is the creative component that leads to your unique contribution

How do you persuade your reader that your data is valid? Without valid data, you can’t build an argument on it

Is your methodology sound enough to convince your reader that your results are trustworthy?

Differing interpretations of the same data can be a result of using different theoretical frameworks that underly the analysis

The proposal should lay out what the researcher wants to do, but should also include limitations i.e. significance (what it will include and why that’s important) vs. limitations (what will be excluded and why)

Negotiate with the supervisors as to what wasn’t done and write it up i.e. explain to examiners and readers what was left out of the study

The literature review is being written and rewritten throughout the process, because you’re reading throughout the journey

Make sure the research question is clear and concise. What is the background to the question? Why is it relevant now and didn’t arise 10 years ago? What makes “now” a good time to try and answer the question?

UWC writing for publication retreat – day 3

Today is the last day of our writing retreat. We had a short session this morning briefly going over the Method, Results, Discussion and Conclusion, before going back to our rooms to spend the last few hours writing. Coming from a more quantitative background, I’m having some difficulty writing up my qualitative responses, so looking forward to feedback (via Google Docs) from my group members.

Here are my notes from this morning.

Data interpretation and alignment

  • Go back to the journal review and decide if this journal accepts your type of paper
  • What type of data is typically presented in that journal?

Method

  • Recognise that there’s a wide range of methods
  • Make sure your methods are aligned with the literature review
  • Explain why that method was used
  • Summarise → “this is what other people have said about this method”
  • “This is what I did” → descriptive account

Data presentation

  • Use only data that is aligned with your introduction and literature review → aim for congruency
  • Let the data speak for itself
    • It should be comprehensible on it’s own
    • It should indicate a general trend
    • Save extended interpretation for the Discussion
  • Avoid data density and overkill
    • Select appropriate data; emblematic data
    • Avoid repetition and tedious prose explaining what is already evident
  • Group data into themes or patterns
  • Think about what sort or data you have e.g. interviews, survey results, observations, and how best to present that. Present data in user friendly ways e.g. graphs and other visuals

What will you do with your data to make sure that you “surface” the message you want to convey?

How do you convince your readers that you’ve designed a rigorous study?

…we collect minds and then we can…” (Student response in an interview about groupwork, a direct translation from another language)

Qualitative studies

  • What can we do to reduce the power differential between students and lecturers, and what strategies did you put in place e.g. focus groups?
  • Be transparent about the process i.e. make issues visible so that the reader can be aware of them
  • How do you get around the problem of interpretation? What is real and what is the researcher creating connections where none exist?

Discussion

Make a compelling argument. What this means and why it’s important

  • Validate and defend your findings
  • comparisons and interpretations
  • Find your niche

Conclusion

“This is what I did, what I found, and some things I might do next”

  • Summary and argument
  • Possible avenues for future research

Return to rewrite the Abstract and Introduction to ensure alignment

How will I align my analysis of the data to the rest of the paper?

What do you think are the best ways of presenting your data?

UWC writing for publication retreat

I’m just finishing up the first day of a 3 day “writing for publication” retreat, hosted at the Mont Fleur conference centre just outside of Stellenbosch. We spent the first half of today covering some of the underlying ideas and concepts around the first sections of an article, which was useful for me because I write what I think sounds good, rather than having a more nuanced understanding of what exactly it is that I’m writing.

After this we spent a few hours getting everyone signed up to Google Docs and sharing the articles we’re currently busy with among our respective group members. We’ll be using Docs over the next few days to provide feedback to the other participants. Even though I’ve done this at a few workshops now, I’m still amazed at how there are always a few more complex cases that take up the bulk of the total time spent.

I’ll be writing an article based on a presentation given at the HELTASA conference in December last year, which was based on a survey I conducted of my fourth year students following a wiki based assignment I’d given to them earlier on in the year.

Here are my notes from the day’s session.

Identify a journal

Identify your journal early on in the process of writing, rather than trying to force an article into a journal

Publication = joining the conversation

  • Who is already participating in the journal (reviewers, editors)?
  • Who decides who can join in?
  • Do you know anyone who is participating?
  • Who has been excluded and on what grounds?
  • What is under discussion?
  • Who do you need to know in order to join

Know the aims and scope of the journal. Does your material suit the journal’s agenda

Email the editor to ask what the interests of the journal are

Finding an argument could involve responding to another publication by another author

Genre = type of expression which has features that all examples of this type share, they shape the thoughts we form and the communications by which we interact

Browsing articles in different publications may give you an idea that’s more creative than you might be used to

Argument = trying to convince your readers of a particular point that you’re trying to make

Abstracts

This was a short exercise where we were asked to “Write your abstract as a bedtime story”. Here’s mine (the underlined sections were provided as cues):

Once upon a time researchers believed that the use of emerging technologies in clinical education would magically create better teaching and learning practices.

But I began to wonder what this magical process was, and if it was as simple as everyone made it out to be.

So what I did was to conduct a small experiment in one of the classes I teach, where students used a wiki to collaboratively construct articles on paediatric conditions.

I discovered that there was little difference in student behaviour as a result of using the wiki, and that the technology wasn’t the problem.

This changed the way I think about integrating technology into my teaching practice.

It was just an idea to begin thinking about the abstract in a different, slightly more creative way.

Make your abstract, concrete. It’s an advertisement for the rest of the work. Is it going to make your reader follow through?

Your work isn’t only about the content and form, it’s also about establishing your identity as an academic. What does this work say about who I am?

Questions to ask about the abstract:

  • What conversation is the researcher in?
  • What is the researcher’s stance?
  • Does the voice sound “expert” enough?
  • Is the research clear?
  • What is the argument? Can it be made stronger?
  • Is the “so what/now what?” question answered?
  • Will the reader want to read the rest of the article?

Begin by establishing a context and / or a conventional idea, and then challenging it.

Identify areas where you should be tentative, and areas where you can be definite.

Some characteristics of an abstract:

  • Locate – what is the relation of this paper to the bigger picture
  • Focus – what questions or problems that will be explored
  • Report – summarise the major findings
  • Argue – open out the argument and indicate a point of view, returning to the angle e.g. the theoretical framework → closing the circle

Introduction

Introductions tend to follow a set pattern, regardless of the discipline. But, be careful of sticking too closely to any one formula or pattern

Create-a-research-space (CARS) model:

  • Establish a territory → highlight work already done in the field
    • Claim centrality
    • Make topic generalisations
    • Review previous research
  • Establish a niche → what are my questions / comments on the topic?
    • Counter claim
    • Indicating a gap
    • Identifying a gap
    • Continuing a tradition
  • Occupy the niche →
    • Announcing present research
    • Outlining the purpose of the present research
    • Announcing principal findings
    • Stating the value of the present research
    • Indicating research article structure

Peer review

A critical friends asks provocative questions and takes time to fully understand the context

Giving feedback:

  • Provides an audience
  • Direct and explicit questions and comments
  • Constructive, rather than destructive
  • Look for meanings, but don’t take over

Open research

I’ve been thinking about the concept of open research since listening to Jon Udell’s interview with Jean-Claude Bradley on his open notebook science project.  The idea is similar to the open approach to writing software in that the process is transparent and open to scrutiny by anyone.  This could have important implications for the soundness of the methodology behind the research, the distribution of results and the potential for massive collaboration on research projects.

Open research makes use of social tools like wikis (wikiresearch), blogs, Google Docs and social networks of like-minded individuals, that allow for collaboration, rapid publication and increased access to information for anyone with an internet connection.  There is also the suggestion that openness in research could lead to more innovation by stimulating ideas that allow others to make contributions to the body of knowledge that may not have been the original intent of the researcher.

However, not everyone is comfortable with the idea of conducting research in an open environment, that is subject to scrutiny by everyone and largely against the culture of secrecy in scientific research.  There are definitely issues with the process and one example of how conflict could arise is by publishing primary data openly.  This has the obvious benefit in that anyone could take that information and use it in ways not intended by the researcher, taking data that may have never seen the light of day and creating new knowledge.  The downside is that someone else could beat you to the finish line by publishing your results and negating your work.

There are other approaches that aren’t as “open” as publishing everything concerned with the project.  For example, you could choose to publish only your methodology or ideas around where the project is headed and request input around that, or raw data could be summarised before publishing online.  Other, similar fields are also becoming more mainstream, like open peer review, in which the peer review process of publication is made public, and open notebook science.

What will the world be like when all knowledge is freely available?