Research project exploring clinicians’ perspectives of the introduction of ML into clinical practice

I recently received ethics clearance to begin an explorative study looking at how physiotherapists think about the introduction of machine learning into clinical practice. The study will use an international survey and a series of interviews to gather data on clinicians’ perspectives on questions like the following:

  • What aspects of clinical practice are vulnerable to automation?
  • How do we think about trust when it comes to AI-based clinical decision support?
  • What is the role of the clinician in guiding the development of AI in clinical practice?

I’m busy finalising the questionnaire and hope to have the survey up and running in a couple of weeks, with more focused interviews following. If these kinds of questions interest you and you’d like to have a say in answering them, keep an eye out for a call to respond.

Here is the study abstract (contact me if you’d like more detailed information):

Background: Artificial intelligence (AI) is a branch of computer science that aims to embed intelligent behaviour into software in order to achieve certain objectives. Increasingly, AI is being integrated into a variety of healthcare and clinical applications and there is significant research and funding being directed at improving the performance of these systems in clinical practice. Clinicians in the near future will find themselves working with information networks on a scale well beyond the capacity of human beings to grasp, thereby necessitating the use of intelligent machines to analyse and interpret the complex interactions of data, patients and clinical decision-making.

Aim: In order to ensure that we successfully integrate machine intelligence with the essential human characteristics of empathic, caring and creative clinical practice, we need to first understand how clinicians perceive the introduction of AI into professional practice.

Methods: This study will make use of an explorative design to gather qualitative data via an online survey and a series of interviews with physiotherapy clinicians from around the world. The survey questionnaire will be self-administered and piloted for validity and ambiguity, and the interview guide informed by the study aim. The population for both survey and interviews will consist of physiotherapy clinicians from around the world. This is an explorative study with a convenient sample, therefore no a priori sample size will be calculated.

Writing a research proposal for T&L

Writing a research proposal for teaching and learning
Prof. Denise Wood (18 July, 2013)

Here are my notes from a presentation by Prof. Denise Wood on developing a research proposal for projects looking at T&L.

Image from Tony Duckle's Flickr photostream
Image from Tony Duckle’s Flickr photostream

Understanding the funding body is important when it comes to applying for funding. Disciplinary specific proposals may not be successful when it comes to T&L projects.

Local evidence of successful projects is important before applying for larger grants. Collaborative teamwork is a great way to build ideas and test concepts. Local resources help you get started and build a track record. Generating pilot data helps to begin publishing. When panels review research proposals, your previous experience in obtaining funding and successful proposals is highly emphasised.

Why are you undertaking the study? Knowing your goals will justify your design decisions. What are your goals:

  • Personal
  • Practical
  • Intellectual / theoretical

Writing proposals is closely tied to career trajectory

How are you using research and research projects to improve your teaching practice?

What conceptual framework are you using:

  • Research paradigm
  • Experiential knowledge
  • Existing theory and research
  • Pilot and exploratory studies

Interested in addressing a gap, bringing in personal reflections that guide and influence the research. If you only think of your conceptual framework as a literature review, then you limit the scope of your research to what others have done.

Research questions:

  • What is the relationship between the goals and the conceptual framework?
  • Help to guide the actual research design / methods
  • Used to connect the problem and practical concerns
  • Should be specific and focused on the study
  • Need to allow flexibility to reveal unanticipated phenomena (if the questions are too focused you may miss emergent ideas)
  • Need to avoid inherent assumptions as they bias the study

Find a balance in the number of questions (3-4 is usually adequate)

Begin with divergent thinking to allow yourself space to explore many possibilities. Mind mapping is useful to identify high-level ideas. Begin reading broadly and then begin narrowing the focus. You can’t answer all possible questions in one study.

Try to avoid getting too caught up in the details of the research methods. Only use methods that you understand.

Note that you will be informed by your own epistemological understanding of what knowledge is and how we come to know. Your methods (quantitative, qualitative or mixed methods)  will most likely mirror your understanding of how we come to know. This will in turn guide how you sample, gather and analyse data.

For local studies, it’s OK to use a pilot within a classroom. Use this to identify a single context. Larger proposals would be better to expand the scope of the study and test the outcome of the pilot. On the basis of the smaller studies, you can make an argument for the larger study. Think laterally about how you can collect data.

  • Validity: How might you be wrong?
  • Bias (what assumptions do you bring with you? Results and interpretation distorted by your own values and preconceptions)
  • Reactivity (quantitative researcher may try to control for the effect of the researcher influence; qualitative researcher looks at how they actually influence the outcomes)

How do you reduce bias and reactivity?

  • Studies should be intensive and long-term (not the same as longitudinal study)
  • Gather rich, thick data (less likely to get from surveys / questionnaires; rather use interviews or focus groups)
  • Respondent validation of outcomes (is what you heard the same as what they meant?)
  • Identifying discrepant cases or evidence (you should take outliers into account, but identify and reflect on them, not necessarily include in the main data and suggest reasons for the discrepancy)
  • Triangulation
  • Comparative data (look at different contexts and populations)

Proposal checklist:

  • Identify a funding body
  • Objectives of the funding body
  • Use the guidelines that the funding body provides
  • Previous funded research and see what has been accepted and / or rejected
  • Links with existing research that the body is involved with
  • Evidence of value, need and benefits (institutional, local, national, international)
  • Background / conceptual framework
  • Methodology
  • Evaluation strategies are valued in educational research
  • Engaged dissemination whereby you share your results as you go, using a variety of methods, including publications, conference presentations, social media and workshops
  • Budget: must meet funding body requirements, realistic, value for money, justify costs
  • Milestones: linked to objectives and outcomes
  • Researcher capabilities: ensure you can deliver what you say you can, track record, previous collaboration, strategic, roles and responsibilities, realistic within workload

Try to model your proposal on successful projects. Learn from the mistakes of others. Sit down with a colleague and ask for constructive feedback.

Explicitly make reference to important and contextually relevant policy documents.

Identify how your research is going to create systemic change.

How are you going to evaluate your process and outcomes?

  • Formative: should be ongoing and used to modify project
  • Summative: can be broad and can go beyond the stated outcomes

Design-based research: can use milestones that are linked to formative evaluation. Identify problems early on and adapt quickly.

How are you going to convince the funding body that the people you’re collaborating with are adding value to the project? You must justify the presence of every team member and highlight how they will contribute.

How are you plugging the holes that funding assessors are going to be looking for?

Differentiate between deliverables (the tangible products that will come from the project) and outcomes (the achievement of stated aims and objectives).

Research development workshop: writing a proposal

These are some of the notes I made during the presentation on preparing a proposal, but include some points that relate to the general process of conducting research

There are 3 things a proposal should try to address:

  • What? – What contribution will this research make? What is it about? What do you want to study?
  • Why? – Why should be bother? Why is it significant? Why do you want to study this?
  • How? – How are you going to study this? What tools and techniques will you use? Who else will be involved? How long is it going to take?

There is no “one way” to write a proposal, rather conform to the norms of your department. How do you find your “voice”, and what is the voice of your department? Bear in mind that the proposal voice is more tentative and uncertain than the thesis voice, which has defiinite ideas to convey

Research = Inquiry

PhD research = Making a contribution to a field, linking it to what already exists

Your research question should be real i.e. the answer is not readily available

Hypothesis = “a bold guess” → I think the answer could be…

Research journey

  1. Ask a question
  2. Begin reading (the “river of words” – who else has asked this question, where do they live, what did they find i.e. what is already out there, what already exists
  3. Could lead to the question changing → you find your question has been answered but there are other, related questions that need answering
  4. Interact with others who are looking at similar questions
  5. Research design is central, may include a redesign following a pilot
  6. Be aware of gatekeepers and how to get around them
  7. Know when to stop gathering data, and when to start analysing it (if this is addressed in part in the proposal, it can give guidance to this process)
  8. Following the analysis, you may have to adapt or even to discard some ideas if the data doesn’t support it
  9. Be careful of despair when your ideas aren’t supported
  10. Following analysis comes synthesis (this is the hardest part where many candidates drop out) → putting the thesis together / linking all the ideas
  11. During the writing / rewriting process, you may have to return to the “river of words” to review significant contributions
  12. What is the contribution of the editor and supervisor of the final document?
  13. What do you have to say when all is said and done i.e. what is your contribution to the current understanding and knowledge base of your field?
  14. Be wary of those who presume to “know it all”. How will you defend the unique nature of your research?

Synthesis involves structuring data (following analysis) and linking it to literature. Analysis follows a formula, whereas synthesis is the creative component that leads to your unique contribution

How do you persuade your reader that your data is valid? Without valid data, you can’t build an argument on it

Is your methodology sound enough to convince your reader that your results are trustworthy?

Differing interpretations of the same data can be a result of using different theoretical frameworks that underly the analysis

The proposal should lay out what the researcher wants to do, but should also include limitations i.e. significance (what it will include and why that’s important) vs. limitations (what will be excluded and why)

Negotiate with the supervisors as to what wasn’t done and write it up i.e. explain to examiners and readers what was left out of the study

The literature review is being written and rewritten throughout the process, because you’re reading throughout the journey

Make sure the research question is clear and concise. What is the background to the question? Why is it relevant now and didn’t arise 10 years ago? What makes “now” a good time to try and answer the question?

Twitter Weekly Updates for 2009-09-21

  • ERN – Networks, CCK09, Memory, Google, Britannica, Location, Immediacy http://bit.ly/1sWDBu #
  • RT @jamesparks101: take a look at this wordle project http://tinyurl.com/n8bsxt #
  • Wordle – Beautiful Word Clouds. Might be useful to determine relative weighting in a document e.g. a CV, or abstract http://bit.ly/NLWzM #
  • @pixlr Been playing with Firefox plugin. Very nice, especially screen capture and express editor. Looking forward to offline availability in reply to pixlr #
  • @pixlr I wasn’t aware that the processing takes place locally, that’s pretty cool. Can the software run in the browser when offline? in reply to pixlr #
  • “Laconica is now StatusNet « StatusNet – Open Source microblogging service” from http://bit.ly/kh0v4 #
  • New blog post: Public libraries on Mendeley http://bit.ly/wCjy7 #
  • “Share recommended readings using Mendeley’s Public Collections | Mendeley Blog” #
  • I just published Abstract – PhD Proposal (M Rowe) to Scribd http://bit.ly/2gLX0W #
  • International Collaborative Undergraduate Student Project 2010 – Physiopedia http://bit.ly/1LqiEY #
  • New blog post: International collaborative undergraduate student project http://bit.ly/fzw7M #
  • I just published Abstract – Wikis and Collaborative Learning to Scribd http://bit.ly/u4OpW #

Powered by Twitter Tools

Proposal presentation

In our department, we’re required to present our research proposals for comment before submission to Higher Degrees.  This allows the group to give feedback before final corrections in the hope that the proposal is accepted without having to make major revisions.

I’ve just shared my proposal presentation that I gave a few days ago on Slideshare.  The feedback I received, although mainly editorial, means that the structure of this presentation is not the same as it will be in the final submission e.g. the Method has received another step in the process.

Would love any feedback.

Proposal coming along slowly

I’ve just shared a third draft of my PhD proposal with my supervisors, using Google Docs. Within a few hours, they’d both submitted feedback, and even though neither is finished reviewing the document, I can already start addressing their comments. Why would you not want to use this tool?

I’ll be presenting the proposal to my department in a few weeks, and aiming for submission to the Higher Degrees Committee on the 07th August. Hopefully there won’t be many corrections to be made.