Categories
reading research

#APaperADay – Conceptual frameworks to illuminate and magnify

Bordage, G. (2009). Conceptual frameworks to illuminate and magnify. Medical Education, 43(4), 312–319. https://doi.org/10.1111/j.1365-2923.2009.03295.x

Conceptual frameworks represent ways of thinking about a problem or a study, or ways of representing how complex things work the way they do.


A nice position paper that emphasises the value of conceptual frameworks as a tool for thinking, not only more deeply about problems, but more broadly, through the use of multiple frameworks applied to different aspects of the problem. The author uses three examples to develop a set of 13 key points related to the use of conceptual frameworks in education and research. The article is useful for anyone interested in developing a deeper approach to project design and educational research.

Frameworks inform the way we think and the decisions we make. The same task – viewed through different frameworks – will likely have different ways of thinking associated with it.

Frameworks come from:

  • Theories that have been confirmed experimentally;
  • Models derived from theories or observations;
  • Evidence-based practices.

We can combine frameworks in order for our activities to be more holistic. Educational problems can be framed with multiple frameworks, each providing different points of view and leading to different conclusions/solutions.

Like a lighthouse that illuminates only certain sections of the complete field of view, conceptual frameworks also provide only partial views of reality. In other words, there is no “correct” or all-encompassing framework for any given problem. Using a framework only enables us to illuminate and magnify one aspect of a problem, necessarily leaving others in the dark. When we start working on a problem without identifying our frameworks and assumptions (can also be thought of as identifying our biases) we limit the range of possible solutions.

Authors of medical education studies tend not explicitly identify their biases and frameworks.

The author goes on to provide three examples of how conceptual frameworks can be used to frame various educational problems (2 in medical education projects, 1 in research). Each example is followed by key points (13 in total). In each of the examples, the author describes possible pathways through the problem in order to develop different solutions, each informed by different frameworks.

Key points (these points make more sense after working through the examples):

  1. Frameworks can help us to differentiate problems from symptoms by looking at the problem from broader, more comprehensive perspectives. They help us to understand the problem more deeply.
  2. Having an awareness of a variety of a conceptual frameworks makes it more likely that our possible solutions will be wide-ranging because the frameworks emphasise different aspects of the problem and potential solution.
  3. Because each framework is inherently limited, a variety of frameworks can provide more ways to identify the important variables and their interactions/relationships. It is likely that more than one framework is relevant to the situation.
  4. We can use different frameworks within the same problem to analyse different aspects of the problem e.g. one for the problem and one for the solution.
  5. Conceptual frameworks can come from theories, models or evidence-based practices.
  6. Scholars need to apply the principles outlined in the conceptual framework(s) selected.
  7. Conceptual frameworks help identify important variables and their potential relationships; this also means that some variables are disregarded.
  8. Conceptual frameworks are dynamic entities and benefit from being challenged and altered as needed.
  9. Conceptual frameworks allow scholars to build upon one another’s work and allow individuals to develop programmes of research. When researchers don’t use frameworks, there’s an increased chance that the “findings may be superficial and non-cumulative.”
  10. Programmatic, conceptually-based research helps accumulate deeper understanding over time and thus moves the field forward.
  11. Relevant conceptual frameworks can be found outside one’s specialty or field. Medical education scholars shouldn’t expect that all relevant frameworks can be found in the medical education literature.
  12. Considering competing conceptual frameworks can maximise your chances of selecting the most appropriate framework for your problem or situation while guarding against premature, inappropriate or sub-optimal choices.
  13. Scholars are responsible for making explicit in their publications the assumptions and principles contained in the conceptual framework(s) they use.

The third example seems (to me) to be an unnecessarily long diversion into the author’s own research. And while the first two examples are quite practical and relevant, the third is quite abstract, possibly because of the focus on educational research and study design. I wonder how many readers will find relevance in it.

In a research context, conceptual frameworks can help to both frame or formulate the initial questions, identify variables for analysis, and interpret results.

The conclusion of the paper is very nice summary of the main ideas. However, it also introduces some new ideas, which probably should have been included in the main text.

Conceptual frameworks provide different lenses for looking at, and thinking about, problems and conceptualising solutions. Using a variety of frameworks, we open ourselves up to different solutions and potentially avoid falling victim to our own assumptions and biases.

It’s important to remember that frameworks magnify and illuminate only certain aspects of each problem, leaving other aspects in the dark i.e. there is no single framework that does everything.

Novice educators and researchers may find it daunting to work with frameworks, especially when you consider that they may not be aware of the range of possible frameworks.

How do you choose one framework over another? It’s important to discuss your problem and potential solutions with more experienced colleagues and experts in the field. Remember however, that some experts may be experts partly because they’ve spent a long time committed to a framework/way of seeing the world, which may make it difficult for them to give you an unbiased perspective.

Reviewing the relevant literature also helps to identify what frameworks other educators have used in addressing similar problems. The specific question you’re asking is also an important means of identifying a relevant framework.

Categories
Publication research

Resource: The Scholarly Kitchen podcast.

The Society for Scholarly Publishing (SSP) is a “nonprofit organization formed to promote and advance communication among all sectors of the scholarly publication community through networking, information dissemination, and facilitation of new developments in the field.” I’m mainly familiar with SSP because I follow their Scholarly Kitchen blog series and only recently came across the podcast series throught the 2 episodes on Early career development (part 1, part 2). You can listen on the web at the links or subscribe in any podcast client by searching for “Scholarly Kitchen”.


Note: I’m the editor and founder of OpenPhysio, an open-access, peer-reviewed online journal with a focus on physiotherapy education. If you’re doing interesting work in the classroom, even if you have no experience in publishing educational research, we’d like to help you share your stories.

Categories
AI research

Comment: Should we use AI to make us quicker and more efficient researchers?

The act of summarising is not neutral. It involves decisions and choices that feed into the formation of knowledge and understanding. If we are to believe some of the promises of AI, then tools like Paper Digest (and the others that will follow) might make our research quicker and more efficient, but we might want to consider if it will create blindspots.

Beer, D. (2019). Should we use AI to make us quicker and more efficient researchers? LSE Impact blog.

I have some sympathy for the argument that, as publication in our respective fields increases in volume and speed, it will become impossible to stay on top of what’s current. I’m also fairly confident that AI-generated research summaries are going to get to the point where you’ll be able to sign up for a weekly digest that includes only the most important and relevant articles for my increasingly narrow area of interest. Obviously, “important” and “relevant” are terms that contain implicit assumptions about who I am and what I’m interested in.

Where I differ from the author of the post I’ve linked to is that I don’t see anyone mistaking the summary of the research for the research itself. No-one is going to read the weekly digest and think that they’ve done the work of engaging with the details. You’ll get a 10 minute narrative overview of recent work published in your area, note the 3-5 articles that grab your attention, read those abstracts and then maybe get to grips with 1 or 2 of them. Of course, there are concerns with this:

  • Who is deciding what is included in the summary overview? Ideally, it should be you and not Elsevier, for example.
  • How long will it be before you really can trust that the summary is accurate? But, you also have no way of trusting summaries written by people, other than by doing the work and reading the original.
  • Whatever doesn’t show up in this feed may be ignored. But you can – and should – have multiple sources of information.

However, the benefit of AI is that it will take what is essentially a firehose of research findings and limit it to something you can make sense of and potentially do something with. At the moment I mainly rely on people I trust (i.e. those who I follow on Twitter, for example) to share the research they think is important. In addition to the value of having a human-curated feed there’s also a serendipity to finding articles this way. However, none of those people are sharing things specifically for me, so even then it’s hit and miss. I think an AI-based system will be better for separating the signal from the noise.

Note: I tried to use the service for 3 of my own open access articles and all 3 times it returned the same result, which wasn’t a summary of any of what I had submitted. So, definitely still in beta.

Categories
Publication research scholarship

Article: Which are the tools available for scholars?

In this study, we explored the availability and characteristics of the assisting tools for the peer-reviewing process. The aim was to provide a more comprehensive understanding of the tools available at this time, and to hint at new trends for further developments…. Considering these categories and their defining traits, a curated list of 220 software tools was completed using a crowdfunded database to identify relevant programs and ongoing trends and perspectives of tools developed and used by scholars.

Israel Martínez-López, J., Barrón-González, S. & Martínez López, A. (2019). Which Are the Tools Available for Scholars? A Review of Assisting Software for Authors during Peer Reviewing Process. Publications, 7(3): 59.

The development of a manuscript is inherently a multi-disciplinary activity that requires a thorough examination and preparation of a specialized document.

This article provides a nice overview of the software tools and services that are available for authors, from the early stages of the writing process, all the way through to dissemination of your research more broadly. Along the way the authors also highlight some of the challenges and concerns with the publication process, including issues around peer review and bias.

This classification of the services is divided into the following nine categories:

  1. Identification and social media: Researcher identity and community building within areas of practice.
  2. Academic search engines: Literature searching, open access, organisation of sources.
  3. Journal-abstract matchmakers: Choosing a journal based on links between their scope and the article you’re writing.
  4. Collaborative text editors: Writing with others and enhancing the writing experience by exploring different ways to think about writing.
  5. Data visualization and analysis tools: Matching data visualisation to purpose, and alternatives to the “2 tables, 1 figure” limitations of print publication.
  6. Reference management: Features beyond simply keeping track of PDFs and folders; export, conversion between citation styles, cross-platform options, collaborating on citation.
  7. Proofreading and plagiarism detection: Increasingly sophisticated writing assistants that identify issues with writing and suggest alternatives.
  8. Data archiving: Persistent digital datasets, metadata, discoverability, DOIs, archival services.
  9. Scientometrics and Altmetrics: Alternatives to citation and impact factor as means of evaluating influence and reach.

There’s an enormous amount of information packed into this article and I found myself with loads of tabs open as I explored different platforms and services. I spend a lot of time thinking about writing, workflow and compatability, and this paper gave me even more to think about. If you’re fine with Word and don’t really get why anyone would need anything else, you probably don’t need to read this paper. But if you’re like me and get irritated because Word doesn’t have a “distraction free mode”, you may find yourself spending a couple of hours exploring options you didn’t know existed.


Note: I’m the editor and founder of OpenPhysio, an open-access, peer-reviewed online journal with a focus on physiotherapy education. If you’re doing interesting work in the classroom, even if you have no experience in publishing educational research, we’d like to help you share your stories.

Categories
AI clinical

Research project exploring clinicians’ perspectives of the introduction of ML into clinical practice

I recently received ethics clearance to begin an explorative study looking at how physiotherapists think about the introduction of machine learning into clinical practice. The study will use an international survey and a series of interviews to gather data on clinicians’ perspectives on questions like the following:

  • What aspects of clinical practice are vulnerable to automation?
  • How do we think about trust when it comes to AI-based clinical decision support?
  • What is the role of the clinician in guiding the development of AI in clinical practice?

I’m busy finalising the questionnaire and hope to have the survey up and running in a couple of weeks, with more focused interviews following. If these kinds of questions interest you and you’d like to have a say in answering them, keep an eye out for a call to respond.

Here is the study abstract (contact me if you’d like more detailed information):

Background: Artificial intelligence (AI) is a branch of computer science that aims to embed intelligent behaviour into software in order to achieve certain objectives. Increasingly, AI is being integrated into a variety of healthcare and clinical applications and there is significant research and funding being directed at improving the performance of these systems in clinical practice. Clinicians in the near future will find themselves working with information networks on a scale well beyond the capacity of human beings to grasp, thereby necessitating the use of intelligent machines to analyse and interpret the complex interactions of data, patients and clinical decision-making.

Aim: In order to ensure that we successfully integrate machine intelligence with the essential human characteristics of empathic, caring and creative clinical practice, we need to first understand how clinicians perceive the introduction of AI into professional practice.

Methods: This study will make use of an explorative design to gather qualitative data via an online survey and a series of interviews with physiotherapy clinicians from around the world. The survey questionnaire will be self-administered and piloted for validity and ambiguity, and the interview guide informed by the study aim. The population for both survey and interviews will consist of physiotherapy clinicians from around the world. This is an explorative study with a convenient sample, therefore no a priori sample size will be calculated.

Categories
writing

Summary: Ten simple rules for structuring papers

Good scientific writing is essential to career development and to the progress of science. A well-structured manuscript allows readers and reviewers to get excited about the subject matter, to understand and verify the paper’s contributions, and to integrate these contributions into a broader context. However, many scientists struggle with producing high-quality manuscripts and are typically untrained in paper writing. Focusing on how readers consume information, we present a set of ten simple rules to help you communicate the main idea of your paper. These rules are designed to make your paper more influential and the process of writing more efficient and pleasurable.

Mensh, B. & Kording, K. (2017). Ten simple rules for structuring papers. PLoS Computational Biology, 13(9): e1005619.

Thank you to Guillaume Christe for pointing to this paper on Twitter. While I’m not convinced that the title should refer to “rules” I thought it was a useful guide to thinking about article structure. I’m also aware that most people won’t have time to read the whole thing so I’m posting the summary notes I made while reading it. Having said that, I think whole paper (link here) is definitely worth reading. And, if you like this you may also like this table of suggestions from Josh Bernoff’s Writing without bullshit. OK, on with the summary.

First, there’s this helpful table from the authors as a very brief overview.

https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1005619.t001

Principles (Rules 1–4)

Rule 1: Focus your paper on a central contribution, which you communicate in the title. Adding more ideas may be necessary but they make it harder for the reader to remember what the paper is about. If the title doesn’t make a reader want to read the paper, all the work is for nothing. A focused title can also help the author to stay on track.

Rule 2: Write for flesh-and-blood human beings who do not know your work. You are the least qualified person to judge your writing from the perspective of the reader. Design the paper for someone who must first be made to care about your topic, and then who wants to understand your answer with minimal effort. This is not about showing how clever you are.

Rule 3: Stick to the context-content-conclusion (C-C-C) scheme. Aim to write “popular” (i.e. memorable and re-tellable) stories that have a clear beginning, middle and end. While there are many ways to tell stories, each of which engages different readers, this structure is likely to be appropriate for most. Also, the structure of the paper need not be chronological.

Rule 4: Optimize your logical flow by avoiding zig-zag and using parallelism. Only the central idea of a paper should be presented in multiple places. Group similar ideas together to avoid moving the reader’s attention around.

The components of a paper (Rules 5–8)

Rule 5: Tell a complete story in the abstract. Considering that the abstract may be (is probably) the only part of the paper that is read, it should tell the whole story. Ensure that the reader has enough context (i.e. background/introduction) to interpret the results). Avoid writing the abstract as an afterthought, as it often requires many iterations to do it’s job well.

Rule 6: Communicate why the paper matters in the introduction. The purpose of the introduction is to describe the gap that the study aims to fill. It should not include a broad literature review but rather narrow the focus of attention to the problem under consideration.

Rule 7: Deliver the results as a sequence of statements, supported by figures, that connect logically to support the central contribution. While there are different ways of presenting results, often discipline-specific, the main purpose is to convince the reader that the central claim is supported by data and argument. The raw data should be presented alongside the interpretation in order to allow the reader to reach their own conclusions (hopefully, these are aligned with the intent of the paper).

Rule 8: Discuss how the gap was filled, the limitations of the interpretation, and the relevance to the field. The discussion explains how the findings have filled the gap/answered the question that was posed in the introduction. If often includes limitations and suggestions for future research.

Process (Rules 9 and 10)

Rule 9: Allocate time where it matters: Title, abstract, figures, and outlining. Spend time on areas that demonstrate the central theme and logic of the argument. The methods section is often ignored, so budget time accordingly. Outline the argument throughout the paper by writing one informal sentence for each planned paragraph.

Rule 10: Get feedback to reduce, reuse, and recycle the story. Try not to get too attached to the writing, as it may be more efficient to delete whole sections and start again, than to proceed by iterative editing. Try to describe the entire paper in a few sentences, which help to identify the weak areas. Aim to get critical feedback from multiple readers with different backgrounds.


And finally, here’s a great figure to show how each section can be structured using the guidelines in the article.

https://journals.plos.org/ploscompbiol/article/figure?id=10.1371/journal.pcbi.1005619.g001
Categories
research

Read: Academic research

Many people think that “ivory tower” intellectuals make little difference in the world. But some of the highest impact people in history have been researchers, and if you have good personal fit with academic research, we think it can be one of the highest-impact paths…In this article, we’ll cover why we think a career in academia has the potential to be very high impact in the right circumstances, how to figure out whether this option is for you, and how to maximise the impact you can have as an academic.

Whittlestone, J. (2018). Academic research. 80,000 hours.

This article is an incredibly deep dive into the relative benefits and disadvantages of an academic career. Whether you’re in the early stages of thinking about starting on an academic career or are already a full professor, you may find some very valuable context for planning your next move. If you think of your career as something that happens to you, then this article may be a good place to start thinking differently about the rest of your working life.

80 000 hours is an organisation that aims to help as many people as possible lead high impact careers. The idea is that you have about 80 000 working hours in your career, which means that your choice of career is one of the biggest decisions you’ll ever make. Therefore, it’s worth spending a bit of time figuring out how to use that time for good. Their About page has links to many more resources that I’ve found very useful for thinking about my career as an academic.

Categories
AI clinical

Translating AI into the clinical setting at UC Irvine – AI Med

Ultimately, many of these shortcomings exist because few if any physicians are actively engaged in developing the next generation of technology, AI or otherwise. It is interesting to note the vast majority of medical startup companies are founded with limited if any physician involvement or oversight.Without experts that deeply understand both the medical and technical aspects of the problem, there is currently a significant gap in translating cutting-edge AI technology to healthcare.

Source: Translating AI into the clinical setting at UC Irvine – AI Med

I’m preparing an article on machine learning for clinicians and one of the recommendations I make is that we must ensure that the 21st century healthcare agenda is not driven by venture capital and software engineers. Even though private corporations and government are probably not malevolent, when surveillance and profit are your core concerns it’s unlikely that you’re going to develop something that truly works in the patients’ best interest. We really do need clinicians to be more involved in guiding the progression of AI integration in the clinical context.

See also: AMA passes first policy guidelines on augmented intelligence.

Note: If you’re interested in this topic, I’ve shared the first draft of my introduction to machine learning for clinicians on ResearchGate and would appreciate any feedback you may have.

Categories
AI

Academic expert says Google and Facebook’s AI researchers aren’t doing science

Google and Facebook, and other corporate research labs are focused on AI for profit, not on advancing science..such laboratories aren’t advancing the field of cognitive science anymore than Ford is advancing the field of physics at the edge.

After all, no matter how impressive neural networks are, they operate on principles that date back decades. Perhaps the greatest good for humanity isn’t in fine-tuning algorithms that make people pay attention to Facebook at the expense of their mental health.

Source: Academic expert says Google and Facebook’s AI researchers aren’t doing science

I tend to agree with the main point that the work being done at Google and Facebook doesn’t count as science, in the sense that it’s not advancing our understanding of the world. The engineers at software companies spend a lot of time optimising algorithms until they get the answer they need. I’m not 100% sure but it sounds a bit like p-hacking.

Having said that I also think that there’s a difference between the kind of academic research that enhances our understanding of the world, and the kind of applied research that has commercial value. I’d also love to see more work devoted to cancer diagnosis (and therapeutic intervention development) than social media optimisation, but that’s not really the point. This is about choosing to make an intellectual contribution to a field of research (in which case, do a PhD at a university) or applying well-understood theoretical principles in the service of  real world application (in which, case go work for a startup).

Categories
AI clinical

AMA Passes First Policy Recommendations on Augmented Intelligence

Combining AI methods and systems with an irreplaceable human clinician can advance the delivery of care in a way that outperforms what either can do alone. But we must forthrightly address challenges in the design, evaluation and implementation as this technology is increasingly integrated into physicians’ delivery of care to patients.

Source: AMA Passes First Policy Recommendations on Augmented Intelligence

The American Medical Association recently released their policy recommendations on the use of agumented intelligence systems in the clinical context. Briefly, the AMA states that it will:

  1. Help set priorities for health care AI.
  2. Identify opportunities to integrate the perspectives of clinicians into the development of health care AI.
  3. Promote the development of thoughtfully-designed, high quality, clinically validated health care AI.
  4. Encourage the education of all stakeholders into the promise and limitations of health care AI.
  5. Explore the legal implications for health care AI.

To me, this looks like a set of objectives or lines of inquiry for anyone interested in a research programme looking at the use of AI in the context of healthcare and health professions education.