For me, mixed methods research (MMR) is about using qualitative and quantitative data to strengthen an argument that is difficult to support with only one type of data. It’s about bringing together the numbers (quantitative) and stories (qualitative) to gain a more complete understanding of the world (research problems and questions). We often think of those two approaches as being separate and distinct, but when combined they produce something greater than the sum of the two parts. Earlier this year we had the opportunity to attend a seminar by John Cresswell & Tim Guetterman. Here are my notes.
Introduction to Mixed Methods Research
Practical uses of mixed methods research:
- Explaining survey results
- Exploring the use of new instruments in new situations
- Confirming quantitative results with qualitative findings (why is it often the quantitative component that comes first; any situations where the quantitative could be used to explain the qualitative results?)
- Adding qualitative data into experiments
- Understanding community health research
- Evaluating programme implementation
What are the major elements of MMR?
- A methodology (popular way of conducting research)
- Collecting and analysing quantitative and qualitative data
- Integrating different sets of data
- Framing the study within a set of procedures (called mixed methods designs)
- Being conscious of a philosophical stance and theoretical orientation
Quantitative data collection (closed ended) makes use of instruments, checklists and records. Quantitative data analysis uses numeric data for description, comparison, relating variables
Qualitative data collection (open ended) uses interviews, observations, documents and audio-visual materials. Data analysis revolves around using text and image data for coding, theme development, and then relating themes.
What does “integration” mean? We can do this by merging (using one set of data with another), connecting (using one set of data to explain or build on another), or embedding (quan within qual, or qual within quan) the data.
What is MMR not?
- Reporting quan and qual data separately (they should be combined)
- Using informal methods (it is systematic)
- Simply using the name (it must be rigorous)
- Collecting either multiple sets of quan or qual data (i.e. not multimethod research, must collect both quan or qual sets of data)
- Collecting qual data and then quantitatively analysing it (instead of content analysis, collecting both forms of data)
- Simply considering it an evaluation approach (it is a complete methodology)
Specific benefits of MMR:
- Quan to qual: make quan results more understandable
- Qual to quan: understand broader applicability of small-sample qual findings
- Concurrent: robust description and interpretation of multiple sets of different data
Popular mixed method designs:
- Basic: convergent (bringing qual and quan data together), explanatory sequential (use one set of data to explain more clearly another set of data), exploratory (use qual data to develop something quantitatively that leads to an intervention design)
- Advanced: intervention, social justice, multistage evaluation
Research questions related to MMR
- Convergent design: To what extent to the quan and qual results converge?
- Explanatory design: In what ways to the qual data help to explain the quan results?
- Exploratory design: In what ways do the quan results generalise the qual findings?
How do we display quan and qual results together (joint display)? Lots of variation in how both sets of data can be presented. MaxQDA is an application that can be used to analyse and display different sets of data.
How do we publish MMR? Consider publishing the different sets of data in different papers.
How do we link writing structure to design? Writing about and publishing mixed methods research may require different approaches to article structure and style of writing.
The importance of qualitative research in mixed methods
Key features of qual research:
- Follows the scientific method
- Listening to participant views
- Asking open ended questions
- Build understanding based on participant views
- Developing a complex understanding of the problem
- Go to the setting to gather data
- Be ethical
- Analyse the data inductively – let the findings emerge
- Write in a user-friendly way
- Include rich quotes
- Researcher presence in the study (reflexivity)
Types of problems that qual research is suited to:
- A need to explore a context
- When it is important to listen
- Unusual / different culture
- Don’t know the questions to ask
- Understanding a process
- Need to tell a story
How do our backgrounds inform the way we interpret the world? There is an element of reflexivity and an understanding that data interpretation is dependent on our individual personal and professional contexts.
Writing a good qual purpose statement:
- Single sentence, often in the form of “The purpose of this study…”
- A focus on one central phenomenon
- “Qualitative words” e.g. explore, describe, understand, develop
- Includes participants and setting
Understanding a central phenomenon:
- Quan: explaining or predicting variables
- Qual: understanding a central phenomenon
Data collection:
- Sampling (purposeful)
- Site selection (gatekeepers, permissions)
- Recruitment (incentives)
- Types of data (observation, interview, public/private documents, audio-visual)
Interview procedures:
- Create a protocol
- 5-7 open ended questions (first question is easy to answer e.g. participant role or experience; last question could be “Who else should I speak to in order to get more information about this?”)
- Allows the participant to create options for responding
- Participants can voice their experiences and perspectives
- Record and transcribe for analysis
Observations:
- Observation protocol
- Descriptive notes (portrait of informant, setting, event) and reflective notes (personal reflections, insight, ideas, confusion, hunches, initial interpretation)
- Decide on observational stance (e.g. outsider, participant, changing roles)
- Enter site slowly
- Conduct multiple observations
- Summarise at the end of each observation
Types of audio-visual material:
- Physical trace evidence
- Videotape or film a social situation, individual or group
- Examine website pages
- Collect sounds
- Collect email or social network messages
- Examine favorite possessions or ritual objects
How to code data:
- Read through the data (many pages of text)
- Divide text into segments (many segments of text)
- Label segments of information with codes (30-40 codes)
- Reduce overlap and redundancy (reduce codes to 20)
- Collapse codes into themes (reduce codes to 5-7 themes)
A good qual researcher can identify fine detail but also step back and see the larger themes
How to write a theme passage:
- Use themes as headings
- Use codes to build evidence for themes
- Use quotes and sources of information to demonstrate themes
Writing up the qual study:
- Description
- 5-7 themes
- Use codes and quotes to support themes
- Tell a good story
Five approaches to qual research:
- Narrative (comes out of literature)
- Phenomenology (psychology)
- Grounded theory (sociology)
- Ethnography (anthropology)
- Case study
- Can also include discourse analysis, participatory approaches
Ethical issues:
- Respect the site, develop trust, anticipate the extent of the disruption
- Avoid deceiving participants, discuss purpose
- Respect potential power imbalances
- Consider incentive for participants
MAXApp is a mobile app for collecting data on Android and iOS devices:
- Take photos
- Write memos
- Audio recording
- Location data (geotagging)
- How is this different to something like Evernote? If you’re already using MAXQDA, it offers integration with the desktop client. If you use another data analysis package, then MAXApp may not be as useful.
Relevant readings
- Best Practices for Mixed Methods Research in the Health Sciences
- Creswell, B.J.W. et al., 2007. Designing and Conducting Mixed Methods Research. Australian and New Zealand Journal of Public Health, 31(4), pp.388–388.
- Creswell, J.W., Fetters, M.D. & Ivankova, N. V, 2004. Designing A Mixed Methods Study In Primary Care. Annals of Family Medicine, 2, pp.7–12.
- Hesse-Biber, S.N., 2010. Mixed Methods Research: Merging Theory with Practice.
- Ivankova, N. V., Cresswell, J.W. & Stick, Sheldon, L., 2006. Using Mixed-Methods Sequential Explanatory Design: From Theory to Practice. Field Methods, 18(1), pp.3–20.
- Johnson, R.B. & Onwuegbuzie, a. J., 2004. Mixed Methods Research: A Research Paradigm Whose Time Has Come. Educational Researcher, 33(7), pp.14–26.
- Meissner, H. et al., Best Practices for Mixed Methods Research in the Health Sciences.
- Schifferdecker, K.E. & Reed, V. a, 2009. Using mixed methods research in medical education: basic guidelines for researchers. Medical education, 43(7), pp.637–44.
- Stange, K.C., Crabtree, B.F. & Miller, W.L., 2006. Publishing Multimethod Research. Annals of Family Medicine, 4(4), pp.292–294.