Mixed Research Methodology

Defining mixed methods research is not an easy task because there are a multitude
of design variations, paradigmatic combinations, and sources of quantitative or qualitative
data. In the research field, there is continuing discussion regarding clarification of mixed
methods research in relationship to mono method research, multi method research, and
mixed model research.

In a mono method research design, one method, either qualitative or quantitative,

with corresponding data collection, analysis, and accompanying procedures, is used to
answer the research question. Multi method research employs different types of data
collecting methods—for example, both survey and archival data. Multi method research
occurs when the research questions are investigated by using two different data collection
procedures (e.g., observations and focus groups) or by combining two research methods
(critical theory, grounded theory, or case study) from the same research tradition
(qualitative or quantitative).

Mixed methods research combines qualitative and quantitative approaches in a

single or multi phased study. The mixing process may occur in any or all phases of the
research, including the methodology, the logistics of the design, the specified methods to
be conducted, the procedures or data collection, and the analysis. However, mixed
methods research is more than quantitative and qualitative research mixed: It incorporates
and embraces blends of paradigms, philosophical assumptions, and theoretical
perspectives directly driven by the purpose of the study and the intended audience.

PURPOSES FOR CONDUCTING MIXED METHODS RESEARCH

The concept of purpose gets to the heart of why a mixed methods approach is chosen:
whether to seek corroboration of findings, to elaborate or clarify findings, to further
develop interpretations, to investigate contradictions, or to expand the breadth or depth of
a study. The five general purposes discussed here were identified by Greene, Caracelli,
and Graham (1989) and have been reiterated by others.

Triangulation seeks to examine the convergence of evidence from different

methods that study the same phenomenon or to corroborate findings from one method by
examining the findings using a different method. The value of triangulation is that it
allows the researcher the opportunity to examine whether findings converge, are
inconsistent, or contradict. The purpose of triangulation is to collect, analyze, and merge
results to better understand a research problem. Triangulation allows researchers to collect
separate forms of data at separate times or simultaneously. A researcher might conduct
focus groups and also gather quantitative ratings using a survey form to assess program
participants’ perceptions. Or, for example, teacher survey responses might indicate that
particular classroom practices are routinely used. Classroom observations might provide
information that triangulates to corroborate (or contradict) the reported behaviors.
 Complementarity involves seeking elaboration, illustration, enhancement, or
clarification of findings from one method using results from the other. Different
approaches are used to measure different facets of a single phenomenon. For example,
one might conduct qualitative interviews with parents to determine influences on their
perceptions of the quality of education their child is receiving in the school combined
with a quantitative questionnaire to determine how they perceive their child’s school
compares in quality ratings with other schools.

Development uses the results from one approach to develop or inform the other
approach.The study can begin with qualitative data analyzed in the first phase as a way to
establish constructs to be measured quantitatively in the second phase. Alternatively,
quantitative survey results from an initial phase of a study might be used for purposeful
selection of participants for a second qualitative phase.

Initiation is aimed at discovering paradoxes or contradictions in findings that

might lead to reframing a theory. This approach is used to add breadth and depth to the
inquiry. The focus is on intentionally analyzing quantitative and qualitative data that are
inconsistent or contradictory. Such a focus might lead to fresh or new insights into a
phenomenon. For example, finding through qualitative data that parents held negative
perceptions of their child’s school experiences while they simultaneously rated the school
as “good” on a quantitative survey could initiate a mixed methods design to find an
explanation.

Expansion is focused on expanding the breadth and range of the inquiry by using
different methods for different components of the study. For example, a researcher might
be interested in examining the effectiveness of a professional development program in
raising student achievement. Qualitative observations might be used to determine whether
and how well the teachers are using the professional development approaches in the
classroom, which is sometimes referred to as implementation fidelity. Quantitative test
scores could be used as a quantitative measure of program effectiveness.

MIXED METHODS DESIGNS

In considering the design of a mixed methods study, the researcher must look at a
number of factors and ask him- or herself a number of questions: Will the study involve
one or more methods of data collection? Will the study have one phase, two phases, or
multiple phases? Will the data collection occur sequentially or concurrently? Will data
conversion occur? Will the study be mixed in the initial stages only, across stages, in the
end stages only, or in some other combination? Does the qualitative or quantitative
component have priority or are they of equal importance? Teddlie and Tashakkori (2006)
discuss several key areas that the researcher should clearly articulate:

1. The number and type of data collection approaches that will be used

2. The number of phases or strands in the study.

3. The type of implementation process to be used.

4. The level or stage of integration of the approaches.

5. The priority of the methodological approaches

How the researcher answers these questions determines the most appropriate mixed
methods design to be used. Six mixed methods designs are described here.

In concurrent designs, both qualitative and quantitative data are collected

separately but at approximately the same time. Analyses are conducted separately and
interpretations are made for each set of data. Results from one set of data are not used to
build on during analysis. Following separate collection, analysis, and interpretation
phases, the researcher integrates the inferences. Concurrent triangulation occurs when
quantitative and qualitative data are collected and analyzed separately but at the same
time, with the findings converging in the conclusions in order to answer an overarching
research question. The challenge associated with using a concurrent design is that it
requires a significant level of expertise on the part of the researcher to analyze both the
quantitative and the qualitative data.



Parallel designs are those in which data are collected and analyzed separately,
similar to concurrent designs. However, in concurrent designs, inferences are made in a
more integrated manner, whereas in parallel designs each data set leads to its own set of
inferences. Indeed, sometimes the results may be reported in two separate write-ups in the
same report. Some authors have referred to this design as a quasi-mixed method design.

In sequential designs, data that are collected and examined in one stage inform the
data collected in the next phase. Data analysis begins before all data are collected and the
analysis may influence choices made in conducting the next phase of the study. Each
separate phase may shape the conceptual and methodological approaches used in
following phases. Different forms of data are collected in sequence at different phases in a
study. For example, a quantitative survey collected and analyzed in the first phase may
inform the second qualitative phase of the study, or qualitative observations conducted in
the first phase may inform development of a quantitative survey in the second phase. The
sequential order- ing provides information necessary to conduct a more thorough study.

Fully mixed designs or fully integrated designs involve mixing of the qualitative
and quantitative approaches in an interactive way throughout the study. At each stage, one
approach may influence the implementation of the other. There are multiple points of
integration throughout the study, from data collection to data analysis and interpretation
and inferences.

Conversion designs involve transforming data, which means that data collected in
one form (e.g., numbers or text) are converted to a different form and then analyzed.
Qualitative data may be converted to quantitative data or vice versa. Data are analyzed
from both perspectives and inferences are made based on both sets of analyses. For
example, qualitative data might be quantitized by counting the number of times a
particular word is used or the number of times a particular theme is identified.
Quantitative data from a survey might be used to create a written profile of a group. Data
transformation is a term used in mixed methods research to describe when a researcher
begins with qualitative data, such as interviews or secondary source documents, and
through analysis trans- forms the words into numerical values for comparative or
statistical analysis. The transformed data are then compared with data from a comparable
method (quantified qualitative data compared with statistical results from a separate
phase, or qualitized descriptive statistics from a survey compared to coded themes that
emerge from analysis of interviews). Conversion designs require that at least one of the
data sources be transformed into another type of data. Some researchers argue that it is
inappropriate to quantify qualitative data because it considerably reduces the value of the
data as a tool for understanding a phenomenon.

In an embedded design, one form of data supports a second form of data within a
single study. For example, if the purpose for the research project is to inform
administration or change policy, quantitative and qualitative data may be required to
convince those with the power to make changes that the results are credible. The rationale
for an embedded design is that a single data set is not sufficient to answer different
questions, and each type of question requires different types of data. The most common
form of embedded mixed methods research is when quantitative data are embedded
within a qualitative case study and the quantitative data are supportive of the major
qualitative findings. Alternatively, qualitative data can be supportive of statistical results
by addressing questions that are unanswerable using experimental or correlation research.
The hallmark of an embedded design is that the different data sets are mixed at the design
level rather than in the conclusions so that one of the data sources is relegated to playing a
supplemental role to the prominent data source. One challenge that must be dealt with
when conducting embedded research is that the purpose of the study may not be clear to
the participants or research team members involved in the project.

NOTATION SYSTEM

The most widely accepted notation system used in mixed methods designs, first
introduced by Morse (1991), uses a plus sign (+) to indicate that the data collection and
analysis of methods occur at the same time. An arrow (→) indicates that data collection
and analysis occur in sequence. Morse also indicates that the weight or importance of the
methods within the study should be denoted by using uppercase letters for prominence
and lowercase letters to indicate less dominant methods. There are no specific rules that
determine appropriate proportions of qualitative and quantitative research in a mixed
methods study. Some researchers use parentheses to indicate methods that are embedded
within other methods. The notation system rules are shown here

Weighting priority

QUAL+QUAN (both are equally important) QUAN+qual (quantitative approach is
dominant) QUAL+quan (qualitative approach is dominant)

Sequence

QUAN → Qual (quantitative collection or analysis occurs first followed by qualitative
collection/analysis)

QUAL → Quan (qualitative collection or analysis occurs first followed by quantitative
collection/analysis)

Embedded QUAL (quan) QUAN (qual)

Visually representing a mixed methods design is a useful tool for helping the
researcher to determine and show logic in the relationship between the components. The
visual interactive model allows for a clear representation of how the purpose, conceptual
framework, methodology, methods, and issues including validity/credibility interact with
the central overarching research question.

DATA ANALYSIS IN MIXED METHODS RESEARCH

In general, because mixed methods research is a combining of qualitative and quantitative
methods, the basic analysis techniques used in those approaches hold true in mixed
methods research as well. The primary difference is in integrating the data. The basic
analysis techniques described in previous chapters would be used to analyze the various
types of data collected in a mixed methods study.

However, Onwuegbuzie and Teddlie (2003) present a seven-stage conceptualization of

mixed methods data analysis that is useful to consider, particularly in conversion
designs.

Stage 1, data reduction, involves analyzing the qualitative data via theme analysis or
thematic coding while also analyzing the quantitative data via descriptive statistics, factor
analysis, etc.

Stage 2, data display, involves using tables and graphs to display the quantitative data
and using other forms, such as matrices, rubrics, and lists, to describe the qualita- tive
data.

Stage 3 is data transformation, in which qualitative data (words) may be transformed

into quantitative data (numbers) and/or quantitative data (numbers) may be transformed
into qualitative data (narrative).

Stage 4 is data correlation, which involves comparing the data from the different analy-
ses (quantitizing and qualitizing compared to the originals). The authors use the term
correlation in a different sense than typically considered in quantitative research. Here,
the researcher is comparing, for example, the original qualitative data and the quantitized
qualitative data to determine whether the two sets seem to reflect similar findings.

Stage 5 is data consolidation, in which both sets of data are combined to create a new set
of data or variables.

Stage 6, data comparison, involves comparing data from the qualitative and quantitative
data sources.
Finally, stage 7 is data integration, in which the data and interpretations are integrated
into either a coherent whole or reported in two separate sets (qualitative and quantitative)
of coherent wholes. Figure 19.1 and Table 19.2 present qualitative data (McKenzie, 2008)
that were quantitized in order to show how the data transformation stage might work.

STRENGTHS AND WEAKNESSES OF MIXED METHODS

RESEARCH
As with any research approach, there are both strengths and weaknesses in
conducting mixed methods research. Johnson and Onwuegbuzie (2004, p. 21) provide a
good chart that describes the strengths and weaknesses of mixed methods research. On
the positive side, the use of words, narrative, and images can be used to add meaning to
numbers, and using numbers can add precision to the qualitative data. Mixed methods
research can take advantage of the combined strengths of qualitative and quantitative
approaches and can use the strengths of one method to overcome the weaknesses of
another. A broader range of research questions can be examined because the researcher is
not confined to a single method. Mixed methods research can provide stronger evidence
for a conclusion through corroboration of findings. The researcher may have insights that
could have been missed with only a single method. The combination may produce more
complete understandings of phenomenon or more complete knowledge to inform theory
or practice.

However, there are also a number of weaknesses. It is difficult for a single

researcher to carry out both quantitative and qualitative research. It is difficult to have
equal skill sets in both methods, and especially if data are collected simultaneously, there
is a great time commitment. The researcher must be able to understand the complexities
of both approaches so as to make wise decisions about how they can appropriately be
mixed. Conducting a mixed methods study is likely to be more expensive than using a
single approach. Quantitizing and qualitizing data can have its own problems. Also,
interpreting conflicting results may be difficult.

References and Further readings

Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed
methods. Journal of Mixed Methods Research, 1(1), 112–133.

ohnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm
whose time has come. Educational Researcher, 33(7), 14–26.

Creswell, J. & Plano Clark, V. (2007). Designing and conducting mixed methods research.
Thousand Oaks, CA: Sage.

Morse, J. M. (1991). Approaches to qualitative quan- titative methodological triangulation.

Nursing Research, 40(2), 120–123.