IMPROVING STUDENT CRITICAL THINKING AND PERCEPTIONS OF

CRITICAL THINKING THROUGH DIRECT INSTRUCTION IN RHETORICAL

ANALYSIS

by

Lauren A. McGuire

BONITA WILCOX, Ph.D., Faculty Mentor and Chair

MAUREEN MCGLYNN, Ph.D., Committee Member

HOWARD JACOBS, Ph.D., Committee Member

Barbara Butts Williams, Ph.D., Dean, School of Education

A Dissertation Presented in Partial Fulfillment

Of the Requirements for the Degree

Doctor of Philosophy

Capella University

May 2010
 UMI Number: 3408479

All rights reserved

INFORMATION TO ALL USERS

The quality of this reproduction is dependent upon the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.

UMI 3408479
Copyright 2010 by ProQuest LLC.
All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.

ProQuest LLC
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106-1346
© Lauren A. McGuire, 2010
 Abstract

This study investigated the effect of direct instruction in rhetorical analysis on students’

critical thinking abilities, including knowledge, skills, and dispositions. The researcher

investigated student perceptions of the effectiveness of argument mapping; Thinker’s

Guides, based on Paul’s model of critical thinking; and Socratic questioning. The explicit

goals of the study were to promote critical thinking knowledge, skills, and dispositions

and to determine improvement in student perceptions of critical thinking knowledge,

skills, and dispositions. A mixed methodology teacher action approach was used in this

one-group pretest-posttest design research study which consisted of a small sample size

(N = 15) and pertained to a 1-semester critical thinking college course that was taught by

the researcher in a naturalistic setting. Changes in critical thinking were evaluated using

the pretest and posttest scores on the California Critical Thinking Skills Test 2000 and the

Pre- and Post- Critical Thinking Surveys. Data from the research findings indicated there

were no significant differences between the pretest and posttest scores for induction,

analysis, and evaluation; however, there was a slight difference between the pretest and

posttest scores for inference and deduction with a negligible significance level of (p =

0.2). Findings from the Pre- and Post- Critical Thinking Surveys indicated a 2.2 mean

gain in critical thinking knowledge, a 1.8 mean gain in critical thinking skills, and a 1.8

mean gain in critical thinking dispositions. These data suggest that purposeful

implementation of the above named interventions could strengthen students’ perceptions

of critical thinking and of their own critical thinking abilities.

 Acknowledgments

I wish to express my deepest gratitude to Dr. Bonita Wilcox, who served as my

mentor during the dissertation process and who has been truly inspirational. I will be

forever grateful for her knowledge, expertise, insightful guidance, and patience. She has

been instrumental in my achievements, and I consider myself incredibly lucky to have

had her as my mentor. I also wish to thank the other members of my committee, Dr.

Maureen McGlynn and Dr. Howard Jacobs, for their positive feedback, assistance, and

counsel. Finally, I wish to thank my husband, Terry, for his love and support and for his

unremitting encouragement and tolerance throughout this long, arduous, yet gratifying

educational journey.

iv
 Table of Contents

Acknowledgments iv

List of Tables viii

List of Figures ix

CHAPTER 1. INTRODUCTION 1

Background of the Study 4

Statement of Problem 7

Purpose of the Study 7

Research Questions 8

Significance of the Study 8

Definition of Terms 9

Assumptions and Limitations 12

Theoretical Framework 14

Nature of the Study 18

Organization of the Remainder of the Study 19

CHAPTER 2. LITERATURE REVIEW 20

Introduction 20

The Critical Thinking Movement 21

Defining Critical Thinking 24

Importance of Teaching Critical Thinking 30

Teaching Critical Thinking 34

Specific Strategies 38

Teaching Critical Thinking: Related Studies 42

v
 Conclusion 49

CHAPTER 3. METHODOLOGY 52

Introduction 52

Statement of Problem 52

Research Questions 53

Research Design and Methodology 54

Population and Sample 55

Instrumentation 57

Field Test 60

Data Collection 60

Data Analysis 63

Ethical Considerations 64

Limitations of the Study 65

Summary 66

CHAPTER 4. DATA COLLECTION AND ANALYSIS 68

Introduction 68

Statement of the Problem 69

Research Questions 69

Overview of Data Collection 70

Data Collection and Instruments 70

Demographic Characteristics 72

Results 75

Summary 96

vi
CHAPTER 5. RESULTS, CONCLUSIONS, AND RECOMMENDATIONS 98

Introduction 98

Overview of Methodology 99

Summary of Findings 99

Open-Ended Questions 109

Summary of Pre- and Post- Critical Thinking Surveys: Knowledge,

Skills, and Dispositions 111

Limitations 119

Recommendations 120

Conclusions 123

REFERENCES 125

APPENDIX A. PRE- CRITICAL THINKING SURVEY 132

APPENDIX B. POST- CRITICAL THINKING SURVEY 136

APPENDIX C. REQUEST FOR PERMISSION TO DR. KIMBERLY METCALF 142

APPENDIX D. LETTER OF PERMISSION FROM DR. KIMBERLY METCALF 143

vii
 List of Tables

Table 1. Sample Demographic Characteristics 73

Table 2. Sample Demographics: Skills and Abilities 74

Table 3. Student Perceptions of Critical Thinking: Argument Mapping 77

Table 4. Student Perceptions of Critical Thinking: Paul’s Thinker’s

Guides 79

Table 5. Student Perceptions of Critical Thinking: Socratic Questioning 80

Table 6. Responses to Critical Thinking Survey: Knowledge 83

Table 7. Responses to Critical Thinking Survey: Skills 84

Table 8. Responses to Critical Thinking Survey: Dispositions 86

Table 9. Results of Pre- Critical Thinking Survey 88

Table 10. Results of Post- Critical Thinking Survey 89

Table 11. Pre- and Post- Critical Thinking Surveys: Gains 90

Table 12. CCTST–2000 Pretest Scores 92

Table 13. CCTST–2000 Posttest Scores 92

Table 14. t-Test Analyses: CCTST Pretest and Posttest Totals 93

Table 15. t-Test Analyses: Analysis, Inference, and Evaluation 94

Table 16. T-Test Analysis: Induction and Deduction 95

viii
 List of Figures

Figure 1. Comparison of mean scores by question type 95

ix
 CHAPTER 1. INTRODUCTION

Educators agree that the development of higher order or cognitive intellectual

abilities is of utmost importance and that critical thinking “is central to both personal

success and national needs” (Paul, 2004, p. 2). In his article, Paul (1995f) cited

Kennedy’s research findings; the first finding indicated

National assessments in virtually every subject indicate that, although our

students can perform basic skills pretty well, they are not doing well on thinking
and reasoning. American students can compute, but they cannot reason. . . . They
can write complete and correct sentences, but they cannot prepare arguments. . . .
Moreover, in international comparisons, American students are falling behind . . .
particularly in those areas that require higher order thinking. . . . Our students are
not doing well at thinking, reasoning, analyzing, predicting, estimating, or
problem solving. (p. 19)

Educators are beginning to explore those pedagogical practices that could

effectively develop student critical thinking knowledge, skills, and dispositions across the

academic disciplines. Further, instructional strategies that advance critical thinking

pedagogy on a consistent basis could enable instructors with the ability to encourage in

their students the transfer of those critical thinking skills learned in the academic

environment to their professional and personal lives.

In order to advance critical thinking pedagogy and encourage students’ critical

thinking abilities, however, educators must possess a clear definition of what critical

thinking is. As the concept of critical thinking is highly complex, a variety of definitions

exist, so it is difficult to pinpoint the exact meaning of the skills involved in this intricate

1
process. P. A. Facione (2000), using the two-sentence definition of critical thinking

generated from the Delphi Report, identified critical thinking as “the process of

purposeful, self-regulatory judgment. This process gives reasoned consideration to

evidence, context, conceptualizations, methods, and criteria” (p. 5). P. A. Facione argued

that a true definition of critical thinking involves both skills and habits of mind or

dispositions. Similar to P. A. Facione and his concern with the necessity for an

inclination toward critical thinking, Paul (1993) asserted that critical thinking is an

intellectually disciplined process “of actively and skillfully conceptualizing, applying,

analyzing, synthesizing or evaluating information” (p. 3). Halpern (1999) believed that

critical thinking is “purposeful, reasoned, and goal-directed” (p. 70), while maintaining

that “it is the kind of thinking involved in solving problems, formulating inferences,

calculating likelihoods, and making decisions” (p. 70). In a statement in 1997 for the

National Council for Excellence in Critical Thinking Instruction, Scriven and Paul (1987)

defined critical thinking as “self-guided, self-disciplined thinking which attempts to

reason at the highest level of quality in a fair-minded way” (p. 1). As with P. A. Facione,

Scriven and Paul contended that “critical thinking varies according to the motivation

underlying it” (p. 1), it is “never universal in any individual” (p. 1), and “the development

of critical thinking skills and dispositions is a life-long endeavor” (p. 1). Critical thinking

is not something that can be learned immediately; however, educators can encourage

students to be aware of their thinking so that it becomes intentional rather than accidental.

Critical thinking is a complex skill that takes a lifetime to master and that involves

deliberate practice and engagement (van Gelder, 2005).

2
 The greatest contribution of the community college, according to Elder (2000),

has been its emphasis on teaching students those abilities needed for competency in the

workplace, yet Elder claimed that “training students for job performance in narrowly

defined skill areas no longer serves students well” (p. 1). Rather, current instructional

practices need to begin to place more emphasis on those critical thinking skills necessary

for survival in the new economy. What follows is the necessity for educators to begin to

teach those intellectual skills that encourage mental flexibility and intellectual discipline

so they can deal appropriately with complex intellectual tasks.

A primary goal for educators, then, is to provide students with opportunities to

struggle with concepts, find meaning, distinguish bias, and use logic in arguments so they

may gain a deeper understanding of the world in which they live. The question is what

teaching methodologies can be used by educators to promote the development of

students’ critical thinking skills?

Educators are beginning to place more emphasis on increasing students’ critical

thinking abilities while attempting to include critical thinking curriculum into a variety of

academic disciplines. Although colleges and universities offer critical thinking courses,

critical thinking can be embedded in the instruction of a variety of academic disciplines,

and faculty can engineer their course focus so that it is more thinking-skills-based

(Halpern, 1999). Mendelman (2007) believed critical thinking should be taught in every

course in the humanities, and stated

In a day and age in which more and more children grow up engaged with
primarily passive activities like television, video games, and the Internet, teaching
critical reading is one of the most important, and most difficult, burdens of the
classroom. (p. 300)

3
 Critical thinking abilities are necessary for success in college and for responsible

citizenry in a highly competitive society. As Elder suggested, true critical thinkers

Recognize the complexities in developing as thinkers, and commit themselves to

life-long practice toward self-improvement. They embody the Socratic principle:
The unexamined life is not worth living, because they realize that many
unexamined lives together result in an uncritical, unjust, dangerous world. (as
cited in Scriven & Paul, 1987, p. 2)

Background of the Study

Students entering postsecondary institutions often lack the necessary level of

literacy and cognitive skills required for academic success or for successful living in a

global economy. National assessments indicate that “although students can perform basic

skills pretty well, in international comparisons, American students are not doing well at

thinking, reasoning, analyzing, predicting, estimating, or problem solving” (Paul, 1995g,

p. 3). Van Gelder (2005) maintained that “a majority of people cannot, even when

prompted, reliably exhibit basic skills of general reasoning and argumentation” (p. 2).

Students are, or seem to be, unable to provide the essential evidence needed to support

their reasoning. Jackson (2008) discussed the need for educators to provide opportunities

for students to grapple with complexities so they may cultivate their problem-solving

abilities. Jackson interviewed Norbert Elliot, a teacher at New Jersey Institute of

Technology (NJIT). Elliot had launched a vigorous campaign at NJIT for information

literacy. His chief concern with students was their seeming inability to “sift, use, and

make meaning out of the information around them, in other words to read their world” (as

cited in Jackson, 2008, p. 163). In an internal assessment of information literacy, Elliot

observed that

4
 The writing portfolios of NJIT’s seniors scored an average of 6.14 on a scale of
12, or just below “satisfactory.” Students could find and cite sources better than
they were able to judge their relevance and authority, and were even less able to
use information to support their argument. (as cited in Jackson, 2008, p. 165)

College students, when confronted with controversial issues, are all too often prone to

shallow, narrow-minded assumptions. The challenge for educators, then, is to encourage

disciplined thought and to teach students that “uncertainty is an ally of good thinking and

knowledge is an evidence-based construction” (Jackson, 2008, p. 229).

A strong critical thinking pedagogy that encourages students’ critical knowledge,

skills, and dispositions may improve students’ academic success while encouraging those

abilities needed for transfer and for competency in the workplace. Elder (2000)

contended, however, that traditional education is not nurturing the intellectual capabilities

needed for personal and academic success. Often, students are merely asked to write

down facts rather than to question or reflect on their reading, and, as a result, they are

incapable of drawing inferences and of engaging in complex conversations about the

literature (both fiction and nonfiction prose) they read. Elder suggested further that “as

the economic structure of the world becomes more complex” (p. 1) and “as we become

increasingly more interdependent both at home and abroad, ‘training’ students for job

performance in narrowly defined skill areas no longer serves students well” (p. 1).

Willsen (1997) discussed the increasing complexities of the world and suggested

these new global realities “are rapidly working their way into the deepest structures of

our lives: economic, social, and environmental realities—realities with profound

implications for teaching and learning” (p. 1). The question is whether educational

institutions are preparing students to adapt to and accommodate for these complexities.

5
Cultivating students’ abilities to develop higher order thinking skills and problem-solving

strategies could better prepare students for survival in this complex world. As Willsen

(1997) reasoned

We must sooner or later abandon the traditional attempt to teach our fellow
citizens what to think. Such efforts cannot prepare us for the real world we must,
in fact, face. We must concentrate instead on teaching ourselves how to think,
thus freeing us to think for ourselves, critically, fairmindedly, and deeply. We
have no choice, not in the long haul, not in the face of the irrepressible logic of
accelerating change and increasing complexity. (p. 16)

It is the responsibility of educational institutions to promote and develop students’

critical thinking abilities. Sternberg (2003), in his discussion of the future of education in

the United States, argued that educational institutions far too often emphasize rote

memorization; while “rote memorization requires recital and repetition” (p. 1), critical

thinking “requires skillful analysis, evaluation, and interpretation” (p. 1). Although all

individuals need a “knowledge base” (Sternberg, 2003, p. 1). or store of information, this

knowledge base must prove useful for living, and instead of emphasizing the

memorization of trivial facts, Sternberg encouraged educators to teach usable information

that students can transfer into their lives. Rather than being taught how to think critically,

students are far too often being taught to do little more than recall and recognize; they

“are becoming highly susceptible to the commission of cognitive fallacies” (Sternberg,

2003, p. 1) and, as a result, they may tend to “act on their prejudices and their fears”

(Sternberg, 2003, p. 1) rather than on reasoned judgment.

6
 Statement of Problem

It is not known whether direct instruction in rhetorical analysis will improve

learners’ critical thinking knowledge, skills, and dispositions. This study explored

instructional pedagogy to promote students’ critical thinking abilities. The researcher

investigated whether direct instruction in rhetorical analysis in a college critical thinking

course could make a difference in students’ critical thinking skills. The interventions

included argument mapping; Thinker’s Guides, based on Paul’s (1995f) model of critical

thinking; and Socratic questioning. These interventions were supplemented with course

work that involved reader/writer workshops and peer review.

Purpose of the Study

The purpose of this study was to determine whether there would be improvement

in students’ critical thinking abilities, including knowledge, skills, and dispositions, after

the intervention of direct instruction in rhetorical analysis. The researcher investigated

student perceptions of the effectiveness of the following instructional strategies:

argument mapping; Thinker’s Guides, based on Paul’s (1995f) model of critical thinking;

and Socratic questioning. The goal of the study was, on the one hand, to promote critical

thinking knowledge, skills, and dispositions, and, on the other hand, to determine

improvement in student perceptions of critical thinking knowledge, skills, and

dispositions so that they may transfer those skills across the academic disciplines and into

their personal and professional lives.

7
 Research Questions

The following research questions guided this study:

1. What were the students’ perceptions of critical thinking before and after direct
instruction in argument mapping?

2. What were the students’ perceptions of critical thinking before and after direct
instruction in Paul’s Thinker’s Guides?

3. What were the students’ perceptions of critical thinking before and after direct
instruction in Socratic questioning?

4. What did the students report about the changes in their knowledge, skills, and
dispositions relating to critical thinking after the direct instruction in rhetorical
analysis?

Significance of the Study

A primary goal for educators is to instill in their students a questioning, reflective,

critical mind. The American educational system has been a target for reform to teach

students to think critically, and the educational arena is beginning to take the issue of

critical thinking seriously. The California State University General Education

Requirements include a graduation requirement in critical thinking. In light of the

increasing demands and accelerating changes occurring in the 21st century, educators are

experiencing profound challenges, and the question continues as to which pedagogical

practices would be most effective for the instruction and assessment of critical thinking.

Paul (1995a) believed that “only through an explicit shift to a critical conception

of education, with an explicit critique and rejection of the assumptions of didactic

education, can we achieve significant reform” (p. 278). As national assessment of student

achievement far too often focuses on lower order thinking and learning, Paul (1995a)

8
argued that assessment must focus on higher order thinking, reasoning, and authentic

performance. Paul (1995c) further contended that “critical thinking is the heart of well-

conceived educational reform and restructuring because it is at the heart of the changes of

the 21st Century” (pp. 97–98). At the request of the U.S. Department of Education, Paul

(1995f) developed A Model for the National Assessment of Higher Order Thinking. This

paper outlined the problem of lower order learning; explained 21 criteria for higher order

thinking assessment; defended the view that a clear, substantial concept of critical

thinking meets those criteria; articulated four domains of critical thinking; offered

recommendations as to how to assess the domains of critical thinking; and discussed the

value of the proposed recommendations for educational reform (Paul, 1995f).

The development and implementation of pedagogy that promotes student

engagement in the learning process could encourage student critical thinking abilities and

the transfer of those abilities necessary for academic achievement, personal success, and

success in the work force. Although critical thinking abilities can be encouraged in all

academic disciplines, many community colleges offer specific courses in critical

thinking. Using direct instruction in rhetorical analysis, this study adds to the knowledge

of instructional practices in critical thinking courses that encourage student engagement

in critical thinking and that may alter student perception of critical thinking and promote

awareness for making critical thinking a lifelong endeavor.

Definition of Terms

Argument. A connected series of statements intended to establish a definite

proposition. Arguments “constitute a body of evidence in relation to some proposition (an

9
idea that is true or false). The proposition is expressed in some claim, and the evidence is

expressed in other claims” (van Gelder, 2005, p. 4). Paul (1995d) asserted that

In emphasizing critical thinking, we continually try to get our students to move

from the first sense of the word to the second; that is, we try to get them to see the
importance of giving reasons to support their views without getting their egos
involved in what they are saying. (p. 522)

Argument mapping. A visual representation of the structure of an argument in

informal logic. It is a map that “makes the logical structure of the argument completely

explicit” (van Gelder, 2005, p. 4). These maps are often used in the teaching of reasoning

and critical thinking, and can support the analysis of pros and cons when deliberating

over problems.

Critical thinking. In 1990, a panel of experts, who included 46 men and women

from throughout the United States and Canada and who represented a variety of scholarly

disciplines, participated in a research project conducted on behalf of the American

Philosophical Association. This 2-year Delphi project was conducted by P. A. Facione

(2006), and the final Delphi Report generated a consensus definition of critical thinking

(P. A. Facione, 2006). The final Delphi Report defined critical thinking as “purposeful,

self-regulatory judgment that results in interpretation, analysis, evaluation, and inference,

as well as explanation of the evidential, conceptual, methodological, criteriological, or

contextual considerations upon which that judgment is based” (P. A. Facione, 1990, p. 2).

Critical Thinking Course. The critical thinking course is designed explicitly to

develop the student’s critical thinking, reading, and writing skills beyond the level

achieved in a basic English course. It focuses primarily on the analysis and evaluation of

10
expository and argumentative discourse and on writing analytical and argumentative

essays.

Direct instruction. Created by Engelmann and Becker, direct instruction is a

model for teaching that emphasizes well-developed, highly scripted, and carefully

planned lessons designed around small learning increments and clearly defined and

prescribed teaching tasks. Through the use of carefully prescribed instructional practices,

direct instruction can improve student academic performance (Engelmann, n.d.).

Paul’s Thinker’s Guides. Paul’s model of critical thinking is comprised of three

basic concepts: elements of reasoning, intellectual standards, and intellectual traits.

Thinking can be divided into seven components, including purpose; point of view;

assumptions, implications, and consequences; data and information; inferences and

interpretations; concepts; and the question at issue. Moreover, thinking can be measured

against standards, such as clarity, accuracy, precision, relevance, depth, breadth, logic,

significance, and fairness (Paul, 1995g). Paul (1995g) maintained that all thinkers should

cultivate positive intellectual traits such as intellectual humility, intellectual perseverance,

intellectual integrity, intellectual courage, confidence in reason, and intellectual empathy.

Rhetorical analysis. Aristotle defined rhetoric as “the faculty of observing in any

given case the available means of persuasion” (as cited in Shea, Scanlon, & Aufses, 2008,

p. 1). Rhetoric is skillfully constructed written, spoken, or visual argument; it is the

means by which individuals appeal to an audience; and it includes a coherent, balanced

exchange of opposing views. Effective rhetoric has a context and a purpose, and it offers

“tools to resolve conflicts without confrontation, to persuade readers or listeners to

support [a] position, or to move others to take action” (Shea et al., 2008, p. 1). Rhetorical

11
analysis is the act of carefully scrutinizing the tools and techniques the author, speaker, or

visual artist has used to appeal to the audience. It is, according to Shea et al. (2008),

nurturing the student’s ability to discover the “nutritional value in the books, stories,

essays, and poems [they] study in school” (p. 35).

Scaffolding. Structure and guidance provided by educators to support students

with challenging tasks. Many theorists believe that scaffolding may improve problem-

solving performance. Ways educators can assist students with difficult problems include

simplifying or dividing a task into smaller units, modeling problem solutions, asking

probing questions, pointing out errors, keeping students focused and motivated, and

giving frequent feedback (Ormrod, 2004).

Socratic questioning. “A mode of questioning that deeply probes the meaning,

justification, or logical strength of a claim, position, or line of reasoning” (Paul, 1995d, p.

18). Socratic questioning encourages rigorous, thoughtful dialogue, and involves the

practice of thoughtful, probing questioning, which enables students to examine the logic

and validity of their ideas.

Assumptions and Limitations

Students enrolled in community college courses in California constituted the

research sample. Although the findings should prove useful, it is doubtful they can be

generalized to a target population in other locations unless the sample sizes share similar

characteristics. Students enrolled in California community colleges are similar to students

enrolled in courses elsewhere; however, as Gall, Gall, and Borg (2003) maintained,

“Generalizations to other cases can be done, but it must be done on a case-by-case basis”

12
(p. 10). Therefore, generalization may not be possible, nor is it necessary as it is truly up

to the reader to find similarities in an action research study to a different site or location.

Possible limitations exist due to the modest sample size of 15 students and the

short term of instruction necessitated by an 18-week semester course. Research suggests

that critical thinking is a lifelong process and that successful interventions may need to

occur over an academic year or over several years. Ideally, a longitudinal study would be

most beneficial in determining growth in critical thinking knowledge, skills, and

dispositions as a result of the interventions. The researcher was the teacher of an intact

class, and no attempt at randomization occurred; therefore, the study was limited to a

description of an existent situation rather than creating an experimental situation. In

teacher action research studies, the data are collected and analyzed to assess and improve

educational practice. Since the researcher was the instructor in the critical thinking

course, there was potential for researcher bias as the researcher may have authority over

the participants. The researcher understood this and attempted to approach the study with

caution and without preconceived assumptions of possible outcomes or results. Care was

taken to maintain clarity and objectivity through sharing with research participants and

through personal reflection, assessment, and modification of instructional practices. For

further consideration is the Hawthorne effect which refers research situations where

“experimental conditions are such that the mere fact that individuals are aware of

participating in an experiment, are aware of the hypothesis, or are receiving special

attention improves their performance” (Gall et al., 2003, p. 376). The researcher

accommodated for this concern by providing equal guidance and attention to all students

in the course, and research participants were not given special attention. Finally, this

13
study was limited to examination of the interventions employed in a critical thinking

course and did not attempt to measure the influence of out-of-classroom experiences in

growth in critical thinking.

Theoretical Framework

P. A. Facione’s (2006) definition of and approach to critical thinking offers a core

of critical thinking skills that includes analysis, interpretation, inference, explanation,

evaluation, and self-regulation. Interpretation occurs when the individual comprehends

the meaning of a variety of experiences, and includes categorizing, decoding, clarifying,

recognizing a problem without bias, and distinguishing the main idea from subordinate

ideas. Analysis is when the learner identifies the intended and inferential relationships

among statements, and includes the examination of ideas, the analysis of arguments and

the ability to examine alternative approaches to a problem and identify any unstated

assumptions. Evaluation includes the learner’s ability to consider the credibility of

assumptions and to compare strengths and weaknesses of alternative views or beliefs.

Inference refers to a person’s ability to construct meaning and to identify the implications

of a particular position (P. A. Facione, 2006).

P. A. Facione (2006) suggested that “we should be cautious about proposals

suggesting oversimplified ways of understanding how humans think” (p. 12). P. A.

Facione’s method of critical thinking involves two systems that are considered essential

decision-making tools. System 1, the intuitive, is reactive, deliberate, and holistic,

whereas System 2 is reflective, analytical, and evaluative. System 1 thinking relies on

what P. A. Facione referred to as “cognitive shortcuts [which include] key situational

14
characteristics, readily associated ideas, and vivid memories to arrive quickly and

confidently at a judgment” (p. 12). This system is most helpful when decisions must be

made quickly and when immediate action is required. System 2 thinking is more

reflective and is useful when individuals are in “unfamiliar situations and have more time

to figure things out” (P. A. Facione, 2006, p. 13). It allows deliberation, planning ahead,

and considering options; “it is reasoning based on what we have learned through careful

analysis, evaluation, explanation, and self-correction; and it values intellectual honesty

and fair-mindedness” (P. A. Facione, 2006, p. 13). P. A. Facione asserted that individuals

who have a disposition toward critical thinking are inquisitive, systematic, judicious,

truth seeking, analytical, open-minded, and confident in their reasoning.

The ideal critical thinkers, according to P. A. Facione (2006), are characterized by

their cognitive skills and by their approach to life. They illustrate

Clarity in stating a question, orderliness in working with complexity, diligence in

seeking relevant information, reasonableness in selecting and applying criteria,
care in focusing attention on the concern at hand, persistence through difficulties
encountered, and precision to the degree permitted by the subject and the
circumstances. (P. A. Facione, 2006, p. 9)

Finally, P. A. Facione (2006) asserted that good critical thinkers can misuse those

superior skills in immoral and unethical ways through manipulation and exploitation.

Ultimately, being a good critical thinker is dependent on the character and integrity of the

individual. For this reason, “knowledge and skills are not sufficient. We must look to a

broader set of outcomes including habits of mind and dispositions, such as civic

engagement, concern for the common good, and social responsibility” (P. A. Facione,

2006, p. 11).

15
 P. A. Facione’s (2006) definition of critical thinking, generated by the scholars

involved in the Delphi project, was the working definition for the study. The Delphi

consensus definition offers a means by which to address the U.S. Department of

Education’s Education Goals: 2000 mandate (P. A. Facione, 2000). The California

Critical Thinking Skills Test–Form 2000 (CCTST–2000), which was used in this study,

derived its construct validity from this critical thinking conceptualization. However,

although the CCTST–2000 targets the core critical thinking skills of analysis,

interpretation, inference, evaluation, and explanation, it does not measure critical thinking

dispositions. P. A. Facione’s reasoning motivation tool, the California Critical Thinking

Dispositions Inventory (CCTDI), was designed to measure the dispositional dimension of

critical thinking; however, the CCTST–2000 proved a valid and reliable instrument for

the research purposes as “high scores on the CCTST are often correlated with a strong

disposition toward critical thinking (high scores on the CCTDI). Individuals with high

scores on the CCTST–2000 and low scores on the CCTDI are relatively rare” (P. A.

Facione, 2000, p. 11).

Paul and Elder (2006a), leaders in the critical thinking movement, developed a

model for critical thinking that provides a practical approach for developing students’

critical thinking abilities. Paul and Elder applied the core elements of critical thinking and

the characteristics of good critical thinkers from the Delphi Report, generated by scholars

in critical thinking, to address the needs of educators and students (P. A. Facione, 2006).

Paul’s (2004) model insists there are universal standards for critical thinking, which

include clarity, accuracy, precision, relevance, consistency, depth, and breadth. Paul

maintained that critical thinkers analyze, assess, and upgrade their thinking, while

16
advocating that instructors design curriculum so that students understand the sense and

logic of what they learn, and that this learning will increase comprehension and insight.

Teaching critical thinking appeals to reason and evidence, encourages students to

discover and process information, and provides occasions for students to think through to

conclusions, defend positions, consider alternative viewpoints, analyze concepts, clarify

issues, solve problems, and transfer ideas to new concepts.

Van Gelder (2005) drew on cognitive science for teaching critical thinking, and

reasoned that because critical thinking is a difficult, lifelong process, it requires repeated

engagement, practice, and some theoretical knowledge for transfer. As arguments are

presented in “streams of words, whether written or spoken” (van Gelder, 2005, p. 4), van

Gelder advocated handling arguments through argument mapping. The logic of

arguments is expressed in sequences, and those sequences can be diagrammed or mapped

out for better, more explicit understanding. Argument maps follow a particular set of

conventions in which the main point is put at the top of the argument tree. Arrows then

indicate that a claim is evidence. The use of the color green and the word reason indicate

they are supporting evidence (van Gelder, 2005). Van Gelder claimed argument maps

clearly and visibly make the logic of arguments more straightforward; as a result,

students’ critical thinking skills and abilities improve when they practice this skill.

In his article, Twardy (2003) claimed that “argument mapping greatly enhances

student critical thinking, more than tripling absolute gains made by other methods” (p. 1).

The results of his study, conducted at the School of Computer Science and Software

Engineering at Monash University in Australia, indicated that students who used van

Gelder’s argument mapping to analyze arguments showed significant gains on the

17
posttest scores using the CCTST. The CCTST is the most widely used measurement of

critical thinking skills as it targets “those core critical thinking skills regarded to be

essential elements in college education” (P. A. Facione, 1992, p. 2). Argument maps

could prove to be useful tools for the analysis of formal arguments as this practice has the

potential to improve students’ ability to closely diagnose and map out rationale while

constructing their own arguments.

Nature of the Study

The mixed-methodology approach was used in this teacher action research study.

The study pertained to one semester of a college critical thinking course that examined

the effect of direct instruction in rhetorical analysis in students’ critical thinking

knowledge, skills, and dispositions. The study occurred in a naturalistic setting, and the

sample was comprised of students enrolled in the critical thinking course. A variety of

works were used to encourage analysis of authors’ rhetorical strategies and stylistic

choices. To reflect the increasing importance of graphics and visual images in texts

published in print and electronic media, students were asked to analyze how such images

both relate to written texts and served as alternative forms of texts themselves. The

interventions were supplemented with course work that involved reader/writer workshops

and peer review.

A Pre- Critical Thinking Survey was given at the beginning of the course and a

Post- Critical Thinking Survey was administered at the end of the course. The qualitative

survey was implemented for analysis of patterns of students’ responses; the observations

made by the survey results offered insights into student perceptions of the most effective

18
instructional interventions. The CCTST–2000 was administered as a pretest at the

beginning of the course and measured the students’ critical thinking skills. The same test

was given as a posttest and measured growth in critical thinking abilities at the end of the

course. Statistical analysis was done on the CCTST–2000 pretest and posttest scored by

Insight Assessment.

Null Hypothesis: There will be no difference in scores on the pretest and posttest

on the California Critical Thinking Skills Test–Form 2000 indicating changes in student

critical thinking skills before and after direct instruction in rhetorical analysis.

Organization of the Remainder of the Study

The remainder of the study is organized into four chapters, in the following

manner: Chapter 2 presents a discussion, evaluation and critique of prior research dealing

with the trends in critical thinking, direct instruction in rhetorical analysis, argument

mapping, Paul’s Thinker’s Guides, and Socratic questioning classroom methodologies;

chapter 3 explains how the study was conducted; chapter 4 includes data analysis and

results and presents a report of the data supported by tables and figures and an

explanation of the statistical analysis; and chapter 5 offers conclusions and

recommendations, evaluates the work, and provides insight into and interpretation of the

study’s results.

19
 CHAPTER 2. LITERATURE REVIEW

Introduction

Ideally, individuals possess the ability to fairly view issues and their world with

an open mind and from a variety of perspectives. It is unknown whether direct instruction

in critical thinking can meet the challenges educators encounter in their attempts to

encourage and develop those essential abilities for reflective, informed thinking. This

need for promoting students’ critical thinking knowledge, skills, and dispositions, and the

responsibility of educational institutions for developing these abilities is addressed by a

variety of scholars and philosophers (P. A. Facione, 2006; Halpern, 2006; Paul, 1995c;

Perkins, 1993; van Gelder, 2005). This chapter explores critical thinking and is divided

into the following sections: an overview of the critical thinking movement, definitions for

critical thinking, the importance of critical thinking, teaching critical thinking, specific

strategies for teaching critical thinking, and related studies of critical thinking. This is

followed by a brief summary that emphasizes the need for teaching or embedding critical

thinking in instructional methodologies. The purpose of this study was to determine

whether there would be improvement in student critical thinking knowledge, skills, and

dispositions on pretest and posttest scores before and after direct instruction in argument

mapping, Paul’s Thinker’s Guides, and Socratic questioning.

20
 The Critical Thinking Movement

A fundamental understanding of the critical thinking movement begins with

contributions from Socrates, who developed a method of asking meaningful questions,

where “confused meanings, inadequate evidence, or self-contradictory beliefs often

lurked beneath smooth but largely empty rhetoric” (Paul, Elder, & Bartell, 1997a, p. 1).

Socrates questioned the assumptions and beliefs of those in authority and established

Socratic questioning, which is a methodology that advocates the importance of asking

probing questions and seeking evidence to examine rhetoric. Socrates’s search for the

essence of reason and truth encouraged a thorough examination of statements, and an

understanding of their evidence, assumptions, theories, reasoning, and implications.

Socrates’s practice and method of reflective, well-reasoned, systematic thinking

influenced the writings of Plato, Aristotle, Thomas Aquinas, Erasmus, Francis Bacon,

Descartes, Sir Thomas Moore, Hobbes and Lock, Robert Boyle, and Sir Isaac Newton

(Paul et al., 1997a).

Twentieth-century theorists whose writings have contributed significantly to

critical thinking theory and education are William Graham Sumner, John Dewey, Edward

Glaser, Jean Piaget, and Lev Vygotsky. Dewey’s pragmatic approach to critical thinking

advocates student-centered rather than subject-centered education. He believed that

genuine education comes about through experience; however, “the quality of any

experience has two aspects. There is an immediate aspect of agreeableness or

disagreeableness, and there is its influence upon later experiences” (Brookfield, Tennant,

& Pogson, 2005, p. 326). Education and life are interrelated, and educators must design

and carefully monitor positive educational experiences. For Dewey, “knowledge is the

21
product of an interaction between the experiencing subject and the external objective

world” (Brookfield et al., 2005, p. 333). Students learn best by doing, and continuity of

experience is essential to growth.

Glaser contributed significantly to critical thinking research. He encouraged

teaching critical thinking and introduced the Watson–Glaser Critical Thinking Appraisal.

He defined the ability to think critically as

[a] an attitude of being disposed to consider in a thoughtful way the problems and
subjects that come within the range of one’s experiences, [b] knowledge of the
methods of logical inquiry and reasoning, and [c] some skill in applying those
methods. (as cited in Scriven & Paul, 1987, p. 2)

Piaget was a biologist who contributed extensively to educational theory and to

the study of child maturation and development (Atherton, 2005). Piaget’s theory is

considered “the single most global theory of intellectual development; it incorporates

such diverse topics as language; logical reasoning; moral judgments; and conceptions of

time” (Ormrod, 2004, p. 144). Piaget proposed the idea of scheme and believed that

knowledge can be understood in terms of structures or operations that, through

assimilation and accommodation, alter during a child’s cognitive development. The

Piagetian view of cognitive development espouses that there are four cognitive structures

that correspond to the stages of child development, and that “children progress through

different types of thinking as they develop toward mature adult thought” (Brookfield et

al., 2005, p. 27). Piaget’s view emphasized individual thought and autonomy. He

believed “the individual will interpret and act accordingly to conceptual categories or

schemas that are developed in interaction with the environment” (Oxford, 1997, p. 39),

that people are intrinsically motivated and actively involved in the learning process, and

22
that cognitive development results from the interactions that individuals have with their

physical environments.

In contrast to Piaget’s view of learning as an individual endeavor, Vygotsky

stressed the importance of past experiences, prior knowledge, society, and culture on

promoting cognitive growth (Dahms et al., 2008). Whereas Piaget was concerned with

the characteristics exhibited by children of a particular age, Vygotsky focused on the

process of child development. Vygotsky believed knowledge is developed through social

interaction, that learning occurs through language and shared experiences, and adults

“foster children’s learning and development in an intentional and somewhat systematic

manner” (Ormrod, 2004, p. 150). Individuals react to, alter, and adapt to their

environment. Vygotsky’s sociocultural perspective includes the concept of the Zone of

Proximal Development (ZPD), which assumes that learning is social and human potential

is limitless, “but the practical limits of human potential depend upon quality social

interactions” (Dahms et al., 2008, p. 3). In Vygotsky’s view, cognitive processes begin to

develop as a result of social interaction within specific cultures, cognitive development

occurs as children discuss and internalize these processes, and “through both informal

conversations and formal schooling, adults convey to children the ways in which their

culture interprets and responds to the world” (Ormrod, 2004, p. 151). Although children

are unable to perform certain tasks independently, their ZPD changes through the

assistance and support of others who are more knowledgeable, and children begin to

reach optimum performance. As students’ abilities develop and certain tasks are

mastered, they begin to acquire the readiness for other more complex skills and to solve

problems through a process referred to by theorists as scaffolding. The assistance of their

23
peers, the teachers who model the processes in a collaborative classroom environment,

and “through both informal conversations and formal schooling, adults convey to

children the ways in which their culture interprets and responds to the world” (Ormrod,

2004, p. 151).

Defining Critical Thinking

Cognitive psychologists and philosophers have offered a variety of definitions for

critical thinking (Chance, 1986; Ennis, 1992; P. A. Facione, 2000; McPeck, 1981; Paul,

1995f; Paul et al., 1997a; Scriven & Paul, 1987; van Gelder, 2005). Chance focused on

argument and defined critical thinking as “the ability to analyze facts, generate and

organize ideas, defend opinions, make comparisons, draw inferences, evaluate arguments,

and solve problems” (p. 6). Chance reasoned that critical thinking involves the “ability to

analyze facts, generate and organize ideas, defend opinions, make comparisons, draw

inferences, evaluate arguments, and solve problems” (p. 6).

Ennis (2002) defined critical thinking as “reasonable reflective thinking focused

on deciding what to believe or do” (p. 1). Ennis viewed an ideal critical thinker as one

who is open-minded; mindful of alternatives; well-informed; judges the credibility of

sources; identifies conclusions, reasons, and assumptions; judges the quality of an

argument, including the acceptability of its reasons, assumptions, and evidence; can

develop and defend a reasonable position; asks appropriate clarifying questions;

formulates plausible hypotheses; plans experiments well; defines terms in a way

appropriate for the context; draws conclusions when warranted, but with caution; and

integrates all items in this list when deciding what to believe or do.

24
 McPeck (as cited in Hatcher, 2000) defined critical thinking as “the skill and

propensity to engage in an activity with reflective skepticism” (p. 2). Hatcher, on the

other hand, believed reflective skepticism is a “vague notion [that has] negative

connotation[s]” (p. 2) and that critical thinking is constructive and its purpose “is not

always negative, but is often to arrive at a positive judgment on an issue” (p. 2). Hatcher

viewed critical thinking as open-ended and that focuses on areas where “evidence and

arguments are appropriate” (p. 3). He believed critical thinking is “thinking that attempts

to arrive at a judgment only after honestly evaluating alternatives with respect to

available evidence and arguments” (p. 3).

As with McPeck, P. A. Facione (2000) was concerned with the dispositions or

habits of mind the individual has toward critical thinking. He maintained the cognitive

skills of “analysis, interpretation, inference, explanation, evaluation, and of monitoring

and correcting one’s own reasoning are at the heart of critical thinking” (p. 2). Individuals

must possess skill in critical thinking, but they must also have the internal motivation to

think. Critical thinking skills can be developed through practice and guidance as good

critical thinking is a complex, purposeful process. Although humans may possess critical

thinking abilities, they may not always choose to act on reasoned judgment for a variety

of personal or professional reasons. N. C. Facione and Facione (2001) suggested

argument mapping and analysis can provide useful ways to understand and interpret the

rationality of a set of arguments involving significant human problems, while maintaining

that “once the argument has been mapped and analyzed, the reasoning it manifests can

then be evaluated for its logical merit” (p. 268).

25
 P. A. Facione served as the lead investigator to coordinate an international effort

to determine the extent to which experts agreed on the definition of critical thinking for

the purposes of college-level teaching and assessment. The result was the 1990 APA

Delphi Report. An international group of experts, who included 46 men and women

throughout the United States and represented a variety of scholarly disciplines, was asked

to generate a consensus definition of critical thinking, including its core cognitive skills.

The report included a description of the Delphi research methodology, addressed the skill

dimension of critical thinking, focused on the dispositional dimension of critical thinking,

and concluded with 15 recommendations pertaining to critical thinking instruction and

assessment. The research project lasted 2 years, was conducted on behalf of the American

Philosophical Association, and continues to influence critical thinking theory, teaching,

and assessment (P. A. Facione, 2006). The panel’s consensus statement regarding critical

thinking and the ideal critical thinker is as follows: “We understand critical thinking to be

purposeful, self-regulatory judgment that results in interpretation, analysis, evaluation,

and inference, as well as explanation of the evidential, conceptual, methodological,

criteriological, or contextual considerations upon which what judgment is based” (P. A.

Facione, 1990, p. 2). The Delphi panel further concluded that the ideal critical thinker is

Habitually inquisitive, well-informed, trustful of reason, open-minded, flexible,

fair-minded in evaluation, honest in facing personal biases, prudent in making
judgments, willing to reconsider, clear about issues, orderly in complex matters,
diligent in seeking relevant information, reasonable in the selection of criteria,
focused in inquiry, and persistent in seeking results which are as precise as the
subject and the circumstances of inquiry. (P. A. Facione, 1990, p. 2)

Critical thinking cognitive skills include interpretation, analysis, evaluation, inference,

explanation, and self-regulation.

26
 Scriven and Paul (1987), like P. A. Facione, asserted that individuals may be

subject to undisciplined thought and that critical thinking varies upon the motivation

underlying it. Paul (1995c) was concerned with the metacognitive aspects of critical

thinking and characterized critical thinking as “thinking about your thinking while you’re

thinking in order to make your thinking better” (p. 91). In line with the metacognitive

aspects of critical thinking, Paul et al. (1997a) maintained that critical thinking requires

“the systematic monitoring of thought; that thinking, to be critical, must not be accepted

at face value but must be analyzed and assessed for its clarity, accuracy, relevance, depth,

breadth, and logicalness” (p. 5). Finally, Paul (1995f) defined critical thinking as “the

intellectually disciplined process of actively and skillfully conceptualizing, applying,

analyzing, synthesizing, and/or evaluating information gathered from, or generated by,

observation, experience, reflection, reasoning, or communication, as a guide to belief and

action” (p. 8).

Paul and Elder (2004) contributed significantly to educational practices for

promoting critical reflection, metacognition, and literary analysis. Their contention was

that few readers have the abilities to skillfully read and translate the author’s intended

meaning in the text, that how students read should be determined by what they read, and

that “skilled readers do not read blindly; they read purposely. They have an agenda, goal,

or objective” (p. 36). Paul and Elder emphasized the importance of students

understanding the purpose of their reading and the author’s purpose in writing, perceiving

ideas in a text as being interconnected, connecting with a text while reading, and

formulating questions and seeking answers to those questions while reading. Further,

27
they encouraged explicating the thesis of a paragraph, analyzing the logic of what is

being read, and evaluating and assessing the logic of a reading.

Critical thinking is the process of “actively and skillfully conceptualizing,

applying, analyzing, synthesizing, and/or evaluating information” (Scriven & Paul, 1987,

p. 1). People who think critically can analyze their own thinking and realize they can

improve their own reasoning; a highly cultivated critical thinker raises vital questions and

problems, gathers and assesses relevant information, thinks open-mindedly, and

communicates effectively (Elder, 2007; Scriven & Paul, 1987). Carroll (2007) suggested

that “one of the key elements of critical thinking is the recognition that one’s worldview

can be a major hindrance to being fair-minded” (p. 4); hence, students must be willing to

negotiate previously held positions and beliefs while considering opposing viewpoints.

Teaching critical thinking through direct instruction in rhetorical analysis could improve

students’ critical thinking ability, for in order to teach students to think critically, “we

must teach them to try to understand how one’s worldview is likely to be embedded with

prejudices, biases, and false notions” (Carroll, 2007, p. 4).

Van Gelder (2005) contended that critical thinking is a complex skill that involves

deliberate practice for transfer. As P. A. Facione suggested that argument mapping is a

useful approach to understanding and interpreting the rationality of a set of arguments,

van Gelder proposed that the effective core to critical thinking is in learning how to

handle arguments. Argument maps make reasoning more easily understandable, students

can see the reasoning and can identify important issues, students can follow extended

critical thinking procedures because the arguments are presented in diagrammatic form,

and instructors can see the student’s line of reasoning when it is laid out in diagrams. Van

28
Gelder’s contention was that student critical thinking abilities improve when instruction

is based on argument mapping, and that “one semester of instruction based on argument

mapping can yield reasoning skill gains of the same magnitude as would normally be

expected to occur over an entire undergraduate education” (p. 5).

Socratic teaching is a powerful instructional strategy that encourages students to

discover their own reasoning through a series of questioning. The Socratic questioner

“acts as the logical equivalent of the inner critical voice which the mind develops when it

develops critical thinking abilities” (Paul & Elder, 1997, p. 1). Through a series of

questions, students are forced to think in a reasonable fashion. The Socratic questioner, or

the instructor, attempts to stimulate and encourage discussion while exploring students’

reasoning with probing questions. At the same time, the instructor/questioner should

ensure the discussion remains focused and intellectually responsible and summarize what

has or has not been accomplished (Paul & Elder, 1997). Socratic questioning has the

potential to develop students’ critical thinking abilities because it helps students to

monitor their thinking, to become more rational, and to pay more attention to that “inner

voice of reason” (Paul & Elder, 1997, p. 1).

Socratic questioning can be used for a variety of purposes, which include

exploring complex ideas, getting to the truth of some concept, uncovering assumptions,

analyzing concepts, and distinguishing “what we know from what we don’t know”

(Elder, 2006, p. 5). Elder encouraged educators to use Socratic questioning so they can

use it in their everyday lives when confronted with complex issues, “understand the

thinking of others [and] trace the implications of what they and others think and do” (p.

5).

29
 Importance of Teaching Critical Thinking

Studies indicate that the United States has not been performing as well as other

industrialized nations, in terms of critical thinking skills (National Center for Education

Statistics, 2007). Students must learn to think skillfully and independently so they may

cultivate their own problem-solving abilities and come to reliable conclusions. The goals

of this study were to provide students with the analytical, problem-solving skills needed

in a variety of academic settings and in their everyday lives. Specifically, the researcher

investigated whether the interventions implemented in the study are effective in

promoting student critical thinking knowledge, skills, and dispositions, and whether there

is growth in student perceptions of critical thinking knowledge, skills, and dispositions.

Sternberg (2003) reasoned that “a future of successfully intelligent thinkers is

important to personal satisfaction and national achievement” (p. 434). Teaching students

to think reflectively and critically should be a primary goal of educational institutions, for

although students may be knowledgeable, they may not have been taught how to think

analytically; hence, they could become highly vulnerable to the fallacious reasoning

exhibited by political leaders and within the media in its various forms (Sternberg, 2003).

The ultimate goal for educators is to promote lifelong learning by enhancing students’

problem-solving abilities so they may “apply these steps not just in school problems, but

in problems [they encounter] in everyday life” (Sternberg, 2003, p. 3).

Brookfield (2003) referred to the transfer of those skills learned in the classroom

and lifelong learning as “the organizing concept for adult education” (p. 2). Brookfield

claimed “there are forms of learning we engage in that are visible in a much more

heightened form in adulthood as compared to childhood and adolescence” (p. 2). Adult

30
learning is distinct to childhood and adolescent learning in that it includes the capacity to

think dialectically, to employ practical logic, to know how one knows what one knows,

and to think reflectively. Brookfield defined critical reflection as “the process by which

adults become critically reflective regarding the assumptions, beliefs and values which

they have assimilated during childhood and adolescence” (p. 10). This critical reflection

occurs over a period of time as a result of interpersonal, work-related, and political

experiences.

Society is experiencing tremendous economic, political, technological, and social

changes. These complexities demand a healthy, democratic populace who exercise

critical thought. When discussing the importance of critical thinking, Brookfield et al.

(2005) deduced that

Without the capacity to think and act critically, we would never move beyond
those assumptions we assimilated uncritically in childhood. We would believe
totally in the myths, folk wisdom, and values we encountered in authority figures
in our early lives. We would make no attempt to change social structures or to
press for the collective social action. (p. 46)

Under these circumstances, individuals might believe they are victims of circumstances

that seem beyond their control, unaware that they have the capacity to prompt action

through social change.

Brookfield (1995) encouraged educators to take a critical reflective stance toward

teaching and to help students confront their world or environment with compassion,

understanding, and fairness; nevertheless, the sincerity of intentions “does not guarantee

the purity of practice” (p. 1). When teachers reflect critically on their practice and model

the critical thinking process, students begin to understand what is involved in a critical

analysis of assumptions. For educators, the reflective process involves questioning their

31
practice and discovering assumptions about their practice and their students. This

reflective habit will lead to informed action, add meaning to instruction, and encourage

the creation of democratic classrooms.

Paul (1995a) believed critical thinking is the “essential foundation for education

because it is the essential foundation for adaptation to the everyday personal, social, and

professional demands of the 21st Century and thereafter” (p. xi). In view of the rapidly

changing world and the new global realities, there is a vital need for individuals to

develop those skills and abilities that enable them to respond and adapt to these changes.

Research findings from “Policy Issues in Teaching Education” indicated that

Although students can perform basic skills pretty well, they are not doing well on
thinking and reasoning. American students can compute, but they cannot reason.
. . . They can write complete and correct sentences, but they cannot prepare
arguments. . . . Moreover, in international comparisons, American students are
falling behind . . . particularly in those areas that require higher order thinking. (as
cited in Paul, 1995g, p. 19)

In order to adapt to the complexities and demands of the 21st century, Paul

(1995b) recommended that educators “cultivate minds that habitually probe the logic of

the systems of the status quo as well as the logic of the possible variations and alternative

systems” (p. xii), and that rather than memorizing the conclusions of others, “students

should reason to those conclusions on the basis of their own disciplined thought” (p. xii).

However, not only are many of the educational institutions ill-prepared for the rapid

changes occurring in society, but educators and students, seemingly, fail to have a clear

idea of what critical thinking is. Students have an obligation to be responsible for their

thoughts, their conduct, and their lives, and educators can attempt to encourage this

32
accountability and improve students’ approach to problem solving through direct

instruction in argument and critical thinking.

Elder (2000) discussed the emphasis of teaching students the skills needed to

become competent employees at the community college level. As society becomes more

complex, and as a rapid change in technology occurs, “training students for job

performance in narrowly defined skill areas no longer serves students well” (Elder, 2000,

p. 1). Elder’s contention was that students are not prepared for the challenges of the

current job market. Therefore, educators should encourage in their students the

intellectual tools that “will render them mentally flexible and intellectually disciplined”

(Elder, 2000, p. 2). Successful employees must be able to utilize disciplined reasoning

and the metacognitive process so they can direct and redirect their thinking. Rather than

emphasizing the transfer of information, educators should encourage students to rethink

their thinking and to reason, analyze, judge, and interpret that information.

P. A. Facione (2006) discussed the value of critical thinking and the need for an

informed citizenry who can make good judgments while offering practices for nurturing

the habits of mind or dispositions for critical thinking. He suggested the ideal critical

thinker “can be characterized not merely by her or his cognitive skills but also by how

she or he approaches life and living in general” (p. 9). P. A. Facione related a study of

over 1,100 college students that indicated “scores on a college level critical thinking

skills test significantly correlated with college GPA” (pp. 17–18). His contention was that

there is a “significant correlation between critical thinking and reading comprehension”

(p. 18); however, this goal may be somewhat limited. Apart from the college experience,

students must learn to stand on their own, think for themselves, and make their own

33
contributions to society; hence, “learning critical thinking, cultivating the critical spirit, is

not just a means to this end, it is part of the goal itself” (P. A. Facione, 2006, p. 18).

To date, there are a number of researchers who have examined the effect of direct

instruction in critical thinking. This study, however, examined this issue by determining

whether there was improvement in students’ critical thinking knowledge, skills, and

dispositions on pretest and posttest scores after direct instruction in argument mapping,

Paul’s Thinker’s Guides, and Socratic questioning. Of added importance was the

researcher’s investigation of the students’ perceptions of the effectiveness of the

instructional interventions and of their own growth in critical thinking abilities at the end

of the course.

Teaching Critical Thinking

An assumption among many educators is that students who attend college will

develop the necessary critical thinking skills simply by attendance in class and

participation in class discussion. Conversely, with the current emphasis on improving

critical thinking and student achievement, educational institutions, colleges, and

universities have been offering courses designed to improve students’ critical thinking

skills, either as complete, isolated courses or within academic disciplines. Although

improving students’ critical thinking abilities has become a primary goal for education,

there is continual debate over the most appropriate or effective pedagogical strategies

educators can use to improve those abilities.

Gardner (1993) advocated that people have a variety of different intelligences. He

identified seven components of intelligences that he believed are distinct from each other.

34
Although Gardner valued individuals who can who can think critically about literature

and world events, he cautioned that “one must be careful not to assume that it is a

particular, dissociable variety of human cognition” (p. 44), while suggesting that

“particular domains of human competence seem to require their own brand of critical

thinking” (p. 44). The kind of critical thinking required for a musician might be quite

different from the critical thinking required for a historian or biologist because “each

domain exhibits its own particular logic of implications” (Gardner, 1993, p. 14); hence,

training for one domain does not necessarily provide transfer to other domains. Further,

instructors should not assume that critical thinking skills taught in a standalone critical

thinking course will transfer to those skills needed for a history course; rather, Gardner

believed that

Only if the lessons of critical thinking are deliberately revisited in each of the
relevant classes or exercises is there any possibility that a more general virtue like
reflectiveness or taking the perspective of the other has any chance of emerging.
(p. 44)

As with Gardner, Sternberg (2008) believed that conventional notions of

intelligence and intelligence tests do not necessarily reflect talent or wisdom. Similar to

Gardner’s theory of multiple intelligences, Sternberg’s theory of successful human

intelligences, often referred to as the triarchic theory, consists of three components:

practical intelligence, experiential intelligence, and componential intelligence. Practical

intelligence refers to “internal abilities [or] mental mechanisms” (Sternberg, 2008, p. 24)

and can lead to more-or-less intelligent behavior; experiential intelligence examines

individuals’ “experience in handling a task” (Sternberg, 2008, p. 24); and componential

intelligence refers to the external world and includes “environmental adaptation,

35
environmental selection, and environmental shaping” (Sternberg, 2008, p. 24). His theory

takes into account the strengths and differences of individuals while considering the

sociocultural context in which they live. Sternberg believed intelligence can be increased

through study and practice, and encouraged educators to assist students so they can use

and develop all of their skills and perform well in all areas.

Sternberg (2003) defined the ideal critical thinker as a good problem solver;

however, students must be taught to transfer the problem-solving skills they learn in

school to their everyday lives. Successful intelligent thinkers have “the creative skills to

generate new ideas, analytical abilities to know whether they are good ideas, and the

practical abilities to know how to implement the ideas and to persuade others of the value

of their ideas” (Sternberg, 2003, p. 4). Yet, although Sternberg believed these to be

valuable skills, he advocated that schools should include wisdom-related skills in the

school curriculum.

If appropriately taught, intelligence and critical thinking can lead to wisdom.

Sternberg (2003) defined wisdom as “the application of intelligence and experience

toward the attainment of a common good” (p. 7). The dilemma exists when students use

their intelligence in foolish ways; hence, it is the responsibility of educators to encourage

students to be responsible and to balance their intrapersonal, interpersonal, and

extrapersonal interests so their knowledge is used for good purposes. Sternberg’s research

involved teaching wisdom as part of a course in American history. Students in the

experimental group learn to understand things from different points of view, whereas

students in the control group learn the historical material in a standard way. The

definitive goal of the study was to teach students that “resolution of difficult life

36
problems requires people to want to understand each other and to reach a resolution,

whenever possible, with which all of those people can live” (Sternberg, 2003, p. 7).

Halpern (1999) believed critical thinking can be taught as rhetorical analysis,

problem solving, decision making, or cognitive process, and suggested that

Regardless of the academic background of the instructor or the language used to

describe critical thinking, all of these approaches share a set of common
assumptions: there are identifiable critical thinking skills that can be taught and
learned, and when students learn these skills and apply them appropriately, they
become better thinkers. (p. 70)

As with Sternberg, Halpern (1999) maintained critical thinking skills should be

taught within a variety of contexts and for transfer so that students can apply the

knowledge and skills into other domains. In agreement with Paul (2004) and P. A.

Facione (2006), Halpern affirmed that critical thinking instruction must address student

dispositions, while acknowledging “it is not enough to teach college students the skills of

critical thinking if they are not inclined to use them” (p. 72). Further, students should not

only have a disposition toward critical thinking, they should also possess the willingness

to apply it. It is, therefore, crucial that students be taught the value of critical, reflective

thinking, and the essential effort needed to achieve it.

As indicated by Halpern (1999), it is vital that teachers encourage students to

apply the knowledge and skills learned in one context to other situations. Students should

be encouraged to transfer critical thinking knowledge, skills, and dispositions learned in

the educational environment to their personal and professional lives. Perkins (1993)

offered a variety of strategies to accomplish this complex task. He viewed the teacher as a

guide and mentor and suggested the three basic tools that can elicit thoughtful learning

are Socratic method-discussion, didactic instruction, and coaching for understanding

37
performances through practice, self-assessment, and informative feedback. Perkins

asserted that “knowledge and skill in themselves do not guarantee understanding” (p. 2),

and encouraged educators to promote transfer by helping students discover the

connections between their lives and the subject matter they are being taught. Perkins

noted understanding a topic of study is “a matter of being able to perform in a variety of

thought-demanding ways with the topic, for instance to: explain, muster evidence, find

examples, generalize, apply concepts, analogize, represent in a new way” (p. 4). Perkins

outlined six priorities for educators who teach for understanding: making learning a long-

term, thinking-centered process; providing for rich, ongoing assessment; supporting

learning with powerful representations; paying heed to developmental factors; inducting

students into the discipline; and teaching for transfer.

Specific Strategies

Argument Mapping

Van Gelder (2005) drew on lessons from cognitive science while discussing the

importance of developing students’ critical thinking abilities, and, as with Paul (1995c),

emphasized the need to improve teaching and educational institutions. He indicated that

individuals, as a rule, are not naturally critical and that critical thinking is a difficult, yet

not impossible, skill that takes time and deliberate practice to master. Students do not

become better thinkers merely through exposure to good critical thinking; they must

engage in critical thinking itself. He further contended that knowledge of the critical

thinking theory “is the basis for self-monitoring and correction” (p. 4) and that this

knowledge of theory encourages and improves teacher guidance and feedback.

38
 Van Gelder (2005) believed a central part of improving this skill is in handling

arguments, that “arguments are presented or expressed in streams of words” (p. 4), and

that the structure of the arguments is expressed in “sequences of words or sentences” (p.

4). Arguments are, generally, linear and can be mapped out so that students can view the

logical structure of the rhetoric. Van Gelder claimed that “critical thinking skills improve

faster when instruction is based on argument mapping” (p. 5) and that “students in classes

based heavily on argument mapping consistently improve their skills much faster than

students in conventional classes” (p. 5). The advantages of argument maps are that they

make reasoning more easily understandable; students can see the reasoning, so they can

more easily identify important issues; arguments presented in diagrammatic form allow

students to follow the extended critical thinking procedures; and arguments laid out in

diagrams allow the teacher to see what the student is thinking (van Gelder, 2005).

Argument maps offer a straightforward, effective tool for improving students’ critical

thinking abilities.

Paul’s Model of Critical Thinking

Paul (2004) stated studies of higher education demonstrate that most college

faculty lack a substantive concept of critical thinking, most college faculty do not realize

they lack this substantive concept, and that lecture and memorization are still the norm in

college instruction. Paul’s (2004) model of critical thinking was created in an attempt to

provide a concept of critical thinking and a common model for instructional design. His

purpose for designing this model was to encourage educators to place emphasis on higher

order learning as “it is precisely these higher order thinking skills that are routinely

39
sacrificed when coverage and lower order recall dominate the classroom” (Paul, 1995f, p.

105).

Paul (1995f) discussed the dangers of a nonsubstantive concept of critical

thinking, while emphasizing that students’ critical thinking abilities cannot be refined

without reference to the “values, traits of mind, and dispositions that underlie those

skills” (p. 8). Assessment must be specific and consistent, an accurate concept of critical

thinking must be fostered, and appropriate assessment strategies must align with a true

concept of critical thinking so it is incorporated into testing and teaching. A real danger

exists when “departments of education incorporate a non-substantive concept of critical

thinking into statewide curriculum” (Paul, 1995f, p. 119).

Paul’s model of critical thinking is a metacognitive approach that consists of four

domains of critical thinking: elements of thought, abilities, affective traits, and

intellectual standards. Elements of thought refer to the kind of thinking that is conceptual

and inferential, and they are the basic building blocks of thinking; abilities include

thinking about complex issues; affective dimensions are the essential attitudes,

dispositions, and traits of mind needed for critical thinking; assessment of affective

dimensions is an important part of the assessment of higher order thinking and refers to

the validity and reliability in terms of the elements of thought; and intellectual standards

are the standards implicit in the assessment.

Paul and Elder took the essential elements of good critical thinkers and developed

The Thinker’s Guide to the Art of Socratic Questioning: Based on Critical Thinking

Concepts & Tools (Paul & Elder, 2006b) and The Thinker’s Guide to Analytic Thinking:

How to Take Thinking Apart and What to Look for When You Do (Elder & Paul, 2007).

40
The researcher used both of these Thinker’s Guides in the study. The Thinker’s Guides

address the following: critical thinking concepts and tools, how to study and learn a

discipline, the art of asking essential questions, active and cooperative learning, how to

improve student learning, how to write a paragraph, fallacies: the art of mental trickery

and manipulation, the human mind, critical and creative thinking, analytic thinking,

scientific thinking, how to read a paragraph, and ethical reasoning (Bessick, 2008).

Socratic Questioning

Paul and Elder (2000) asserted that educators concentrate far too much on

“coverage” (p. 1) of course material over “engaged thinking” (p. 1) partly because they

believe “answers can be taught separate from questions” (p. 1). Conversely, Paul and

Elder (2000) argued that questioning encourages thinking and learning because all

assertions and statements are, ultimately, answers to questions. They reasoned that

thinking is driven by questions as questions stimulate thinking, “define tasks, express

problems and delineate issues” (p. 1). Paul and Elder (1997) referred to Socratic

questioning as a highly disciplined, systematic approach to inquiry and “the most

powerful, teaching tactic for fostering critical thinking” (p. 1). Along with allowing

educators a more focused approach to their students and to their instruction, Paul (1995e)

acknowledged that “Socratic questioning is at the heart of critical teaching” (p. 335). as it

presents opportunities for students to develop and evaluate their ideas. Socratic

questioning

Raises basic issues; probes beneath the surface of things; pursues problematic
areas of thought; helps students discover the structure of their own thought; helps
students develop sensitivity to clarity, accuracy, and relevance; helps students
arrive at judgment through their own reasoning; and helps students note claims,
evidence, conclusions, questions-at-issue, assumptions, implications,

41
 consequences, concepts, interpretations, points of view—the elements of thought.
(Paul, 1995e, p. 336)

Similar to Paul and Elder (2000), Newton believed “intelligence is a reaction to

the social and physical world” (p. 289) and that educators, through higher cognitive

questioning, encourage students to adapt and use their knowledge in a problem-solving

manner. While suggesting educators are in need of concrete strategies for improving

critical thinking, Newton (1978) contended that educators should stimulate rather than

indoctrinate learners. In line with Piaget and Dewey, she perceived the learner as an

active participant in the learning process and presented a theoretical basis for higher

cognitive questioning, which includes analysis, synthesis, and evaluation, as an avenue to

critical thinking. Newton referred to Dewey’s definition of reflective thinking as the

“careful and persistent examination of an action, proposal, belief, analysis or use of

knowledge in the light of the grounds to justify it” (p. 287), while encouraging educators

to teach “that knowledge and those skills through discussion of controversial social

issues” (p. 287), literary criticism, and rhetorical analysis.

Teaching Critical Thinking: Related Studies

Paul et al. (1997b) conducted a study of teacher preparation programs to assess

the extent to which they prepare candidates to teach critical thinking and problem-solving

skills. Using both quantitative and qualitative data, the researchers’ objectives were to

assess current teaching practices and knowledge of critical thinking among faculty,

identify representative teaching practices that improve critical thinking, and develop

policy recommendations based on the results of the study. The results of their research

42
were based on a study of randomly selected California professors from 38 public and 28

private colleges and universities. In-depth interviews were utilized to determine the

extent to which students in teacher preparation programs were being taught to encourage

and teach critical thinking instruction. The researchers also sought to determine the

faculty’s conception of critical thinking. Findings of the research study indicated that

although the majority of the faculty claimed critical thinking is a primary objective of

instruction, only a small minority could clearly explain what critical thinking is. Paul et

al. maintained that although teacher educators claim to be committed to teaching critical

thinking, “few have had any in-depth exposure to research on the concept and most have

only a vague understanding of what it is and what is involved in bringing it successfully

into instruction” (p. 10). Paul et al. claimed educators must have a clear concept of what

critical thinking is and must be taught “ways that facilitate skill in critical thinking and

the ability to teach it to others” (p. 2) so that students will graduate and enter the work

force with the problem-solving abilities needed for clarity of thought and intellectual

discipline.

Meinecke (1997), in an attempt to develop an operational definition of critical

thinking for New Mexico State University (NMSU) College of Business Administration

and Economics, designed a naturalistic study to examine existing definitions of critical

thinking and approaches to teaching critical thinking. Meinecke used surveys, interviews,

and classroom observations to collect data to determine existing instructional practices.

The study revealed that “business college faculty did not have a conceptual definition of

critical thinking and often failed to teach critical thinking in their classrooms” (Meinecke,

1997, Abstract). Meinecke’s conceptual definition of critical thinking was “an ability to

43
engage in a process of objectively analyzing a problem with an open mind and

intellectual honesty” (Abstract). She believed critical thinking is a recursive process that

includes reason assessment, a critical spirit, and metacognition, and maintained that

critical thinking can be developed through a reflective pedagogical paradigm and that

students should be “placed at the center of the classroom environment with autonomous

control over their own learning process” (p. 208).

Meinecke (1997) contended that educators at NMSU need to create a community

of inquiry by focusing on combined classes “with an emphasis on hands-on experience

taught with a common language that includes their unified conceptual definition of

critical thinking” (p. 212). Further, she encouraged educators at NMSU to create

classroom environments that promote critical and creative thinking through the

application of the six dimensions of thinking, as developed by Tishman, Perkins, and Jay,

which include a language of thinking, thinking dispositions, mental management,

strategic spirit, higher order knowledge, and transfer.

There is a growing number of educators who are attempting to design

instructional pedagogy that will promote students’ critical thinking skills. Several studies

have focused on direct instruction of critical thinking through the use of argument

mapping, Paul’s Thinker’s Guides, and Socratic questioning (Bessick, 2008; Reed, 1998;

Scanlan, 2006; Tsui, 1998; Van Erp, 2008; Yang, 2008), and their findings suggested that

although these instructional strategies prove useful for promoting critical thinking, further

research is needed in this area.

Bessick (2008) discussed precollege preparation and the responsibility of

educational institutions in improving higher order thinking skills. Her contention was that

44
students entering colleges and universities are deficient in reading, writing, mathematics,

and critical thinking skills; however, “because of the lack of preparedness in the basics,

schools have become so focused on general academics that efforts to enhance the

development of critical thinking skills in their students is minimal” (p. 33). Bessick

believed direct instruction in critical thinking and reasoning can improve student learning

outcomes; however, the ultimate educational goal should be to enable students to transfer

those skills learned in the classroom to their everyday lives. In her study, Bessick

examined the importance of teaching critical thinking skills and their relationship to

academic achievement. Bessick conducted an experimental quantitative study of

freshmen students being tutored in repeat courses in which she examined the effect of

direct instruction in critical thinking on the students’ critical thinking ability and

academic achievement using van Gelder’s rational argument mapping and Paul’s

Thinker’s Guides. Data analysis revealed the intervention programs using the Thinker’s

Guides based on Paul’s model of critical thinking and van Gelder’s rational argument

mapping program revealed no significant improvement in students’ critical thinking

skills. However, the Thinker’s Guides group and the control group demonstrated

improvement in students’ academic achievement. Bessick maintained there is a need to

improve the critical thinking skills and that “instruction in critical thinking, whether

through direct instruction or independent study in addition to tutoring may contribute to

the improvement of students’ academic achievement” (p. 153). Her findings suggested

that further research is needed using a larger sample size to determine the extent to which

direct instruction using the Thinker’s Guides and the rational argument mapping program

can improve students’ critical thinking ability and academic achievement.

45
 Scanlan’s (2006) study was conducted in order to explore strategies to improve

students’ critical thinking skills by incorporating Paul’s elements and standards of

reasoning into all standards-based curriculum. The project researched the effect of

focused critical thinking training on the composition skills of 12th graders, specifically

through educational techniques developed by Paul. Scanlan included persuasive essays,

and assessment was measured through a series of rubrics. The results of his research

indicated the value of incorporating Paul’s elements and standards of reasoning is in the

gains students made in terms of clarity of writing, level of analysis, use of supporting

information, and organization of ideas.

Reed (1998) conducted a study at a community college in central Florida in which

she investigated the effect of integrating Paul’s model of critical thinking into a U.S.

history course on community college students’ abilities to think critically about U.S.

history and about everyday issues, dispositions toward thinking critically, and knowledge

of history content. Reed found that students’ critical thinking abilities improved in a

single course, the students’ end-of-term knowledge of history content did not suffer when

training in critical thinking abilities was integrated into course material, and age and

gender did not play significant roles in developing college students’ critical thinking

abilities. Reed’s study revealed that students’ abilities to think critically can improve in a

single course when provided with deliberate instruction and that Paul’s model “can be

successfully integrated into an introductory history course with statistically significant

benefits to students’ abilities to think critically within a domain and to their general

critical thinking abilities” (p. 160).

46
 Yang (2008) conducted a study designed to examine the effectiveness of teaching

critical thinking skills “through asynchronous discussion forums (ADFs) with the

facilitation of teaching assistants, to investigate student interaction patterns and the depth

of their critical thinking demonstrated via the asynchronous discussion forum” (p. 241).

The major goal was to investigate whether students’ critical thinking skills would

improve after they participated in Socratic dialogues as taught by the instructor and then

as modeled and facilitated by the teaching assistants. The main task of the teaching

assistants was to focus on the quality of the student interaction, including the

assumptions, reasoning, and evidence provided by the students. The qualitative analysis

provided a detailed description of how students’ discussions moved from the lower to the

higher phases of critical thinking. Results from the CCTST indicated that instructors who

use Socratic dialogues during small-group online discussions can successfully improve

students’ critical thinking skills in a large university class. This improvement in learning

is accomplished as “students interact with their peers or instructor and benefit from

combining their levels of expertise, offering support, distributing the thinking load, and

confronting alternative points of view” (Yang, 2008, p. 261). The results also indicated

that “learning is also accomplished as students elaborate, clarify, and justify their

personal responses/solutions” (Yang, 2008, p. 261).

Van Erp’s (2008) study consisted of an investigation of the lived experiences of

the instructor and a group of learners in three online graduate courses in which critical

thinking skills were emphasized alongside course content. Students completed a pre- and

postcourse self-assessment reflection and participated in postcourse interviews. The

courses included strategies that promote critical thinking skills, including asynchronous

47
discussion, instructor modeling and coaching, explicit instruction about critical thinking,

self-assessment writing, Socratic questioning, and scoring rubrics that included critical

thinking criteria. Van Erp used P. A. Facione’s definition in the study while investigating

the correlation between the definitions of critical thinking found in literature and the

themes identified in learners’ definitions; the terms “‘reflective’ and ‘reflection’ were

themes in both the pre and the post course self-assessment reflections” (p. 97). Van Erp

believed the strongest themes that emerged in the precourse self-assessment were open-

mindedness and the nature of asynchronous discussion; moreover, throughout the

interview process, the strongest theme to emerge from the student interviews was

instructor feedback. Learners’ critical thinking ability was enhanced when they were

required to examine assumptions and to be open-minded while considering possibilities.

Further, the conversations and the systematic, carefully crafted questions designed by the

instructor challenged the learners’ assumptions and improved student learning and the

quality discussion. Van Erp recommended embedding critical thinking throughout an

entire program while supporting van Gelder’s (2005) premise that critical thinking is a

difficult skill to master and that instructors should scaffold learning and be systematic

and intentional in their approach through carefully designed discussion questions. Quality

questions that scaffold learning can impact learning because they challenge learners to

think reflectively and critically.

Tsui (1998) suggested that although developing student critical thinking skills is a

primary goal of education, educators lack sufficient knowledge as to how this educational

aim can be achieved. Tsui combined quantitative and qualitative data to investigate the

impact of college on the development of student critical thinking skills and to examine

48
factors that lie inside as well as outside of the classroom. The sample included 13,216

students from more than 200 four-year colleges and universities. Quantitative data were

drawn from longitudinal questionnaire surveys of a national sample of college students.

The qualitative data were gathered by interviewing 55 participants and observing 28

classes at four institutional case study sites.

Tsui (1998) found that critical thinking abilities can be developed positively in

association with substantive writing, critical discussion, class presentations, student-led

inquiry, emphasis on analysis over recall, and the constructivist philosophy to learning;

negative factors for promoting critical thinking include lectures and multiple-choice

examinations. Further, Tsui discovered that infusion of critical thinking into the

curriculum is more successful when “faculty have enough confidence in their students to

challenge them [and are] enthused enough to partake in collegial exchanges about

effective pedagogy” (Abstract). Additionally, Tsui’s study revealed that an important

factor in enhancing critical thinking abilities is engagement in critical dialogue with

peers.

Conclusion

Promoting students’ critical thinking abilities should be of primary concern to all

educators. There are a variety of definitions for critical thinking; however, the researcher

found P. A. Facione’s (2006) metacognitive approach with its emphasis on analysis,

interpretation, inference, explanation, evaluation, and self-regulation to be most suitable

for this study.

49
 Paul (1995b) viewed critical thinking as the essential foundation for education as

individuals must have the ability to adapt to the demands of the 21st century. Frequently,

instructional practices emphasize the content rather than the quality of instruction.

Although students can perform basic skills, pedagogical practices that encourage higher

order thinking skills could enhance student critical thinking. Another key obstacle to

promoting critical thinking is that teachers and students may not have a clear idea of what

critical thinking involves.

As Paul (2004), P. A. Facione (2006), and Perkins (1993) contended, critical

thinking should be taught within a variety of contexts, and teachers should encourage

students to transfer the knowledge and skills learned in the classroom into their everyday

lives. What is essential is that educators develop the appropriate skills and strategies

needed to enhance students’ higher cognitive skills.

While there is continual debate regarding the best methods for improving

students’ critical thinking abilities, critical thinking, although a complex skill, may well

be successfully taught through direct instruction. Bessick (2008) confirmed the

importance of direct instruction in critical thinking skills and academic achievement

using a combination of van Gelder’s argument mapping and Paul’s Thinker’s Guides.

Scanlan (2006) and Reed (1998) discovered that student learning and critical thinking can

be enhanced by using Paul’s model of critical thinking. Van Erp (2008) and Yang (2008)

concluded that Socratic dialogues can improve students’ critical thinking abilities;

however, Van Erp and Yang stressed the importance of timely feedback, coaching, and

frequent modeling. Finally, Tsui (1998) found that critical thinking abilities can be

50
developed positively in association with substantive writing, critical discussion, class

presentations, and student-led inquiry.

Sternberg (2003) argued that teaching students to think critically should be a

primary goal of educational institutions so students acquire the abilities to transfer those

skills into their everyday lives. Paul (1995b) avowed that critical thinking should be the

essential foundation for education, while arguing that educational institutions are ill-

prepared for the rapid changes occurring in society, but educators and students do not

have a clear idea of what critical thinking is. Elder (2000) commented on students being

ill-prepared for the current job market, while suggesting that educators encourage

intellectual discipline. Similarly, P. A. Facione (2006) addressed the value of critical

thinking and the need for an informed citizenry. Direct instruction in critical thinking

using a variety of instructional strategies could promote those critical thinking skills and

traits of mind necessary for productive, successful living.

51
 CHAPTER 3. METHODOLOGY

Introduction

Cultivating critical thinking, intellectual growth, and lifelong learning

opportunities that afford students the knowledge and skills necessary for success in life is

a fundamental goal of all educational institutions. Educators are beginning to explore

those pedagogical practices that could effectively develop higher order thinking skills.

Instructional strategies that advance critical thinking pedagogy on a consistent basis

could positively impact the range and quality of student critical thinking skills’

performance. This study investigated whether direct instruction in rhetorical analysis in a

critical thinking course would improve students’ critical thinking abilities. This chapter

provides a description of the methodology used in the study and includes a statement of

the problem, the research questions, the research design and methodology, a discussion of

the population and sample, and a detailed description of the instrumentation, data

collection, and data analysis. This is followed by a discussion of ethical considerations,

limitations of the study, and a summary.

Statement of Problem

It is not known whether an intervention in rhetorical analysis will improve student

critical thinking knowledge, skills, and dispositions. This study was an exploration and

52
evaluation of instructional pedagogy designed to promote students’ critical thinking

abilities. The researcher investigated whether direct instruction in rhetorical analysis in a

college critical thinking course could make a difference in students’ critical thinking

knowledge, skills, and dispositions. The interventions included argument mapping;

Thinker’s Guides, based on Paul’s model of critical thinking; and Socratic questioning.

These interventions were supplemented with course work that involved reader/writer

workshops and peer review.

Research Questions

The following research questions guided this study:

1. What were the students’ perceptions of critical thinking before and after direct
instruction in argument mapping?

2. What were students’ perceptions of critical thinking before and after direct
instruction in Paul’s Thinker’s Guides?

3. What were the students’ perceptions of critical thinking before and after direct
instruction in Socratic questioning?

4. What did the students report about the changes in their knowledge, skills, and
dispositions relating to critical thinking after the direct instruction in rhetorical
analysis?

Null Hypothesis: There will be no difference in scores on the pretest and posttest

on the California Critical Thinking Skills Test–Form 2000 indicating changes in student

critical thinking skills before and after direct instruction in rhetorical analysis.

53
 Research Design and Methodology

The triangulation methodology involves using quantitative and qualitative

methods and “gives the researcher greater understanding of the issues under

investigation” (Cowden, Chivore, Maravanyika, Nyagura, & Sibanda, 1999, p. 11). This

mixed methodology approach was used in this one-group pretest–posttest design to

determine whether direct instruction in critical thinking, using the aforementioned

interventions, would improve students’ critical thinking knowledge, skills, and

dispositions. The independent variable for this study was the critical thinking

interventions. The dependent variables were the posttest scores on the CCTST–2000 and

on the Pre- and Post- Critical Thinking Surveys. The researcher examined the

relationship between the variables involving teacher instruction of the interventions, the

students’ mastery of those skills, and the students’ perceptions of the effectiveness of the

interventions. This blend of methodologies enhanced the relevance and strength of the

study as a combination of qualitative and quantitative methods were used to cross-check

the validity of data. The quantitative approach is formal and conclusive and involves

“collecting numerical data on observable behaviors of samples and then subjecting these

data to numerical analysis” (Gall et al., 2003, p. 23).

The researcher was the teacher of an intact class, and no attempt at randomization

occurred; therefore, the study was limited to a description of an existent situation rather

than creating an experimental situation. In teacher action research, the data are collected

and analyzed to assess and improve educational practice. Since the researcher was the

instructor of the critical thinking course, there was potential for researcher bias as the

researcher may have had authority over the participants. The researcher understood this

54
and attempted to approach the study with caution and without preconceived assumptions

of possible outcomes or results. Care was taken to maintain clarity and objectivity

through sharing with research participants and through personal reflection, assessment,

and modification of instructional practices.

The researcher administered a Pre- Critical Thinking Survey at the beginning of

the course to measure the dependent variable; implemented the experimental treatment,

or the independent variable; and administered a Post- Critical Thinking Survey at the end

of the course. The qualitative survey was used to support description and analysis by

providing contextual information, and by exploring areas that are impossible to

conceptualize (Cowden et al., 1999). Further, the qualitative data aided in looking for

patterns, and the observations made by the survey results offered insights into student

perceptions of the most effective instructional interventions. The CCTST–2000 was

administered as a pretest at the beginning of the course and measured the students’

critical thinking skills. The same test was given as a posttest and measured growth in

critical thinking abilities at the end of the course. Statistical analysis was administered on

the CCTST–2000 pretest and posttest scored by Insight Assessment.

Population and Sample

The population of this study was a convenience sample of students who attended

a mid-sized culturally diverse community college in southern California. The district

serves a geographic area that is semirural in nature. The college offers two-year academic

transfer programs and corresponding vocational education programs. Students at this

community college reflect the diversity of the surrounding community. The demographic

55
makeup of the college reveals that 66.5% of the student population is White/non-

Hispanic, 8.7% Black, 20% Hispanic, 3.7% Asian, and 1.1% Native American. The total

number of students enrolled in composition classes is approximately 2,000 students;

however, it was not convenient for the researcher to collect data from this entire

population. Hence, the population was limited to those students enrolled in a critical

thinking class taught by the researcher.

Convenience sampling suited the purposes of this study as it was the most

practical approach for the researcher. The sample consisted of students enrolled in an on-

campus critical thinking course taught by the researcher. Total enrollment in the critical

thinking course was 29 students at the beginning of the semester; 19 of those students

agreed to participate in the study; however, three students dropped from the course and

one student did not show up the final day for the posttest; thus, the final research sample

consisted of 15 students.

The critical thinking course was designed to develop the students’ critical

thinking, reading, and writing skills. Basic English is a prerequisite to this course. The

critical thinking course is not a required course; however, it is required for students who

are English majors. The critical thinking course focused primarily on the analysis and

evaluation of expository and argumentative discourse, and on writing analytical and

argumentative essays. The objectives of the course and the purpose of studying

composition was to help students improve their writing, reading, and critical thinking

skills to a level that will enable them to transfer those skills to their professional and

personal lives. All of the students in the sample received the interventions that were

56
provided throughout the semester. Gains in student critical thinking abilities were

expected due to the ongoing instruction of the interventions.

Instrumentation

The CCTST–2000 was ordered from Insight Assessment and was administered by

the researcher as a pretest and posttest. There are three versions of the CCTST: Form A,

Form B, and Form 2000. The CCTST–2000 is based on the APA Delphi consensus

conceptualization of critical thinking. This conceptualization of critical thinking is an

expression of expert consensus based on the participation of 46 leading theorists,

teachers, and critical thinking assessment specialists, and was reaffirmed in the national

survey conducted by the National Center for Higher Education Teaching, Learning and

Assessment at Pennsylvania State University. The Delphi study consensus definition of

critical thinking has been endorsed by educators and scholars around the world (N. C.

Facione, Facione, Blohm, & Gittens, 2008). The CCTST–2000 is specifically designed to

measure the skills dimension of critical thinking. The items on the CCTST–2000

emphasize a variety of critical thinking skills, which include analysis of the meaning of a

given sentence, drawing a correct inference from a set of assumptions, and evaluating or

justifying the inference provided.

According to N. C. Facione et al. (2008), “the primary use of the CCTST is to

gather valid and reliable data about the baseline, entrance-level, or exit-level critical

thinking skills of various groups of people, commonly college level students and working

adults” (p. 11). The items are given in standard English and are set in contexts and

address topics that are familiar to college-age students. The CCTST–2000 is appropriate

57
for college undergraduate and graduate students and for the researcher’s study (N. C.

Facione et al., 2008).

The CCTST–2000 targets the core critical thinking skills of analysis,

interpretation, inference, evaluation, and explanation. The CCTST–2000 takes 45

minutes to complete under timed conditions and can also be administered untimed. The

CCTST–2000 can be taken in the booklet using a paper-and-pencil format with a

Scantron® sheet or through an online formal set up by Insight Assessment.

Reliability

Items in the CCTST–2000 include the critical thinking skills identified by the

Delphi experts: interpretation, analysis, evaluation, explanation, and inference. They are

discipline-neutral; sex-role and social class stereotypic context have been avoided; and

“equal numbers of male and female referents are used to decrease possible gender and

cultural bias” (N. C. Facione et al., 2008, p. 27). Validation studies “have produced

consistency estimates . . . ranging from .68 to .80” (N. C. Facione et al., 2008, p. 27).

According to N. C. Facione et al.

Reliability ratings of .65 to .75 have been suggested to be considered sufficient

for placing confidence in instruments of this type. The reliability coefficients
demonstrated by the CCTST is more than sufficient to inspire confidence in the
internal consistency of its items to measure the overall construct. (p. 27)

The CCTST has been tested by its developers as well as by independent

researchers (Bondy, Koenigseder, Ishee, & Williams, 2001; N. C. Facione & Facione,

1997; P. A. Facione, Facione, Blohm, Howard, & Giancarlo, 1998; Jacobs, 1995, 1999).

According to Pike (as cited in Laird, 2005), the CCTST–2000 is considered to be more

reliable than Forms A and B. Laird suggested the instrument is suitable for educational

58
assessment; however, “test administrators should pay particular attention to the reliability

and inter-correlation of the total score and subscales within the group under study” (p. 4).

Validity

The validity of the CCTST–2000 can be examined in terms of content validity,

construct validity, and criterion validity. Content validity refers to how well the test items

represent the critical thinking domain. CCTST–2000 test items were chosen for their

theoretical relationship to the Delphi panel’s conceptualization of critical thinking (N. C.

Facione et al., 2008). Construct validity refers to the extent the CCTST–2000 measures

what it claims to measure, in other words, the Delphi experts’ definition or

conceptualization. As more students enroll in critical thinking courses, and as their test

results improve as a result of this enrollment, well-crafted multiple-choice items, as those

in the CCTST–2000, can validly and reliably measure this improvement in higher order

thinking skills. Further, face validity is supported by the types of questions asked on the

CCTST–2000 that encourage students to make judgments, identify correct analyses, draw

inferences, evaluate reasoning, and justify inferences and evaluations (N. C. Facione et

al., 2008). Criterion validity refers to a test’s ability to predict external criteria, such as a

higher level of college success. The CCTST–2000 is reported to have a content validity

of .74 and moderate criterion validity with grade point average and SAT math and verbal

scores (N. C. Facione et al., 2008).

The second and third instruments that were used in the study, the Pre- and Post-

Critical Thinking Surveys, were qualitative in nature. The second instrument was

comprised of two sections: a student demographic survey and the Pre- Critical Thinking

Survey. The Post- Critical Thinking Survey included three sections: the student

59
demographic survey, the Critical Thinking Survey, and a Student Perceptions of Critical

Thinking Survey. The student demographic portion of the survey included questions

concerning gender, race, age, current year in college, academic major, student status,

students’ perceptions of their writing and comprehension skills, and previous academic

experience in critical thinking. The Pre- and Post- Critical Thinking portion of the survey

offered useful information regarding the students’ critical thinking knowledge, skills, and

dispositions. Finally, the Student Perceptions of Critical Thinking Survey provided

valuable qualitative feedback concerning the effectiveness of the instructional

interventions included in the study.

Field Test

To determine whether the surveys had merit and to correct any flaws, the Pre- and

Post- Critical Thinking Surveys were administered to four teachers at the community

college level for review before conducting the research study. The participants of this

field test were encouraged to make criticisms and recommendations for improving the

surveys. Following the review, respondents indicated the surveys were appropriate,

understandable, and relevant to the research study.

Data Collection

On July 13, 2009, a research permission letter was sent to the dean of humanities

at the study college requesting permission to conduct research in a critical thinking

course during the Fall semester, along with a detailed explanation of the purpose of the

research. During the first week of the course, students and the researcher read and

60
discussed the critical thinking course syllabus, a detailed description of the research study

was given, and a request was made for those students who wished to participate in the

study. Participation in the research by students was strictly voluntary. Data collection

began and an informed consent letter was given to those who were interested in

participating in the study. Students were assured that participation in the study was

strictly voluntary, that participation or nonparticipation would in no way affect their

grade, and that their personal identity would not be released in the dissertation. At the end

of the first week of the semester, the informed consent letters were collected from those

students who wished to participate in the study.

A pre-investigation of student dispositions toward critical thinking was utilized. A

collection of data to determine student views of their own critical thinking dispositions

and abilities was gathered during the first week of the course. Students completed the

Pre- Critical Thinking Survey (Appendix A) self-assessment, which included a Student

Demographic Survey and questions concerning critical thinking dispositions, knowledge,

and skills. During the second week of the course, the CCTST–2000 was administered as a

pretest.

Embedded within critical thinking course curriculum, each of the independent

variables (interventions) was systematically included in the researcher’s instructional

strategies at 4-week intervals throughout the semester. Utilization of argument mapping

was introduced during the first 4 weeks; this was followed by instruction in Paul’s

Thinker’s Guides; during the final 4 weeks, instructional content included Socratic

questioning.

61
 During the final week of the semester, the CCTST–2000 was again administered

as a posttest. The paper-and-pencil version consisted of the test booklets (10-page

documents that contain directions and the 34 multiple-choice questions) and scannable

answer forms. Before beginning the test, the students were given an identification

number, and a master list was kept of the test takers’ names matched with their personal

identification numbers (PINs). This list was kept in a secure place to protect the privacy

of the test takers.

Upon completion of the CCTST–2000 pretest and posttest, exams were shipped

back to Insight Assessment, where they were scanned and scored using the CapScore

system. Insight Assessment then sent the research data file and a report summarizing the

survey results. Reports were generated that included descriptive statistics for the group as

a whole as well as subscale statistics. The CCTST–2000 is normed for 4-year college

students, 2-year college students, graduate students, and workplace manager-level

employees. These norm groups were used in the analysis of the test-taker data, and the

comparison with norm sample information was included in the Basic CapScore (N. C.

Facione et al., 2008).

Students also completed a Post- Critical Thinking Survey (Appendix B) during

the final week of the semester that consisted of the following sections: the Student

Demographic Survey; self assessment questions concerning critical thinking knowledge,

skills, and dispositions; and questions concerning the extent to which instruction in

argument mapping, Paul’s Thinker’s Guides, and Socratic questioning increased their

understanding of and ability to apply critical thinking concepts to practical situations in

62
life. The data from the Pre- and Post- Critical Thinking Surveys (Appendixes A & B) was

analyzed using tables in a Microsoft Word document to identify significant themes.

Data Analysis

Research Questions 1–3 focused on student perceptions regarding a positive

relationship between argument mapping, Paul’s Thinker’s Guides, and Socratic

questioning, and the acquisition of critical thinking skills. Research Question 4 focused

on the students’ perceptions of the changes in their knowledge, skill, dispositions, and

attitudes after direct instruction in critical thinking. These questions were addressed in the

Likert-scaled Student Perceptions of Critical Thinking and were analyzed using tables in

a Microsoft Word document and the highlighting tool to identify significant themes and

instructional methodologies most beneficial for developing students’ critical thinking

abilities. Additional charts and graphs were used to investigate patterns and insights. This

tool was particularly effective in recognizing which critical thinking instructional

strategies are most effective.

Research Question 5 focused on the extent to which the pretest and posttest scores

indicate changes in the students’ knowledge, skills, and dispositions relating to critical

thinking after direct instruction in argument mapping, Paul’s Thinker’s Guides, and

Socratic questioning. For both the pretest and posttest, the CCTST–2000 was the sole

quantitative instrument used. The CCTST–2000 involves an overall score on one’s

critical thinking skills and five subscales. The five subscales are Analysis, Evaluation,

Inference, Deductive Reasoning, and Inductive Reasoning. Analysis, Inference, and

Evaluation “draw together the major core skills identified in the theory of critical

63
thinking advanced in The APA Delphi Report. Inductive Reasoning vs. Deductive

Reasoning is represented in the fourth and fifth CCTST–2000 sub-scale” (N. C. Facione

et al., 2008, p. 12). The total score on the CCTST–2000 provides the overall measure of

students’ critical thinking skills.

Students were assigned a PIN for reasons of confidentiality; additionally, the

group indicator on the CapScore response form allowed for institution-specific

information to be coded onto the response form. For the pretest, each student indicated

001 on his or her response form. For the posttest, each student indicated 002 on his or her

response form. CapScore provided a total and scale CCTST–2000 scores for each test

taker by PIN and descriptive statistics for both Groups 1 and 2 separately. The CapScore

answer forms were returned along with a PC-formatted disk with an Excel and a tab-

delimited text file, including total and subscale CCTST–2000 scores for each test taker by

PIN (N. C. Facione et al., 2008). Changes in students’ critical thinking scores were

analyzed comparing the mean pre-CCTST–2000 score to the mean post-CCTST–2000

score using a t test.

Ethical Considerations

Certain procedures had to be followed in order to obtain permission for this

research study. A written description of the research design was submitted to the

administration. In a research study in which human participants are involved, students

must be informed about what will occur during the research study. Consent was obtained

from the students and the administrative faculty. Participants received an explanation of

the test and course room procedures to be used and a letter describing the research and

64
conditions of their participation (Gall et al., 2003). Meetings with the administration

occurred in order to obtain their interest and cooperation.

The research study was examined by Capella University’s Institutional Review

Board to ensure that it complied with “institutional regulations, professional standards of

conduct and practice and the participants provisions of the Code of Federal Regulations

for the Protection of Human Subjects” (Gall et al., 2003, p. 66). Student confidentiality

was respected, and participants were told who would have access to data and that their

privacy would be protected. Random codes were given to participants in the study to

ensure participant anonymity.

Limitations of the Study

A limitation to this study was the small sample size, which limits the conclusions

that can be generalized to the target population. A random sample from the accessible

population in the critical thinking course was used in the study. The researcher gathered

data from the Pre- Critical Thinking Survey (Appendix A) to determine the degree of

similarity between the accessible sample population with regard to age, gender

distribution, and ethnic distribution in the critical thinking course to the target population.

However, generalization may not be possible, nor is it necessary. It is truly up to the

reader to find similarities in an action research study to a different site or location.

In view of the fact that the researcher adopted a teacher action research approach

in which the researcher was the instructor, there was the potential for researcher bias as

the researcher could have real or perceived authority over the participants; however, the

researcher administered the CCTST–2000 and the Student Perceptions of Critical

65
Thinking Surveys as pre- and posttests. Since the Student Perceptions of Critical

Thinking Survey is qualitative in nature, the researcher provided a thorough description

of the findings. These findings were checked by the results generated by the CCTST–

2000, which is a standardized test and in which there is no room for bias. This

triangulation method contributed to the validity of the study as it extracted multiple

perspectives and data sources and helped to “eliminate biases that might result from

relying exclusively on any one data-collection method” (Gall et al., 2003, p. 464).

Suitable controls over extraneous variables that could have threatened the validity

of the experiment needed to be established in order to allow a precise interpretation of

data. Experimental designs often provide adequate controls for sources of internal

validity, but the results have restricted external validity. Another possible limitation of

the study was that the researcher had a certain amount of control over what occurred in

the classroom and very little control over what occurred in the students’ lives outside of

the classroom.

Other possible limitations to the research study included (a) the length of the

study as the development of critical thinking skills requires practice over an extended

period of time and (b) the response rate of the participants involved in the final data

analysis.

Summary

This study investigated whether direct instruction in rhetorical analysis in a

critical thinking course can improve students’ critical thinking skills. Argument mapping,

Paul’s Thinker’s Guides, and Socratic questioning were used as treatment interventions in

66
increments for 12 weeks during the semester. Data were collected from the pretest at the

beginning of the course and from the posttest during the final week of the course using

the results from the CCTST–2000. Students were also required to complete Pre- and

Post- Student Surveys as a means to better conceptualize students’ perceptions of what

they believed to be the most beneficial and applicable method for enhancing their critical

thinking skills. Final scores on the CCTST–2000 posttest and the Student Perceptions of

Critical Thinking Surveys were used to determine student achievement in critical

thinking.

67
 CHAPTER 4. DATA COLLECTION AND ANALYSIS

Introduction

The purpose of this study was to determine whether there would be improvement

in students’ critical thinking abilities, including knowledge, skills, and dispositions, after

the intervention of direct instruction in rhetorical analysis. The researcher investigated

student perceptions of the effectiveness of the following instructional strategies:

argument mapping; Thinker’s Guides, based on Paul’s model of critical thinking; and

Socratic questioning. The goal of the study was, on the one hand, to promote critical

thinking knowledge, skills, and dispositions and, on the other hand, to determine

improvement in student perceptions of critical thinking knowledge, skills, and

dispositions so they may transfer those skills across the academic disciplines and into

their personal and professional lives.

This chapter provides a statement of the problem, a review of the research

questions, an overview of data collection and instruments, and a discussion of student

demographics. This is followed by an overview of the data and the results as they relate

to the four research questions and the null hypothesis.

68
 Statement of the Problem

It is not known whether direct instruction in rhetorical analysis will improve

learners’ critical thinking knowledge, skills, and dispositions. This study explored

instructional pedagogy to promote students’ critical thinking abilities. The researcher

investigated whether direct instruction in rhetorical analysis in a college critical thinking

course could make a difference in students’ critical thinking skills. The interventions

included argument mapping; Thinker’s Guides, based on Paul’s model of critical

thinking; and Socratic questioning. These interventions were supplemented with course

work that involved reader/writer workshops and peer review.

Research Questions

The following questions directed the study:

1. What were the students’ perceptions of critical thinking before and after direct
instruction in argument mapping?

2. What were the students’ perceptions of critical thinking before and after direct
instruction in Paul’s Thinker’s Guides?

3. What were the students’ perceptions of critical thinking before and after direct
instruction in Socratic questioning?

4. What did the students report about the changes in their knowledge, skills, and
dispositions relating to critical thinking after the direct instruction in rhetorical
analysis?

Null Hypothesis: There will be no difference in scores on the pretest and posttest

on the California Critical Thinking Skills Test–Form 2000 indicating changes in student

critical thinking skills before and after direct instruction in rhetorical analysis.

69
 Overview of Data Collection

The mixed-methodology approach was used in this action research study. The

study pertained to one semester of a college critical thinking course that was taught in a

naturalistic setting by the researcher. The overall goal of the study was to examine the

effect of direct instruction in rhetorical analysis in students’ critical thinking knowledge,

skills, and dispositions. Twenty-nine students were initially enrolled in the critical

thinking course. Of the 29 students, 19 agreed to participate in the study. Three of those

students dropped the course and one student was not present the final day of the course

and did not take the posttest; thus, the final research sample was comprised of 15 (N =

15) of the 26 students enrolled in the critical thinking course.

Data Collection and Instruments

The triangulation or mixed-methodology approach was used in this one-group

pretest–posttest design to determine whether direct instruction in critical thinking, using

the aforementioned interventions, would improve students’ critical thinking knowledge,

skills, and dispositions. The independent variable for this study was the critical thinking

interventions. The dependent variables were the posttest scores on the CCTST–2000 and

on the Pre- and Post- Critical Thinking Surveys. The researcher examined the

relationship between the variables involving teacher instruction of the interventions, the

students’ mastery of those skills, and the students’ perceptions of the effectiveness of the

interventions.

Research Questions 1–3 focused on student perceptions regarding a positive

relationship between argument mapping, Paul’s Thinker’s Guides, and Socratic

70
questioning, and the acquisition of critical thinking skills. Research Question 4 focused

on the students’ perceptions of the changes in their knowledge, skill, dispositions, and

attitudes after direct instruction in critical thinking. These questions were addressed in the

Likert-scaled Pre- and Post- Critical Thinking Surveys and were analyzed using tables in

a Microsoft Word document to identify significant themes and instructional

methodologies most beneficial for developing students’ critical thinking abilities.

Research Question 5, the null hypothesis, focused on the extent to which the

pretest and posttest scores indicate changes in the students’ knowledge, skills, and

dispositions relating to critical thinking after direct instruction in argument mapping,

Paul’s Thinker’s Guides, and Socratic questioning. For both the pretest and posttest, the

CCTST–2000 was the sole quantitative instrument used. The CCTST–2000 test involves

an overall score on one’s critical thinking skills and five subscales. The five subscales are

Analysis, Evaluation, Inference, Deductive Reasoning, and Inductive Reasoning.

Analysis, Inference, and Evaluation “draw together the major core skills identified in the

theory of critical thinking advanced in The APA Delphi Report. Inductive Reasoning vs.

Deductive Reasoning is represented in the fourth and fifth CCTST–2000 sub-scale” (N.

C. Facione et al., 2008, p. 12). The total score on the CCTST–2000 provided the overall

measure of students’ critical thinking skills.

Students were assigned a PIN for reasons of confidentiality; additionally, the

group indicator on the CapScore response form allowed for institution-specific

information to be coded onto the response form. For the pretest, each student indicated

001 on his or her response form. For the posttest, each student indicated 002 on his or her

response form. CapScore provided a total and scale CCTST–2000 scores for each test

71
taker by PIN and descriptive statistics for both Groups 1 and 2 separately. The CapScore

answer forms were returned along with a PC-formatted disk with an Excel and a tab-

delimited text file, including total and subscale CCTST–2000 scores for each test taker by

PIN (N. C. Facione et al., 2008). Changes in students’ critical thinking scores were

analyzed comparing the mean pre-CCTST–2000 score to the mean post-CCTST–2000

score using a t test.

The second and third instruments used in the study, the Pre- and Post- Critical

Thinking Surveys (Appendixes A & B) were qualitative in nature. The Post- Critical

Thinking Survey included three sections: the student demographic survey, the Critical

Thinking Survey, and a Student Perceptions of Critical Thinking Survey. The Student

Perceptions of Critical Thinking portion of the Post- Critical Thinking Survey provided

valuable qualitative feedback concerning the effectiveness of the instructional

interventions included in the study.

Demographic Characteristics

The characteristics of the sample were based on the data obtained from the

student demographic portion of the Pre- Critical Thinking Survey. This portion of the

survey included questions concerning gender, race, age, current year in college, academic

major, student status, students’ perceptions of their writing and comprehension skills, and

previous academic experience in critical thinking. Tables 1 and 2 provide information

concerning student demographic characteristics.

72
Table 1. Sample Demographic Characteristics

Category n

Gender
Male 8
Female 7

Race
Black 1
Indian/Alaskan 1
Asian/Pacific Islander 0
Hispanic 2
White/non-Hispanic 11

Age
17–19 10
20–22 3
23–25 1
26–30 1
Over 30 0

Year in college
Freshman 2
Sophomore 13
Junior 0
Senior 0

Discipline major
Humanities 3
Sciences 4
Mathematics 2
Education 3
Social Sciences 3

Student status
Full-time, degree-seeking 13
Part-time, degree-seeking 2
Full-time, other credit 0
Part-time, other credit 0

73
Table 2. Sample Demographic: Skills and Abilities

Category N

English skills
Excellent 2
Very good 4
Good 6
Fair 2
In need of improvement 1

Writing skills
Excellent 0
Very good 6
Good 7
Fair 1
In need of improvement 1

Reading comprehension
Excellent 0
Very good 6
Good 6
Fair 3
In need of improvement 0

Taken Critical Thinking before

Yes 2
No 13

Taken Critical Thinking in Grades 9–12

that taught critical thinking explicitly
Yes 6
No 9

Taken Critical Thinking course in college

that taught critical thinking explicitly
Yes 2
No 13

74
 Of the 15 research participants, 8 were male and 7 were female, 11 were

White/non-Hispanic, 2 were Hispanic, 1 was Indian/Alaskan, and 1 was Black. Students’

academic majors were comprised of a variety of disciplines, including humanities,

sciences, mathematics, education, and social sciences. The majority of the students (13)

had never taken a college critical thinking course, were full-time degree-seeking

sophomores between the ages of 17 and 22, and believed their English and writing skills

and reading comprehension were good to excellent.

Results

This section has been organized around the study research questions. The

following format is used for each of the research questions: restatement of the research

question, a paragraph provided for clarification, a visual representation of the data, and an

explanation of the information provided by the visual.

Research Questions 1–3 focused on student perceptions regarding a positive

relationship between argument mapping, Paul’s Thinker’s Guides, and Socratic

questioning, and the acquisition of critical thinking skills. Research Question 4 focused

on the students’ perceptions of the changes in their knowledge, skill, dispositions, and

attitudes after direct instruction in critical thinking. These questions were addressed in the

Likert-scaled Student Perceptions of Critical Thinking portion of the Post- Critical

Thinking Survey. The null hypothesis focused on finding no difference in student critical

thinking skills before and after direct instruction in rhetorical analysis. This hypothesis

was addressed in the CCTST–2000 pretest and posttest.

75
 Research Question 1: What were the students’ perceptions of critical thinking

before and after direct instruction in argument mapping?

Argument mapping is a way of laying out visually reasoning and evidence for and

against a statement or claim. Good argument maps clarify and organize thinking by

showing the logical relationships between thoughts that are expressed simply and

precisely. They help students organize and navigate around complex information, clarify

reasoning, and communicate reasoning effectively while supporting critical thinking. The

independent variable used to address this question was the instruction and

implementation of argument mapping. Student responses on the Pre- and Post- Critical

Thinking Surveys (Appendixes A & B) served as dependent variables. Students were

asked to circle the appropriate number for each item in response to the overall

effectiveness of argument mapping with 1 being the low score and 5 being the high score.

A score of 1 or 2 indicated the intervention was ineffective; 3 indicated effective, and 4

or 5 indicated highly effective. Table 3 provides student responses concerning students’

perceptions of critical thinking after direct instruction in argument mapping.

There were 10 questions in this portion of the survey. Analysis using this Likert-

type scale revealed that 9 (60%) of the 15 students in the course designated argument

mapping as an effective to highly effective strategy for identifying simple and complex

arguments; the author’s central thesis; the rhetorical function of each paragraph; claims,

objections, counterarguments, and inferences made by the author; the structure of the

reasoning process; and the missing layers in an argument. Six (40%) of the students had

mixed feelings, 3 students indicated the intervention was ineffective in helping them

identify a simple argument, 2 a complex argument, 3 the author’s central thesis, 3 the

76
rhetorical function of each paragraph, 1 the claims made by the author, 2 the objections

made by the author, 2 the counterarguments, 3 the inferences made by the author, 2 the

structure of the reasoning process, and 4 the missing layers in an argument.

Table 3. Student Perceptions of Critical Thinking: Argument Mapping

Student responses

Low High
score score
To what extent does argument mapping help
you identify . . . 1 2 3 4 5

A simple argument 1 2 5 5 2

A complex argument 2 4 7 2

The author’s central thesis 3 3 6 3

The rhetorical/argumentative function of 3 7 5

each paragraph

The claims made by the author 1 5 6 3

The objections made by the author 2 4 7 2

The counter arguments 2 5 7 1

The inferences made by the author 1 2 3 5 4

The structure of the reasoning process 2 2 10 1

The missing layers in an argument 1 3 5 4 2

77
 Research Question 2: What were the students’ perceptions of critical thinking

before and after direct instruction in Paul’s Thinker’s Guides?

Paul’s model of critical thinking is a metacognitive approach that consists of four

domains of critical thinking: elements of thought, abilities, affective traits, and

intellectual standards. Paul and Elder took the essential elements of good critical thinkers

and developed The Thinker’s Guide to the Art of Socratic Questioning: Based on Critical

Thinking Concepts & Tools (Paul & Elder, 2006) and The Thinker’s Guide to Analytic

Thinking: How to Take Thinking Apart and What to Look for When You Do (Elder &

Paul, 2007). The researcher used both of these Thinker’s Guides in the study. The

independent variable used to address this questions was the instruction and

implementation of Paul’s Thinker’s Guides, which was the second intervention used in

this study. Student responses on the Post- Critical Thinking Survey (Appendix B) served

as the dependent variable. Table 4 provides students’ responses concerning their

perceptions of critical thinking after direct instruction in Paul’s Thinker’s Guides.

Analysis using this Likert-type scale indicated that 6 (40%) of the students

believed Paul’s Thinker’s Guides had been effective to highly effective in helping them

identify the author’s purpose, questions or problems the author had, point of view,

assumptions, inferences, bias or narrowness, contradictions in point of view, author’s

evidence and conclusions, and the elements of thought and how they assist in critical

thinking skills. Nine (60%) of the students offered mixed responses, indicating the

Thinker’s Guides had been ineffective in helping them identify the point of view,

unstated assumptions, inferences, bias or narrowness, and contradictions in point of view.

78
Table 4. Student Perceptions of Critical Thinking: Paul’s Thinker’s Guides

Student responses

Low High
score score
To what extent does Paul’s Thinker’s Guides
help you identify . . . 1 2 3 4 5

Author’s purpose 2 5 8

Author’s questions or problem 2 6 7

Point of view 1 2 4 8

Unstated assumptions 1 6 3 5

Inferences 2 4 6 3

Bias or narrowness 2 2 9 2

Contradictions in point of view 3 9 3

Author’s evidence 5 5 5

Author’s conclusions 2 6 7

Elements of thought and how they assist in 11 4

critical thinking abilities

Research Question 3: What were the students’ perceptions of critical thinking

before and after direct instruction in Socratic questioning?

Socratic questioning is a highly disciplined, systematic approach of inquiry that

aids students in developing and evaluating their ideas. The Socratic method provides

opportunities for instructors to explore student thinking and model the necessary moves

79
for Socratic questioning as they question, analyze, and speculate out loud. The instruction

and implementation of Socratic questioning served as the independent variable. Student

responses on the Post- Critical Thinking Survey (Appendix B) served as the dependent

variable. Table 5 provides student responses to this question.

Table 5. Student Perceptions of Critical Thinking: Socratic Questioning

Student responses

Low High
score score
To what extent does Socratic questioning
help you identify . . . 1 2 3 4 5

Goals and purposes of the writer 3 9 3

Nature of the question 3 9 3

Problem being addressed 1 11 3

Data and information to support 3 11 1

Alternative interpretations of data and 1 3 8 3

information

Key concepts and ideas 5 6 4

Assumptions being made 1 2 7 5

Implications and consequences of what is 1 11 3

being said

Alternative points of view 2 6 7

Effectiveness of Socratic questioning in 3 7 5

preparation for discussion, writing, and for
developing your arguments

80
 Analysis using this Likert-type scale indicated that 12 (80%) of the students

indicated Socratic questioning was effective to highly effective in helping them identify

the goals and purposes of the writer, the problem being addressed, relevant data and

information to support, key concepts and ideas, alternative points of view, and preparing

them for discussion, writing, and developing arguments. Three (20%) of the students

offered mixed responses, suggesting that Socratic questioning was ineffective in helping

them identify alternative interpretations of data and information, assumptions being

made, and implications and consequences of what is being said.

Students were also asked to rate which of the instructional interventions extended

their knowledge of critical thinking. The rating criteria consisted of 1 = true, 2 = false,

and 3 = I don’t know. Eight (53%) of the students believed argument mapping had

extended their knowledge of critical thinking, 13 (86%) indicated the Thinker’s Guides

had extended their knowledge of critical thinking, and all 15 (100%) of the students

believed Socratic questioning had extended their knowledge of critical thinking.

The final question on the Post- Critical Thinking Survey (Appendix B) offered

additional insights into students’ perceptions of the instructional interventions presented

in the course. Students were asked to identify which of the three strategies taught in the

critical thinking course had been most effective in helping them become better thinkers.

Responses to this question revealed the students preferred Paul’s Thinker’s Guides or

Socratic questioning and that argument mapping had not been effective in helping them

become better thinkers. Nine (60%) of the students designated the Thinker’s Guides as

being most effective in helping them become better thinkers and 6 (40%) reported

Socratic questioning had been most effective in helping them become better thinkers.

81
 Research Question 4: What did the students report about the changes in their

knowledge, skills, and dispositions relating to critical thinking after the direct instruction

in rhetorical analysis?

Student responses to the Knowledge, Skills, and Dispositions portions of the Pre-

and Post- Critical Thinking Surveys (Appendixes A & B) served as dependent variables.

Students were asked to rate each of the questions for how it described them using a 3-

point scale in which 1 = true, 2 = false, 3 = I don’t know.

Table 6 presents student responses to the Knowledge portion of the Pre- and Post-

Critical Thinking Surveys.

Data from the Knowledge portion of the Pre- and Post- Critical Thinking Surveys

revealed an increase in 11 of the critical thinking elements addressed in the critical

thinking skills questions. There were 6-point gains in students’ understanding of

synthesis as a higher level thinking skill and of fallacious reasoning; 4-point gains were

achieved in students’ ability to clearly define critical thinking and evaluation and in their

perceptions of strong critical thinkers; students demonstrated 3-point gains in their

definitions of comprehension and analysis as they relate to Bloom’s taxonomy; and there

were 1-point gains in students’ understanding of evaluation, bias, inference, and premise.

82
Table 6. Responses to Critical Thinking Survey: Knowledge

Correct responses

Question Pretest Posttest Gain

Comprehend synthesis as it relates to Bloom’s taxonomy 5 11 6

Recognize the elements of critical thinking 10 14 4

Understand comprehension as it relates to Bloom’s 11 14 3

taxonomy

Understand the connection between evaluation and 12 13 1

making judgments based on analysis

Understand the relationship between synthesis and 10 10 0

creative thinking

Comprehend the qualities of strong critical thinkers 4 8 4

Distinguish analysis as it relates to Bloom’s taxonomy 2 5 3

Identify the essential elements of an argument 14 14 0

Define bias 12 13 1

Define conclusion 14 13 –1

Define evaluation 9 13 4

Define evidence 14 14 0

Define fallacious reasoning 6 12 6

Define inference 8 9 1

Define premise 10 11 1

83
 Table 7 presents student responses to the Skills portion of the Pre- and Post-

Critical Thinking Surveys.

Table 7. Responses to Critical Thinking Survey: Skills

Student responses

Pretest Posttest

Don’t Don’t
Question True False know True False know Gain

Recognize connotation & 9 4 2 14 1 5

denotation

Recognize author’s purpose 10 3 2 15 5

Consider points of view 14 1 15 1

Evaluate information for 8 4 3 14 1 5

relevance

Evaluate source credibility 9 4 2 9 4 2 1

Comprehend point of view 8 4 3 13 2 5

Difficulty recognizing 2 12 1 3 12 0
difference between
assumptions & facts

Avoid generalizations 10 4 1 11 2 2 1

Analyze arguments, 12 2 1 14 1 2
interpretations, & beliefs

Difficulty comparing 2 12 1 2 11 2 –1
perspectives & considering
opposing points of view

84
Table 7. Responses to Critical Thinking Survey: Skills (continued)

Student responses

Pretest Posttest

Don’t Don’t
Question True False know True False know Gain

Offer relevant evidence 8 5 2 11 3 1 3

Transfer insights learned in 13 1 1 12 2 1 1

classroom to real life

Difficulty recognizing 6 8 1 8 7 –1
contradictions

Fair-minded 12 1 2 12 1 2 0

As indicated in Table 7, for the Skills portion of both Pre- and Post- Critical

Thinking Surveys, students illustrated an understanding of the importance of considering

opposing points of view, with scores of 14 and 15, respectively, out of 15 possible points.

The highest gains, 5 points, occurred in recognizing the connotative and denotative

meaning of words, recognizing the author’s purpose, evaluating information for its

relevance, and accurately comprehending one’s point of view. A 3-point gain occurred in

students’ acknowledgment of the necessity to offer relevant, constructive evidence; and

there was a 2-point increase in students grasping the importance of analyzing and

evaluating arguments, interpretations, and beliefs. Further, there were 1-point gains in

students understanding the importance of considering alternative points of view,

85
acknowledging the merit of evaluating the credibility of sources of information, and

avoiding generalizations and oversimplifications.

Table 8 presents student responses to the Dispositions portion of the Pre-and Post-

Critical Thinking Surveys.

Table 8. Responses to Critical Thinking Survey: Dispositions

Student responses

Pretest Posttest

Don’t Don’t
Question True False know True False know

Consider myself to be thoughtful 15 15

Think of ways to improve thinking 9 5 1 13 2

Prefer complex to simple problems 7 7 1 10 4 1

Enjoy situations that requires a lot of 6 7 2 10 5

thinking

Prefer tasks which challenge my 8 5 1 11 3 1

thinking abilities

Avoid situations which require in- 4 11 4 8 3

depth thinking

Enjoy long deliberation 1 12 2 4 5 6

Prefer thinking about small, daily 4 9 2 4 10 1

projects

Prefer projects which require very 4 10 1 2 13

little thought

86
Table 8. Responses to Critical Thinking Survey: Dispositions (continued)

Student responses

Pretest Posttest

Don’t Don’t
Question True False know True False know

Relying on thought to make my way 11 3 1 12 2

to the top is appealing

Enjoy discovering solutions to 12 1 2 13 1 1

problems

Do not enjoy learning new ways to 1 14 14 1

think

The notion of thinking abstractly is 2 11 2 1 14

frightening

Prefer tasks that are intellectual 11 3 1 13 1 1

Frequently deliberate about issues 9 4 2 12 1 2

On the pretest and posttest surveys, all of the research participants (N = 15)

indicated they considered themselves to be thoughtful, and 14 of the 15 students on both

surveys reported they enjoyed learning new ways to think. There were 4-point gains in

students’ willingness to discover ways to improve their thinking and to handle situations

that require a lot of thinking. Data from the survey reflected 3-point gains in students’

preference for tasks that challenge their thinking abilities, for attempting projects that

require considerable thinking, for abstract thinking, and for deliberating about issues even

when those issues do not affect them personally. Further, there were 2-point gains in

87
students’ preference for tasks that are intellectually difficult, for solving complex

problems, and for tasks that challenge their thinking abilities. Finally, the data revealed 1-

point gains in student preference for thinking about long-term projects, relying on

thought to make their way to the top, and discovering new solutions to problems.

Table 9 provides the total scores for student responses to the Knowledge, Skills,

and Dispositions portions of the Pre- Critical Thinking Survey.

Table 9. Results of Pre- Critical Thinking Survey

Survey Total score M

Knowledge 141 9.4

Skills 156 10.4

Dispositions 145 9.6

The total sum of all scores for the Knowledge portion of the Pre -Critical

Thinking Survey for the 15 students was 141 points (62%) out of a possible 225 points.

The measure of central tendency, or mean score of 9.4 was calculated by dividing the

sum of all scores by the number of participants (N = 15). The total sum of all scores for

the Skills portion of the Pre- Critical Thinking Survey was 156, with an average or mean

score of 10.4. Finally, the total sum of all scores for the Dispositions portion of the Pre-

Critical Thinking Survey was 145, with a mean score of 9.6.

Table 10 presents the total scores of student responses to the Knowledge, Skills,

and Dispositions portions of the Post- Critical Thinking Survey.

88
Table 10. Results of Post- Critical Thinking
Survey

Survey Total score M

Knowledge 174 11.6

Skills 184 12.2

Dispositions 171 11.4

The total sum of all scores for the 15 research participants on the Knowledge

portion of the Post- Critical Thinking Survey was 174 (77%). The average or mean score

was 11.6. The total sum of all scores for the Skills portion of the Post- Critical Thinking

Survey was 182, with an average or mean score of 12.3. The total sum of all scores for

the Dispositions portion of the Post- Critical Thinking survey was 171, yielding a mean

score of 11.4.

Table 11 provides a display of the total raw scores and gains for the Pre- and Post-

Critical Thinking Surveys.

Table 11 illustrates a 33-point (2.2 mean) gain in student scores in the Knowledge

portion of the Post- Critical Thinking Survey. Data obtained from the Skills portion of the

Post- Critical Thinking Survey conveyed a 26-point (1.8 mean) increase in student scores,

and data acquired from the Dispositions portion of the Post- Critical Thinking Survey

denoted a 26-point (1.8 mean) increase in student scores.

89
Table 11. Pre- and Post- Critical Thinking Surveys: Gains

Pretest Posttest

Total Total Total score Mean

Survey score M score M gain gain

Knowledge 141 9.4 174 11.6 33 2.2

Skills 156 10.4 184 12.2 26 1.8

Dispositions 145 9.6 171 11.4 26 1.8

On the Post- Critical Thinking Survey, students were asked to rate the

interventions used in the research project using a 3-point scale. Fourteen of the 15

students (93%) indicated their knowledge, skills, and dispositions had improved through

direct instruction in critical thinking.

Null Hypothesis: There will be no difference in scores on the pretest and posttest

on the California Critical Thinking Skills Test–Form 2000 indicating changes in student

critical thinking skills before and after direct instruction in rhetorical analysis.

The independent variable used to address this question was direct instruction in

rhetorical analysis using the treatment interventions, which included argument mapping,

Paul’s Thinker’s Guides, and Socratic questioning. The dependent variable was the

CCTST–2000 posttest total ranked score.

The CCTST–2000 was ordered from Insight Assessment and was administered by

the researcher as a pretest and posttest. The CCTST–2000 was based on the APA Delphi

90
consensus conceptualization of critical thinking. The CCTST–2000 targets the core

critical thinking skills of analysis, interpretation, inference, evaluation, and explanation.

At the end of the semester, the batch of CapScore CCTST answer forms was

returned to Insight Assessment. Insight Assessment then scanned and scored those

answer forms and returned a digital file reporting total and scale CCTST scores for each

test taker by PIN and descriptive statistics for the group of test takers as a whole.

The five subscales reported by Insight Assessment for the CCTST are analysis,

evaluation, inference, deductive reasoning, and inductive reasoning. A total score for

overall critical thinking was provided by Insight Assessment.

The CCTST–2000 consists of 34 multiple-choice questions. Each question is

worth 1 point, for a total raw score of 34. The test takes 45 minutes to complete under

timed conditions; however, for the purposes of this study, the test was administered

untimed using a paper-and-pencil format with a Scantron sheet.

Insight Assessment provided an Excel file and a Word file. The Excel file

included the scored pretest and posttest data for each test taker by PIN. The Word file

included descriptive statistics as well as additional t-test analyses. The basic descriptive

statistics provided as part of the CapScore scoring service were mean, median, trimmed

mean (the mean of the middle 90% of the data, used to remove outlier effect), standard

deviation, standard error of the mean, minimum, maximum, and first and third quartiles.

The median is the second quartile. This split the data into four groups, with 25% of the

data in each group.

The pretest scores for the CCTST–2000 are provided in Table 12. Note the total

mean score is 14.9, with a deviation of responses of 4.55.

91
Table 12. CCTST–2000 Pretest Scores

Variable N M Mdn Trimmed mean SD

Induction 15 9.000 10.000 9.154 2.699

Deduction 15 5.867 6.000 5.769 2.642

Analysis 15 3.800 4.000 3.846 1.373

Inference 15 6.600 6.000 6.615 2.720

Evaluation 15 4.467 4.000 4.462 1.885

Total1 15 14.870 14.000 14.920 4.550

Table 13 displays the statistical analysis of the posttest scores. Note the total mean

score is 16.1, with a standard deviation of 6.35. As the total mean score for the CCTST–

2000 pretest was 14.9, these data suggested a 2.2 mean gain in student scores.

Table 13. CCTST–2000 Posttest Scores

Variable N M Mdn Trimmed mean SD

Induction 15 9.000 9.000 8.923 2.928

Deduction 15 7.067 7.000 6.923 3.674

Analysis 15 3.800 4.000 3.769 1.699

Inference 15 7.667 7.000 7.538 3.109

Evaluation 15 4.600 4.000 4.385 2.230

Total2 15 16.070 15.000 15.850 6.350

92
 A paired samples t test and confidence interval (CI) were calculated and analyzed

by Insight Assessment for the CCTST–2000 pretest and posttest composite and subscale

scores to determine whether the difference in mean scores was statistically significant.

The CI was calculated to determine if all of the values within the range were defined by

the confidence limits of a sample statistic. The confidence limits “define the upper and

lower value of a range of values for a sample statistic that is likely to contain a population

parameter” (Gall et al., 2003, p. 638). The results of the t-test analyses are presented in

Tables 14, 15, and 16.

Table 14. T-Test Analyses: CCTST–2000 Pretest and Posttest Totals

95% CI for mean

difference

Variable N M SD SE mean Upper Lower t value p value

Total2 15 16.07 6.35 1.640 –0.788 3.188 1.29 0.216

Total1 15 14.87 4.55 1.170

Difference 15 1.20 3.59 0.927

There did not appear to be a statistical significance between the pretest and

posttest in any of the five categories at the 5% significance level; however, a paired t test

revealed a p value of 0.123 for the difference in values from the pre- to the posttest for

the variable deduction and a p value of 0.112 for the variable inference.

93
Table 15. T-Test Analyses: Analysis, Inference, and Evaluation

95% CI for mean

difference

Variable N M SD SE mean Upper Lower t value p value

Analysis

Analysis2 15 3.800 1.699 0.439 –0.936 0.936 0.00 1.000

Analysis1 15 3.800 1.373 0.355
Difference 15 0.000 1.690 0.436

Inference

Inference2 15 7.667 3.109 0.803 –0.281 2.415 1.70 0.112

Inference1 15 6.600 2.720 0.702
Difference 15 1.067 2.434 0.628

Evaluation

Evaluation2 15 4.600 2.230 0.576 –0.868 1.134 0.29 0.779

Evaluation1 15 4.467 1.885 0.487
Difference 15 0.133 1.807 0.467

This indicated the researcher can be approximately 88% confident there is a

difference in deduction scores from the pretest to the posttest scores and approximately

89% confident there is a difference in inference scores from the pretest to posttest scores.

There were no significant differences between the pretest and posttest scores for

induction, analysis, and evaluation. Although Figure 1 illustrates a slight difference

between the pretest and posttest scores for inference and deduction, the significance level

was negligible (p > 0.2).

94
Table 16. T-Test Analysis: Induction and Deduction

95% CI for mean

difference
SE
Variable N M SD mean Upper Lower t value p value

Induction

Induction2 15 9.000 2.928 0.756 –1.256 1.256 0.00 1.000

Induction1 15 9.000 2.699 0.697
Difference 15 0.000 2.268 0.586

Deduction

Deduction2 15 7.067 3.674 0.949 –0.369 2.769 1.64 0.123

Deduction1 15 5.067 2.642 0.682
Difference 15 1.200 2.833 0.732

Figure 1. Comparison of mean scores by question type.

95
 In this study, the null hypothesis was accepted. No statistical evidence was found

to suggest changes in student critical thinking skills; however, there was a difference

between the pretest and posttest scores for the variables deduction and inference.

Summary

This chapter provided a description of the research sample along with a discussion

of student demographics. This was followed by a summary of data analysis and

procedures and an overview of the data and results as they relate to the four research

questions. Research Questions 1–3 were concerned with students’ perceptions of critical

thinking before and after direct instruction in argument mapping, Paul’s Thinker’s

Guides, and Socratic questioning. Student responses to the survey questions indicated

that although students found all three of the instructional strategies to be effective, the

majority of the students preferred Socratic questioning and Paul’s Thinker’s Guides as

highly effective interventions for developing their knowledge of critical thinking and for

helping them become better thinkers.

The fourth question was concerned with students’ perceptions of the changes in

critical thinking knowledge, skills, and dispositions after direct instruction in rhetorical

analysis. For this question, the researcher examined the Knowledge, Skills, and

Dispositions portions of the Pre- and Post- Critical Thinking Surveys. Student responses

on the Knowledge portion of the Post- Critical Thinking Survey revealed a 2.2 increase in

student scores, the Skills section suggested a 1.8 increase in student scores, and the

average increase in student scores on the Dispositions portion of the survey was 1.8.

96
 Data from the CCTST was used to address the null hypothesis: There will be no

difference in scores on the pretest and posttest on the California Critical Thinking Skills

Test–Form 2000 indicating changes in student critical thinking skills before and after

direct instruction in rhetorical analysis. Results from the CCTST–2000 revealed that none

of the paired t tests was significant at the 0.05 level of significance; however, a paired t

test revealed a p value of 0.123 for the difference in values from the pre- to the posttest

for the variable deduction and a p value of 0.112 for the variable inference. There were

no significant differences between the pretest and posttest scores for induction, analysis,

and evaluation. Although there was a slight difference between the pretest and posttest

scores for inference and deduction, the significance level was negligible (p > 0.2).

97
 CHAPTER 5. RESULTS, CONCLUSIONS, AND RECOMMENDATIONS

Introduction

The purpose of this study was to determine whether there would be improvement

in students’ critical thinking abilities, including knowledge, skills, and dispositions, after

the intervention of direct instruction in rhetorical analysis. The researcher investigated

student perceptions of the effectiveness of the following instructional strategies:

argument mapping; Thinker’s Guides, based on Paul’s model of critical thinking; and

Socratic questioning. The goal of the study was, on the one hand to promote, critical

thinking knowledge, skills, and dispositions and, on the other hand, to determine

improvement in student perceptions of critical thinking knowledge, skills, and

dispositions so they may transfer those skills across the academic disciplines and into

their personal and professional lives.

This chapter presents a summary of the findings, discussion, and important

conclusions that were drawn from the data presented in chapter 4. Further, it provides

suggestions for further research, recommendations for practice, and connections to what

is known from the review of literature.

98
 Overview of Methodology

The mixed methodology approach was used in this action research study. The

study pertained to one semester of a college critical thinking course that was taught in a

naturalistic setting by the researcher. The overall goal of the study was to examine the

effect of direct instruction in rhetorical analysis in students’ critical thinking knowledge,

skills, and dispositions. Twenty-nine students were initially enrolled in the Critical

Thinking 104 course. Of the 29 students, 19 agreed to participate in the study. Three of

those students dropped the course and one student was not present the final day of the

course and did not take the posttest; thus, the final research sample was comprised of 15

(N = 15) of the 29 students enrolled in the critical thinking course.

The triangulation or mixed-methodology approach was used in this one-group

pretest–posttest design to determine whether direct instruction in critical thinking, using

the aforementioned interventions, would improve students’ critical thinking knowledge,

skills, and dispositions. The independent variable for this study was the critical thinking

interventions. The dependent variables were the pre- and posttest scores on the CCTST–

2000 and Pre- and Post- Critical Thinking Surveys. The researcher examined the

relationship between the variables involving teacher instruction of the interventions, the

students’ mastery of those skills, and the students’ perceptions of the effectiveness of the

interventions.

Summary of Findings

Research Question 1 addressed students’ perceptions of critical thinking before

and after direct instruction in argument mapping. There were 10 questions in this portion

99
of the Post- Critical Thinking Survey (Appendix B). Analysis using a Likert-type scale

revealed that 9 (60%) of the 15 students in the course designated argument mapping as an

effective to highly effective strategy for identifying simple and complex arguments; the

author’s central thesis; the rhetorical function of each paragraph; claims, objections,

counterarguments, and inferences made by the author; the structure of the reasoning

process; and the missing layers in an argument. Six (40%) of the students had mixed

responses; students indicated that the intervention was ineffective in helping them

identify (a) simple and complex arguments; (b) the author’s central thesis and the

rhetorical function of each paragraph; (c) the claims, objections, and inferences made by

the author; (d) the counterarguments; (e) the structure of the reasoning process, and (f)

the missing layers in an argument.

In an effort to improve argument analysis skills, students were explicitly taught

how to construct argument maps. Repeated individual and collaborative practice

encouraged students to evaluate and modify the premise, structure, support, and logic of

their own written arguments; as well, it offered them practical tools for evaluating the

validity of arguments presented in the text and in a variety of editorials and media

images. Once students became familiar with and grew accustomed to practicing this

instructional intervention, their responses to the assigned readings and analysis of

required essays revealed some improvement in argument analysis skills; however,

argument mapping was the first intervention presented by the teacher/researcher during

the first 5 weeks of an 18-week college course. At this point in time, students were

gradually becoming accustomed to close prose rhetorical analysis. Van Gelder (2005)

acknowledged “representing arguments in diagrams tends to be slow and cumbersome”

100
(p. 5), and student responses to argument mapping indicated they found this process to

be, at times, somewhat tiresome. Although the majority of the students believed argument

mapping contributed to their perceptions of critical thinking, students may not have been

given a sufficient amount of time to practice and develop the skills necessary for

identifying simple and complex arguments, the author’s thesis, the rhetorical functions of

each paragraph, objections, counterarguments, inferences made by the author, and the

structure of the reasoning process.

As with this current study, Bessick’s (2008) study examined the effect of direct

instruction in critical thinking using van Gelder’s rational argument mapping and Paul’s

Thinker’s Guides. Her study consisted of freshman students who took repeat courses and

who received individual tutoring throughout the semester. Bessick’s study revealed no

significant effect of the intervention group on subjects’ critical thinking skills as

measured by the CCTST–2000, yet students demonstrated improvement in academic

achievement, which may have been a result of the individual tutoring students received

throughout the semester and the fact they were taking repeated courses. Although Bessick

believed direct instruction in critical thinking and reasoning can improve student learning

outcomes, she also acknowledged the ultimate educational goal should be to enable

students to transfer those skills learned in the classroom to their everyday lives.

Similarly, van Gelder (2005) acknowledged individuals are not naturally inclined

toward critical thinking; nonetheless, students’ critical thinking can improve through

guided instruction, which exposes students to good critical thinking and engages them in

the critical thinking process. Argument mapping offers students the essential tools for

recognizing the logical structure of rhetoric; however, the students had a limited amount

101
of time to practice and develop this skill as each intervention was facilitated at 5-week

intervals. Student improvement in and appreciation for argument mapping occurs as a

result of effective instruction, repeated modeling of the procedures involved, and practice

over an extended period of time.

Research Question 2 targeted students’ perceptions of critical thinking before and

after direct instruction in Paul’s Thinker’s Guides. The independent variable used to

address this question was the instruction and implementation of Paul’s Thinker’s Guides,

which was the second intervention used in this study. Student responses on the Post-

Critical Thinking Survey served as the dependent variable. Analysis using a Likert-type

scale indicated that 6 (40%) of the students believed Paul’s Thinker’s Guides had been

effective to highly effective in helping them identify the author’s purpose, questions or

problems the author had, point of view, assumptions, inferences, bias or narrowness,

contradictions in point of view, the author’s evidence and conclusions, and the elements

of thought and how they assist in critical thinking skills. Nine (60%) of the students

offered mixed responses, indicating the Thinker’s Guides had been ineffective in helping

them identify the point of view, unstated assumptions and inferences, bias or narrowness,

and contradictions in point of view.

Students appeared to enjoy the application of Paul’s Thinker’s Guides as the

guides offer a comprehensible set of guidelines for analysis of textual as well as visual

arguments. The teacher/researcher taught Paul’s model explicitly, provided handouts, and

guided the students through frequent modeling of the process. Student written responses

and in-class discussion of the required readings suggested that the use of Paul’s elements

of reasoning provided a more thorough understanding of the rhetorical devices used by

102
authors; nevertheless, students appeared to have the most difficulty with identifying

unstated assumptions and inferences, bias, and contradictions in point of view and,

initially, found Paul’s model to be difficult and even frustrating. For all of the required

essays for the course, students were guided through a variety of activities to assist them in

creating final drafts. The teacher/researcher engaged the students in prewriting activities;

students then used their prewriting as a guide as they developed a rough draft of their

essays. This was followed by peer-editing sessions in which students examined the drafts

of their peers using Paul’s elements of reasoning. Teacher/researcher observations of the

peer-editing sessions and of student final drafts suggested this intervention assisted

students in offering a more systematic analysis and constructive criticism of their peers’

essays while increasing the thoroughness of assessment, evaluation, and modification of

their own written essays.

Correspondingly, Reed’s (1998) one-semester study investigated the effect of

integrating Paul’s model of critical thinking into a U.S. history course on community

college students’ ability to think critically about U.S. history and everyday issues,

dispositions toward thinking critically, and knowledge of history content. Unlike the

current study, Reed’s study consisted of four sections: two sections were experimental

groups and two sections were control groups. Students participating in the study took

three pretests and four posttests to determine the effectiveness of Paul’s model. Students

in both the control and experimental groups took the Ennis–Weir Critical Thinking Essay

Test as a pretest and posttest. Unlike the current study, results from Reed’s study revealed

that “posttest means increased by 3.28 points in the experimental group” (p. 143). One

possible reason could be this researcher’s small sample size of (N = 15) compared to

103
Reed’s size of (N = 52). Another significant contributing factor for Reed’s success could

have been that the instructor for Reed’s study participated in intensive training in Paul’s

model, while this researcher had limited preparation, which consisted of 1 week of

training, for implementing this intervention. Regardless, Reed’s study revealed that

students’ ability to think critically can improve in a single course when provided with

deliberate instruction and that Paul’s model “can be successfully integrated into an

introductory history course with statistically significant benefits to students’ abilities to

think critically within a domain and to their general critical thinking abilities” (p. 160).

Paul’s model of critical thinking and the Thinker’s Guides are invaluable tools for

facilitating and encouraging substantive critical thinking skills and for providing a

common model for instructional design, which can contribute to higher level learning.

Nevertheless, it is vital that students possess the “values, traits of mind, and dispositions

that underlie those skills” (Paul, 1995f, p. 119). As assessment should be specific,

consistent, and appropriate, it is necessary for students to possess the intrinsic motivation

and dispositions that contribute to the changes in perspective needed for substantial

learning. As the semester progressed, students’ performance on essays and homework

assignments reflected an increase in their ability to think critically and analytically and to

analyze arguments effectively. As with argument mapping, Paul’s Thinker’s Guides

demand continued practice over a period of time.

Student essays and responses to homework assignments revealed a marked

improvement in evaluation, analysis, and synthesis. Teacher/researcher observations of

class discussion and of students’ written homework assignments and essays revealed that

students’ ability to analyze the logic of articles, essays, and their own written work

104
improved. Concurrently, the students who illustrated this growth in analytical skills were

also those who appeared to be intrinsically motivated and who approached the course

work and the in-class discussions and collaboration with vigor and enthusiasm.

Research Question 3 was concerned with students’ perceptions of critical thinking

before and after direct instruction in Socratic questioning. The instruction and

implementation of Socratic questioning served as the independent variable. Student

responses on the Post- Critical Thinking Survey served as the dependent variable.

Analysis using a Likert-type scale indicated that 12 (80%) of the students revealed

Socratic questioning was effective to highly effective in helping them identify the goals

and purposes of the writer, the problem being addressed, relevant data and information to

support, key concepts and ideas, alternative points of view, and preparing them for

discussion, writing, and developing arguments. Three (20%) of the students offered

mixed responses, suggesting that Socratic questioning was ineffective in helping them

identify alternative interpretations of data and information, assumptions being made, and

implications and consequences of what is being said.

There are three kinds of Socratic questioning that can be utilized by educators to

probe student thinking: spontaneous, exploratory, and focused. Spontaneous questioning

provides opportunities for teachers to explore with students “how [they] might find out if

something is true, logical, or reasonable” (Paul & Elder, 2006b, p. 48). This type of

questioning provides opportunities for educators to “listen critically” (Paul & Elder,

2006b, p. 48) and for students to “become self-correcting” (Paul & Elder, 2006b, p. 48).

Spontaneous Socratic questioning moves include asking for examples, evidence, or

reasons; asking a group whether they agree, suggesting parallel examples; and providing

105
an analogy, asking for a paraphrase, or rephrasing a student response (Paul & Elder,

2006b).

Exploratory Socratic questioning is used by teachers to explore student thinking

on diverse issues. It can be used to explore bias or student values, to identify clarity of

thought, or to probe controversial issues (Paul & Elder, 2006b). This type of questioning

can be used for introducing a subject to the class and for analyzing a topic. An issue can

be raised for discussion and students could be required to write about the issue or to form

groups to discuss the issue. As with spontaneous Socratic questioning, exploratory

questioning requires minimal preplanning, although teachers might consider preparing

appropriate questions that could be used when discussing a topic or an issue.

Focused Socratic questioning occurs when instructors wish to explore and discuss

specific topics or issues. It includes having students engage in a discussion that

encourages them to “clarify, sort, analyze and evaluate thoughts and perspectives,

distinguish the known from the unknown [and] synthesize relevant factors and

knowledge” (Paul & Elder, 2006b, p. 50). Focused questioning requires preplanning,

which could include reflecting on a variety of perspectives, conclusions, concepts,

implications, and consequences, and constructing a set of questions for the Socratic

dialogue to allow students to think through a specific concept.

There are also a variety of patterns in which Socratic dialogue can be facilitated in

the classroom. To encourage reasoning and discussion, Socratic questioning can begin

with whole-class discussion, which then leads to small-group speaking and listening,

whole-class Socratic speaking and listening, individual writing, and small-group speaking

and listening. An alternative to this would be a schema that begins with whole-class

106
Socratic discussion to clarify students’ understanding of the question; this would be

followed by small-group research, individual writing, and then large-group Socratic

evaluation of the research. A third pattern would begin with whole-class Socratic

discussion with an introduction of the lesson and concepts involved, whole-class listening

and observing as the teacher models the task, followed by individual or group practice

(Paul & Elder, 2006b).

Socratic questioning in the critical thinking course included a combination of

Spontaneous questioning and exploratory questioning. Spontaneous questioning occurred

as the teacher/researcher discussed the concepts and ideas taken from the required

reading assignments, which generally consisted of a variety of arguments found in the

text, on the Internet, or in the news media. The researcher would introduce the key

concepts or questions from the readings to stimulate student thinking. In order to allow

students to reflect on the concepts or issues, students were given 5–10 minutes to free-

write a response. This was followed by class discussion of the issue as the

teacher/researcher asked for clarification, examples, and evidence, while offering

examples, providing analogies, asking for a paraphrase, or rephrasing student responses.

Exploratory Socratic questioning was used when whole group discussion was

used to introduce key questions or concepts to the class and to stimulate thinking.

Students were often given a prompt that dealt with a controversial issue discussed in a

previous reading assignment. They were then allowed 5–10 minutes to free-write their

response. This was often followed by small-group speaking and listening in order to

encourage students to think and reason cooperatively and to assess their ideas. For the

purpose of disciplining the questioning, students were given a script that required them to

107
focus on Paul’s elements of reasoning. The teacher/researcher assigned a questioner and a

recorder for each group. In this way, students worked with their peers in an informal

setting rather than having the teacher/researcher address focused formal Socratic

questioning techniques. Thus, students reasoned individually and cooperatively, applying

and assessing their ideas using preassigned questions to assist them in developing clarity,

accuracy, precision, relevance, depth, breadth, logic, significance, and fairness. This was

followed by class discussion for review and assessment of the small-group work and for

introducing new questions for further thought, discussion, and writing.

The collaborative process was used most frequently for the Socratic questioning

in the critical thinking course and, at times, it was highly productive; however, using the

single Socratic questioner to the whole group might have been more powerful and

engaging for the students. Further, researcher observations revealed that in order for the

collaborative process to be more effective, Socratic questioning should be consistently

modeled and monitored, and students must clearly comprehend their assigned roles in the

Socratic process; otherwise, the likelihood exists that students will either enter into a

debate or that certain students will monopolize the dialogue. Everyone should be allowed

to participate and to go beneath the surface of what is being discussed, constantly probing

the complexities of their ideas and the questions. Students must also be assured they do

not need to come to a group consensus, only that open discussion is allowed and that

students stay on topic.

In support of the merits of implementing Socratic questioning into instructional

pedagogy, Yang’s (2008) study was designed to investigate the effects of teaching critical

thinking skills, with the major goal of investigating whether students’ critical thinking

108
skills would improve after they participated in Socratic dialogues. Yang measured the

critical thinking scores using the CCTST; however, Yang achieved positive results,

indicating that instructors who use Socratic dialogues during small-group online

discussions can successfully improve students’ critical thinking skills. This improvement

in learning was accomplished “as students interact with their peers or instructor and

benefit from combining their levels of expertise, offering support, distributing the

thinking load, and confronting alternative points of view” (Yang, 2008, p. 261).

Paul (1995e) believed Socratic questioning embodies the spirit of critical thinking

as it presents opportunities for students to develop and evaluate their ideas; nevertheless,

the vital component for success with this instructional strategy is appropriate modeling of

the necessary moves for this kind of questioning. Moreover, rather than indoctrinating

learners, Newton (1978) contended that educators should stimulate learners while

encouraging them to become active participants in the learning process. Although higher

cognitive questioning is a viable strategy for teaching critical thinking, and “intelligence

is a reaction to the social and physical world” (Newton, 1978, p. 289), learners must

possess the willingness to acknowledge alternative perspectives while they adapt, refine,

and apply their newly acquired knowledge in a problem-solving manner (Newton, 1978).

Open-Ended Questions

Students were also asked to rate which of the instructional interventions extended

their knowledge of critical thinking. Eight (53%) of the students designated argument

mapping as a valuable instructional approach for contributing to their knowledge of

critical thinking, 13 (86%) of the students indicated the Thinker’s Guides had extended

109
their knowledge of critical thinking, and all 15 (100%) of the students suggested Socratic

questioning had played a significant role in furthering their knowledge of critical

thinking.

Concurrently, students were asked to identify which of the three strategies that

had been taught in the critical thinking course had been most effective in helping them

become better thinkers. Responses to this question revealed argument mapping had not

been effective in helping them become better thinkers. Nine (60%) of the students stated

Paul’s Thinker’s Guides had been most effective in helping them become better thinkers,

and 6 (40%) of the students designated Socratic questioning as being effective in helping

them become better thinkers.

As mentioned, argument mapping was the first intervention presented by the

teacher/researcher, and students were gradually becoming accustomed to the close

scrutiny required for rhetorical analysis. Once students had been exposed to Paul’s

Thinker’s Guides and Socratic questioning, they might have been more accustomed to

this process. Further, students may not have been given a sufficient amount of time to

practice and develop the skills necessary for identifying the structure and logic of an

argument and for effective argument mapping. As well, students may have preferred

Paul’s Thinker’s Guides because they appear to encourage more reflection on the

elements of thinking, on key concepts, and on the assumptions and inferences being

made, whereas argument mapping involves a more visual examination of rhetoric.

Argument maps are spatial layouts for presenting information, they rely on encouraging

the reader or writer to view the structure and the overall essence of the argument, and

they assist in conveying the essential logic of the rhetoric. Students may not have found

110
this process to be as interesting as Paul’s model, which appears to be more analytical and

to allow for the development of higher level thinking skills.

Research Question 4 was concerned with students’ perceptions of the changes in

their knowledge, skills, and dispositions relating to critical thinking after the direct

instruction in rhetorical analysis. Student responses to the Knowledge, Skills, and

Dispositions portions of the Pre- and Post- Critical Thinking Surveys (Appendixes A &

B) served as dependent variables.

Halpern (1999) contended that critical thinking skills can be taught as rhetorical

analysis, problem solving, and decision making, yet her contention is that these skills

should be taught within a variety of contexts and for transfer. However, students must not

only be taught the necessary skills needed for critical thinking, they must also have a

disposition toward and a willingness to apply them.

Summary of Pre- and Post- Critical Thinking Surveys:

Knowledge, Skills, and Dispositions

Data from the Knowledge portion of the Pre- and Post- Critical Thinking Surveys

revealed an increase in 11 of the critical thinking elements addressed in the critical

thinking skills questions. Students experienced gains in the understanding of synthesis as

a higher level thinking skill and of fallacious reasoning. Gains were also realized in

students’ ability to clearly define critical thinking, to evaluate, to distinguish strong

critical thinkers, and to define comprehension and analysis as they relate to Bloom’s

taxonomy. Further, there was an increase in students’ comprehension of evaluation, bias,

inference, and premise.

111
 For the Skills portion of both the Pre- and Post- Critical Thinking Surveys,

students illustrated an understanding of the importance of considering opposing points of

view, with the highest gains in recognizing the connotative and denotative meaning of

words, recognizing the author’s purpose, evaluating information for its relevance, and

accurately comprehending one’s point of view. There were also gains in students’

acknowledgment of the necessity to offer relevant, constructive evidence; grasping the

importance of analyzing and evaluating arguments and interpretations; understanding the

importance of considering alternative points of view; acknowledging the merit of

evaluating the credibility of sources of information; and avoiding generalizations and

oversimplifications.

On the pretest and posttest Dispositions portions of the surveys, all of the research

participants (N = 15) indicated they considered themselves to be thoughtful, and 14 of the

15 students on both surveys reported they enjoyed learning new ways to think. There

were also gains in students’ willingness to discover ways to improve their thinking, for

attempting tasks that challenge their thinking abilities, for abstract thinking, and for

deliberating about issues even when those issues do not affect them personally.

Additionally, there was an increase in students’ willingness to solve complex problems,

think about long-term projects, rely on thought to make their way to the top, and discover

new solutions to problems.

The total sum of all scores for the Knowledge portion of the Pre- Critical

Thinking Survey for the 15 students was 141 points (62%) out of a possible 225 points.

The measure of central tendency, or mean score of 9.4 was calculated by dividing the

sum of all scores by the number of participants (N = 15). The total sum of all scores for

112
the Skills portion of the Pre- Critical Thinking Survey was 156, with an average or mean

score of 10.4. Finally, the total sum of all scores for the Dispositions portion of the Pre-

Critical Thinking Survey was 145, with a mean score of 9.6.

The total sum of all scores for the 15 research participants on the Knowledge

portion of the Post- Critical Thinking Survey was 174 (77%). The average or mean score

was 11.6. The total sum of all scores for the Skills portion of the Post- Critical Thinking

Survey was 182, with an average or mean score of 12.3. The total sum of all scores for

the Dispositions portion of the Post- Critical Thinking survey was 171, yielding a mean

score of 11.4. These data revealed a 33-point (2.2 mean) gain in Knowledge, a 26-point

(1.8 mean) increase in Skills, and a 26-point (1.8 mean) increase in Dispositions on the

Post- Critical Thinking Survey.

Data from the Post- Critical Thinking Survey suggested instructors can make a

positive impact on students’ perceptions of their own critical thinking abilities through

direct instruction in rhetorical analysis. The highest gains were achieved in the

Knowledge portion of the Post- Critical Thinking Survey. This 2.2 mean gain in student

scores indicated an increase in students’ comprehension of synthesis and fallacious

reasoning, ability to clearly define critical thinking and evaluation, and discernment of

strong critical thinkers. Students also experienced gains in their ability to characterize

evaluation, comprehension, and analysis as they relate to Bloom’s taxonomy, and in their

understanding of evaluation, bias, inference, and premise.

The 1.8 mean gain in the Skills portion of the Post- Critical Thinking Survey

implied an increase in understanding of the importance of considering opposing

viewpoints. The highest gains, 5 points, occurred in students’ ability to distinguish the

113
connotative and denotative meaning of words, recognize the author’s purpose, evaluate

information for relevance, and accurately comprehend point of view. Further, gains were

realized in students’ ability to offer constructive evidence, recognize the importance of

analyzing and evaluating arguments, consider alternative points of view, acknowledge the

importance of evaluating the reliability of sources, and avoid generalizations.

The 1.8 mean gain in the Dispositions portion of the Post- Critical Thinking

Survey revealed an increase in students’ desire to enjoy learning new ways to think. The

highest gains were achieved in students’ willingness to discover ways to expand their

thinking and to handle situations that require extensive thinking. Additional gains were

made in students’ preference for tasks that challenge their thinking abilities for

attempting projects that require considerable thinking, for abstract thinking, and for

deliberating about issues. Finally, gains were realized in students’ preference for

intellectually difficult tasks, for solving complex problems, for deliberating about long-

term projects, for relying on thought for successful living, and for discovering new

solutions to problems.

The final question on the Post- Critical Thinking Survey asked students if they

believed their knowledge, skills, and dispositions had improved through direct instruction

in critical thinking. Fourteen of the 15 students (93%) indicated direct instruction had

been instrumental in developing their critical thinking.

Although the students’ overall scores on the CCTST–2000 were not at the level of

significance, the implementation of the research interventions—argument mapping,

Paul’s Thinker’s Guides, and Socratic questioning—did enhance students’ knowledge of

critical thinking and their perceptions of their own critical thinking abilities. Explicit

114
teaching of the components of critical thinking provided the students with the necessary

vocabulary, tools, and awareness of their own critical thinking dispositions essential for

effective argument analysis, and it could provide them with the knowledge of how to

apply that vocabulary and knowledge to the analysis of their own writing, to the rhetoric

in written texts, and to the rhetoric with which they are confronted in the media.

The null hypothesis focused on the extent to which the pretest and posttest scores

indicate changes in the students’ knowledge, skills, and dispositions relating to critical

thinking after direct instruction in argument mapping, Paul’s Thinker’s Guides, and

Socratic questioning. For both the pretest and posttest, the CCTST–2000 was the sole

quantitative instrument used. The total score on the CCTST–2000 provided the overall

measure of students’ critical thinking skills. Changes in students’ critical thinking scores

were analyzed comparing the mean pre-CCTST–2000 score to the mean post-CCTST–

2000 score using a t test. The independent variable used to address the null hypothesis

was direct instruction in rhetorical analysis using the treatment interventions, which

included argument mapping, Paul’s Thinker’s Guides, and Socratic questioning. The

dependent variable was the CCTST–2000 posttest total ranked score.

Data from the pretest scores for the CCTST–2000 indicated the total mean score

was 14.9, with a deviation of responses of 4.55. Statistics from the CCTST–2000 posttest

scores revealed a total mean score of 16.1, with a standard deviation of 6.35. As the total

mean score for the CCTST–2000 pretest was 14.9, these data suggested a 2.2 mean gain

in student scores.

A paired samples t test and CI were calculated and analyzed by Insight

Assessment for the CCTST–2000 pretest and posttest composite and subscales scores to

115
determine whether the difference in mean scores was statistically significant. The CI was

calculated to determine if all of the values within the range were defined by the

confidence limits of a sample statistic. The resulting data suggested there does not appear

to be a statistical significance between the pretest and posttest in any of the five

categories at the 5% significance level; however, a paired t test revealed a p value of

0.123 for the difference in values from the pre- to the posttest for the variable deduction

and a p value of 0.112 for the variable inference. This indicated the researcher can be

approximately 88% confident there is a difference in deduction scores from the pretest to

the posttest scores and approximately 89% confident there is a difference in inference

scores from the pretest to posttest scores. There were no significant differences between

the pretest and posttest scores for induction, analysis, and evaluation. Although there was

a slight difference between the pretest and posttest scores for inference and deduction, the

significance level was negligible (p > 0.2).

In this study, the null hypothesis was accepted. No statistical evidence was found

to suggest changes in student critical thinking skills; however, there was a slight

difference between the pretest and posttest scores for the variables deduction and

inference. Although N. C. Facione et al. (2008) contended that “measures of CT skills

have been demonstrated to capture gains in CT over time periods of one quarter or one

semester” (p. 28), there were no significant differences in students’ pretest and posttest

scores on the CCTST–2000. Nonetheless, this research study still could provide insight

for educators who are interested in instructional pedagogy that may well promote

students’ critical thinking skills.

116
 The critical thinking course could be revised to include more emphasis on

instructional strategies that target the core critical thinking skills of analysis,

interpretation, inference, evaluation, and explanation. Specifically, more time could be

spent on the analysis of the logic of a variety of arguments, on recognizing bias, and on

evaluating rhetoric for its credibility. Further, students often have a tendency to base their

decisions on preconceived notions, assumptions, and inferences, and, often, they appear

to have a difficult time accommodating for opposing points of view. Encouraging

students to be fair-minded through discussion and modeling and helping them to

recognize the logic or the fallacious reasoning in texts, in the media, and in their own

thinking and writing could promote the quality of fair-mindedness and the critical

thinking skills needed for intelligent, productive decision making. As P. A. Facione,

Facione, and Giancarlo (2000) affirmed, “To the open-minded, imaginative, and

intellectually adventurous, reflection on human experience can reveal a rich array of

possibilities” (p. 7).

Encouraging students’ critical thinking abilities and designing pedagogy that can

promote those skills is a noteworthy cause; however, teacher/researcher observations in

the critical thinking course suggested that student motivation was a contributing factor to

student success and achievement. P. A. Facione et al. (2000) argued it is not sufficient to

focus on critical thinking skill development without nurturing the “internal motivation to

use those skills in the appropriate circumstances” (p. 34). An effective tool for nurturing

students’ critical thinking dispositions, such as modeling good critical thinking behaviors,

could have been used more frequently and deliberately by the researcher in the critical

thinking course.

117
 Tucker’s (2008) study at Kent School of Social Work examined the development

of graduate-level social work students’ critical thinking abilities via a critical-thinking-

infused curriculum. Tucker utilized the CCTST–2000 to measure the students’ critical

thinking performance on pre- and posttest scores. Similar to this researcher’s study,

Tucker’s pretest and posttest scores revealed no significant differences except for the

inference subscale. Students improved in their ability to “develop hypotheses, deliberate

and question relevant information from a variety of sources, consider alternatives and

potential consequences, and draw conclusions” (Tucker, 2008, p. 225). Although

Tucker’s study did not yield an overall improvement in critical thinking, as with the

current study, data analysis revealed a slight increase in deductive scores. Tucker

maintained the reason for the lack of improvement in scores was, in part, due to an

ineffective curriculum and lack of motivation on the part of teachers and faculty.

Regardless of this, she believed students’ critical thinking skills can be improved by

providing a course specifically designed for critical thinking. She further claimed that

“critical thinking skills are not the result of merely rewording goals and objectives, but of

explicit attention to skill development in each individual course and session through the

creation of purposeful learning activities” (p. 234). Student motivation to be open-minded

is essential; however, the need exists for the faculty to be motivated to create critical

thinking pedagogy that repeatedly encourages thoughtful analysis and purposeful

reflection.

118
 Limitations

Students enrolled in community college courses in California constituted the

research sample. Although the findings should prove useful, it is doubtful they can be

generalized to a target population in other locations unless the sample sizes share similar

characteristics. Students enrolled in California community colleges are similar to students

enrolled in courses elsewhere; however, as Gall et al. (2003) maintained, “generalizations

to other cases can be done, but it must be done on a case-by-case basis” (p. 10).

Therefore, generalization may not be possible, nor is it necessary, as it is truly up to the

reader to find similarities in an action research study to a different site or location.

Possible limitations exist due to the modest sample size of 15 (N = 15) students

and the short term of instruction necessitated by an 18-week semester course. Research

suggests that critical thinking is a lifelong process and that successful interventions may

need to occur over an academic year or over several years. Ideally, a longitudinal study

would be most beneficial in determining growth in critical thinking knowledge, skills,

and dispositions as a result of the proposed interventions. The researcher was the teacher

of an intact class, and no attempt at randomization occurred; therefore, the study was

limited to a description of an existent situation rather than creating an experimental

situation. In teacher action research, data are collected and analyzed to assess and

improve educational practice. Since the researcher was the instructor in the critical

thinking course, there was potential for researcher bias as the researcher may have

authority over the participants. The researcher understood this and attempted to approach

the study with caution and without preconceived assumptions of possible outcomes or

results. Care was taken to maintain clarity and objectivity through sharing with research

119
participants and through personal reflection, assessment, and modification of

instructional practices. This study was limited to examination of the interventions

employed in a critical thinking course and did not attempt to measure the influence of

out-of-classroom experiences in growth in critical thinking. For further consideration is

the Hawthorne effect, research participants were aware of the fact they were participating

in a research study, and this may have affected their performance (Gall et al., 1996).

Recommendations

Recommendations for Future Research

This study examined whether there would be improvement in students’ critical

thinking abilities, including knowledge, skills, and dispositions, after the intervention of

direct instruction in rhetorical analysis using argument mapping; Thinker’s Guides, based

on Paul’s model of critical thinking; and Socratic questioning. Further research should be

conducted to investigate instructional pedagogies that could promote critical thinking

knowledge, skills, and dispositions so students may transfer those skills across the

academic disciplines and into their personal and professional lives. Additional

consideration should be given to the examination of the relationship between critical

thinking skills and critical thinking dispositions and designing research studies that

investigate instructional pedagogy that explicitly addresses both of these variables.

This research study consisted of a modest sample size of 15 students and a short

term of instruction necessitated by an 18-week semester course. Further research is

needed using a larger sample size and, possibly, a control group and an experimental

group to determine the extent to which direct instruction using argument mapping, Paul’s

120
Thinker’s Guides, and Socratic questioning can improve students’ critical thinking

abilities.

To increase student learning and achievement in critical thinking and the overall

effectiveness of the course, the length of the study could be extended to an entire year, or

a longitudinal study could be conducted across several critical thinking courses. This

could provide instructors with more time to implement the instructional interventions,

enhance student comprehension and application of those interventions, and provide

deeper insight as to the effectiveness of the research study.

Moreover, a clear definition of critical thinking should be agreed upon by faculty

members who wish to teach critical thinking. The Delphi Report’s definition of critical

thinking could prove useful for those who are attempting to conceptualize and implement

direct instruction in critical thinking in a variety of academic disciplines.

Recommendations for Practice

Although the students’ overall scores on the CCTST–2000 were not at the level of

significance, the implementation of the research interventions—argument mapping,

Paul’s Thinker’s Guides, and Socratic questioning—did improve students’ knowledge of

critical thinking and their perceptions of their own critical thinking abilities. Educators

need to consider explicit teaching of the components of critical thinking as it could

provide the students with the necessary vocabulary, tools, and awareness of their own

critical thinking dispositions essential for effective argument analysis, and with the

knowledge of how to apply that vocabulary and knowledge to the analysis of their own

writing, to the rhetoric in written texts, and to the rhetoric with which they are confronted

in the media.

121
 To improve students’ critical thinking knowledge, skills, and dispositions,

educators could develop instructional pedagogy with purposeful learning activities that

encourage critical thinking abilities. Students must be taught how to think critically, and

frequent and explicit teacher modeling of argument mapping, Paul’s Thinker’s Guides,

and Socratic questioning is essential for student comprehension and understanding.

Furthermore, students need to be given opportunities for consistent, repeated practice of

these skills over an extended period of time.

As well, support from the administrative staff along with the implementation of

teacher training in critical thinking instructional strategies, such as argument mapping,

Paul’s Thinker’s Guides, and Socratic questioning, could improve the quality of

instruction and student learning. Ideally, staff members could be encouraged to observe

and collaborate with instructors from other colleges and universities who have

successfully implemented critical thinking into their course curriculum. In California, the

budget deficit is hindering optimum teacher performance. Appropriate resources,

including serviceable and functional technology, are not necessarily imperative but

certainly beneficial for teacher/learner performance and achievement.

What is of utmost importance is creating a classroom that encourages

collaboration, open dialogue, and an acceptance of diverse values, beliefs, and

perspectives. Students should be allowed to openly express their opinions without fear of

judgment, censure, or reproach, and educators can encourage optimal critical thinking

behaviors and attitudes through effective modeling of those behaviors.

122
 Conclusions

There is a growing need for educators to attempt to design instructional pedagogy

that will promote students’ critical thinking skills. The researcher employed a variety of

instructional strategies to support and promote the implementation of argument mapping,

Paul’s Thinker’s Guides, and Socratic questioning.

Data from the Post- Critical Thinking Survey suggested instructors can make a

positive impact on students’ perceptions of their own critical thinking abilities through

direct instruction in rhetorical analysis. The highest gains were achieved in the

Knowledge portion of the Post- Critical Thinking Survey. This 2.2 mean gain in student

scores indicated an increase in students’ comprehension of synthesis and fallacious

reasoning, in their ability to clearly define critical thinking and evaluation, and in their

discernment of strong critical thinkers. There was also a 1.8 mean gain in the Skills

portion of the survey. This gain in student scores occurred in students’ ability to

distinguish the connotative and denotative meaning of words, recognize the author’s

purpose, evaluate information for relevance, and accurately comprehend point of view.

As well, there was a 1.8 mean gain in the Dispositions portion of the Post- Critical

Thinking Survey. The highest gains in this category were achieved in students’

willingness to discover ways to expand their thinking and to handle situations that require

extensive thinking. Finally, when asked if they believed their knowledge, skills, and

dispositions had improved through direct instruction in critical thinking, 14 of the 15

students (93%) indicated direct instruction had been instrumental in developing their

critical thinking.

123
 Although data from the research findings indicated there were no significant

differences between the pretest and the posttest scores for induction, analysis, and

evaluation, there was a slight difference between the pretest and posttest scores for

inference and deduction; however, the significance level was negligible (p > 0.2).

Nonetheless, findings from the study suggested that repeated, purposeful implementation

of instructional pedagogy, such as argument mapping, Paul’s Thinker’s Guides, and

Socratic questioning, could strengthen students’ perceptions of critical thinking and of

their own critical thinking abilities. Further, the results and the procedures used in this

study could encourage teachers to develop a critically reflective classroom environment

by participating in teacher training courses that support and facilitate the development of

effective pedagogies and assessment tools for the encouragement of student critical

thinking abilities.

124
 REFERENCES

Atherton, J. S. (2005). Learning and teaching: Piaget’s developmental theory. Retrieved

March 1, 2009, from http://www.learningandteaching.info/learning/piaget.htm

Bessick, S. C. (2008). Improved critical thinking skills as a result of direct instruction

and their relationship to academic achievement. Retrieved from Dissertations &
Theses: Full Text database. (AAT 3303591)

Bondy, K. N., Koenigseder, L. A., Ishee, J. H., & Williams, B. G. (2001). Psychometric
properties of the California Critical Thinking Tests. Journal of Nursing
Measurement, 9, 309–328.

Brookfield, S. (1995). Becoming a critically reflective teacher. The getting of wisdom:

What critically reflective teaching is and why it’s important. Retrieved May 11,
2009, from National-Louis University Web site: http://www.nl.edu/academics/
cas/ace/facultypapers/StephenBrookfield_Wisdom.cfm?RenderForPrint=1

Brookfield, S. (2003). Adult cognition as a dimension of lifelong learning. In J. Field &

M. Leicester (Eds.), Lifelong learning: Education across the lifespan (pp. 89–
101). New York: RoutledgeFalmer.

Brookfield, S. D., Tennant, M., & Pogson, P. (2005). Theory and methods of educating
adults. New York: Wiley.

Carroll, R. T. (2007). Teaching critical thinking. Retrieved November 11, 2008, from
http://www.skepdic.com/essays1

Chance, P. (1986). Thinking in the classroom: A survey of programs. New York:

Teachers College Press.

Cowden, E. M., Chivore, B. R. S., Maravanyika, O. E., Nyagura, L. M., & Sibanda, I. M.
(1999). Overview of educational research methodologies and assessment criteria
of the quality of educational research. In Zimbabwe Education Sector Analysis
Review Group (Ed.), Review of education sector analysis in Zimbabwe 1990–
1996 (pp. 9–14). Retrieved March 26, 2010, from http://unesdoc.unesco.org/
images/0011/001177/117705eo.pdf

125
Dahms, M., Geonnotti, K., Passalacqua, D., Schilk, J. N., Wetzel, A., & Zulkowsky, M.
(2008). The educational theory of Lev Vygotsky: An analysis. Retrieved March 1,
2009, from http://www.newfoundations.com/GALLERY/Vygotsky.html

Elder, L. (2000). Why critical thinking is essential to the community college mission and
why it will be difficult to achieve. Retrieved December 7, 2008, from http://www
.criticalthinking.org/page.cfm?PageID=529&CategoryID=73

Elder, L. (2006, April). Socratic dialogue in instruction. Paper presented at the

Cambridge Academy on Critical Thinking, England. Retrieved March 27, 2010,
from http://www.criticalthinking.org/print-page.cfm?pageID=571

Elder, L. (2007). Why critical thinking? Retrieved March 26, 2010, from http://www
.criticalthinking.org/page.cfm?PageID=796&CategoryID=103

Elder, L., & Paul, R. (2007). The thinker’s guide to analytic thinking: How to take
thinking apart and what to look for when you do. Dillon Beach, CA: Foundation
for Critical Thinking.

Engelmann, K. (n.d.). What is direct instruction (DI)? Retrieved December 2, 2008, from
http://www.nifdi.org/

Ennis, R. (1992, March). Critical thinking: What is it? Proceedings of the 48th annual
meeting of the Philosophy of Education Society, Denver, Colorado. Retrieved
May 26, 2006, from http://www.ed.uiuc.edu/PES/92_docs/Ennis.HTM

Ennis, R. (2002). An outline of goals for a critical thinking curriculum and its
assessment. Retrieved October 12, 2009, from http://faculty.ed.uiuc.edu/rhennis/
outlinegoalsctcurasses3.html

Facione, N. C., & Facione, P. A. (1997). Critical thinking assessment in nursing

education programs: An aggregate data analysis. Millbrae, CA: The California
Academic Press.

Facione, N. C., Facione, P. A., Blohm, S. W., & Gittens, C. A. (2008). California Critical
Thinking Skills Test: CCTST test manual 2008 edition. Millbrae, CA: California
Academic Press.

Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of

educational assessment and instruction: Executive summary: The Delphi report.
Retrieved March 1, 2009, from http://www.insightassessment.com/9dex.html

Facione, P. A. (1992). Manual, the California Critical Thinking Skills Test, Form A and
Form B. Millbrae, CA: California Academic Press.

126
Facione, P. A. (2000). The California Critical Thinking Skills Test: CCTST—Form 2000.
Millbrae, CA: Insight Assessment/The California Academic Press

Facione, P. A. (2006). Critical thinking: What it is and why it counts. Retrieved October
20, 2006, from www.calpress.com/pdf_files/what&why.pdf

Facione, P. A., Facione, N. C., Blohm, S. W., Howard, K., & Giancarlo, C. A. (1998).
The California Critical Thinking Skills Test manual (revised). Millbrae, CA: The
California Academic Press.

Facione, P. A., Facione, N. C., & Giancarlo, C. A. (2000). The disposition toward critical
thinking: Its character, measurement, and relationship to critical thinking skill.
Informal Logic, 20(1), 61–84. Retrieved June 12, 2009, from http://www.insight
assessment.com/9articles2.html

Gall, M. D., Gall, J. P., & Borg, W. R. (1996). Educational research: An introduction
(6th ed.). New York: Allyn & Bacon.

Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An introduction
(7th ed.). New York: Allyn & Bacon.

Gardner, H. (1993). Multiple intelligences: The theory in practice. New York: Basic
Books.

Halpern, D. F. (1999, December). Teaching for critical thinking: Helping college students
develop skills and dispositions of a critical thinker. New Directions for Teaching
and Learning, 80, 69–74. Retrieved November 18, 2008, from http://www3
.interscience.wiley.com/journal/101523939/abstract?CRETRY=1&SRETRY=0

Halpern, D. F. (2006). Halpern critical thinking assessment using everyday situations:

Background and scoring standards. Mahwah, NJ: Erlbaum.

Hatcher, D. L. (2000). Critical thinking: A new definition and defense. Retrieved October
12, 2009, from The Center of Critical Thinking Web site: http://www.bakeru.edu/
crit/literature/dlh_ct_defense.htm

Jackson, M. (2008). The erosion of attention and the coming dark age: Distracted. New
York: Prometheus Books.

Jacobs, S. S. (1995). Technical characteristics and some correlates of the California

Critical Thinking Skills Test, Forms A and B. Research in Higher Education, 36,
89–108.

127
Jacobs, S. S. (1999). The equivalence of Forms A and B of the California Critical
Thinking Skills Test. Measurement and Evaluation in Counseling and
Development, 31, 211–222.

Laird, T. F. N. (2005). The California Critical Thinking Skills Test (CCTST). Retrieved
February 1, 2009, from Wabash College, Center of Inquiry in the Liberal Arts
Web site: http://liberalarts.wabash.edu/cila/displayStory_print.cfm?news_ED=
5121

McPeck, J. (1981). Critical thinking and education. New York: St. Martin’s.

Meinecke, J. S. (1997). Toward an operational definition of critical thinking for the

NMSU college of Business Administration and Economics. Retrieved from
Dissertations & Theses: Full Text database. (AAT 9818805)

Mendelman, L. (2007, December). Critical thinking and reading. Journal of Adolescent &
Adult Literacy, 51(4), 300–302. Retrieved October 1, 2008, from Academic
Search Premier database.

Metcalfe, K. C. (2007). Assessing a feasible effective critical thinking protocol for

community colleges. Retrieved from Dissertations & Theses @ Capella University
database. (AAT 3291948)

National Center for Education Statistics. (2007). Digest of education statistics: 2007.
Retrieved March 2, 2009, from http:/nces.ed.gov/programs/digest/d07/

Newton, B. T. (1978). Theoretical bases for higher cognitive questioning—An avenue to

critical thinking. Education, 9(3), 286–291. Retrieved September 20, 2008, from
Academic Search Premier database. (AAT 4721735)

Ormrod, J. E. (2004). Learning theory and the educational process. Boston: Pearson.

Oxford, R. (1997). Constructivism: Shape-shifting, substance, and teacher education

applications. Peabody Journal of Education, 72, 35–36. Retrieved March, 1,
2009, from http://www.newfoundations.com/GALLERY/Piaget.html

Paul, R. (1993). Critical thinking, moral integrity, and citizenship: Teaching for the
intellectual virtues. Retrieved October 25, 2006, from http://www.criticalthinking
.org/resources/articles/ct-moral-integrity.shtml

Paul, R. (1995a). The critical connection: Higher order thinking that unifies curriculum,
instruction, and learning. In J. W. Willsen & A. J. A. Binker (Eds.), Critical
thinking: How to prepare students for a rapidly changing world (pp. 103–151).
Santa Rosa, CA: Foundation for Critical Thinking.

128
Paul, R. (1995b). Introduction. In J. W. Willsen & A. J. A. Binker (Eds.), Critical
thinking: How to prepare students for a rapidly changing world (pp. xi–xv). Santa
Rosa, CA: Foundation for Critical Thinking.

Paul, R. (with Binker, A. J. A.). (1995c). Critical thinking: Basic questions and answers.
In J. W. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to prepare
students for a rapidly changing world (pp. 91–100). Santa Rosa, CA: Foundation
for Critical Thinking.

Paul, R. (with Binker, A. J. A.). (1995d). Glossary: A guide to critical thinking terms and
concepts. In J. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to
prepare students for a rapidly changing world (pp. 521–552). Santa Rosa, CA:
Foundation for Critical Thinking.

Paul, R. (with Binker, A. J. A.). (1995e). Socratic questioning. In J. Willsen & A. J. A.

Binker (Eds.), Critical thinking: How to prepare students for a rapidly changing
world (pp. 335–365). Santa Rosa, CA: Foundation for Critical Thinking.

Paul, R. (with Nosich, G.). (1995f). A model for the national assessment of higher order
thinking. In J. W. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to
prepare students for a rapidly changing world (pp. 103–151). Santa Rosa, CA:
Foundation for Critical Thinking.

Paul, R. (with Willsen, J.). (1995g). Critical thinking: Identifying the targets. In J.
Willsen & A. J. A. Binker (Eds.), Critical thinking: How to prepare students for a
rapidly changing world (pp. 17–36). Santa Rosa, CA: Foundation for Critical
Thinking.

Paul, R. (2004). The state of critical thinking today: As the organizer in developing
blueprints for institutional change. Retrieved December 11, 2008, from http://
www.criticalthinking.org/professionalDev/the-state-ct-today.cfm

Paul, R. (2005). Choosing a model of critical thinking. Retrieved November 7, 2008,

from Surry Community College Web site: http://www.surry.edu/about/ct/
choosingn/choosing.html

Paul, R., & Elder, L. (1997, April). Socratic questioning. Retrieved December 7, 2008,
from http://www.criticalthinking.org/page.cfm?PagelID=606&CategoryID=64

Paul, R., & Elder, L. (2000). The role of questions in teaching, thinking and learning.
Retrieved December 6, 2008, from http://www.criticalthinking.org/articles/
thinking-some-purpose.cfm

129
Paul, R., & Elder, L. (2003, Winter). Critical thinking . . . and the art of close reading,
part I. Journal of Development Education, 27(2), 36–39. Retrieved October 1,
2008, from Academic Search Premier database. (AAT 11542636)

Paul, R., & Elder, L. (2004, Fall). Critical thinking . . . and the art of close reading, part
III. Journal of Developmental Education, 28(1), 36–37. Retrieved October 1,
2008, from Academic Search Premier database. (AAT 14576885)

Paul, R., & Elder, L. (2006a). First International Academy on Critical Thinking.
Retrieved February 8, 2007, from http://www.criticalthinking.org/conference/
cambridge.shtml

Paul, R., & Elder, L. (2006b). The thinker’s guide to the art of Socratic questioning:
Based on critical thinking concepts & tools. Dillon Beach, CA: Foundation for
Critical Thinking.

Paul, R., Elder, L., & Bartell, T. (1997a). A brief history of the idea of critical thinking.
Retrieved November 8, 2008, from http://www.criticalthinking.org/aboutCT/brief
HistoryCT.cfm

Paul, R., Elder, L., & Bartell, T. (1997b). Research findings and policy
recommendations: Study of 38 public universities and 28 private universities to
determine faculty emphasis. Retrieved December 7, 2008, from http://www
.criticalthinking.org/page.cfm?PageID=506&CategoryID=69

Perkins, D. (1993, Fall). Teaching for understanding. American Educator: The

Professional Journal of the American Federation of Teachers, 17(3), 8, 28, 35.
Retrieved November 25, 2008, from http://www.exploratorium.edu/IFI/resources/
workshops/teachingforunderstanding.html

Reed, J. H. (1998). Effect of a model for critical thinking on student achievement in

primary source document analysis and interpretation, argumentative reasoning,
critical thinking dispositions, and history content in a community college history
course. Retrieved from Dissertations & Theses: Full Text database. (AAT
9911510)

Scanlan, J. S. (2006). The effect of Richard Paul’s universal elements and standards of
reasoning on twelfth grade composition. Master’s thesis, Alliant International
University, San Diego, CA. Retrieved March 2, 2009, from http://www.critical
thinking.org/research/Abstract-JSCANLAN.cfm

Scriven, M., & Paul, R. (1987). Defining critical thinking. Retrieved November 9, 2008,
from http://www.criticalthinking.org/aboutCT/define_critical_thinkng.cfm

130
Shea, R. H., Scanlon, L., & Aufses, R. D. (2008). The language of composition: Reading,
writing, rhetoric. Boston: Bedford/St. Martin’s.

Sternberg, R. J. (2003). Four alternative futures for education in the United States: It’s
our choice. School Psychology Quarterly, 18(4), 431–445. Retrieved October 11,
2008, from ProQuest Education Journal database. (AAT 611229291)

Sternberg, R. J. (2008, January). Interdisciplinary problem-based learning. Liberal

Education, 94(1), 12–17. Retrieved October 11, 2008, from ProQuest Education
Journals database. (AAT 1429139651)

Texas State Technical College. (2004). Critical Thinking Surveys. Retrieved January 4,
2009, from Center for Excellence in Teaching Web site: http://www.harlingen.tstc
.edu/cetl/about.aspx

Tsui, L. (1998). Fostering critical thinking in college students: A mixed-methods study of

influences inside and outside of the classroom. Retrieved from Dissertations &
Theses: Full Text database. (AAT 9917229)

Tucker, T. (2008). Predictors of critical thinking as a component of an outcomes

assessment in a graduate level school of social work. Retrieved from
Dissertations & Theses: Full Text database. (AAT 3333829)

Twardy, C. (2003). Argument maps improve thinking. Teaching Philosophy, 27(2), 1–19.

Van Erp, N. L. (2008). Critical thinking in online graduate courses: A phenomenological

study. Retrieved from Dissertations & Theses @ Capella University database.
(AAT 3316338)

van Gelder, T. (2005). Teaching critical thinking. College Teaching, 53(1), 41–46.
Retrieved September 2, 2005, from Academic Search Premier database. (AAT
15348172)

Willsen, J. (1997, March). Chapter 1: Accelerating change, the complexity of problems,

and the quality of our thinking. Community College Times, 9(5). Retrieved
October 25, 2006, from
http://www.criticalthinking.org/resources/articles/accelerating-change.shtml

Yang, Y. (2008, June 1). A catalyst for teaching critical thinking in a large university
class in Taiwan: Asynchronous online discussion with the facilitation of teaching
assistants. Educational Technology Research and Development, 56(3), 241–264.
Retrieved January 28, 2009, from ERIC database. (ERIC Document Reproduction
Service No. EJ791102)

131
 APPENDIX A. PRE- CRITICAL THINKING SURVEY

(Modified from Metcalfe, 2007)

Part 1: Student Demographic Survey

The following information is being gathered for statistical purposes only. Please answer
each question on the form provided.

1. Gender
a. Male
b. Female

2. Race
a. African-American
b. American Indian/Alaskan Native
c. Asian or Pacific Islander
d. Hispanic
e. White – Non-Hispanic

3. Current Age
a. 17-19
b. 20-22
c. 23-25
d. 26-30
e. Over 30

4. Current year in college

a. Freshman
b. Sophomore
c. Junior
d. Senior

5. In what discipline is your major?

a. Humanities
b. Sciences (including nursing, computer science).
c. Social Sciences (including psychology)
d. Mathematics
e. Education

132
6. Student status:
a. Full-time degree seeking student
b. Part-time degree seeking student
c. Full-time other credit student
d. Part-time other credit student

7. I would consider my English skills to be

a. Excellent
b. Very good
c. Good
d. Fair
e. In need of improvement

8. I would consider my writing skills to be

a. Excellent
b. Very good
c. Good
d. Fair
e. In need of improvement

9. I would consider my reading comprehension skills to be

a. Excellent
b. Very good
c. Good
d. Fair
e. In need of improvement

10. Have you ever taken a Critical Thinking course (or a course similarly labeled) that
was devoted to teaching critical thinking skills or abilities?
a. Yes
b. No

11. Did you take one or more courses in grades 9-12 that explicitly taught critical
thinking skills while incorporating them into regular course work?
a. Yes
b. No

12. Have you taken one or more courses at a two or four year college or university
that explicitly taught critical thinking skills while incorporating them into regular
course work?
a. Yes
b. No

Part II: Pre- and Post- Critical Thinking Survey

133
Please rate each of the following statements for how it describes you. Use a 3-point scale
in which:

1 = True. 2 - False. 3 = Don’t Know.

Choose a number for each statement.

123

3: Knowledge
13) _____Synthesis is a higher-level thinking skill than summary.
14) _____Critical thinking is using disciplined, self-directed thinking to achieve a desired
outcome.
15) _____Comprehension is a lower-level thinking skill than knowledge.
16) _____Evaluation focuses on making a judgment based on analysis of a statement or
proposition.
17) _____Synthesis can be equated with creative thinking.
18) _____Strong critical thinkers are those who use the intellectual skills of critical
thinking selectively to promote and serve their own personal interests.
19) _____Analysis is a higher level skill than synthesis.
20) _____An argument consists of reasons for or against something
21) _____To be biased toward a particular idea or issue is to be fair-minded toward
opposing points of view.
22) _____A conclusion is the last step in the reasoning process.
23) _____To evaluate is to judge or determine the worth or quality of something.
24) _____Evidence is the data on which a judgment or conclusion might be based or by
which proof or probability might be established.
25) _____Fallacies or fallacious reasoning conforms to the rules of good reasoning.
26) _____An inference is an intellectual act by which one concludes that something is
true in light of something else being true
27) _____A premise is the last step in the reasoning process.

Critical Thinking: Skills

28) _____I can recognize the connotative and denotative meaning of words when I read.
29) _____I can recognize the author’s purpose for writing a specific piece of literature.
30) _____Considering alternative points of view is helpful in problem solving.
31) _____Evaluating information for its relevance is a valuable critical thinking ability.
32) _____When exploring an issue, I frequently evaluate the credibility of the sources of
information.
33) _____I can accurately comprehend one’s point of view or frame of reference.
34) _____I have difficulty recognizing the difference between assumptions, beliefs, and
facts.
35) _____I avoid generalizations and oversimplifications.
36) _____I analyze and evaluate arguments, interpretations, beliefs, or theories.
37) _____I have difficulty comparing perspectives, interpretations, or theories.

134
38) _____I have difficulty taking into account opposing points of view.
39) _____I find it easy to offer relevant, constructive evidence to support my points of
view.
40) _____I frequently compare or transfer insights learned in the classroom to real life
situations.
41) _____When exploring an issue, I sometimes have difficulty recognizing
contradictions..
42) _____I consider myself to be fair-minded.

Critical Thinking: Dispositions

43) _____I consider myself a thoughtful person.
44) _____I frequently think about ways to improve my thinking.
45) _____I would prefer complex to simple problems.
46) _____I like to have the responsibility of handling a situation that requires a lot of
thinking.
47) _____I would rather do something which challenges my thinking abilities.
48) _____I try to anticipate and avoid situations where there is a likely chance I will have
to think in-depth about something.
49) _____I find satisfaction in deliberating hard and for long hours.
50) _____I prefer to think about small, daily projects than long-term ones.
51) _____I prefer projects which require very little thought.
52) _____The idea of relying on thought to make my way to the top appeals to me.
53) _____I really enjoy a task that involves coming up with new solutions to problems.
54) _____Learning new ways to think doesn’t excite me very much.
55) _____The notion of thinking abstractly is frightening to me.
56) _____I would prefer a task that is intellectual, difficult, and important to one that is
somewhat important but does not require much thought.
57) _____I usually end up deliberating about issues even when they do not affect me
personally.

Note. The questions in the “Dispositions” portion of the Pre- Critical Thinking Survey are from Assessing a
Feasible Effective Critical Thinking Protocol for Community Colleges (p. 227), by K.C. Metcalfe, 2007,
Ph.D. Dissertation, Capella University, United States – Minnesota. Adapted for use with permission.

135
 APPENDIX B. POST- CRITICAL THINKING SURVEY

(Modified from Metcalfe, 2007)

Part 1: Student Demographic Survey

The following information is being gathered for statistical purposes only. Please answer
each question on the form provided.

1. Gender
a. Male
b. Female

2. Race
a. African-American
b. American Indian/Alaskan Native
c. Asian or Pacific Islander
d. Hispanic
e. White – Non-Hispanic

3. Current Age
a. 17-19
b. 20-22
c. 23-25
d. 26-30
e. Over 30

4. Current year in college

a. Freshman
b. Sophomore
c. Junior
d. Senior

5. In what discipline is your major?

a. Humanities
b. Sciences (including nursing, computer science).
c. Social Sciences (including psychology)
d. Mathematics
e. Education

6. Student status:
136
 a. Full-time degree seeking student
b. Part-time degree seeking student
c. Full-time other credit student
d. Part-time other credit student

7. I would consider my English skills to be

a. Excellent
b. Very good
c. Good
d. Fair
e. In need of improvement

8. I would consider my writing skills to be

a. Excellent
b. Very good
c. Good
d. Fair
e. In need of improvement

9. I would consider my reading comprehension skills to be

a. Excellent
b. Very good
c. Good
d. Fair
e. In need of improvement

10. Have you ever taken a Critical Thinking course (or a course similarly labeled) that
was devoted to teaching critical thinking skills or abilities?
a. Yes
b. No

11. Did you take one or more courses in grades 9-12 that explicitly taught critical
thinking skills while incorporating them into regular course work?
a. Yes
b. No

12. Have you taken one or more courses at a two or four year college or university
that explicitly taught critical thinking skills while incorporating them into regular
course work?
a. Yes
b. No

Part II: Pre- and Post- Critical Thinking Survey

137
Please rate each of the following statements for how it describes you. Use a 3-point scale
in which:

1 = True. 2 - False. 3 = Don’t Know.

Choose a number for each statement.

123

Critical Thinking: Knowledge

13) _____Synthesis is a higher-level thinking skill than summary.
14) _____Critical thinking is using disciplined, self-directed thinking to achieve a desired
outcome.
15) _____Comprehension is a lower-level thinking skill than knowledge
16) _____Evaluation focuses on making a judgment based on analysis of a statement or
proposition.
17) _____Synthesis can be equated with creative thinking.
18) _____Strong critical thinkers are those who use the intellectual skills of critical
thinking selectively to promote and serve their own personal interests.
19) _____Analysis is a higher level skill than synthesis.
20) _____An argument consists of reasons for or against something
21) _____To be biased toward a particular idea or issue is to be fair-minded toward
opposing points of view.
22) _____A conclusion is the last step in the reasoning process.
23) _____To evaluate is to judge or determine the worth or quality of something.
24) _____Evidence is the data on which a judgment or conclusion might be based or by
which proof or probability might be established.
25) _____Fallacies or fallacious reasoning conforms to the rules of good reasoning.
26) _____An inference is an intellectual act by which one concludes that something is
true in light of something else being true
27) _____A premise is the last step in the reasoning process.

Critical Thinking: Skills

28) _____I can recognize the connotative and denotative meaning of words when I read.
29) _____I can recognize the author’s purpose for writing a specific piece of literature.
30) _____Considering alternative points of view is helpful in problem solving.
31) _____Evaluating information for its relevance is a valuable critical thinking ability.
32) _____When exploring an issue, I frequently evaluate the credibility of the sources of
information.
33) _____I can accurately comprehend one’s point of view or frame of reference.
34) _____I have difficulty recognizing the difference between assumptions, beliefs, and
facts.
35) _____I avoid generalizations and oversimplifications.
36) _____I analyze and evaluate arguments, interpretations, beliefs, or theories.
37) _____I have difficulty comparing perspectives, interpretations, or theories.
38) _____I have difficulty taking into account opposing points of view.

138
39) _____I find it easy to offer relevant, constructive evidence to support my points of
view.
40) _____I frequently compare or transfer insights learned in the classroom to real life
situations.
41) _____When exploring an issue, I sometimes have difficulty recognizing
contradictions..
42) _____I consider myself to be fair-minded.

Critical Thinking: Dispositions

43) _____I consider myself a thoughtful person.
44) _____I frequently think about ways to improve my thinking.
45) _____I would prefer complex to simple problems.
46) _____I like to have the responsibility of handling a situation that requires a lot of
thinking.
47) _____I would rather do something which challenges my thinking abilities.
48) _____I try to anticipate and avoid situations where there is a likely chance I will have
to think in-depth about something.
49) _____I find satisfaction in deliberating hard and for long hours.
50) _____I prefer to think about small, daily projects than long-term ones.
51) _____I prefer projects which require very little thought.
52) _____The idea of relying on thought to make my way to the top appeals to me.
53) _____I really enjoy a task that involves coming up with new solutions to problems.
54) _____Learning new ways to think doesn’t excite me very much.
55) _____The notion of thinking abstractly is frightening to me.
56) _____I would prefer a task that is intellectual, difficult, and important to one that is
somewhat important but does not require much thought.
57) _____I usually end up deliberating about issues even when they do not affect me
personally.

Part III: Student Perceptions of Critical Thinking

Please circle the appropriate number for each item. Low High
Score Score
Argument Mapping
To what extent does Argument Mapping help you identify
58) a simple argument 1 2 3 4 5
59) a complex argument 1 2 3 4 5
60) the author’s central thesis 1 2 3 4 5
61) the rhetorical/argumentative function of
each paragraph 1 2 3 4 5
62) the claims made by the author 1 2 3 4 5
63) the objections made by the author 1 2 3 4 5
64) the counter arguments 1 2 3 4 5
65) the inferences made by the author 1 2 3 4 5
66) the structure of the reasoning process 1 2 3 4 5

139
67) the missing layers in an argument 1 2 3 4 5

Richard Paul’s Thinker’s Guides

To what extent do Thinker’s Guides help you identify
68) the author’s purpose 1 2 3 4 5
69) the questions or problem the author has 1 2 3 4 5
70) the point of view or frame of reference 1 2 3 4 5
71) unstated assumptions 1 2 3 4 5
72) inferences 1 2 3 4 5
73) bias or narrowness 1 2 3 4 5
74) contradictions in point of view 1 2 3 4 5
75) the author’s evidence 1 2 3 4 5
76) the author’s conclusions 1 2 3 4 5
77) the elements of thought and how they can assist
in your critical thinking abilities 1 2 3 4 5

Socratic Questioning
To what extent does Socratic Questioning help you identify
78) goals and purposes of writer 1 2 3 4 5
79) the nature of the question 1 2 3 4 5
80) the problem being addressed 1 2 3 4 5
81) relevant data and information to support 1 2 3 4 5
82) alternative interpretations of the data and information 1 2 3 4 5
83) key concepts and ideas 1 2 3 4 5
84) assumptions being made 1 2 3 4 5
85) implications and consequences of what is being said 1 2 3 4 5
86) alternative points of view 1 2 3 4 5
87) How effective was Socratic Questioning in preparing
you for discussion, writing, and for developing your
arguments 1 2 3 4 5

Please rate each of the following statements for how it describes you. Use a 3-point scale
in which:

1 = True. 2 - False. 3 = Don’t Know.

Choose a number for each statement.

123

88) _____The instruction in Socratic Questioning has extended my knowledge of critical

thinking.
89) _____The instruction in argument mapping has extended my knowledge of critical
thinking.

140
90) _____The instruction in Richard Paul’s Thinker’s Guides has extended my
knowledge of critical thinking.
91) _____I believe that my knowledge, skills, and dispositions have improved through
direct instruction in critical thinking.
92) _____The critical thinking process that I have learned in English 104 has helped me
to make better and more informed decisions in my everyday life.
93) _____I will be able to apply what I have learned in this course about thinking
critically to a practical situation in my life.

Please answer the following questions:

94) What has been the most frustrating or difficult aspect of learning to think more
critically?

95) Of the three strategies that you learned in Critical Thinking 104, which has been the
most effective in helping you become a better thinker?

Note. The questions in the “Dispositions” portion of the Post-Critical Thinking Survey are from Assessing a
Feasible Effective Critical Tthinking Protocol for Community Colleges (p. 227), by K.C. Metcalfe, 2007,
Ph.D. Dissertation, Capella University, United States – Minnesota. Adapted for use permission.
141
 APPENDIX C. REQUEST FOR PERMISSION TO DR. METCALF

March 31, 2010

Dr. Metcalfe,

I am a doctoral student at Capella University. For my dissertation I am currently

conducting a study concerning entitled Improving Student Critical Thinking and
Perceptions of Critical Thinking Through Direct Instruction in Rhetorical Analysis at
Victor Valley College in Apple Valley, California in a Critical Thinking 104 course. The
quantitative measure of the study is the California Critical Thinking Skills Test-2000, the
qualitative measure is a Pre- and Post- Critical Thinking Skills Survey, and both
assessment will be administered at the beginning and at the end of the semester. The Pre-
and Post- Critical Thinking Survey consists of three sections: “Knowledge,” “Skills, “and
“Dispositions.” For the “Dispositions” portion of the survey, I would like your
permission to use some of the questions from your Capella Dissertation, Assessing a
Feasible Effective Critical Thinking Protocol for Community Colletes, Appendix E. Need
for Cognition Scale, page 227, published in 2007. Specifically, I would like to use the
following questions:

I consider myself a thoughtful person.

I frequently think about ways to improve my thinking.
I would prefer complex to simple problems.
I like to have the responsibility of handling a situation that requires a lot of
thinking.
I would rather do something which challenges my thinking abilities.
I try to anticipate and avoid situations where there is a likely chance I will
have to think in-depth about something.
I find satisfaction in deliberating hard and for long hours.
I prefer to think about small, daily projects than long-term ones.
I prefer projects which require very little thought.
The idea of relying on thought to make my way to the top appeals to me.
I really enjoy a task that involves coming up with new solutions to problems.
Learning new ways to think doesn’t excite me very much.
The notion of thinking abstractly is frightening to me.
I would prefer a task that is intellectual, difficult, and important to one that is
somewhat important but does not require much thought.
I usually end up deliberating about issues even when they do not affect me
personally.

If you agree, I would appreciate an email indicating your permission.

Thank-you,

Lauren McGuire

142
 APPENDIX D. LETTER OF PERMISSION FROM DR. METCALF

March 31, 2010

Dear Lauren McGuire,

Yes, you are free to use whatever you need from my work. Be sure to contact Dr. Diane
Halpern, McKenna College Claremont University, (here in CA.), as much of my work
was built on hers. She will be delighted, as am I, that you are engaged in this type of
research and that you have found our work together useful.

Best of luck to you!

Kim C. Metcalfe
Assistant Professor of Early Childhood Studies & Psychology
Educational Psychology Ph.D.
Assistant Chair Health Human & Public Services

143