Research Matters - to the Science Teacher
No. 9003     March 1, 1990

 

Definition and Assessment of the Higher-Order Cognitive Skills
by Audrey B. Champagne, Professor of Science Education, State University of New York - Albany, Albany, NY

Imagine glass cylinders containing different quantities of sand. Students are supplied with three cylinders, one of which is completely filled with sand, and an inclined board. They are encouraged to observe the motion of the cylinders as they roll down the incline. Then students are shown a fourth cylinder and asked to predict how that cylinder will roll down the incline. How would your students respond to this exercise? Will they agree with your interpretations of the students' behavior? My experience suggests that you and your colleagues will probably not agree. However, this is an empirical question-if you doubt my premise, try the experiment. The exact manipulative exercise is not important, but having the students and some colleagues participate is crucial.

Observers interpret performance on tests of the higher-order cognitive skills differently because there are no agreed-upon operational definitions of this skills. Developing such definitions is difficult, because our understanding of the skills is limited. For example, we know very little about the relationships between the higher-order skills and the lower-order skills. Improved instruction and assessment of higher-order cognitive skills is contingent on developing operational definitions of those skills. This is the aim of this paper.

• Are the two levels distinguished by age? Do children, by definition, engage only in lower-order thinking and adults engage only in higher-order thinking?
• Are the two levels distinguished by the frequency with which we observe them in the population? Are higher-order skills by definition rarer than lower-order ones?
• Is content the distinguishing characteristic? Are higher-order skills only exhibited in the context of a formal discipline-mathematics, physics, economics-or can they be exhibited in practical situations such as automobile repair or dressmaking?
• Are definitions of higher-order skills idiosyncratic-that is, does an observer categorize a skill as higher order if it produces a solution to a problem that the observer could not do?

If you accept the assertion that operationally defining the higher-order thinking skills is difficult but potentially useful; then, you might try collecting and categorizing cognitive skills like naturalists collect and categorize plant and animal specimens. You can collect samples from practitioner journals, research journals in science education and the cognitive sciences, teachers' manuals for science textbooks, curriculum guides, and technical manuals for standardized science tests. When a number of skills have been identified, it is possible to build a taxonomy for the skills.

I found such a collection and categorization activity useful in illuminating some of the uncertainties in our understanding of the higher-order cognitive skills. The ordered list of skills that I collected is displayed in Figure 1. The list is quite long, but undoubtedly incomplete. You will have little difficulty collecting new skills and adding them to my list.

Higher-Order Cognitive Skills
Lower-Order (Algorithmic)
Generic
Metacognitive
Assess Understanding
Assess Validity of Generalizations
Test Facts Against Rules of Evidence
Reasoning
Logical
Inductive
Deductive
Analogical
Creative
Verbal
Spatial
Qualitative
Quantitative
Discipline (Content) Specific
Task Specific
Problem Solving
Patterns of Knowledge
Generic Problem Schemata
Rate
Limit
Proportion/Ratio
Discipline
Formulas
Algorithms
Facts
Rules
Procedural Skills
Heuristics (Strategic Knowledge)
Evaluate Progress
Constraint Satisfaction
Progressive Refinement
Means-ends Analysis
Setting Goals
Monitoring Progress
Making and Adapting Plans
Problem Decomposition
Problem Decontextualization
Elaboration
Reasoning (Rule Based Information Processing)
Inquiry
Generic
Discipline Specific
Scientific
Discipline Knowledge
Procedural Knowledge (Conducting an Inquiry)
Planning and Implementing an Investigation
Problem Definition
Hypothesis Generation
Apparatus Selection
Observation
Data Management and Analysis
Identify Patterns
Graphing
Extrapolation
Generalization
Modeling
Mathematical Learning

The first sort in my taxonomy produces two categories of cognitive skills, higher-order and lower-order. Leaving aside for the moment which skills belong in which category, and which criteria influence this decision, the relationship between the higher- and lower-order skills is an issue central to the practice of science teaching and to theory in psychology. Are the higher-order skills simple concatenations of lower-order ones or are the two kinds of skills qualitatively different? Your answer to this question will profoundly effect the way you teach these skills. If you believe that higher-order skills are concatenations of the lower ones, your teaching strategy will probably be based on analysis of higher-order skills that breaks into simpler skills. Each of the simpler skills is then taught. The assumption underlying this strategy is that when the individual skills are all learned, the higher-order ones will be also. This basic idea of building skills from the bottom up pervades our educational practice and was the theoretical basis for the instructional model used in Science: A Process Approach (SAPA).

If, on the other hand, you believe that higher-order cognitive skills are qualitatively different from concatenations of lower-order ones, your instruction will probably be more like that used in the Elementary Science Study (ESS) materials. The assumption underlying the instructional model used in the ESS materials, is that the higher-order skills develop as the result of understanding related phenomenon.

My taxonomy proceeds to sort the higher-order skills into three categories: problem solving, learning, and inquiry (See Figure 2). There are other ways in which they can be sorted, such as according to the kind of cognitive task, they might have been sorted according to the discipline or context in which the skills are demonstrated. The result of a discipline-based sort is illustrated in Figure 3. These sorts produce different categories and illuminate another important issue that has both practical and theoretical importance: Are the higher-order skills task or discipline specific?

higher-order skills

math
physics
problem solving chemistry
economics
(etc)

math
physics
higher-order learning chemistry
skills economics
(etc)
math

math
physics
inquiry chemistry
economics
(etc)

learning

chemistry problem solving
inquiry

learning
higher-order physics problem solving
skills inquiry

learning
economics problem solving
inquiry

This question is the subject of major research efforts in cognitive psychology. The goal of this research is to understand the relationship between knowledge about a discipline and performance of problem solving, learning, and inquiry skills in the context of the discipline. The current research in cognitive science implicates discipline knowledge in successful performance of most higher-order thinking skills. Studies of experts' problem solving confirms the importance of discipline knowledge and illuminates important differences in the discipline knowledge "of experts and novices". Experts' knowledge is highly structured in ways that facilitate problem solution. This suggests that when students practice the solution of problems, they are not only learning how to solve the problems. They are also restructuring knowledge of the discipline in ways that will facilitate future problem solution. Problem solution serves as both a method for learning (structuring) content and a way of demonstrating understanding of the content. Another significant characteristic of experts' knowledge is that much of it is automatic. The simple formulas, algorithms, facts, and rules of the discipline are recalled and applied so rapidly that most experts do not even mention that they are using them unless specifically asked. This suggests that some memorization is in order if students are to act like experts. They just can't do problems without basic information.

Experts have another kind of knowledge that facilitates problem solution. This is knowledge about problem schemata. Upon reading a textbook problem, an expert will often identify it as a momentum problem, or a kinetic energy problem, or a mixture problem, or a rate problem. Experts categorize the problem by comparing it with problem schemata stored in their memories. No matter what the physical context described in the particular problem type in her repertoire of problem schemata. Having categorized the problem the expert also has available a method for solving the problem.

With all of this talk about the importance of knowledge, you might get the idea that there are no generic problem skills. This is not the case. Expert problem solvers also exhibit procedural skills that facilitate problem solving. Some of those skills are listed in Figure 1. Some investigators in the field call these procedural skills heuristics. Others call them strategic knowledge. The use of these different terms further confuses the distinction between knowledge and cognitive skills.

The many theoretical issues surrounding the relationship between knowledge and cognitive skills are by no means resolved. In addition, the theoretical issues have corresponding instructional ones. Do the higher-order skills transfer? In practice, that is, in teaching and testing, the skills are generally treated as if they are discipline specific. However, educators and laymen alike are often talk about the skills as if they are generic. If students learn to think scientifically in science class, will they think scientifically in context outside science class? Popular belief aside, the mounting evidence is that students leave their scientific knowledge and thinking skills in science class and use other knowledge and skills in their encounters with the real world.

Renewed emphasis on teaching the higher-order skills requires that science teachers reconsider some basic questions about the relationship between cognitive skills and discipline learning. Are these skills best taught/learned in the context of a discipline, or in separate skills-development courses that focus on rational thinking, problem solving, inquiry, and critical or creative thinking? Do the higher-order cognitive skills transfer? If so, under what conditions? Are the sciences particularly good disciplines for learning the higher-order skills? If so, how should instruction be modified to produce better results? The answers to these questions are not at all clear.

Most of the work in cognitive psychology suggests that use of the higher-order cognitive skills is closely linked with discipline specific knowledge. This conclusion is based primarily on research on problem-solving and learning-to-learn skills. Consequently, the conclusion is limited to these specific higher-order thinking skills. The findings may be quite different for higher-order skills such as metacognition, and logical, analogical, inductive or deductive reasoning.

All of this comes back to the original point: As a community of educational practitioners and researchers, we need to be more scientific in our approach to teaching and assessing the higher-order cognitive skills. Common operational definitions for these skills are a necessary condition for more scientific teaching and research to attain the all-important goal of science education: scientific thinking in students.

Segal, J. W., Chipman, S. F., & Glaser, R. (Eds.) (1985). Thinking and Learning Skills. Volume 1: Relating Instruction to Research. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

deBono, E. (1976). Teaching Thinking. London: Temple Smith.

Lockhead, J. & Clement, J. (eds.) (1979). Cognitive Process Instruction. Philadelphia: Franklin Institute Press, 1979.

Linn, M. C., Pulos, S., Gans, A. (1981). Correlates of formal reasoning: Content and problem effect, Journal of Research in Science Teaching, 18, 435-447.