Text of NSF course-development project (J. Jadrich, PI)

A COURSE IN SCIENTIFIC ANALYSIS

FOR PROSPECTIVE ELEMENTARY SCHOOL TEACHERS

 

PROJECT NARRATIVE

OVERVIEW

In concordance with the recommendation put forth by the National Science Teachers Association [NSTA] (NSTA, 1992), many science courses have been developed specifically for pre-service elementary school teachers. An essential requirement of such courses is that they be inquiry based. This condition helps to assure that the pedagogies appropriate to teaching at the elementary school level will be modeled to prospective teachers. It is anticipated that if prospective teachers are exposed to methods of scientific inquiry, then they, too, will be able to engage in scientific inquiry with their future students.

Now that the methods and outcomes of teaching through scientific inquiry have been studied for several years, a consensus is emerging. Students apparently do learn scientific content and concepts better through the inquiry approach than they do through traditional methods. (Tobin et al., 1994) They also develop a more favorable impression of science and a confidence in their own ability to teach it. Both of these results are important in the development of good science teachers.

However, it has also been found that most students do not develop the ability to actually perform systematic scientific inquiry from having experienced the process. (Millar, 1997) From a scientist’s perspective, these students still do not know how to analyze a system scientifically. By this it is meant that they are not proficient at recognizing the variables pertinent to the behavior of a system, nor are they adept at recognizing if a system is open or closed and if, therefore, the variables might be changing. They have difficulty setting up controlled experiments and discerning which variables are impacting the system in significant ways. They have trouble distinguishing between fair and unfair testing, and they typically do not see the need for repetition when measuring, for averaging results, and for considering experimental uncertainties. Their problem solving abilities in science are also deficient, and they lack the metacognitive skills which would allow them to become more efficient in all the areas listed above.

In summary, students exposed to inquiry-based science do not automatically develop the ability to conduct their own scientific inquiries or analyses; skills which will be crucial to their success as science teachers. We believe that pre-service science teachers need explicit training in scientific analysis in order to become proficient in this area, and we are proposing to develop materials appropriate for this. While there have been many attempts in the past to teach science process skills to students, to the best of our knowledge no courses exist for pre-service teachers in any form which are devoted to teaching the integrated process skills needed for successful inquiry-based science and which have been designed in accordance with the research to date.

Guided by the vast amount of research in the areas of critical thinking, metacognition, and problem solving, we plan to develop units in scientific analysis for prospective elementary school teachers. Working with appropriate concepts and within familiar contexts, a modeling process will occur as the expert (instructor) models and builds a scaffold of cognitive skills which will enable the students to attain these same skills. The result will be pre-service teachers better able to teach science, because they will have a thorough grounding in the skills and techniques needed for scientific inquiry.

Students successfully completing the units produced will gain proficiency in the proper use of evidence in scientific investigations, including the formulation of hypotheses and the testing of inferences. They will be able to frame experimental questions and to conduct controlled experiments utilizing scientific concepts appropriate for study in elementary school. They will also develop scientific problem solving skills, which they may model to their own students and which they will utilize in their teaching as they create an environment of scientific inquiry in their own classrooms.

The units produced must be sequenced to follow after students have developed a reasonable grasp of the relevant scientific concepts to be utilized in their inquiry and analysis. (Inquiry-based courses which teach appropriate content already exist, and some are described during the course of this narrative.) Given this sequencing constraint, and given the diversity of science content courses which students will be exposed to at other institutions, the units proposed here will be made general enough in nature so that they may be utilized at any institution which provides a relevant science course for elementary education students. The developed units may stand alone as a complete course which is sequenced to come after an appropriate content course, or the individual units may be infused into existing courses, provided the necessary content is covered first.

Science content courses for prospective teachers should accomplish several things. From these courses students should gain a firm foundation of the concepts and theories which they themselves will teach one day. But foundational knowledge alone is not enough. Students should also gain an understanding of the processes of science so that they can both teach these processes to their future students, and teach science to their students through an inquiry, hands-on approach. As was mentioned previously, there is considerable evidence that teaching prospective teachers via inquiry-based science courses helps these students to learn science content more thoroughly than traditional approaches do. Constructivist learning models predict as much, since the process of inquiry into concepts should help students construct understanding about the concepts.

It does not follow, however, that simply putting students through the paces of scientific inquiry will help them learn how to engage in scientific inquiry themselves. It seems that we hope that they will simply "catch on" to how they should do this. However, research clearly shows that students do not just "catch on" to the processes of science. (Millar, 1997) They need explicit instruction in them. An extensive review of the literature available in this regard reveals the following ideas, which serve as premises for our proposed course.

The above four points can serve as general guidelines for the structure of a course in scientific analysis. However, they are not complete in that they do not indicate how analysis skills should be taught. Research into this question is far from complete. However, several programs and curriculum projects in the past have attempted to teach some subset of the analytical skills described in this narrative. These attempts have been directed toward both children and prospective teachers. Many of the more prominent ones are described below along with an analysis of their outcomes.

More and more courses are being offered through national distributors that are based on scientific inquiry and which are directed towards prospective elementary school teachers. (See, for example, AAPT, 1996 and McDermott, 1996). As mentioned previously, we have created and implemented such a course at Calvin College which will be described in more detail later. (Jadrich and Haan, 1998) All of these courses are similar in that they teach science content through inquiry without explicitly teaching scientific inquiry or analysis. As was already discussed, students taking these courses do not automatically pick up analytical skills from the experience. More explicit teaching is necessary. Therefore, while these courses are useful and even necessary for the pre-service training of elementary school teachers, they are not sufficient if we wish to teach scientific analysis skills.

Science - A Process Approach (S-APA) (AAAS, 1967) represented an attempt to teach the scientific process to elementary level children through explicit instruction and practice with the basic science process skills (observation, classification, measurement, etc.). In a sense, this was an attempt to teach general analytical skills outside of the context or domain of a particular subject (science). As stated in the previous section, the transfer of these skills has been found to be very limited. The meta-analysis of Shymansky et al., (Shymansky, 1983) showed that teaching the process skills in isolation in S-APA did little to affect the students’ ability to carry out open-ended scientific inquiry.

Some researchers have attempted first to break down the steps of problem solving as exhibited by expert problem solvers and then to teach these steps as algorithms to students. (See, for example, Halloun and Hestenes, 1987 and Van Heuvelen, 1991) Common to these approaches is a very narrow, perhaps incomplete definition, of a scientific problem. What these approaches do (and seem to do very well) is teach students how to solve the kind of exercises found at the end of the chapter of a typical physics or chemistry book. However, educational psychologists are more apt to refer to these exercises as puzzles rather than scientific problems. These exercises tend to have unique solutions; they are solvable through the application (without understanding) of an algorithm; and they lack real-world authenticity.

Therefore, these approaches better serve science majors needing specialized skills within a specific discipline of science than they do prospective elementary school teachers not having or needing such a disciplinary focus. However, the metacognitive skills fostered by the approaches described can be important in the course proposed here. Some of these will be utilized, and a description of them is given in the section entitled Course Description.

Curricula coming under the heading of Science, Technology, and Society (STS) attempt to teach problem solving within a real-world context. (National Science Teachers Association, 1992-93) Students analyze real life issues (such as human population densities, global warming, etc.) which are naturally interdisciplinary within science and which have technological and societal overtones as well. (Campbell et al., 1997) Students typically spend weeks or even months on a single issue due to the depth and complexity of the topic. Students finishing a course utilizing a STS curriculum are found to be much more confident problem solvers than students passing through a traditional course. (Yager, R.E., 1988) As was pointed out in the previous section, confidence in problem solving is a necessary ingredient to insure that students will be good problem solvers. However, research has not shown that such students actually are better problem solvers. Also at issue is the length of time a typical STS investigation takes. Very few problems can be addressed in this manner when a single one can take weeks to bring to closure. Finally, since STS investigations address issues which transcend science, there may not be enough time to focus on scientific phenomena as entities themselves. We suggest, therefore, that smaller, more closed-ended investigations need to be addressed initially and primarily to insure that students meet with success before they move onto the large scale investigations found in STS curricula.

Current NSTA recommendations suggest spending about 15% of science class time on STS issues. This recommendation, in conjunction with the open-ended nature of STS investigations, suggests that one might investigate an STS issue as a final project in the course to be described here. Another possibility would be to pursue a Science and Technology (ST) issue as described below.

Science and Technology (ST) curricula (See, for example, SAE, 1990) are similar to STS investigations, but they differ in that they primarily emphasize technological problem solving as opposed to consideration of the societal impact of issues. As such, they are more focused on science, but still very open-ended in the types of investigations which may be attempted. As was the case for STS issues described above, it would be reasonable to consider performing a ST investigation near the end of the course proposed here.

In summary, we note that there have been many attempts to teach portions of what we refer to here as scientific analysis. However, these attempts have not always been based on the research findings available in regard to how this should be done, and for the most part, the curriculum designs have not had the interests of a prospective elementary school teacher in mind. Below, we describe the outline of a course which takes these two factors into account. Some aspects of the programs described above can be modified to fit the needs of the course described below. In addition to the programs referenced above, other research findings have indicated methodologies that seem helpful in teaching skills associated with scientific analysis. These will be incorporated into the course design as well.

The course units we are proposing will give prospective elementary school teachers a firm and wide grounding in the process of scientific analysis. Attention will be focused primarily in the areas of I) Developing Evidence in Scientific Investigations, II) Framing Scientific Questions and Conducting Experiments, and III) Scientific Problem Solving. Although these three areas are not independent of each other, they will be treated separately and sequentially in order to consolidate the skills and processes which need be taught. The scientific content and concepts providing the context for these skills and processes will be drawn almost exclusively from those which are appropriate for investigation in an elementary science classroom. This will achieve four things: 1) Students will be more likely to take ownership of the processes, and therefore they will be more likely to exert the cognitive effort they will need in order to meet with success in the course, since they will have a vested interest in learning the material. 2) Since the content domain will be familiar to the students, they will be able to devote more of their short term memory and working memory to the difficult cognitive tasks associated with these three areas. 3) Students will have an easy time adapting materials from the course to their own science teaching after they graduate. This is an added bonus, and it also feeds back into #1 above. And, 4) since the national standards and benchmarks effectively prescribe the level and breadth of science content to be experienced in elementary schools nation-wide, basing our scientific analysis units on this content base assures that institutions around the country will be able to easily adopt the units to be developed here.

Before giving a detailed description of the content and methodologies to be incorporated into the units, we need to provide an overview of the objectives within each unit of the course. Within Developing Evidence in Scientific Investigations, students will learn to:

Within the area of Framing Scientific Questions and Conducting Experiments, students will learn to:

Within the area of Scientific Problem Solving, students will learn to:

I) The area of Developing Evidence in Scientific Investigations comes first in the unit sequence, because within this area students are required to keep track of the fewest number of variables (usually one or two), and thus they will be able to devote a greater amount of their cognitive resources to the tasks at hand. (Roth, 1990) At the same time, the skills and processes they learn in this section will act as a foundation to the other areas of study. Within the area described here, students will primarily focus their attention on the observation, inference, prediction cycle utilized in the scientific process. [Note: students will not be led to believe that scientists routinely investigate phenomena using such a simple cycle, since that has been to shown to be overly naive. (Finlay, 1982 and Millar and Driver, 1987) Instead, students will view this cycle as a summary of scientific "proofing".] Sub-skills to be learned include differentiating between inferences and predictions, and learning to assess the quality of evidence needed to warrant an inference.

Demonstrations and activities in the form of discrepant events will be used often when teaching in this area. (Friedl, 1991) Examples of these would be seeing water flow uphill or discovering that a cotton ball cannot be blown into what appears to be an empty bottle. Discrepant events have the tendency to attract students’ attention and cause them to look for explanations. This is the environment needed to begin teaching about the differences between an observation and an inference, and the further need to support inferences (i.e. the explanations furnished to explain the discrepant event) based on sound evidence and testing through the cycle described above. Several groups have utilized discrepant events in ways similar to what is described above. (Palmer, 1965 and Thompson, 1989)

Students will learn to sharpen their analysis of scientific evidence by posing counter examples of inferences previously submitted. (Case and Fry, 1973) Here students are asked to come up with as many possible explanations of a phenomenon as they can, and at the same time provide evidence which refutes these inferences. For example, students might observe that if a jar is placed over a burning candle placed in the center of a dish of water, the water level in the jar will go up when the candle burns out. After making this observation, students must provide a variety of explanations for this phenomena, such as; oxygen in the jar got used up, or the heat from the candle pulled up the water. For every explanation offered, students will have to design another activity, or make a logical argument from what has already been observed, that proves that explanation cannot be valid (assuming it is not). Students will also be asked to design demonstrations and give explanations such that no counter examples to the explanations are possible. Case and Fry have shown that, in both of these scenarios, students become much more adept at the use of scientific evidence.

Within this first unit, students will also focus on the role of prediction in science. They will learn the differences between guessing (one extreme), purely logical deductions (the other extreme), and the more common form of actual scientific prediction which utilizes deduction but is not based on pure logical certainties. (MacDonald, 1993) For example, students shown two closed but seemingly identical containers can only guess at which container might weigh more, because they do not know the contents of either. If students are shown that one container is filled with lead and that the other container is filled with feathers, then it is nearly a logical certainty that the leaded one will weigh more than the other one. However, when students are asked to predict which container will fall faster if they are dropped to the ground, they must now make a prediction which is neither a logical certainty nor a simple guess. Instead, their prediction must be based on an application of their inferences concerning the interactions among gravity, weight, inertia, and any other relevant variables they might imagine. Their observation of the actual falling event will then be used to modify their appropriate schema in ways that cannot occur with the extreme forms of predicting, but which do occur in the course of science. Through these types of activities, students will increase their ability to utilize prediction in scientific endeavors.

Finally, students can be expected to master the skills in this area (and in the other two areas of study) only if they have extensive opportunities for practice. Therefore, students must be provided with many activities and phenomena to further their mastery of these skills. The student and instructor manuals written for the course described at the end of this section will catalogue these activities. Activities for this area will include many multivariate systems where the instructor controls all but one of the variables (since the students would not yet have this skill) and the students must infer relationships based on their observations and also make predictions about what is to be expected. These investigations could include such activities as determining meal worm responses to light and moisture, the population of frogs to be expected in a pond, and the contents of unknown powders based on their reactions to various liquids.

II) We will begin area II (Framing Scientific Questions and Conducting Experiments) by first teaching the students how to recognize the variables inherent in a system and how to vary these in a controlled way. These skills are at the heart of scientific experimentation, and they are necessary ingredients in the process of framing new scientific questions and experiments. Unfortunately, these skills are difficult to learn, and students must attain a level of formal thought in order to be successful at this. (Karplus et al., 1979 and Wollman, 1977) Research has shown that students become proficient at these only if they are adequately prepared and have ample opportunity for practice. (Bailey and Millar, 1996) Thus, we propose a deliberate strategy to be followed in order to help the students gain mastery in this area.

The learning process will begin by exposing the students to many different but simple systems that are governed by only a few variables. (For example, the rate at which a pendulum swings, the rate at which a mass on a spring oscillates, the distance water will squirt from a hole in a bottle, etc.) It is crucial that only simple systems are explored initially so that the students’ working memories is not overly taxed as they attempt to recognize variables and decide on their relevancy to the phenomenon studied. (Ross and Maynes, 1983 and Staver, 1986)

The instructor will model the process of determining what variables are present and which of these variables are most relevant. This will not be accomplished through polished lectures (which is the way most scientists present this type of analysis). Instead, instructors for the course will utilize "talk aloud" strategies. (Whimbey and Lochhead, 1991 and Ronning and McCurdy, 1985) In this method the expert (instructor) speaks all the thoughts which pass through her mind as she analyzes a system for the first time. This method has been shown to be extremely effective in modeling these processes to novices. Not only do the students observe the care with which the expert analyzes, but they also see that experts make many false starts, and they discover that even experts rely on cognitive "crutches" to get through the process. For example, an instructor will reveal that he counts on his fingers, jots down simple notes to remind himself of things, and that he is constantly talking to himself. This modeling process frees up the novices to do the same sorts of things that experts do themselves.

Of course, after a given instructor has analyzed the same system several times, the "talk aloud" methodology becomes less authentic. It is very difficult, if not impossible, to pretend that you are thinking through a situation for the first time when you are really not. And, when this occurs, the analysis process is not properly modeled to the students. To guard against this, we will make video tapes of scientists (often future instructors of the course) who have practiced the "talk aloud" methodology and who are analyzing systems for the first time. This will insure that students always observe authentic analysis events, and it will also give the class the opportunity to revisit the process if it is so desired. Copies of these videos will be made available to any institution wishing to implement the course.

We will also introduce the method of "hunching" to the students at this time. (Wilson and Koran, 1976) "Hunching" occurs when an individual explicitly writes down the variable he thinks may be most important in determining the outcome of an event. This hunch is made explicit before the individual begins to systematically check for the relevancy of the variables inherent within a system, and it has been shown that this initial guesswork has a significant impact on the ability of a student to control and to test the variables in a system. This "hunching" process has been shown to be commonly used by experts (although they often do so unconsciously), and therefore it also models exemplary behavior for the students. After students write down their hunches, they are more careful in controlling the variables involved, and they will be less likely to accept small variations in data as if they were significant.

As an example of the "hunching" process, students may be presented with a written activity (adapted from a published source) which states that one can prove that carbon dioxide is a greenhouse gas by observing how much more a sealed aquarium filled with carbon dioxide will heat up than an identical aquarium sealed and filled with normal air. (Both aquaria are exposed to a bright light source.) Before testing to see if this is indeed the case, the students would be asked to make their best guess as to which variable(s) might be most important in determining how much either of the two aquaria will actually heat up. (Students are likely to consider variables such as the amount of carbon dioxide present, the intensity of the light, the volume of the aquaria, or a host of other variables present within the systems.) It is during this analysis and commitment stage that students become aware of the multitude of variables which might affect their results, and they become better attuned to how these variables affect the results of their investigations.

Once students become adept at recognizing and controlling variables, they will move on to conducting complete experiments. An important aspect of this is learning to frame well posed experimental questions. To practice this, students will learn to change naively posed questions into ones that can be investigated rigorously. (Karlan, 1995) For example, the question "What is your body temperature?" might be transformed into "What is your core body temperature?" or "What is your body temperature first thing in the morning when you awake?" In these cases, the students are learning how to frame a scientific question such that the relevant variables are made explicit, and therefore they are less likely to be overlooked. The question, "How long would the air in the classroom last if all the windows and doors were suddenly sealed?" could be changed to, "How long would it take before half the people in this room passed out, assuming the room were completely sealed and everyone remained sitting quietly in their seats?" Framing good experimental questions not only encourages students to carefully consider the variables inherent in a system, it will also be a necessary skill for designing inquiry-based science lessons once they are teachers.

Many of the questions the students have framed can eventually be investigated in full-fledged experiments. For this to happen, the students must come to grips with their need to make estimations when actual measurements cannot be made. For example, in the question concerning breathing in a sealed room, students will have to estimate a variety of things, such as the exact size of the room, and the number of times a volume of air can be re-breathed before it lacks enough oxygen. Being willing to estimate measurements is as difficult for students to do as it is for them to recognize that all measurements contain uncertainties. Therefore, students will receive explicit exercises that give them practice in making reasonable estimations, and additional exercises will also be given which will make explicit to them the need to recognize that all measurements and data contain errors and uncertainties.

To solidify the skills to be developed in this area, students will be given ample opportunity to frame additional scientific questions for investigation, and they will be allowed to conduct as many experiments as time permits. The student and instructor manuals for the course will list enough possibilities to allow flexibility among different instructors and institutions conducting the course. Included in the list of investigations would be ones such as "What variables affect the rate at which a cart will roll down a hill?", "How does the strength of the leaves on a tree vary as a function of their position on the tree?", "Which freezes faster, cold water or hot water?", and "Which brand of paper towel is ‘best’?"

III) The area of Scientific Problem Solving will follow area II because students must be able to recognize and manipulate variables in order to do problem solving at this level. Many of the techniques utilized in area II will again be applied, including the "talk aloud" strategies and the "hunching" method described earlier. Students will begin by investigating very simple problems so that they learn problem solving techniques and gain confidence in their problem solving abilities. Group work is essential at this point so that students benefit from peer resources and do not have to carry the entire cognitive load. Some simple problems, which will foster confidence (without being trivial), and which can be used to learn problem-solving techniques include things such as operationally proving one object is more dense than other, trying to pour air from one cup into another cup under water, and proving that removal of two white spaces from a checker board prohibits one from completely covering the board with dominoes which are large enough to exactly cover two spaces on the board.

Gradually, more difficult problems will be introduced as students gain in confidence in their ability to use problem-solving strategies effectively. Some of these problems will ask students to maximize an output (such as, "Starting with this basic formula for making soap bubbles, find the right proportion of ingredients to maximize the life of the bubbles blown."), but many of these problems will come in a form where the students are asked to make a demonstration or activity found in an elementary science textbook actually work. This experience will dramatically demonstrate the need for the teachers to have problem-solving skills if they wish to engage in scientific inquiry with their students in the future.

The final unit of the course will allow students to carry out investigations of their own using "hi-tech" equipment from the various science departments at the college. The purpose here is to introduce a technology component to the course while still maintaining the emphasis on performing investigations in areas which are appropriate to elementary school. Our goal is to allow students to experience the power and excitement that hi-tech equipment can afford in scientific investigations, and it is our hope that this excitement can carry over to when these students are mentoring their own students. These activities can take on the form of an STS investigation. For example, students may use a thermal desorption - gas chromatography - mass spectrometry unit in order to study air quality around campus. For this investigation, the students could determine the variation of air quality in their environment in a short amount of time and correlate this result with mechanical or biological sources in the vicinity. Students may use an ion chromatography unit in order to study water quality and look for trace quantities of lead or phosphates. Another possibility is to have students study the power consumption of various appliances by plugging them into a power meter which automatically integrates over time and displays the power usage for each item. Of course, any school wishing to implement the course described here will have its own unique set of equipment which might be used in an endeavor such as this. The point is to allow the students to do "real life" investigations with modern equipment while still limiting the investigations to topic areas that are relevant to the prospective teachers.

Part of the development of these units will include the creation of student and instructor materials for use at Calvin College and for dissemination to other institutions. It is important that flexibility be built into these materials so that a variety of instructors and institutions may find them useful. Therefore, although the methodologies used in teaching the skills and processes relevant to the course are not expected to vary much from one instructor to the next and must therefore be fairly prescribed, the actual activities and experiments performed may vary considerably depending on the desires of the instructors and the resources available. Therefore, the student manual will consist of worksheets and detailed descriptions of many standard activities which are crucial to the development of the course or are easily accomplished no matter what institution adopts the course, but it will also have numerous activities described with some detail which can be attempted depending on the desires of the instructor. In the same way, the instructor manual will consist of all the same elements as the student version, and it will also contain detailed notes to help the instructor set up any of the investigations listed in the manual.

Course materials will also consist of VHS format videos of experts describing their thought processes during "talk aloud" investigations. These will reflect activities within the experimental and problem solving sections of the course. These videos may be adapted to CD or laser disc format in the future, depending on the desires of future publishers.

The personnel associated with the project are divided into three groups: senior personnel, the Science Advisory Board, and the Evaluation Board. The senior personnel will have primary responsibility for the development, implementation, and dissemination of the course. The Science Advisory Board will provide key input in regard to the science content and processes inherent in the course, and the Evaluation Board will be primarily responsible for advising the senior personnel in developing analysis skills in students and in developing evaluation instruments to test student outcomes.

The PI for this project (J. Jadrich) is an Associate Professor of Science Education and Physics at Calvin College. He has a Ph.D. in physics, but he also has over ten-years experience in science education. He has been involved in numerous large and small scale science curriculum development projects at every level, from elementary school through university, and in every capacity including design, implementation, and evaluation. (See biographical sketch for complete details.) Several of these curriculum projects were NSF supported. The PI also has extensive experience in elementary education teacher development, observation, and evaluation for both pre-service and in-service teachers.

S. Haan (co-PI) is Professor of Physics and Chair of the Physics Department at Calvin College. In addition to a well established research program in theoretical atomic physics, Haan has spent more than twelve years working in the area of elementary science education, including teaching methods and content courses for prospective teachers, performing classroom observations and evaluations of student teachers, presenting workshops at local elementary schools and at conferences, and serving as a consultant for elementary science curriculum development.

Together, Jadrich and Haan have developed a physical and earth science content course for prospective elementary school teachers. The course is novel in its thematic approach as it covers the physical and earth science content deemed appropriate by the national standards in an inquiry and interdisciplinary manner. The course has been extremely well received at Calvin and is receiving national exposure through presentations at state and national conferences, through dissemination to individuals at other institutions (e.g. Northern Kentucky University and University of Michigan at Dearborn), and via an article in The Journal of College Science Teaching. (Haan and Jadrich, In Press) Other articles have and will be submitted to additional journals [e.g. The Physics Teacher (Jadrich and Haan, In Press)] as examples of activities and methodologies utilized within the course. A publisher for the course materials is currently being sought.

An Evaluation Board consisting of C. Joldersma (Education Department, Calvin College) and S. VanderStoep (Psychology Department, Calvin College) has been created to assist in the evaluation of the project and to lend expertise in the areas of critical thinking, problem solving, and cognition. Both board members have broad experience and expertise in areas of analysis, critical thinking, cognition, and the philosophy of science, and they will consult with the principal investigators in the areas of course development which overlap with these fields. The primary responsibility of the Evaluation Board will be to develop an evaluation tool which will measure student outcomes in the area of scientific analysis. This tool will generate the main portion of data to be used in evaluating the course and in reporting results in research journals such as The Journal of Research in Science Teaching. In order to insure that the evaluation data have integrity, neither of these board members will have a direct hand in the writing of course materials or in the teaching of the course. Members of the Evaluation Board have been designated as senior personnel for the project.

A Science Advisory Board has been created consisting of a chemist (L. Louters), a geologist (R. Stearly), and a biologist (S. Stegink). These individuals were selected because their scientific expertise spans the disciplines covered by the course (physics is represented by the co-principal investigators), and because each of them have experience in science education at the pre-college level. Summaries of their c.v.’s can be found in Appendix A.

The members of the Science Advisory Board will fulfill three mandates: 1) They shall insure that the science presented in the course is accurate within their domain of expertise, 2) as active scientists, they shall insure that the scientific processes presented in the course are authentic, and 3) they shall be knowledgeable enough about the content and processes of the course that they will be able to teach the course after the development stage is complete. Members of the Science Advisory Board are not listed as senior personnel for the project.

A schematic of the groups involved with the project, along with the major duties associated with each group, is shown on the next page.

A continual process of evaluation will be utilized in order to insure that a high quality course is produced and that a meaningful analysis of student outcomes can be reported on in research journals. Formative and summative evaluation will be accomplished both through formal evaluation tools and via more informal methods.

A primary source for formative evaluation will be the daily journals students are required to keep. Following the approach used in other elementary science courses at Calvin College, students' daily journal entries must describe 1) what the students did in class that day, 2) what the students learned that day, and 3) any outstanding questions the students still have or any remarks regarding the course which the students wish to make. From the students’ perspective, these journal entries work to reinforce the day’s activities, and they encourage the students to be more reflective in their learning. As an evaluation tool, the journal entries provide the instructor with a wealth of information regarding student learning and response to a course.

For example, student attitudes about the course and about scientific processes in general can be gleaned from the journal entries. The instructor can also discern whether or not the students perceive the lesson objectives to be the same as the instructor’s objectives. The instructor can also quickly discern if the students’ learning summaries match the intended learning objectives for the course. Whenever mismatches occur in these or other areas, the instructor can consider whether or not changes to the course need to be made.

Students will also fill out general instructor and course evaluations at the end of each semester. These anonymous evaluation forms are regularly used at Calvin College to evaluate instructors, courses, and course materials, and they solicit evaluatory information which students may not ordinarily volunteer on their own.

Formative evaluation will also occur through formal and informal meetings among the principal investigators and the two advisory boards as described in the section entitled Personnel. These meetings will assure that the methods and content employed by the course meet current standards of practice in science, education, and cognitive psychology.

As part of the formal review of the course, the Evaluation Board will create a pre-course/post-course evaluation tool to measure student outcomes. This tool will evaluate how well the course succeeds in fulfilling the major learning objectives for the students, and an analysis of the resulting data will result in one or more research papers concerning the effectiveness of the course. Although all senior personnel will have input into the development of this evaluation tool, its final form and the primary analysis of the resulting data will be determined by the Evaluation Board in order to insure the integrity of the evaluation. The form of this tool will be primarily "pen and paper", as the number of students taking the course each year (approximately 90) will be too large to enable a more personalized test format. However, the tool will ask students to analyze a variety of physical systems and problems composed of physical objects (as opposed to a written format describing systems and problems) in order to insure that students are tested on "real life" phenomena that they are likely to encounter in their teaching. The final form of this tool will be customized to evaluate the effectiveness of this particular course, but in its development the Evaluation Board will make use of a variety of tools and tests previously developed by others which test over areas of scientific processes and analysis and which have been tested for content validity and reliability. Included in these resources would be TIPS (Dillshaw and Okey, 1980), TISP (Tobin and Capie, 1982) and others. (Lunetta et al., 1981 and Molitor and George, 1976)

Members of the Evaluation Board will also track a few students for case studies. These detailed studies will allow the Board to augment data obtained from the general evaluation tool for final analysis, and it will also provide for a feedback mechanism in the development of the course itself and in the development of the evaluation tool.

During Year 1 of the project, two institutions beyond Calvin will be identified and selected for beta testing of the course during the 2000-2001 school year. Identification of testing sites will be done by directly approaching the institutions known to Calvin which have already become familiar with Calvin’s science content course for prospective elementary school teachers, and/or by making general announcements via listserves devoted to individuals involved in science education research or teacher training.

Beta test sites will be selected based on their ability to implement the course as described in the proposal. In particular, institutions must already have in place a science content course for prospective elementary school teachers and a willingness to implement the evaluation instrument developed during Year 1. Each of these institutions will send a representative to Calvin during the summer of 2000 for training in the implementation of the course.

Data collected from the beta test sites will be used in the final evaluation of the course, both in terms of the overall efficacy of the components of the course as well as to measure the transportability of the course to other institutions.

The contents of this course, along with a variety of research findings, will be widely disseminated during the development and following the complete implementation of the course at Calvin College. In addition, all written (student and instructor manuals) and electronic materials (video tapes) produced for the course will be designed such that they can be easily adapted to any institution wishing to implement a similar course.

Progress associated with the development of the course, and then final descriptions and analyses of the course, will be presented at several professional meetings and conventions, including annual Regional Science and Math Update Seminars (State of Michigan) and Michigan Science Teachers Association meetings, and national meetings of the National Science Teachers Association, the American Association of Physics Teachers, and the American Educational Research Association.

Several written publications will follow after the implementation of the course. An article describing the course and its place in teacher training will be submitted to The Journal of College Science Teaching. Results detailing successes of the course in impacting student learning and skills in the area of scientific processes and problem solving will be submitted to The Journal of Research in Science Teaching as well as The Journal of Excellence in College Teaching. It is also anticipated that several "spin-off" articles to publications such as The Science Teacher, School Science and Mathematics, The Journal of Educational Psychology, The Journal of Chemical Education, The Physics Teacher, and the Journal of Geoscience Education will be submitted when certain activities from within the course are found to be especially useful and interesting to selected audiences.

We will also look for a commercial publisher for the materials produced in the course. Since student success with the course will depend on their having a good grounding in elementary-level science content, it is possible that this material would be packaged with the text previously written at Calvin: Physical and Earth Science for Elementary School Teachers. However, as previously mentioned, student success in the course will depend only on students having a good background in elementary level science, and not on their having taken any one particular course.

Finally, while the course proposed here is designed specifically for prospective elementary school teachers, we feel that its impact can be much broader than this. In-service elementary school teachers could also be well served by such a course as part of their continuing education, since most of them have never been exposed to a course, which actually teaches methods of inquiry. In addition, one can imagine using the foundation of this course as a general science literacy course for non-science majors. Thus, the audience for this course could be much greater than just prospective elementary school teachers.

Calvin College is on a 4-1-4 semester system. The Fall and Spring semesters are traditional 14-week semesters, and the January interim semester lasts 3.5 weeks and allows students the opportunity to take a single in-depth course. The course proposed here will normally be conducted during the Fall and Spring semesters, but one January interim semester will be utilized as a trial period to evaluate activities and methods to be used in the final course.

A single section of a preliminary version of the course will be trialed during this time. Many different activities, projects and methods will be explored and evaluated during this time in terms of their usefulness in the final version of the course. This preliminary course has already been officially scheduled, and enrollment is being limited to students who have already completed the elementary level physical and earth science content course at Calvin (Physical and Earth Science for Elementary School Teachers) which is described more fully in the section labeled Personnel. No NSF funds are requested for this preliminary course.

An inaugural version of the student text for the course will be completed during this time based on outcomes from the trial course of January 1999 and from additional research into teaching scientific analysis. This text will be used by students during the 1999-2000 school year. An outline of the final version of the instructor manual will also be completed during this time. Beta test sites for the 2000-2001 school year will be selected.

Funds are requested to support the PI at 2/9 time during this period, as he will do the bulk of the development in consultation with the other senior personnel and the two advisory boards.

A preliminary form of the course evaluation instrument will be completed by the two members of the Evaluation Board. This instrument will be tested out during the 1999-2000 school year, evaluated, and then revised during Summer 2000. Both members of the Evaluation Board will receive three weeks salary for this work.

The PI will teach a single section of the course during the Fall semester. This will mark the first time the course has been taught during a normal semester. The January interim semester will be used by the PI and the other senior person to evaluate the effectiveness of the course, and modifications will be made at that time. The PI is requesting a course reduction from the college so that all 3.5 weeks of the interim semester will be available for this work. The other senior person does not have teaching duties during this time due to his reduced teaching load as chair of the Physics Department.

The co-PI will teach a section of the course during the Spring 2000 semester. The evaluation tool developed by the Evaluation Board will again be utilized during this semester in order to evaluate student outcomes from the course and to provide the information needed to make final modifications and improvements to the evaluation tool. Initial inquiries into commercial publication for course materials will begin during this time. No NSF funds are requested for this entire time period.

Jadrich and Haan will work on producing complete versions of both the student and instructor materials for the course. It is expected that some modifications of these materials may still take place after this time, but the structure of the course, as well as the bulk of the activities to be integrated into the course, will be completed. Both individuals are requesting 2/9 salary support from NSF during this time due to the size of the writing project. The pre-service elementary school teacher will also work for ten weeks during this time.

Individuals from the beta test sites will be hosted on the Calvin campus for one week to familiarize the individuals associated with the project with the course. These individuals will receive a stipend and per diem during their stay.

The Evaluation Board will complete the final version of the evaluation tool for the project. This tool will be fully implemented during the next school year. Both members of the Evaluation Board will receive two weeks salary for this work.

The course will be fully implemented in four course sections per year at Calvin. Sections will be taught by senior personnel and by members of the Science Advisory Board. Data will be collected during this time for use in the overall student outcome and course evaluation. A commercial publisher for the course will sought. No NSF funds are requested for this period.

The final version of the course will be completed at this time. The PI and co-PI are both requesting one month salary to complete the project. A paper describing the course will be submitted to The Journal of College Science Teaching. The senior personnel and the Evaluation Board will begin analysis of the data collected from the evaluation instrument. If it is determined that a sufficient amount of data has been collected to draw definite conclusions concerning the course, preparation for a manuscript intended for The Journal of Research in Science Teaching will proceed. Otherwise, additional data will be collected during the next school year.

Internal funds will be utilized for training other Calvin professors to teach the course.

The course will remain in full implementation at Calvin College. The article for The Journal of Research in Science Teaching will be completed and submitted. It is expected that several "spin off" articles based on activities performed in the course or on student teacher outcomes will be submitted to various journals over this time. Dissemination of the finished course to other institutions will begin. No NSF funds are requested for this period.

REFERENCES

Link to PI's homepage.

NSF Project Information Summary.

Return to the Calvin College Physics and Astronomy Department home page.