Professional Development in Nevada:
The Traveling Science Boxes Program of the Desert Research Institute

by

John R. Cannon
University of Nevada, Reno

Introduction

Science education reform remains at the forefront of education policy debate within the United States of America. With the publication of the Third International Mathematics and Science Study (TIMSS, 1997) examining teaching and curriculum at grade 4, grade 8, and grade 12 levels, more attention has been focused on the quality of science instruction in America as compared to 41 other nations across the globe. TIMSS data analyses discloses that U.S. eighth graders science achievement scores are behind Japan, on par with England, Canada, and France, and above France. Overall, eighth graders from the U.S. score above the international average of all 41 TIMSS countries. Further analyses of TIMSS data will include grade 12 students.

While the grade 8 international comparisons are promising, especially in light of the declining science achievement during the previous decade in the U.S., the goal set by the President George Bush and the 50 state governors in 1989 of becoming "the first in the world in mathematics and science achievement" still remains unreachable. Many both in and out of the science education reform debate have offered suggestions for new programs designed to enhance student learning and teaching in science. Nevada's Desert Research Institute's Traveling Science Boxes (TSB) is such program.

As stated on the Desert Research Institute's World Wide Web site,

The participating teachers were from the states of Nevada, California, Oregon, and Idaho. The project meetings originated in Reno and Las Vegas, Nevada via Distance Education (live video). Off site video feeds also went to Elko and Winnemucca, Nevada. The initial two project meetings took place at Desert Research Institute (DRI) facilities in Reno and Las Vegas respectively on June 12 and 19, 1999. Teachers were exposed to current education reform issues and movements within the science education community.

The TSB program was modeled after the Traveling Study Kits program of the Burke Museum residing at the University of Washington. The museum's web site reports:

The web site also reports having kits ready for check-out to schools pertaining to the study of the following topics:

African Art
Age of Dinosaurs
Bats
Bills and Claws
Central America
Fish
Fossils

Geology of Mt. Rainier
Human Evolution
Indonesia
Insects*

Local Birds
Marine Invertibrates
Masks

Mexico
Native Peoples of Puget Sound
Native Peoples of the Northwest Coast
Native Peoples & the Environment of Washington
Native Peoples of Eastern Washington
Northwest Coast Native

Peopling of the Americas* Prehistoric Animals
Polynesia
Reptiles
Rocks and Minerals
Tropical Shells
Vietnam
Washington State Pioneer History
Whales
What is Archaeology?*

The National Science Education Standards (NSES, 1996) were used as one of the foundations and rationales for the curriculum and instruction included in the TSB. The NSES state:

The previous statements are of most importance to schools who typically have little, or no, science materials for student use. These schools are what the TSB were designed to service, offering contemporary and effective science instruction and the materials required for such experiences.

The recently adopted Nevada Science Content Standards (NSCS) were also reviewed. The NSCS formed the basis of science curriculum materials developed by the teachers over the summer. The project ended on August 7, 1999, totaling 10 weeks of curriculum development by the participating teachers to be included in the TSB.

Current education reform issues introduced to the teachers included presentations on constructivist teaching practices, the learning cycle model of teaching and planning, and alternative assessment. The constructivist teaching practices presented were based upon a constructivist epistemology.

Constructivism is defined as "an epistemology, a theory of knowledge used to explain how we know what we know" (Lorsbach & Tobin, 1992, p. 21). Lorsbach and Tobin maintain that:

The Handbook of Research on Science Teaching and Learning (Gabel, 1994), published by the National Science Teachers Association, refers to constructivism as a "framework...to make sense of teaching and learning" (p. 85). If constructivist teaching practices are noted as sound for elementary and secondary science courses, then it follows to reason that they may also be appropriate for professional development experiences with teachers, both elementary and secondary alike.

Constructivist epistemology has influenced learning in science and mathematics, and many other academic areas. The value of constructivist epistemology upon teaching practice has also been identified and highly praised in many educational reform reports and published papers. (American Association for the Advancement of Science [AAAS], 1993; Glynn, Yeany, & Britton, 1991; Pressley, Harris, & Marks, 1992).

The TSB teachers selected various content strands (e.g., physical, earth science, chemistry, etc.) from the NSCS and were assigned to produce a contemporary science instructional unit. The entirety of the units and teaching materials had to fit inside one suitcase similar in size to that of a piece of carry-on luggage allowed on commercial air carriers. Each participant was given one piece of luggage. The reason for the luggage was to enable DRI to ship the completed Traveling Science Boxes across the state of Nevada to primarily rural school districts where little, if any, science teaching support materials existed for use in school classrooms. The final TSB meeting featured the science units produced by the teachers. Each teacher was asked to briefly present an overview of a sample lesson from the unit and the corresponding hands-on materials included in the Traveling Science Box.

Finally, it should be noted that the National Science Education Standards also address the requirement of school districts in supporting an activity-based, hands-on science program. Program Standard D (NSES) states:

The purpose of this report is to assess the efficacy of TSB model based upon both quantitative and qualitative data analyses.

Method

Subjects

Thirty-six teachers were included in the project during 1999, teaching Kindergarten through high school level science classes. Nine of the teachers were male; 27 were female. Of the total 36 participants, 2 were preservice elementary teachers currently enrolled in teacher preparation programs at the University of Nevada, Reno and the University of Nevada, Las Vegas. One teacher was also selected from the states of Arizona, California, Oregon, Utah, and Idaho. These teachers were included to serve as potential project leaders for future replication of the TSB model outside of Nevada.

Materials

A review of the literature relating to similar projects unveiled no similar research endeavors. Many papers have been written promoting activity-based, hands-on science programs for public school children, however, their is a lack of research based upon the efficacy of innovations that promote curriculum development by teachers and how this activity may influence the teachers' beliefs about science in general and school science in particular.

Therefore, based upon the review of the literature, The Beliefs About Science and School Science Questionnaire (BASSSQ) (Chen, Taylor, & Aldridge, 1997) was chosen as the quantitative instrument for this study. This instrument was of particular significance due to the nature of its ability to measure teachers' and students' beliefs about the nature of science and school science in light of many of the reforms called for in science education reform, namely science taught in line with more of a constructivist epistemology. This active mode of learning is in direct contrast to that of the traditional transfer of knowledge model (lecture) commonly found in many science classrooms today.

The BASSSQ is a Likert-scaled survey consisting of 41 items. Each statement is ranked by the respondent. The instrument contains for sub-scales pertaining to one's beliefs about science and school science. The sub-scales are:

Chen et al. (1997) state:

Simply put, the higher the sub-scale mean scores, the more constructivist in beliefs is the respondent. Due to the small sample size, sub-scale means were pooled for analysis.

Reliability and Validity

Chen et al (1997) report:

Internal consistency (Cronbach's alpha reliability) and discriminant validity (mean correlation with other scales) for each of the scales of the BASSSQ were deemed appropriate for use in this study.

Qualitative Instrument

The qualitative data gathered in this study were from self-reporting, open-ended questionnaires completed by the project teachers. These questionnaires are required by the Nevada Dwight D. Eisenhower (DDE) Program Manager for review. DDE funds are made available to states across the United States to promote science and mathematics teaching and learning. Proposals for funding are awarded on a competitive basis. DDE monies helped support a large part of the TSB program. In addition, funding for external review, support for out-of-state teachers, and half-time support for the TSB coordinator came from the TOYOTA USA Foundation in 1999.

Design and Procedure

Qualitative Design

The following questions come from the 1999 DDE Program Evaluation Forms for the TSB program:

1. Did the workshop meet your expectations?

2. Would you recommend this experience to other teachers?

3. As a result of this workshop, has there been any increase in your knowledge of your selected content strand?

4. As a result of this workshop, do you anticipate any changes in the way you utilize the Nevada Science Content Standards in your lessons?

Due to the small sample size of this study (N=35), ethnicity and gender influences were unsuitable as variables to be studied. Therefore, the data resulting from the questions above will be reported using only descriptive means.

Quantitative Design

A "one group pretest-posttest design" (Campbell & Stanley, 1963; p. 7) was employed in this study. Teachers enrolled in the TSB program defined the one group only pre-experimental model. Each teacher was administered the BASSSQ at the beginning and end of the TSB program during the summer of 1999. Campbell and Stanley (1963) report that there are three threats to validity using this design: history, maturation, and the effects of testing. Due to the short time frame of the TSB program, these potential causes of invalidity were deemed to be acceptable for this study. The manipulated variable, or treatment, in this study was the experience of being exposed to contemporary trends and issues in science education and the resulting curriculum developed by the teachers. The responding variable was the BASSSQ scores of the teachers.

The BASSSQ was first administered during the initial all participants meeting held on June 12 , 1999. These data would serve as a baseline for noting a change of beliefs relating to teachers' beliefs about the nature of science and school science.

Results

Quantitative results for pre-post differences in BASSSQ scores can be found in Table 1.

Table 1

Mann-Whitney test of differences in pre-post BASSSQ scores of teachers (N=35)

Variables Sample Size_____________________

PREBASSSQ 35

POSTBASSSQ 35

Sum of the ranks 1120

Test statistic T 490

Approximate Large Sample Test

Mean of T 612.5

Standard deviation of T 85.1347

z-value -1.438896

Prob( | z | > 1.439 ) 0.1502

_________________________________________________________________

Note: The low probability value of .15 above implies there is no statistically significant differences between pre- and post-BASSSQ scores.

Qualitative Results

Each teacher was administered the DDE survey questions found below. Responses were limited to YES, NO, and MAYBE. Also included are percentages of the respondent's answers.

________________________________________________________________________

1. Did the workshop meet your expectations?

Yes 93%

No 7%

2. Would you recommend this experience to other teachers?

Yes 100%

No 0%

3. As a result of this workshop, has there been any increase in your knowledge of

your selected content strand?

Yes 96%

No 4%

4. As a result of this workshop, do you anticipate any changes in the way you utilize

the Nevada Science Content Standards (NSCS) in your lessons?

Yes 84%

Maybe 8%

No 8%

________________________________________________________________________

Additional, open-ended questions were asked of the teachers. What follows are selected responses from the surveys.

How specifically will your experience benefit you as a teacher?

"It will improve the way I teach science."

"Providing materials."

"It will benefit me by showing me how to package lessons/activities to meet the State

Standards."

"I know how to locate new resources. I received many new ideas. I now have a better approach than the scientific method."

"Gave me additional insight into how to put a science unit together."

"Broaden my knowledge for the content strand in addition to making the Nevada Standards much more approachable."

"Increased knowledge and understanding of current science theory. Knowledge of Nevada Standards."

"It made me realize that it is important to change and improve teaching methods."

"I learned as I worked through the Engagement, Explanation, Expansion, Exploration, and Evaluation teaching model."

"Loved the great information given on writing effective lesson plans. The five E's were fantastic!"

"Utilize science knowledge to improve students interest and achievement."

"I will meet with my administrator and department chair to discuss changes in curriculum."

"It increased my own knowledge, expanded my resources immeasurably, and added

enthusiasm."

Teachers were also given the chance to add any other comments to the survey not yet included in any survey question.

General Comments:

"I have used the Nevada Standards, but I feel I have a better understanding as a result of this workshop."

"I will look at them (Nevada Science Content Standards) more often and have fun with them."

"Wanted the opportunity to explore science concept in a friendly manner while at the same time succeeding in generating useful materials."

"...the experience was both positive and refreshing. I especially appreciate the detail and organization of the whole program."

"...making the Nevada Standards much more approachable."

"I know what they are now (Nevada Standards). I am aware of what levels are what; what students should have already learned and may still need to learn. I can develop lessons accordingly."

"It will improve the way I teach science."

"....staff at DRI have been very helpful and supportive."

"Meeting others, learning new techniques, seeing all of the fantastic kits."

Teachers were urged to make suggestions for future TSB programs. Below is a summary of points from this section of the questionnaire:

Displays

 display Traveling Science Boxes at each site

 show past lessons to new participants

 display all TSB's from all participants on last day

Lesson Plans

 more detail on lesson plan specifics

 revise lesson template

 need assessment formats

 more time between 1st, 2nd & 3rd general meetings

General

 put forms on a disk

 more minority teachers encouraged to participate

 get bigger suitcases

 more workday time in workshop

 use of TV screen (video feeds) seems impersonal

 receipt accounting (for materials) is over done

Conclusion

While the BASSSQ scores between pre and post tests of the TSB program did not show a statistically significant difference in teachers' attitudes towards science and school science, the effect size of .47 (reject practical significance if < .33) is practically significant (Borg, Gall, & Gall, 1993). Practical significance indicates that while a treatment effect may not meet statistical significance, the treatment, nonetheless, has a practical value. Borg, Gall, and Gall state that "an effect size of 1.00 is twice as large as effect size of .50. The mean of the effect sizes...can be calculated to yield an estimate of the effect of the experimental program or method" (p. 171). While the effect size is not twice as large as .66, it remains that there appears to be a positive influence upon the teachers' beliefs about science and school science based upon BASSSQ scores.

This lack of statistical difference cannot be attributed solely on the perceived failures of the TSB program on influencing teachers. To the contrary, the short timeline for the treatment (curriculum development) to influence teachers' beliefs was the main threat to validity, as noted by Campbell and Stanley (1963).

Qualitative data suggest the TSB program was extremely successful, especially in terms of its practicality in teachers gaining a better understanding of the Nevada Science Content Standards. Phrases from the teachers such as "...making the Nevada Standards much more approachable" and "I know what they are now (Nevada Standards). I am aware of what levels are what; what students should have already learned and may still need to learn. I can develop lessons accordingly" clearly reveal that the goal of making the TSB exclusively standards-based was a success.

To close, the Desert Research Institute's Traveling Science Boxes program stands out as an effective model for teacher professional development in both quantitative and qualitative ways. Within this time of science education reform across the nation, it is refreshing to report teachers being excited about learning, teaching, and curriculum development. The Traveling Science Boxes program is an excellent example of the type of reform called for in these changing times.

Creative thinking consultant Peter Lloyd states, "Think right or be left behind" (Miller, 1993). The Traveling Science Boxes program does just that by allowing students and teachers to "think right" about contemporary science education.

References

American Association for the Advancement of Science [AAAS]. (1993). Benchmarks for scientific literacy. New York: Oxford University Press.

Aldridge, J., Taylor, P.C., & Chen, C.C. (1997, March). Development, validation and use of the Beliefs About Science and School Science Questionnaire (BASSSQ). Paper presented at the annual meeting of the National Association for Research on Science Teaching, Chicago, Il.

Borg, W. R., Gall, J. O., & Gall, M. D. (1993). Applying educational research: A practical guide. (3rd ed.) New York: Longman.

Campbell, D. & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Dallas: Houghton-Mifflin.

Gabel, D. L. (1994). Handbook of research on science teaching and learning (Ed.). Washington, D.C.: National Science Teachers Association.

Glynn, S. M., Yeany, R. H., & Britton, B. K. (1991). A constructive view of learning science. In S. M. Glynn et al., (Eds.), The psychology of learning science (pp. 3-20). Hillsdale (NJ): Lawrence Erlbaum Associates.

Lorsbach, A., & Tobin, K. (1992). Research Matters . . . To the science teacher: Constructivism as a referent for science teaching. Narst Monograph, 5, 21-27. National Association of Research in Science Teaching.

Miller, H. G. (1993). Peter Lloyd on being a concept artist. Compuserv Magazine, 12,(5), 31.

National Science Education Standards. (1996). National Academy Press: Washington, D.C.

Nevada Science Content Standards. (1998). Available on-line at http://www.dri.edu

Pressley, M., Harris, K. R., & Marks, M. B. (1992). But good strategy instructors are constructivists!. Educational Psychological Review, 4, 3-31.

Third International Mathematics and Science Study [TIMSS]. (1997, September). U.S. Department of Education: Office of Educational Research and Improvement.

Back to the EJSE