Evaluation Results
We conducted several different types of evaluations as we developed and presented the course. These included:
- Pre and post attitude survey
- Pre and post strategies survey
- Comparison of Chemical Sciences students with General Chemistry students
- Pre and post content exam
- Final exam comparison with traditional class.
The results of these assessments are presented here looking at both affective and cognitive domains.
Affective Evaluations
I. Survey Results on Attitudes toward science and teaching.
Part of becoming a "lifelong learner" of science (goal#2) is the development of an appreciation for science and a curiosity about it. Our attitude survey results indicate that this course is successful at improving, and indeed developing, a positive attitude about science.
The attitude survey (appendix #2) shows a full point increase in response to questions like:
- I feel confident that I can teach concepts related to chemistry to K-6 students. (1.07) Question #9
- I know how to conduct a science investigation. (1.43) Question #16
- I know nothing about chemistry. (-1.17) Question #44
- I understand some key concepts related to chemistry. (1.02) Question #50
(a full point increase is equivalent to moving from "agree" to "strongly agree" or moving from "neutral" to "agree" in regards to the statement.)
There was over a .50 increase in questions:
I would like to take more science. #32
I enjoy talking to scientists. #33
I feel I understand the science involved in most current issues. #34
I enjoy reading articles about science in a newspaper or magazine.#35
I see myself as a scientist. #46
These data certainly support our belief that we are developing in our students an appreciation of chemistry that may lead to "life-long learning".
II. We also conducted a Strategies Survey (appendix#3), which addressed collaborative learning issues. That survey indicated that there was some improvement, but no major increases (over a change of 0.50).
III. In an attitude survey (appendix #4), taken by both the Chemical Sciences and first quarter General Chemistry (Chem 121) students, our results showed:
In regards to understanding, visualizing, and applying concepts in chemistry, our Chemical Sciences students showed significant increases, while the General Chemistry students showed little change.
It is interesting to note that General Chemistry students showed a significant decrease in the their perception of the "importance of performing laboratory experiments". However, the view of the Chemical Sciences students toward the importance of laboratory did not change or may have improved slightly.
Confidence in my ability to: Chemical Sciences Gen Chem Winter Spring Chem121
pre-post pre-post pre-post
Understand key concepts in chemistry. 2.25-1.67 2.25-1.80 1.44-1.37
Visualize key concepts in chemistry. 2.36-1.80 2.36-1.78 1.57-1.59
Apply knowledge too real world. 2.27-1.17 2.29-1.59 1.84-1.82
Perform lab experiments. 2.02-1.69 1.11-1.23
The Importance of :
Performing lab experiments. .85- .78 .84- .82 .73-1.33
(scale from 1 — 5 with 1 = strongly agree and 5 = strongly disagree)
Cognitive Evaluation
We believe the affective evaluation showed that this course increases enthusiasm for both science and teaching. The following evaluations were carried out to determine whether the valued content was still learned (goal #1).
I. Pre/Post Test Results
The pre/post test consisted of 31 (appendix #5) multiple choice questions taken from the traditional course final exam and used as a representative sample of the valued content.
The results indicate no decrease in content
learning.
|
Number of Students |
||
| Winter 1998* |
|
|
| Scores greater than 20 |
|
|
| Scores less than 15 |
|
|
| Scores less than 10 |
|
|
| Spring 1998* | ||
| Scores greater than 20 |
|
|
| Scores less than 15 |
|
|
| Scores less than 10 |
|
|
*The number of students in the class were exactly the same.
Final Exam Comparisons with Traditional Class.
Another measure of content learning was obtained by comparison of the performance of our students in the new course to those in the traditional course. The complete traditional course exam could not be given because the format did not fit the new course. Instead, eight questions from the traditional final were included in the new course final exam. A trained student using the same rubric independently graded the questions.
In the first quarter we found that the new course students did better on the essay questions requiring thinking and expression. Those items are given below.
Changes given are changes from the traditional course compared to the new course.
1 Which of the following elements most closely behaves chemically like chlorine?
a) potassium b) argon c) fluorine d) magnesium e) oxygen
Change from 3.33 to 3.15 out of 4.00
2. The molecular weight (mass of one mole) of carbon dioxide, CO2 is
a) 5 b) 10 c) 17 d) 44 e) 52
Change from 3.70 to 3.84 out of 4.00
3. If the half-life of iodine-131 is 11 days, how much of a 100
gram sample will be left after 55 days?
a) none b) a trace amount c) about 3 d)12.5 e) 25.0
Change from 3.10 to 2.77 out of 4.00
4. What is the basic difference between chemical reactions and nuclear reactions?
Change from 2.50 to 3.08 out of 4.00
5. Draw a picture of a carbon atom.
Change from 5.37 to 5.56
6. Why do chemicals react. Why, for example, does Na react with Cl to make NaCl?
Change from 4.50 to 5.06 out of 6.00
Change from 22 to 39 people making the max 6/6
Change from 16 to 7 people making 3/6 or below
7. What are the four molecules of Life.
Change from 3.96 to 3.56 out of 4.00
8. Name two scientists who made contributions to the area of chemistry and describe their contributions.
Change from 5.46 to 5.88 out of 6.00
Change from 10 people making below 5 to only 2
Change from 43/53 to 48/52 people making 6/6
A preliminary look at the second quarter data indicates that the changes may not be as significant.
Observation Skills Comparison of Chemical Sciences and Gen Chem Students
Students were given an exercise in experimentation and observation having to do with a sucker dissolving in water. Surprisingly, our Chemical Sciences students showed overall better observation skills than third quarter General Chemistry (Chem 123) Students. They reported in greater percentages those items deemed important by the instructor. (appendix #6).