June 28, 2001
*Next Meeting:
TBA
Present:
IC Assessment Task Force: Carol Bednar, Corryn Crosby-Muilenburg, Susan Curzon, Kathleen Dunn, Maria Garrido, Linda Goff, Paula Hammett, Ann Perkins, Ilene Rockman, Gordon Smith, Jo Bell Whitlatch
SBRI: Nancy Hartwig, Linda Pulliam, Barry Saferstein, Linda Shaw, Richard Serpe
Kathleen Dunn introduced Linda Goff, Library Instruction Coordinator, CSU Sacramento,
and welcomed her to the Task Force. Also discussed was the addition of another CSU librarian to the task force, possibly from central California. Kathleen will follow up on this.
Research staff from the Social and Behavioral Sciences
Institute presented a preliminary report on Phase II of the CSU Information
Competence Assessment Project to the task force. The total sample size from Phase II is as follows: Students, 76; Librarians, 20; Faculty, 10.
Pulliam reviewed the research plan for Phase II which included construction of four tasks designed to maximize demonstration of outcomes of the performance indicators of the ACRL Information Literacy Competency Standards for Higher Education. Copies of the four tasks were distributed to the task force members. Linda Pulliam reminded the group that this preliminary report concerns Phase II of a qualitative study of students at twenty-one CSU campuses. The focus of Phase II is to describe and assess how students find and process information, with the goal of providing a baseline of student information competence in the CSU. Pulliam reported that one hundred students were selected as participants in the project. The study was conducted at four CSU campuses: Fullerton, Northridge, Sacramento, and San Jose. Seventy-six students actually participated in the study. Twenty librarians and ten discipline faculty members also participated. Three major parts of the data have been analyzed by SBRI: the written forms and videotapes, the ethnographic data, and the screen capture data.
B. Analysis of data from focus groups: Linda Pulliam
Pulliam reported that she began her analysis of the focus
group material by watching the discipline faculty/library faculty
videotapes. She identified six major
topics that emerged during the analysis of the video-taped data and focused
on two of these topics for the preliminary report: 1) expectations of
librarians and discipline faculty for student performance
and 2) student and librarian perspectives on the use of reference librarians.
In examining the expectations of librarians and faculty, she examined responses
to two questions discussed in the focus groups: "What do you hope students
will do?" and "What do you think students will do?"
“Pulliam presented a series of recurring statements for each of the two
topics. These specific statements will
be included in the preliminary report.
She will add findings for fac/lib differences and will add findings for
faculty/librarian differences and for “best practices.” Linda indicated that she can approach this
data in at least three ways: content
analysis, comparison of faculty/librarian/student differences, and comparison
with the ACRL competencies.
Pulliam stated that SBRI researchers and members of the task force had identified forty (out of a total of seventy-three) ACRL outcomes that could be evidenced from the four tasks presented to the students. The data showed that thirty-four outcomes were embedded in responses.
Additional findings of note:
Of the 76 students studied,
22% had not had instruction on using a college library
78% had attended a presentation on using a college library
54% had physically visited their campus library more than six times during the last term
9% had not visited their campus library at all during the last term
34% would have consulted others to complete their task if they had been given more time
8% named a librarian as a potential person to consult
Pulliam played segments of videotapes of students discussing reasons why they might or might not seek assistance from a librarian during the research process. Linda noted that a description of “best practices” for librarians and discipline faculty members might evolve as a future product of this data.
Linda Shaw presented findings from the ethnographic data for the sixteen students who were tracked by graduate student ethnographers. The ethnographers queried the students about ACRL Standards 1C, 1E, and 3C to discover how students would attempt to meet these standards as they completed their assigned task. Most students followed two steps: they tried to get an overview of the pool of information available for their task; they tried to general a pool of information from which they could draw to complete their task. Nearly all students used computers during the process. Students commented on the flexibility, convenience and efficiency of the Internet. They believed that they could fit computerized research into the demands of the rest of their lives. Seeking information via computer seemed to give students an autonomy or sense of control. However, some students did seek out books to find key words to put into computer search engines. Shaw noted that students struggled over key words. In addition, students often sought one rich, authoritative article that would “have it all.”
Shaw suggested that interviews with students revealed that they establish their information-seeking strategies early and tend not to deviate from those strategies. She said this suggests that interventions should occur early (by the middle school years).
Barry and Nancy used the screen capture software Camtasia to capture “what students did” on the computers. Barry noted that during the two hour period, eight of the 64 students tracked, checked their email in addition to working on their assigned task. Barry and Nancy showed clips of some of the actual screen captures. They also created a database that classified the sites the students searched.
Of the 64 students:
28 used the Web only
4 used library subscription databases only
31 used both Web sites and library subscription databases
Sample size: Gordon Smith asked Richard Serpe if he believed the student population in the study is representative of the CSU student population. Serpe stated that the sample size is representative, but that there is may be some bias in who agreed to participate. Richard described the process that was used to select the sample and expressed confidence in it, but noted that students who agreed to participate likely had no aversion to the library. Richard also commented that the faculty sample was too small to be considered representative, and that generalizations could not be made about the faculty group.
Determination of information competence: Sue Curzon -- Sue stated that our great dilemma is that although we are in an exploratory phase, we need to be able to say that students are “here” in terms of Information Competence, and we are able to move them to “there” through an intervention. We also need to know what kinds of interventions to design. Richard indicated that in Phase III we had planned to work with interventions, and also with an entry/exit test based on the questions suggested two years ago by the task force subgroup chaired by Jo Bell.
Kathleen Dunn summarized that as a result of the Phase II assessment we now have data from the focus groups, sixteen sets of data from ethnographers, and screen captures from sixty-three students She noted that the CSU is interested in concrete data on student information competence; and that at the February 12, 2001 meeting, the task force expressed interest in analyzing the Phase II data for quality of resource choices, methodologies, and comparison with the 2000 ACRL standards. Discussion followed on this. Some felt that task force members would not have time to do this analysis, and thought that SBRI should do this type of analysis. Other task force members expressed interest in doing it. Corryn Crosby-Muilenburg offered to review some of the screen capture data on behalf of the task force to determine how difficult it would be for the task force to analyze this type of data . Barry Saferstein indicated that he thought it a good idea to our perspective on the Phase II data. He and Linda Shaw agreed to send a samples of their data to Corryn for analysis. After Corryn reports on the screen capture data, Kathleen will query task force members on their interest in review phase II data for quality of resource choices, methodologies, and comparison with the 2000 ACRL standards.
Richard emphasized that the data is available to anyone who wants to look at it. Other people can pick it up and run with it. He also noted that a number of people looking at the data can come to different conclusions. He reminded the group that the resources that have been allocated to the project will be up soon and asked if the task force wants SBRI to work beyond those resources? Richard also mentioned that Linda Pulliam has begun work on another major project, but this isn’t to say that SBRI couldn’t do something more.
Linda Pulliam will make some cosmetic changes to the preliminary report, will mail the report to Jo Bell Whitlatch, and email a copy (without graphics) to Kathleen Dunn.
Jo Bell Whitlatch asked if the student forms could be put in the appendix.
Ilene Rockman read the Phase II proposal elements and noted that we started out to discover if our students are information competent or not. Richard said that we can say “these are examples of where students are meeting standards and where they are not.” However, Richard expressed doubt that a measure of information competency exists at this time.
Gordon Smith noted that we can agree on student inadequacies and find strategies to reduce
them.
Kathleen Dunn noted that we can gleaned suggestive data on related areas of interest from this study, such as students use of reference desk services.
Sue Curzon suggested that after the report is issued we meet to see what conclusions we can draw, what interventions are appropriate, and to define phase 3.
Jo Bell Whitlatch noted that as a broad generalization we are asking: What is the quality of the process? What is the quality of the product? Do they always correlate?
The Task Force is to submit input on the preliminary report due to Kathleen Dunn by July 20. After the input is received, Linda Pulliam will give the timeline for the final report to Sue Curzon, Kathleen Dunn, and Gordon Smith.
Next Meeting: Not scheduled at this time. We are waiting on Corryn’s preliminary review of the screen capture data, and the receipt of the final report of Phase II.
Minutes: Carol Bednar recorded the minutes of the meeting. Kathleen Dunn prepared the final report for the web site.