|
Summary of the Information Competence Assessment Task Force Meeting February 12, 2001
**Next Meeting: June 28, 2001, Cal Poly Pomona Present: IC Assessment Task Force: Paul Adalian, Carol Bednar, Kathleen Dunn, Maria Garrido, Ann Perkins, Ilene Rockman, Gordon Smith, Gabriela Sonntag, Jo Bell Whitlatch SBRI: Linda Pulliam, Barry Saferstein, Richard Serpe On Monday, February 12, task force members and research staff from the Social and Behavioral Sciences Institute met at Cal Poly Pomona to review and discuss all planning to date for the Phase II assessment project scheduled to start in March of 2001. I. Overview of work to date on Phase II: Linda Pulliam SBRI has been at work on the methodology and organization of Phase II since the last task force meeting in September 2000. Linda reminded the group that this is Year Two of a planned five-year study, and that it builds on the Phase I CSU-wide questionnaire/phone interview study completed in 2000. Phase II is a qualitative study in which we are looking for real-life behaviors through data from screen captures, field notes taken by ethnographers, and from focus groups and discussion groups that will be videotaped. To date we have developed and refined our methodologies, developed the tasks that are central to the study, recruited staff for the project, contacted the campuses to secure a student sample, and outlined the sequence of activities for the assessment day. II. Review and Discussion of Assessment Methodology and Site Arrangements: Richard Serpe, Barry Saferstein. This portion of the multi-year assessment targets CSU students, faculty and librarians to determine the following: • How students approach and complete information tasks within a given time frame using library and computer resources • How the processes students use are related to the products of their work • What pedagogical needs might be discovered through an examination of student's information-seeking strategies when observed directly • What similarities and differences exist between faculty, librarians, and students in their conceptualization of information-seeking strategies To accomplish this 25 students, recruited from surrounding campuses, will be brought to four geographically dispersed CSU campuses on Saturdays in Spring 2001 to participate in various activities that will extend from 10a.m. to 4 p.m. All activities will be recorded through screen capture, video and audio capture, and ethnographic observations Sample: Sample size about 100. Sample formed from a random sample of about 5 students from each campus, except Maritime. 1/2 will be upper division students and 1/2 lower division. The sample will not be balanced between ethnicity and other demographics. Richard expects, however, that we will have an over representation of students of color, about 40% will be student of color. Students selected may or may not have participated in the Phase I survey. Students will receive $100 for day and travel costs. Method: Four tasks have been developed. 25 students will perform each task. Morning: While students work on the tasks, librarians and faculty will participate in a focus group to discuss the tasks in terms of 1) what student might be expected to do, 2) what it would be possible for them to do, 3) how, ideally, educators would like to see them approach the task, and 4) what barriers students might encounter in performing the tasks. Afternoon: Students participate in focus groups to discuss their experiences with the task (problems, barriers, surprises, etc.) with librarians and faculty observing. Following this students, faculty, and librarians will participate in a give and take session to give the educators a chance to ask questions and wrap-up the day's events. All focus group will be video/audio taped. Assessment locations: Four CSU campuses libraries will serve as sites for the Phase II studies. Students, faculty, and librarians will be brought from surrounding campuses to participate in the daylong activities. The assessment activities will take place from March through May 2001. Campuses selected: Fullerton (March 3), Northridge (April 21), San Jose (April 7), Sacramento (May 5). Fullerton is the test site to see if anything needs refining before the other testing at the other three sites. Campuses aligned with test site follow this summary. Discussion centered on how one faculty member and one librarian from each campus would be selected to participate in the focus groups. Decision: Gordon Smith will inform COLD at the February meeting that each campus needs to send a faculty and a librarian. Using Gabriela's list of the information competence contact for each CSU library, SBRI will contact the librarian and the librarian will select the faculty member. SBRI will send the pertinent information to faculty and librarian participants. Task force members will select the librarian, in consultation with their Directors/Deans, and notify Richard ASAP. III. The Tasks: SBRI researchers and task force members, Gabriela Sonntag and Carol Bednar worked together to develop the four information tasks that are a central part of this study. Linda passed out tasks and explained that the following guidelines informed development of the tasks: 1) must accommodate students from different backgrounds; 2) must fit within a two hour time frame; and 3) must be related to the 2000 ACRL information competence standards. Some of the ACRL standards could not be represented in the tasks because they are too complex to hope to approach in two hours. Each task, which consists of several parts, is preceded by a brief set of background questions and a statement that sets the stage for the task. Task in full are attached at the end of this summary. The task force members then reviewed all 4 tasks, providing input that resulted in some minor modifications to the tasks, their background questions, and introductory statements. Confirmed with SBRI that researchers will answer student questions about the tasks, and that they can use any resource available to them in response to the tasks, including seeking assistance from reference librarians on duty. Some concerns expressed by the librarians present: • How can we distinguish random, inept searching from information-seeking behavior based on a transferable model? Response: three-fold data gathering methods will assist in distinguishing this. Regarding the responses that we might get to the tasks, Linda Pulliam noted that qualitative research will provide information that we weren't expecting, since prior experience or interest will inform their answers. • Can we get at some of the issues raised in Standard Five (addresses ethical and legal issues of information access and use) by asking the following question in the focus groups: “Now that you have worked on this question and know more about the topic, what would have you done differently? Perhaps. • Question raised about deliberately not asking or specifically testing computer resource use. Consensus was that we don't want to focus on that since in reality questions may call for many different types of resources. • Make sure that the students don't need to spend too much time answering form questions, and that we don't ask leading questions that give students the answers. • Won't students want to use their own library homepage? Access will be to the homepage of the test site library, but they can change it to their own library, if they wish. • Shouldn't we find out it the students have had library instruction? We will ask them at the end of the day to self-identify. The question needs to make a very clearly worded question. For example, “Have you ever had instruction on the use of a library for research? By this we mean, a class, a presentation, a workshop, a workbook, a tutorial or other session delivered or taught by a librarian at the college level. IV. Access to Phase II data for comparison with the ACRL standards: Kathleen Dunn, task force chair, requested that the task force have access to the screen capture, video/audio, and ethnographic data for analysis against the 2000 ACRL information competency standards. Considerable discussion followed. Conclusion: we will have access to the data, but how that will be accomplished is still in question. Kathleen also pointed out that the task force and COLD is looking for concrete data on the information competency of CSU students as measured against identifiable standards, such as the ACRL standards. Timelines for Future Work: Upon completion of data gathering from the Phase II assessment, the task force would like to meet to: 1) Review and discuss preliminary results from Phase II and how it relates to Phase I 2) Consider how to organize for analysis of Phase II data against the ACRL standards 3) Consider strategies for follow-up assessment based on year 1 and year 2 results, including the development of a diagnostic measure based on the standards Examples of Richard's ideas for Phase III include: • Identify faculty who will share with us experiences with students in their classes. • Identify students to follow and develop portfolios on them to include GPA, courses enrolled in, including syllabi and assignments, faculty evaluations and test results and papers written, etc. • Will need incentives - get something going with local bookstores, such as gift certificates for supplies, books or whatever.... V. Operational Issues for the Task ForceA. Additional members for the task force: Kathleen suggested that we need more people on the task force in order have wider CSU representation and to have enough members to participate in meetings and assessment activities. Decided to add three new members. Campuses not currently represented that should be considered include Sacramento, San Diego, Fresno, Long Beach, and Los Angeles and names were suggested from these campuses. Kathleen will discuss this with Sue Curzon prior to notifying the new members. B. Publicizing assessment activities to all CSU libraries. Members present decided to adopt campuses not represented on the committee. Adoptions agreed to are as follows: Adalian Stanislaus, Maritime, Channel Islands Dunn San Diego, Bakersfield Whitlatch Chico Rockman Monterey Bay Bednar San Bernardino Perkins Los Angeles, Dominquez Hills Summary prepared by Kathleen Dunn, Chair With thanks to Gabriela Sonntag for taking notes. CAMPUSES ALIGNED WITH TEST SITES Fullerton: March 3 Fullerton Pomona San Bernardino San Diego San Marcos San Jose: April 7 Fresno Hayward Monterey Bay San Francisco San Jose Northridge: April 21 Bakersfield Dominguez Hills Long Beach Los Angeles Northridge Sacramento: May 5 Chico Humboldt Sacramento San Luis Obispo Sonoma Stanislaus |