Summary of CSU IC Assessment Task Force Meeting 
September 1, 2000

 Present: 
Task Force:  Curzon, Dunn, Roth, Smith, Sonntag, Whitlatch
SBRI:  Large, Pulliam, Risley, Saferstein, Serpe, Shaw

 Host:  SBRI,  San Marcos.  Many thanks! 

Follow-up on the Draft Report:

I.   Second draft of the CSU Information Competence Survey Report distributed. Changes and additions in response to our comments on the first draft report noted.  Some discussion followed regarding the inclusion in the final report of  charts/graphs that tie student demographic data to the scenario depth and breadth measures.  Richard affirmed that this could be done by indicating what percent of, for example, males and females, fall in each of the 4 quartiles.  Task force members indicated  that they would like to see this type of data included in the final report. 

Task force Action Time:  Review second draft and submit suggestions to Richard Serpe by September 15.

II. Richard Serpe  responded to some of the questions previously posted to the task force list about the assessment methods and the draft report. 

  1. Can we test students for IC in order to get hard data?   Serpe noted that we do not yet know what measures would say that a student is information competent. For example, is information competence a concept or a set of skills? This is still an open question.   Also, after SBRI looked at the new ACRL Information Literacy Competency Standards, they concluded that they would be difficult to operationalize in an assessment, possibly because they are too classroom-based. 

We see IC at this point as a process, not an outcome.  It is a sliding scale,  making it difficult to decide a cut-off point between competence and non-competence.  Richard advises against hard measures of IC at this stage in our assessment and noted a number of potential difficulties with tests, including the impression that classroom “tests” are not taken seriously except as exit exams, such as that used by SFSU.

  2. What exactly can we expect from the survey?  The survey is a first step in assessing information competence and provides quantitative, aggregate data for the CSU.  It gives us baseline measures of what our students look like, i.e., broad patterns of IC behaviors at a given point in time. 

Clarification on Long Range Assessment Plan for the CSU

This is a  long-term assessment of 5 years in the following phases:

  Year 1:  large nomothetic survey to elicit broad patterns of what our students look like in the system.  The survey was completed in Spring 2000.  Final report due in October 2000.

Year 2:  Secure qualitative, individual data concerned with process, style, understanding, and meaning to provide an understanding of how students approach information tasks in real life.   Planned for 2001.

Year 1 and 2 together give us information to develop methods and technique to follow people and get at change over time

Year 3, 4, 5   Longitudinal Panel Design:  Freshman and transfer students are selected at random in large enough numbers to study over a three year period.  Freshman will constitute one panel, and transfer students will constitute a second panel. Experimental groups could be drawn from the panels and followed in regard to targeted interventions, and/or new groups could be added for specific purposes.   Possible methods to use with panels: phone interviews, surveys, work with faculty on some campuses to set up portfolios of work, etc.   SBRI plans to  include both quantitative and qualitative measures to ensure a variety of measures. 

Expected results of a five year longitudinal study:   

  1. A better understanding of the process of information competence.  In the academic environment, information competence is a process used within an assignment, and we hope to understand this process more fully.
  2. Models that will demonstrate the extent of competencies in individual students or groups of students
  3. Patterns of information behavior and thought processes for those who are and those who are not information competent.

  Task Force request for changes/additions to the long-term study:

  One clear concern of the task force and the IC Work Group is that the study provide  tangible data/information models/tests to use at all levels of the CSU.  We are counting on data that will make sense to CSU administrators and faculty,  and that will reliably identify the IC status and  needs of students.  We would also like the study to experiment with interventions and to test questions for diagnostic tests.     

  1. Questions for a diagnostic test.  Agreed that we will begin in Year 3 and 4 to test certain questions for reliability and validity for use as a diagnostic.  Decided to look again at the questions developed by Jo Bell and the Assessment Questions Subcommittee of the task force in preparation for Year 3.  These questions must be tested for reliability and validity before we can adopt them.  In addition, once we have a battery of questions,  Richard suggests that the task force refer them to the Research Triangle for cognitive testing. 
  2. Interventions will be tested during Years 3,4,5.
  3. Compare graduation rates and GPA with information competence measures
  4. Compare against measures of IC,  campuses with BI/IC programs and campuses without such programs.

  Year Two, 2000-2001, Discussion and Plans

  General Plan:   Four morning and afternoon events on  4 different campuses, to be selected.    Participants:  25 students, 5 faculty, 5 librarians.  Students will be randomly recruited from the region.  Faculty and librarians will be recruited from the region according to interest and commitment to IC.   During the morning students will do a series of tasks. SBRI staff, including ethnographers, will capture their activity through screen capture, audio, and video methods.  At the same time the faculty and librarians will meet as group to think through what they would do with the tasks given to the students and/or what they would expect the students to do with the tasks. This session will be videotaped.    The afternoon session will consist of focus groups of students, faculty, and librarians to debrief students on their morning activity, identify problems and barriers, etc. These sessions will also be videotaped.  The plan is to develop and pilot test one event  before carrying out the other events   Timeline:  Pilot test in February, and complete all of the groups during Spring 2001. 

  Meeting Decisions:  1) tasks assigned to students will be both classroom research tasks and real world tasks. 2) may use scenarios again in this new context; 3) possible campuses for the events: SJSU, CSUN CSUF, CSUS; 4) SBRI staff along and task force members will draft the tasks for review by the larger task force.  

Kathleen Dunn
9/18/2000