assessment approach

data collection

critique of collection and analytical process

Institutional assessment, student learning outcomes assessment, and all the forms of assessment in between are part of the conduct of education at Cal Poly Pomona. Some people in the community regard them as an irritant or a chore, others as a progressive movement, others as a threat to freedom; most appear to have accepted them as the approach to educational management most comprehensible to legislatures, corporate partners, agencies, and foundations to whom we are beholden, a kind of lingua franca.

On the occasion of the WASC Self-Study Workshop of January, 1999, in which five of us participated, we learned that the following assessment issues would be regarded as crucial to our successful evaluation of ourselves and representation of ourselves to the WASC team. For the most part, we find that we have made progress with all of these measures, to varying degrees.

1. Assessment will be a theme in all comprehensive self-studies.

Assessment issues are embedded in all the themes of this study, especially Themes Two and Three (Chapters VI and VII).

2. The institution will have conducted a census.

We are required to conduct a census every instructional quarter. The results are reported to the Chancellor’s Office of the CSU and appear on their web page, as they do on the web page of Institutional Research and Planning, along with enrollment data.

3. The institution will have established its own working definitions of assessment.

Most departments and divisions now have assessment plans, with appropriate definitions of what it is they are assessing, how, and why.

4. Faculty are to be directly involved in assessment efforts.

Faculty are very much involved in such efforts, in some instances in leadership roles.

5. Several specific assessment efforts already are underway.

Student Affairs, Administrative Affairs, and Academic Affairs have all carried out extensive divisional assessment exercises; within them, at least a dozen units (Graphics, Student Health, Student Enhancement Services, Human Resources, Facilities Management, Public Safety, Enrollment Services, Admissions, and several colleges and departments) have done so. Many offices have undergone audits by the Trustees very recently, including the Foundation and the College of the Extended University. The Colleges of Business Administration and Engineering are both accredited by professional accrediting organizations (AACSB and ABET). Programs in Environmental Design and Science are also externally accredited.

6. Develop institutional plan, or set of plans, for assessment.

These are being developed; most departments have assessment plans. The erstwhile Assessment Planning Group had proposed as one of its functions the coordination of the development of such plans across the campus. We are holding this proposal in abeyance until the Academic Senate has decided how it wants to interact with the administration on assessment as an institutional activity. The question may become moot in the context of the accountability indicators required by the CSU; the resultant reporting process may subsume assessment planning.

7. Incorporate assessment data in periodic evaluations of the effectiveness of the general education program.

This is a facet of the latest proposed revision of general education. Discussion on the Senate floor indicates that there is little opposition to this part of the proposed program.

8. Incorporate assessment techniques into program review procedures.

Departments in professionally accredited colleges have been quick to convert to an assessment format for their periodic reviews. Other programs across the colleges are gradually doing so. Assessment can be conducted in lieu of one cycle of regular program review, by agreement between the Academic Senate and Academic Programs Office (Appendix C4). Several programs are opting for this arrangement.

9. Develop an assessment program to review the co-curricular program of the institution.

Review of Existing Data

Nearly every office of the university, or office they report to, was contacted near the beginning of the self-study and asked for data in any form. Most complied with this request eventually. We found the systematic, routine collection of information on the student body, personnel, physical plant, bureaucratic structures and processes, informal organization, curriculum, policy framework, relations between offices and/or groups, socioeconomic context, and educational issues to be well developed in a few places, absent in others, scattered between. The quality and availability of all kinds of documents and statistics improve dramatically after about 1995. Prior to that, there are major gaps in institutional history, unless one relies entirely on oral tradition.

Addition of New Data

As mentioned, new data was procured via the focus groups and campus climate surveys. We also sought documents on a continual basis. One of the methodological problems of community-based research we have experienced is that information is scattered and there is no hub or source for locating it. This is true of classical communities (e.g., a village, a neighborhood), and it is sometimes true of organizations, though highly compartmentalized structures such as a university can be expected to do a better job of directing seekers of information to the appropriate compartments. Cal Poly Pomona falls somewhere between these extremes of high compartmentalization and total scattering. The deliberate program of decentralization undertaken in the early ‘90s had many positive benefits, but also produced a kind of dis-connectedness and dis-coordination that have made it hard to find information or know where to look, in some cases. The Joint Review process (Annexes D3, 4, 5), with all its faults, at least discovered this feature of our work environment, as did the International Programming Task Force (Team Room), the ACE Project (Appendix D3), and probably other efforts.

Compilation of Data

A Team Room was set up in the University Library for the use of the Steering Committee during the second research phase, described above. We began to compile and organize written materials here. Many different approaches have been tried — file by Standards, by Themes, by Goals, by units of the university, by serendipity. The Writers’ Group made use of the document library in different ways, according to their needs.

Critique of Collection and Analytical Process

The use of ethnographic method was at times in conflict with standard assessment methodology. The quantitative preference of the latter left many members of our community, even scholars proficient in quantitative analysis, with the sense that the human factor, the creative spark and the meaningful relationship, had been diminished in the formula for educational effectiveness. We have tried to produce a good mix of interaction-intensive research and measurement.

One of our other goals was to make a case for self-study as internal accountability (further discussed in Theme Four, Chapter VIII). If we, as a community, had a basis on which to hold ourselves, as parts and as a whole, accountable for our own performance of our mission(s), we would be in a better position to confront the Chancellor’s Office, state legislature, public, and external accreditors/auditors without emergency preparation. We would already, and always, know most of the answers they seek. This internal accountability rests on the ready availability of reliable, honest information about what everybody does, how they think it affects the educational program, and how it is supported. Secrecy and disrespect are anathema to internal accountability. Such an expectation applies evenly to all quarters of the campus, though people have different responsibilities, different amounts of responsibility and privilege, and different characteristics and levels of compensation.