methodology

Our mission has been "to create an account of our university that will be meaningful and inspirational to us and informative to WASC. The self-study will generate recommendations and new directions that have the potential to make our university a better place to study and work." (cf. "Interesting Times" 2, Jan. 2000, Annex A8). The specific values of the study that were promulgated from early in the research period were based on:

A constructive understanding of how we function or why we don’t (learning),
A heightened sense of connectedness to each other and to our context (community),
Better service and performance, as the community defines them (quality).

These values were a significant foundation for the process we employed, one emphasizing openness, collective exchange, and scholarly rigor. The formal general goals we identified include:

Implant the self-study process as monitoring/assessment for internal accountability.
Identify the most useful and meaningful indicators of progress toward goals.
Contribute to a greater understanding of students and to shared knowledge of the organization among community members.
Identify movements leading to establishment of more effective educational management procedures.
Identify strengths and weaknesses of our educational program and design road maps to improvement.
Measure successes of learning outcomes assessment initiatives, and specify weaknesses in these initiatives.
Re-affirmation of our accreditation (external accountability).

We realized that the university did not have a pre-existing data base that would be sufficient to our informational needs. A community-based project was therefore initiated, consisting of a combination of document collection and analysis; focus group discussions; a campus-wide survey; interviews and meetings with students and personnel; and participant observation. No study of this sort can hope to proceed without creating a stir of its own beyond the stirrings it attempts to record. We therefore conducted our inquiry publicly, so that the community could compensate for being studied by participating themselves. Personal communications were kept in confidence. The critical general processual, methodological, political, and structural issues are addressed in the following sections of this chapter, and specific research methods or data and documentation problems discussed further in connection with the appropriate themes of the study (Chapters V through VIII).

III. METHODOLOGY OF THE SELF-STUDY
introduction framework role of steering committee formation of sub-committees assessment approach process of self study campus climate survey	EVOLUTION AND ENGAGEMENT A Self-Study in Preparation for an Accreditation Review by the Western Association of Schools and Colleges October, 2000
Introduction	Introduction to the Self-Study Methodology
Introduction	Our mission has been "to create an account of our university that will be meaningful and inspirational to us and informative to WASC. The self-study will generate recommendations and new directions that have the potential to make our university a better place to study and work." (cf. "Interesting Times" 2, Jan. 2000, Annex A8). The specific values of the study that were promulgated from early in the research period were based on: A constructive understanding of how we function or why we don’t (learning), A heightened sense of connectedness to each other and to our context (community), Better service and performance, as the community defines them (quality). These values were a significant foundation for the process we employed, one emphasizing openness, collective exchange, and scholarly rigor. The formal general goals we identified include: Implant the self-study process as monitoring/assessment for internal accountability. Identify the most useful and meaningful indicators of progress toward goals. Contribute to a greater understanding of students and to shared knowledge of the organization among community members. Identify movements leading to establishment of more effective educational management procedures. Identify strengths and weaknesses of our educational program and design road maps to improvement. Measure successes of learning outcomes assessment initiatives, and specify weaknesses in these initiatives. Re-affirmation of our accreditation (external accountability). We realized that the university did not have a pre-existing data base that would be sufficient to our informational needs. A community-based project was therefore initiated, consisting of a combination of document collection and analysis; focus group discussions; a campus-wide survey; interviews and meetings with students and personnel; and participant observation. No study of this sort can hope to proceed without creating a stir of its own beyond the stirrings it attempts to record. We therefore conducted our inquiry publicly, so that the community could compensate for being studied by participating themselves. Personal communications were kept in confidence. The critical general processual, methodological, political, and structural issues are addressed in the following sections of this chapter, and specific research methods or data and documentation problems discussed further in connection with the appropriate themes of the study (Chapters V through VIII).
	A. Framework of the Self-Study
<top of page	The university decided to adopt an experimental approach to our self-study following the WASC Self-Study Workshop of May 1997. Nine people from the faculty and administration were in attendance. They concluded, as a result of the discussions of the relative merits of different approaches to self-study, that a thematic orientation rather than a traditional, Standards-based analysis and presentation would be a greater learning experience for the university. In January 1998, we composed the proposal that eventually was accepted by representatives of the faculty and administration and then approved by our liaison at WASC (Annex A2). The composition of the Steering Committee, the tentative Plan of Action and timeline, and a rough outline of the Self-Study were also endorsed. Modifications to these have been made subsequently. By June 1998, the Self-Study Coordinator had implemented the data collection phase of the research and presented a specific division of the Steering Committee into task forces for the forthcoming phases of analysis and compilation/writing. (more...)
	B. Role of the Steering Committee
<top of page	The final specialization within the Self-Study Steering Committee was presented at the beginning of this volume. Ultimately, our project resolved itself into the work of writing and various support tasks. (Annex A9 contains an early history of the project.) At all times, there was student representation, though the student members were very different as regards their level of active participation. Staff were under-represented as a category, and this resulted in criticism of the first draft of the report as being deficient in attention to staff issues. Faculty and administrators were about evenly represented. Ex officio committee members contributed immensely to the effort at all stages, as did our consultants from off campus. The list of people and groups the researchers interviewed, met with, or consulted (i.e., our on-campus respondents) is provided in Annex A5. Students played a useful and powerfully symbolic role in the process of the self-study. We were at pains to include them for several reasons. One, student leaders have in recent years been very active in campus affairs, and are knowledgeable and capable contributors to all phases of endeavor. Their activism has propelled issues ranging from diversity to shared governance to retention. Two, the university seems to be sincere in putting the interests, learning, and democratic participation of students in first place in our thinking. Were we to have neglected them in our research, we would have ended with a blind man’s view of the elephant. Third, the student representatives on our committee and committees doing related work (e.g., the Campus Climate Survey Committee) contributed a good deal of brain power and energy to the projects we accomplished; we could not have done as much without them. We consulted them through their focus group and other meeting participation, as well. The "WASC and You" video (the script is contained in Annex A6) and other pieces of publicity targeted students as the primary audience, since we thought it was proper that they have up-to-date information on the accreditation of their institution. Finally, we have found that if we cannot explain our purposes and methods to students, perhaps they are not very clear to us, either.
	Formation of Sub-Committees
<top of page	Compliance with WASC Standards The division of labor went through a complex evolution, with at least two formal stages evidenced by two assignment matrices. The first matrix, [Figure 5] placed members of the Steering Committee in charge of organizing topical items and data according to the four themes (two per theme), and others in charge of organizing materials called for by the nine standards and relating them to the concepts of the themes (two per standard). This exhausted (in both senses of the word!) the personnel of the Steering Committee, which at that time consisted of twenty-six staff, faculty, students, and administrators. This initial phase of research, lasting from Spring, 1998 through Fall, 1998, amounted to an internal compliance review, familiarizing us with the meaning and requirements of the Standards and assuring us as to the institution’s basic capacity and characteristics in relation to the foci of the Standards. During this first phase, we held and attended a large number of meetings, facilitated ten focus groups, and began to assemble documentation for the Portfolio. The second phase of research began in the Winter of 1999 with the formulation of the Goals Matrix, [Figure 6] and the organization of the Campus Climate Survey Committee. We identified consultants from outside the university who would be able to bring neutral and expert perspectives to bear on some of the ideas in our study. We began a more intensive round of internal consultation, targeting committees (e.g., the GE Committee), groups (e.g., the Associate Deans), and individuals who were well-placed or well-informed on critical matters. We also designed and inaugurated a program of activities meant to spread information about WASC, about the self-study, and about how people could interact with it/us. We took advantage of existing campus venues for presentation, such as Campus Forum, Fall Conference and the Faculty Center Brown Bag, and created some of our own opportunities. We interviewed most of the officers of the university individually, many faculty and staff, and many students. Our "ethnographic" approach inserted us into the regular meetings of a number of pieces of the organization as participant observers. Naturally, the membership of the Steering Committee gave us entree into several key committees, task forces, councils, and other groups, as it was supposed to do. The selection of a Writers’ Group, although this has not been a perfectly stable phenomenon, took place during this second research phase. Actual composition of narrative began following the Retreat of May, 1999, inaugurating the third phase of the project, the production of a first draft. Pieces of that draft were disseminated during Fall Conference, 1999, after another summer retreat, but a skeletal version of the whole thing wasn’t ready until Winter, 2000, when we began to prepare for the final series of consultations. We feel we have made a herculean effort to inform people, listen to them, and reflect their interests and concerns. Applicability of the Themes and Standards to Campus Life We also recognized the value of building on existing plans and studies that others at the university had carried out, and which had some general cogency for the majority of students and workers. Although the university has engaged in quite a lot of planning, reporting, and research exercises (Master Plan, Joint Review, Academic Senate surveys, Campus Life, Cal Poly 2001 project, annual report, … ), many of these were of a restricted scope, focused on college- or unit-specific issues, and were unanalyzed, or non-scholarly. The Strategic Planning phase of 1993-97 has had the widest impact and represents the most convincing statement of university-wide mission, goals, values, and vision, along with action plans or strategies and specific objectives. Most divisions, colleges, and departments have also aligned their Plans with the University Strategic Planning Guidelines of 1997, so we selected this document as our index to the sense of the university. The six Goals articulated in the Strategic Plan offered another lens through which to look at our themes and Standards, albeit a fly’s eye image that complicated our division of labor. The following table is our effort to correlate the themes, Standards, and Goals in a single matrix, which also generated a refinement of data and issues to be explored. The notations in the cells refer to objectives of the Strategic Planning goals or topics related to them.
	Assessment Approach
<top of page	Institutional assessment, student learning outcomes assessment, and all the forms of assessment in between are part of the conduct of education at Cal Poly Pomona. Some people in the community regard them as an irritant or a chore, others as a progressive movement, others as a threat to freedom; most appear to have accepted them as the approach to educational management most comprehensible to legislatures, corporate partners, agencies, and foundations to whom we are beholden, a kind of lingua franca. On the occasion of the WASC Self-Study Workshop of January, 1999, in which five of us participated, we learned that the following assessment issues would be regarded as crucial to our successful evaluation of ourselves and representation of ourselves to the WASC team. For the most part, we find that we have made progress with all of these measures, to varying degrees. (more...)
	Process of Self-Study
<top of page	Format for Campus Participation In addition to collection of documents and other data, the Steering Committee carried out basic ethnographic research in the process of self-study. This consisted of frequent attendance at regular and occasional meetings of the most prominent committees and other structures of the university’s students, staff, faculty, and administration. We regarded ourselves as participant observers of campus life, collecting all sorts of first- and second-hand data to supplement our document collection. We arranged a variety of meetings, conferences, and interviews to help us organize and increase our data base. (more...)
	Campus Climate Survey
<top of page	Here we present parts of two documents developed during the survey process, whose preliminary results are offered in Theme One (Chapter V). The first, following, is from materials used in presentations on and off campus, subsequent to the administration of the survey and intended to acquaint the subject population and any interested parties with the purposes and methods of the survey. Our unstated hope was that, through an open educational campaign, we would assure people that our committee and the university would take the findings to heart and arrive at a course of action, where called for. The second is contained, except for a summary, in Annex 5 of this chapter. (more...)