Before you Begin
Page Contents
The Why and the How
When asked about assessment, many professionals begin by speaking of surveys and questionnaires. I will take a different approach, one that integrates both the design and the implementation of a comprehensive assessment program. What I mean by this is that excellence in assessment, as in student affairs practice, involves as much attention to answering the "why" of what we do (design) as to the how (implementation). Indeed, as Nietzsche once said, "He (or she) who has a why to live can bear with almost any how." Assessment is similar - by focusing more attention to the why of student affairs work, the how of putting it into practice and the how of assessing student learning become more natural processes.
Terminology
My goal is to make this web guide as practitioner-friendly as possible, so I will not make extensive use of assessment jargon. That said, there are a few terms that are important to define up front.
Learning
- Learning Reconsidered defined learning as "a comprehensive, holistic, transformative activity that integrates academic learning and student development, processes that have often been considered separate, and even independent of each other."
This important shift in the definition of learning has key implications for how we understand and use the following terms. While there is no universal consensus for the definitions, I suggest the following:
Evaluation and Assessment
- Assessment is a subset of broader evaluation practices that focuses specifically on student learning as defined by Learning Reconsidered. Thus, assessment has to do with the organized collection of data in order to measure and document student learning (academic learning and student development).
- Evaluation is the broader of the two concepts and has to do with the organized collection of data to measure program, departmental, or institutional effectiveness. While student learning is an important component of an evaluation process, evaluation efforts may also consider factors such as student satisfaction, retention and graduation rates, staffing and budgetary ratios, etc.
Goals, Learning Objectives, and Learning Outcomes
- Goals, as I use the term, are broad areas of strategic emphasis (e.g. "learning," "diversity," "civic engagement," etc.). At many campuses, you might call these "strategic initiatives" as they often fit better with strategic planning efforts than with assessment or evaluation.
- Student learning objectives are general statements of what students should be able to know or do as a result of engagement in broadly defined programs, departmental services, or institutional experiences (e.g. counseling, residence life, EOP&S, new student orientation, etc.).
- Student learning outcomes are specific statements of what students will be able to know and do as a result of engagement in a narrowly defined programmatic intervention (e.g. a workshop, a counseling session, an internship, etc.).
Goals, Objectives, & Outcomes - An Example
A university or department might first identify the four goal areas of diversity, learning, leadership, and citizenship through its strategic planning efforts. While these are quite broad, the institution or department might reasonably assume that this set of four goals distinguishes it from other regional institutions or departments.
The department of student affairs might next expand upon each of these, developing 3-5 learning objectives for each. In this case, let's say the department student services expanded upon the diversity goal by writing the following four learning objectives.
- Students will gain a deeper understanding of their own culture.
- Students will develop a rich appreciation of the various cultures represented on campus.
- Students will develop an understanding of privilege and oppression.
- Students will, at a minimum, embrace tolerance and, at the ideal, promote the value of diversity.
Finally, student affairs professionals, when designing learning interventions, will write learning outcomes that put the learning objectives to practice in clear and measurable ways. For example, in a diversity training exercise, a student affairs professional might ask a group of students to each develop a personal "coat of arms" by drawing pictures and writing words on a shield that they will later share with others in the group. Each picture or word should represent a cultural connection that is important to that student. Later in the activity, the professional might ask students to separate their cultural connections into groups that are "very important," "somewhat important," and "not very important" to their sense of identity. The practitioner might also ask students to share their coats of arms with other students, identifying cultural connections that are unique to smaller sets of students from the larger group, and into those that are shared by most of the students in the group. Learning outcomes for this activity might include the following:
- Students will be able to identify at least 10 cultural connections that have shaped their sense of identity.
- Students will assess the degree to which various cultural connections have influenced their sense of identity.
- Students will be able to list at least two cultural connections that are unique to no more than 25% of the participants.
- Students will be able to list at least two cultural connections that are shared by at least 50% of the participants.
Note that it is this last group of learning outcomes that is most directly measurable. Affective Objectives and Outcomes
In addition to what students should be able to know and do, many professionals will add affective objectives or outcomes for what students should think or feel (e.g. learning objective #2 in the example above is affective, while learning objectives 1, 3, and 4 are cognitive or behavioral). While such affective objectives clearly have value and should be a part of more advanced assessment efforts, they are more difficult to measure and require a high degree of political sensitivity. Thus, I suggest that practitioners creating new assessment efforts begin by focusing on cognitive and behavioral learning objectives and outcomes, and later integrate affective learning objectives.
The Assessment Team
When beginning the assessment process, choose the members of the assessment team carefully. While the process should be as inclusive as possible in its implementation, identifying a small, select group of individuals to lead the effort is wise. Consider including the following:
- A supportive member of the faculty -- someone who values the work of student affairs and who can be an effective bridge to other members of the faculty.
- A member of the student affairs staff -- I suggest someone 1.) who has strong relationships with other members and is able to sell the value of the process, and 2.) who is capable of successfully implementing efforts in her or his department early on, thus creating important and symbolic early victories.
- A student -- this should be obvious, but I have observed few assessment teams that include a student, and even fewer that effectively use students.
- A graduate intern -- if you have access to a regional graduate preparation program for student affairs or higher education professionals, a graduate intern can be a great asset. She or he should have an academic background in assessment and may be able to integrate the work of the internship into class assignments, creating a nice win-win situation. This might also become a nice incentive for someone to do some of the grunt work necessary early in the process.
- Someone who works in the fund raising department of the institution -- this individual may help in two important areas: 1.) early in the process, this person should have some expertise in collecting, managing, and using data to demonstrate results, and 2.) later in the process, this person may have suggestions for using the data in grant-writing or fund raising efforts for student life.
Political Considerations
There are a broad set of reasons why various individuals may or may not want to engage in an assessment program. Some fear the workload, others fear the results or how results may be used against them. Still others see assessment as a means of advancing an agenda. None of these fit well with the true purpose of assessment - enhancing student learning. For this reason, it is important to discuss the purpose of the assessment effort up front including how the results will and will not be used. For example, consider the following:
- First, you absolutely must use the results of assessment efforts to inform future decision-making processes. Practitioners will become discouraged if they feel that politics (coalitions, favorites, personal agendas, etc.) somehow trump the results of assessment efforts in future decision-making efforts.
- Second, plan to use assessment activity rather than assessment results in personnel evaluations. Staff members who conduct assessments, find significant gaps in student learning, and then begin to develop new strategies to meet those learning needs should be rewarded for their activity, not punished for lack of results.
- Finally, if at all possible, develop a "growth fund" in the department's budget. One surefire means to kill or stigmatize an assessment process is to use it to create winners and losers. Assume, for a moment, that two departments conduct assessments and discover that students in one department are learning and succeeding while students in the other department are not. Some might penalize the unsuccessful program and invest more dollars in the one that is successful . Others might reallocate dollars from the successful program to the unsuccessful one. Both of these actions create winners and losers. A growth fund, on the other hand, might be an appropriate source to either provide needed support for unsuccessful programs or to build on the work of highly successful programs without taking money away from anyone.
|
|
Contents
|