One of the FLPEP staff (Yukiko Watanabe) was invited to International Christian University (ICU) in Tokyo, Japan as a visiting scholar for one week to deliver program evaluation workshops and presentations for language programs, writing center, and graduate seminars. Interested in what she presented and what she learned? Read further!
-----------------------------------------
Many ICU faculty and graduate students were enthusiastic to learn about program evaluation and accreditation practices in the United States. The one-week visiting scholar program was a very rewarding experience!
Below is a line up of presentations and workshops I did during my visiting scholar week.
- Program evaluation 101: Getting started workshop (2 language programs)
- Developing and evaluating a writing center
- Graduate seminar presentations
- College-driven initiatives on building evaluation culture
- A nation-wide evaluation needs and capacity survey study - One-day open-public program evaluation event
- A 3-hour workshop on utilization-focused program evaluation
- Panel presentations
- A discussion session
The two ICU language programs I separately interacted are both going through major program reform (one in the middle of it and one at the beginning of reform). I often see program innovation and evaluation go hand-in-hand. When there is an agreement to innovate a program, faculty are likely to be open to program changes on the basis of program evaluation. And most likely, if you innovate a program, you want to learn how effective the innovation is/was! Agreement to program change/innovation makes it easier to gain buy-in from program staff to take on an evaluation project when they see that evaluation is use-driven and is based on internal needs. So it was very timely that my visit coincided with their innovation efforts. Since one of the programs was at the beginning stage of program reform, integrating program evaluation from the get-go can provide empirical basis for making decisions on what to change, what to keep, and how to reform the program.
The workshop for these two programs aimed at building understandings on how to conduct situational analysis and how to focus on program evaluation use and questions. Of course, they both had their own unique programmatic issues, but what was interesting was that improving student learning outcomes and faculty collaboration were the two major evaluation themes that emerged out of the discussion. In higher education, student learning outcomes assessment is a common one (especially in the United States), but evaluating organizational collaboration is rare and is a challenge.
Learning from the literature on organizational management, the sine qua non of organizational collaboration is shared vision/purpose and opportunities for co-construction. Some of the evaluative questions on faculty collaboration that immediately come to my mind are: Do faculty have shared understanding of the program goals and student outcomes? Do faculty have sufficient time for fruitful discussion enabled by constructive meaning-making process? Are there any missing voices in decision-making process? How are knowledge shared and stored within and between teams?
Unfortunately, we didn't have time to prioritize evaluation questions during the workshop, but if they decide to evaluate faculty collaboration in the future, it will be very interesting.
One of the challenges for me putting together the workshop materials was to translate evaluation concepts from English to Japanese. Assessment and evaluation are both "hyouka" (評価、ひょうか) in Japanese, and we don't distinguish the two terms. My solution? For student learning outcomes assessment, I translated "assessment" as "調査" (ちょうさ、 "inquiry/study"）in order to avoid the common misconceptions of assessment with testing.
Here is a question to multi-lingual readers of this blog. How do other languages handle the two terms, "assessment" and "evaluation," and how are they translated?
