UV9917V4 – Micro-Analytic Methods in Large-Scale Educational Assessment
Course content
This workshop will introduce ‘micro-analytic’ methods in large-scale educational assessments, their application in testing and underlying theory. The focus of the workshop will be on the use of micro-analytic methods in international large-scale assessments (ILSAs). However the workshop will also be of value to those students who are interested in other large-scale assessments – including both paper-based and computer-based modes.
Learning outcome
The workshop will introduce the students empirical data on small-scale observations of assessment practice, including the study of verbal and non-verbal interaction (including body posture facial expression) obtained in real-life assessment events. This will be informed theoretically by studies of interaction and ecology of testing situations (drawing from micro-ethnographic traditions in anthropology and Conversation Analysis, and ecological theories of assessment).
The workshop will introduce participants to how such methods can be complement and combine with other sources of data – including computer-generated log files and psychometric data on validity. The teaching methods will be participative – including short lecture presentations, the use of animations, and discussion of research papers. The participants will be given the opportunity for hands-on analysis and practice of micro-analytic data including video-observation and ethnographic transcription.
The objective of the workshop will be to introduce students to micro-analytic methods, to understand how they are produced, and how they can be integrated into other data on assessment performance and validity.
- The learning outcomes will include;
- Knowledge about the significance of interaction and affect in large-scale assessments.
- Knowledge about the impact of ecological settings on assessment performance and its relevance for understanding of assessment validity.
- Understanding about the strengths and limitations of computer-generated log files and large-scale psychometric data.
- Skills in observation and analysis of small-scale interactions and assessment - including transcription, data-coding and data analysis.
- Knowledge and skills about how micro-analytic data can be combined with other data sources.
- Knowledge and skills in the conduct of micro-ethnographic observation.
Admission
Participation is free of charge and is open to all, with a maximum of 20 participants. Depending on participation numbers, we might need to disappoint some applicants as priority will be given to students and participants applying from the Nordic region.
Ph.d.-students from the University of Oslo apply through Studentweb. Other apply though Nettskjema.
Registration deadline: June 5, 2017
Teaching
Organizer: CEMO (Centre for Educational Measurement at University of Oslo)
Responsible: Professor David Rutkowski
Guest Lecturer: Bryan Maddox, School of International Development, University of East Anglia. NR1 7TJ. UK. B.Maddox@uea.ac.uk
Dates: June 12th -13th
Time: 09.00-16.00 both days
Location: Sophus Bugge, seminarrom 5 (123)
Course group: Qualitative Methodology
Course credits: 3 ECTS with documentation, 1 ECTS without documentation
Participants: maximum 20
Readings:
- Goodwin, C. (2007) ‘Environmentally coupled gestures’. In S.Duncan J.Cassell, and E.Levy (eds) Gesture and the dynamic dimension of language. Amsterdam & Philadelphia, John Benjamin’s publishing. pp 195 – 212.
- Goodwin, M., Cakaite, A. and Goodwin, C. (2012). ‘Emotion as Stance’ Chapter 2 in M.Leena Sorjonen and A. Perakyla (Eds) Emotion in Interaction. Oxford, Oxford University Press. pp. 16-41.
- Maddox, B. (2014). ‘Globalising Assessment: an ethnography of literacy assessment, camels and fast food in the Mongolian Gobi’. Comparative Education. 50 (4), 474-489.
- Maddox, B. (2015) ‘The neglected situation: assessment performance and interaction in context’. Assessment in Education.
- Maddox, B. Zumbo, B., Tay-Lim, B., and Qu, D. (2015). ‘An Anthropologist among the Psychometricians: Assessment Events, Ethnography, and Differential Item Functioning in the Mongolian Gobi’. International Journal of Testing.
- Maddox, B. and Zumbo, B.D. (in Press). Observing Testing Situations: Validity as Jazz. In B.D. Zumbo and A.M. Hubley (Eds.), Understanding and Investigating Response Processes in Validation Research. Springer Press. (copies will be provided).
- McNamara, T. F. (1997). “Interaction” in second language performance assessment: Whose performance? Applied Linguistics, 18(4), 446–466.
A list with optional further reading will also be provided.