This is an evaluation guide.
Pat Hutchings’ (2000) taxonomy of educational inquiry questions can be helpful in specifying the main purpose of your evaluation.
- “What works” questions are evaluative; they set out to prove effectiveness. One example is as a comparison between existing teaching modes in order to decide how to allocate resources. Another is how to most effectively teach a difficult concept to a large cohort. ‘What works’ questions might be formulated as, “Did students’ use of assessment feedback change after […]?” or “Was increasing the module structure associated with reduced attainment gaps?”
- “What is” questions focus on understanding the features or constituent parts of a given approach or intervention, but do not set out to prove effectiveness. Examples include observing the dynamics of classroom discussion about a difficult topic, understanding interdisciplinarity from a student point of view, understanding how students move from grasping others’ theories to developing their own. ‘What is’ questions might be formulated as “What use do my students make of assessment feedback?” or “What are my students’ study behaviours?”.
- “Possibilities” questions build on knowledge from “What is” questions, to concentrate on particular dimensions which support students towards their goals. One example is how students might come to view difficulties in their learning as opportunities rather than morale-destroying deficits. Another is how falling student engagement might be rekindled.
- “Theory building” questions build on “Possibilities” questions to formulate a new model conceptual framework about a given aspect of practice. Examples of frameworks emerging from this kind of work include Schommer’s epistemological dimensions of learning, Laurillard’s conversational learning framework, Kirkpatrick’s levels of evaluation, and Miller’s pyramid of clinical competence.
Cohen et al (2017, p336) recommend distilling general purposes into a specific central aim. “To explore students’ views about assessment feedback” (a ‘What is’ question) would be nebulous, whereas “To describe the factors motivating students to use assessment feedback in their future work” would be more specific.
Once you have that specific, central aim, the guides on questionnaires and interviews make suggestions for developing questions to ask students directly.
References
- Cohen, L., Manion, L., & Morrison, K. (2017). Research methods in education. 8th edition (8th Ed). London ; New York: Routledge.
- Hutchings, P., & Carnegie Foundation for the Advancement of Teaching (Eds.). (2000). Opening lines: Approaches to the scholarship of teaching and learning. Retrieved from https://eric.ed.gov/?id=ED449157
Image source: “Question mark” by chrisinplymouth is licensed under CC BY-NC-SA 2.0
Leave a Reply