“Asking a computer to combine facts and values with evidence and standards to make a judgment or determine value... it's just not there yet.”
Foxes, Hedgehogs and AI in Evaluation
Bianca Montrosse-Moorhead · Professor of Research Methods, Measurement, and Evaluation, University of Connecticut
About this episode
What does it mean to be an evaluator, and how do we think about our professional identity in a rapidly changing field? In this episode, Dr Bianca Montrosse-Moorhead helps us look beneath the surface of evaluation practice. We explore the classic fox and hedgehog metaphor and what it reveals about how evaluators operate, why our tendencies matter, and how artificial intelligence is reshaping the judgments we make and the competencies we need.
Key takeaways
The fox and hedgehog metaphor offers a lens for evaluator identity
Hedgehogs burrow deep into one area while foxes move across contexts and bring many things together. Evaluation thrives when we have both orientations, and recognising your tendency can help you understand friction or fit in your work.
AI will not make evaluators redundant any time soon
Humans remain essential for interpretation, evaluative reasoning, and making judgments that combine facts, values, evidence and standards. The technology is not yet capable of the interpretive and values-based work that sits at the heart of evaluation.
New competencies are required for working with AI
Evaluators need to develop skills in prompt engineering, understand when AI is fit for purpose, and navigate new ethical considerations around consent, transparency, and participant data.
Simple language gets us to the heart of the work
Evaluation loves to name and coin new things, but the definition should be fit for context. Sometimes "helping people understand the value of what they're doing" is more powerful than academic terminology.
AI introduces ethical considerations beyond the obvious
These include environmental impact from energy and water use, bias amplification from training data, and the question of whether participants should consent to AI being used on their data, even when commissioners approve it.
Topics covered
- Evaluator identity
- Fox and hedgehog metaphor
- AI in evaluation
- Evaluative reasoning
- Evaluation competencies
- Professional development
- Ethics