These choices can create or reinforce advantages for some people, organisations and sectors, while disadvantaging others.

It made me think about how we approach evaluation. Many of the programs we evaluate are designed to address disadvantage or respond to an identified problem. Understandably, our evaluation questions often follow that framing. We ask whether a program has improved outcomes for disadvantaged groups, reduced barriers or addressed needs.

But I wonder whether we sometimes inherit the assumptions behind a program without questioning them. When we focus on disadvantage, we can end up concentrating on what people or communities lack, rather than examining the systems and structures that have contributed to their circumstances. We might ask whether a program has helped people overcome barriers, without asking why those barriers exist or whose interests are served by the existing arrangements.

A program might achieve its intended outcomes while leaving the broader conditions that contribute to disadvantage largely unchanged. It might also benefit its intended participants but create or reinforce disadvantage for others who are not the focus of the evaluation. We might conclude that it has been successful, without necessarily asking whether it has challenged or reinforced the arrangements that create advantage for some and disadvantage for others.

The Privilege Toolkit encourages us to look at these questions from a different angle. Rather than focusing only on who is disadvantaged, it asks us to consider how advantage is created and maintained. Who has access to resources and opportunities? Who holds decision-making power? Whose knowledge and contributions are valued? And who carries the costs and risks?

I particularly like the toolkit's attention to what is absent. In evaluation, we spend time much of our time examining what a program is intended to achieve. But it's also important to ask what was left out of the original problem definition, or which alternative solutions were never considered?

This doesn't mean every evaluation needs to examine the entire policy system. But it does suggest that we could sometimes ask different questions about the programs we evaluate.

For example, alongside our usual questions about effectiveness, outcomes and impact, what if we included a key evaluation question such as:

How do the choices and assumptions underpinning the program shape who benefits, who holds influence and whose interests are overlooked?

This isn't entirely new territory for evaluation. For example, equity-focused, feminist and transformative approaches have long encouraged us to examine power, assumptions and structural inequality. But I like the way the Privilege Toolkit brings these different dimensions together and offers a practical lens for making them visible.

Perhaps the value of the Privilege Toolkit for evaluators isn't necessarily in applying its five steps, but in encouraging us to ask different questions. Not just whether a program works, but how the choices behind it shape who benefits and whose interests are served. 

This post was written by Carina Calzoni, AES Policy Officer