Welcome to the AES Blog
The ten success factors for building evaluation capabilities in the public sector
The appetite for evaluation capacity building is growing but faces challenges
- making the case for long-term investment in an environment of heightening fiscal pressures
- balancing quick wins that demonstrate the value of evaluations with laying the foundation for long-term success
- competition when recruiting for specialist evaluation skills in a tight labour market.
Meeting these challenges can require some difficult strategic decisions and deft management.
There is a spectrum of evaluation operating models
A fundamental decision that agencies need to make is on the size and operating model for its evaluation capabilities. Government agencies that commission and conduct regular evaluations typically employ one of three operating models:
There are several benefits to this model. A centralised evaluation function can:
- provide economies of scale relative to having discrete evaluation staff located across an agency
- more easily drive capability development and the application of consistent evaluation practice across an agency
- be more easily located alongside complementary functions, such as economic analysis, strategic policy, data and analytics, and performance measurement to increase the likelihood of value-adding cross-pollination
- provide an additional level of independence by being separated from policy development and program delivery, which supports the credibility and objectivity of the evaluations
- establish an identity and brand that can be used to drive change internally and attract external talent to the agency.
The challenge in implementing a centralised model is to retain strong links and knowledge of activity area to ensure evaluation is practical and valuable, from the front-line to strategic decision-making.
There are ten success factors
– PLANNING (first three months)
- Get early agreement on the why. Engage early with your leadership team to clarify their expectations and get shared agreement on the business case for building evaluation capabilities.
- Be clear about the trade-offs. In a resource-constrained environment, you need to understand what you want your evaluation function to focus on. For example, do you want broad coverage across all your activities and deeper coverage of a few? Do you want to conduct more frequent evaluations of program delivery and quality, or do you want them to be less frequent and explore in-depth what outcomes are occurring and why?
- Get the governance right. Establish internal governance arrangements that will ensure the evaluation unit's work is viewed as independent, credible and useful – by staff, ministers, program participants and the general public. This governance includes getting regular executive engagement to determine which projects the evaluation function should focus on, to increase the likelihood of sustained support from senior leaders.
– BUILDING (first six months)
- Have a clear strategy. Establish an agency-wide strategy that establishes clear and consistent expectations about how, when and why evaluations are conducted.
- Tailor the value proposition. Understand the diverse needs of business areas, tailor intellectual property and capability development to the needs of different parts of your agency, and strive to continually demonstrate the value of evaluations and evaluative thinking to staff across the agency.
- Identify champions in the senior executive. Leverage the knowledge and enthusiasm of senior leaders who understand the value of evaluations to champion the function across the agency. Identifying multiple champions is important given the average turnover in most agencies.
– EMBEDDING (first twelve months)
- Develop accessible evaluation guidance. Develop easy-to-use templates and guidance materials that empower staff to build evaluative thinking into their business areas.
- Communicate and demonstrate value. Clearly communicate the evaluation function's value and how much the evaluation process can contribute to agency-wide decision-making.
- Engage early. Engage with policy and program areas during the early stages of policy development or program implementation to ensure they are building monitoring and evaluation into their delivery plans.
- Communicate and clearly link the findings of evaluations. Make evaluation findings easily accessible across the agency, and, ideally, publicly. Get agreement during evaluation planning about the timing and nature of the decisions that evaluation findings will inform. It is also important to consider the implementation of recommendations and anticipate (where possible) the accountability, priority and levels of effort associated with each recommendation.
Agencies need to source the key skills