by Florent Gomez
Have you ever tried to grow evaluation capacity across your organisation? And this, with very limited resources?
At the recent AES International Evaluation Conference in Sydney, I shared some learnings from our successful Evaluation Community of Practice in the NSW Department of Customer Service (previously NSW Department of Finance) and other soft approaches to evaluation capacity building we are using in our department.
When I started as an internal evaluator with the department in February 2017, I quickly realised that contrary to other government departments, such as in community services, health or education, we didn’t have an established central evaluation unit; only pockets of evaluation capacity here and there. However, there was definitely a need and appetite to learn more about evaluation across the department!
This is why we decided to put together an Evaluation Community of Practice at the department level. The Community of Practice is an open and informal forum allowing staff from different roles and with varying levels of evaluation experience to share and learn about good evaluation practices. Quarterly events are the main component, supported by a Yammer group (corporate social media platform) that we use as a blog to keep the community engaged between events, and an Intranet page with key resources, templates and presentations from the events.
After one-and-a-half years of existence, we decided to evaluate ourselves and see how we were travelling in terms of building evaluation capacity across the department. We refined our intended outcomes by developing a detailed program logic. The evidence we gathered for the different outcome levels showed that this low-cost approach effectively contributed to raising evaluation awareness and capability across the department. Our evaluation showed that:
- A wide range of people participate in the community and continue to do so (an average of 56 participants attend each event and there are 127 members of the Yammer group).
- Participants learn some good evaluation practices and report applying these back in their work place (42% already and a further 56% probably will).
- There is also some evidence that the quality of evaluation deliverables produced within the department increased with, for instance, a more frequent use of program logic.
- For the first time, the Department was represented at the AES conference!
As one key stakeholder summed up: the Evaluation Community of Practice collectively played the role of the evaluation centre of excellence the department didn’t have.
The following key success factors we identified could be applied to other organisations facing similar challenges:
- Keeping this community open and informal increases people’s confidence and contributes to reducing ‘imposter syndrome’ – this is, I believe, a core mechanism that made it work in our particular context.
- Leadership support and licence to innovate – this is critical in the context of limited resources. In our case, we had to streamline processes and constantly innovate to deliver successful events within existing resources.
- Having a rotating chair – helps to reach out to all parts of the organisation and also shares the responsibility for the organisation of the events
- Having a few staff dedicated to coordinating the community on top of their daily priorities – helps ensure continuity and efficient delivery of the events, although this is a risk factor at the same time because of the reliance on a few individuals.
This is definitely a low-cost evaluation capacity building approach I’d recommend, in particular in organisations with less established evaluation capacity. What tips can you share from successful approaches to evaluation capacity building in your organisation?
Florent is a Manager for planning, evaluation and reporting at the NSW Department of Customer Service. Before that, he worked as an external evaluator for over 10 years in Europe then Australia.