AES Awards for Excellence in Evaluation

Past Award Recipients

Evaluation Systems Award

2023

Kara Scally-Irvine and iPEN – Impact Planning & Evaluation Network for 'iPEN "turbocharging impact" - evaluation capability and capacity building programme'

Evaluator: Kara Scally-Irvine, KSI Consulting | Commissioners: Impact Planning and Evaluation Network (iPEN)
IPEN across the 7 Crown Research Institutes (CRIs) in NZ has been developed on a strong and broadly based theoretical approach. This, in turn, has been systematically applied to the operation of iPEN and the evaluation studies done within the iPEN institutions. There is strong evidence of proactive work with Maori, and broadly based, high stakeholder satisfaction. Strong and sustainable partnerships have been developed across multiple institutions but there is at present less evidence of the influence of the iPEN system on evaluation practice within the CRIs except in shared knowledge, language and skills. The iPEN has made very strategic and effective use of limited resources.

Mission Australia’s Centre for Evidence and Insights for 'Mission Australia: An integrated organisational approach to Monitoring, Evaluation and Learning'

This is a very comprehensive and well thought through MEL system, strongly based on current theory. The use of MEL is very appropriate for this organisation, which is spread across Australia in the very complex domain of homelessness. MEL is well integrated into Mission Australia's strategy and practice. It ticks all the boxes of a sustainable system which provides clear guidance but isn’t overly prescriptive. The MEL system is designed to work nationally but at the same time remain the organisation’s desire to focus on outcomes for the individual.

Public Sector Evaluation Award

2022
Te Ihuwaka | Education Evaluation Centre, Education Review Office – for Evaluation of learning in residential care The Education Review Office is the public service department of New Zealand charged with reviewing and publicly reporting on the quality of education and care of students in all New Zealand schools and early childhood services. This evaluation study looked at the quality of education for students in Oranga Tamariki Care and Protection and Youth Justice residences and how it can be improved. Children and young people who are placed in Oranga Tamariki residential care are among the most at risk of poor outcomes later in life. This was the first nationwide, system-level evaluation undertaken by the ERO. The study developed an excellent theory of change based on a published systems framework and used this effectively to develop the evaluation methodology which was robust and of a very high quality. The study developed a unique set of indicators of education performance for the residences, using a 4-point rubric to define the quality of practice, which will be useful in future evaluation studies and service delivery. The rubric was developed in consultation with the education providers to ensure their support for its use. Given the target population of school aged children, the evaluation team developed processes to give voice to these students including an appropriate reporting process. The evaluation team recognized the need to strongly engage stakeholders in the design and implementation of the study. Importantly, the team communicated regularly with the stakeholders on the study findings and developed a range of hard copies and online publications including the substantive report, a summary document, and four guides for students; parents and whanau (community members); leaders and teachers; and social workers. The evaluation team used a range of innovative approaches to enhance utilization of the study findings and products. As a result, there are a number of impacts that have already resulted from the study both in developing improved evaluation practice and decisions about the future of the residential care program.

EJA Publication Award

2023

Kathryn Erskine & Matt Healey for 'Conversations that count: Lessons from evaluating a men’s digital mental health response during COVID-19'

This article by Kathryn Erskine, Cube Group and Matt Healey, First Person Consulting, demonstrates flexibility and adaptation of evaluation approach and methodology when faced with challenging circumstances, in this case COVID-19 lockdowns.

It shows that evaluation re-design can be done in an adaptive, iterative and agile way whilst maintaining rigour and relevance through co-design approaches. The use of cleverly designed digital technology with an in-built evaluation data collection component enabled sequential check-ups in the in the evaluation process. The article is timely and provides excellent learning points for evaluators facing system disruptions, whether due to COVID-19 or for other reasons.

Emerging New Talent Award

2023

Mallory Notting, First Person Consulting

There is evidence of strong stakeholder satisfaction with Mallory’s work, and examples of projects she has worked on with sensitivity (e.g. in feedback and also the internal 360 deg process she manages). Mallory has demonstrated effective engagement with different evaluation approaches (eg impact, utilisation focused, developmental eval). Also evidence of evaluation leadership (supporting other staff members as a 'go to' colleague and contribution to eval - writing a chapter on impact eval for a pro bono project). In her short time as an evaluator she has worked on an impressive range of evaluations as a lead or in a key evaluation role.

Award for Enhancing the Social Good

2022

Centre for Health Service Development, Australian Health Services Research Institute, University of Wollongong – for Evaluation of the Pathways to Community Living Initiative

The evaluation makes a strong contribution to the social good by building up knowledge and support for the deinstitutionalisation process for those with severe and persistent mental illness (SPMI). The evaluation informed and supported the development of new service models for appropriate care in the community. It is noted that community-based settings aid recovery of those with SPMI, and historically limited support to this context has reinforced inequalities experienced by those with long-term mental health challenges. The overall quality of the evaluation was excellent. The team adopted a useful formative approach with a strong commitment to add value and contribute to the on-going refinement of the program. The evaluation team's focus on the significant role of clinicians and the need to build support and reflective practice within rehabilitation psychiatry are impressive. Similarly, theoretical and methodological attention to the drivers of transformation in a complex health system were important to identify and maintain focus on areas critical to change. Attention to including consumer and carer perspectives were pivotal in building a valid, credible, and useful evaluation.

Indigenous Evaluation Award

2022

Curijo Research Monitoring and Evaluation Team

This was an innovative submission for the AES Indigenous Evaluation Award. The submission was not for the conduct of a particular evaluation, nor for the commonly recognised types of evaluative thinking process documentation such as a monitoring and evaluation plan. This Award recognises the company’s holistic approach to developing its Research, Monitoring and Evaluation Framework and how it has shaped the organisational settings. We have awarded the nomination as an example of an excellent evaluation product that has been prepared and used by a privately-owned Indigenous Australian monitoring and evaluation company. The company is courageously embarking on influencing the qualities and competitiveness of the Indigenous evaluation market through a culturally specific research, monitoring and evaluation framework. It employs a model of economic independence to build upon the growing national commitment to improve the quality of Indigenous-related programs and services through better evaluation. Curijo’s commitment has translated into measured organisational staffing structures, staff training in philosophical and practical Indigenous methodological approaches, as well as provision of community support beyond the requirements of the commercial contract. The internal application of the model against the business is still developing but it is excellent to see an Indigenous research and evaluation company that has developed a holistic, cross cultural approach to centring Indigenous perspectives, priorities and knowledges. This company is making a significant contribution to promoting the accountability and learning functions of Indigenous evaluation, as a means to realising beneficial impacts and self-determination for the First Nations peoples of Australia.

Evaluation Study or Project Award

2018

Queensland Government’s Office of the Commonwealth Games, Department of Innovation, Tourism Industry Development, and the Commonwealth Games; and the evaluation team of Mark Douglas, Robert Grimshaw, Nicolette Pavlovski, Sean Conway, Kelly Reynolds, Joanne Ryan and Meghan Purcell – for the Evaluation and Monitoring Framework for the Embracing 2018 Legacy Program

The Queensland Government’s Embracing 2018 Legacy Program aims to ensure the Queensland community realised lasting benefits from hosting the 2018 Commonwealth Games. The reach of this event extends beyond the Games host and event cities to provide measurable outcomes for Queensland and Australia. An Evaluation and Monitoring framework for this program has evolved since 2013 and is designed to guide implementation of the Embracing 2018 Legacy Program and assess its outcomes over a 10-year period.

The assessors were impressed by the evaluation’s sound use of evaluation theory and approaches, including the ability to incorporate emergent findings into a results’ framework and ensure ongoing connections between projects. There was evidence of strong and sustainable connections being developed; for example, with the evaluation team building on work started in Glasgow with the 2014 Commonwealth Games, and their commitment to developing a framework that can be used by future hosts of the Commonwealth Games. The assessors noted the evaluation team’s commitment to publicly share the methods behind their framework.

Evaluation Policy and Systems Award

2019

The Victorian Department of Health and Human Services (DHHS) Centre for Evaluation and Research

The DHHS Centre for Evaluation and Research led the development and implementation of evaluation policy and systems in this large Victorian state government department. It played a pivotal role in transforming the evaluation culture of the department through policy guidance, advice and support, design and delivery, training and knowledge translation.

The Centre for Evaluation and Research demonstrated excellence in applying theory-based principles and approaches to developing and implementing evaluation policy and systems. The judges were impressed with the high-quality evaluation resources and pragmatic support that departmental staff have access to, including tools and tailored advice. The nomination included good evidence of stakeholder satisfaction with the team’s work. The judges were also impressed with the significant contribution of the team to transferring knowledge across the department and broader community and public sectors. Overall, the judges considered this Award winner as an exemplary case of evaluation policy and systems contributing to service delivery and improving community wellbeing, including among those most vulnerable to the risks of poorer outcomes.

Community Development Award

2012

Palmerston/ Tiwi Island Communities for Children (C4C) Participatory Evaluation conducted by the Communities for Children (C4C) Local Committee, Pandanus Evaluation & Planning Services (Nea Harrison) and the Australian Red Cross (Rachel Dunne)

The Award judges noted that the work was well-thought through, extremely thorough and comprehensive. A local committee, the elders and young women were all actively involved in the project in a strong collaboration with the facilitator of the evaluation. The nomination indicated an understanding of how an evaluation design contributes to community development goals and how the use of community development processes enables the accomplishment of the evaluation.

Areas that stand out as exemplary and examples of good evaluative practice in this evaluation include:

  • culturally appropriate evaluation design and methodology
  • strong community engagement at all stages of the evaluation from inception to conclusion, including closure and reporting back to the community
  • a developmental approach working sensitively and in sympathy with the local community
  • outcomes resulting in sustainable benefits for local participants (e.g. capacity building)
  • use of advanced techniques –logic framework, quality rubrics, the community report.

Evaluation Publication Award (Caulley Tulloch Award)

2015

Samantha Abbato – for 'The case for evaluating process and worth: evaluation of a programme for carers and people with dementia'

This is a book chapter published in the most recent volume of the prestigious series Advances in Program Evaluation, edited by Trisha Greenhalgh & Saville Kushner. Its thesis is the utility of the case study approach as a major component of a mixed-methods evaluation. What makes the chapter worthy of the award is the author's careful analysis and demonstration of the role of case studies in mixed methods evaluations, including the way she contrasts them with the quantitative methods more frequently used in the sector covered by the project (community health/dementia). This chapter will be particularly valuable for people new to evaluation, or to those coming from a quantitative background who wish to gain an understanding of the role of case study research in evaluation. The nomination explicitly and convincingly addresses the specified filters and the seven specified criteria of excellence.

Outstanding Contribution to Evaluation Award

2018

Jess Dart

Jess is a recognised leader in evaluation with over 25 years of experience in the collaborative design and evaluation of programs that seek to bring about a more equitable and just society. The judges were impressed by this nominee’s sustained application of authentic inclusivity, and her high level of ethical standards so clearly evidenced through her practice. The judges noted this nominee’s skill in being able to combine deep evaluation knowledge and theoretical understandings with straightforward communication. She has undertaken more than 30 external evaluations and overseen over 120 evaluations in countries around the world. The judges noted the high level of professionalism and stakeholder satisfaction evidenced in this nominee’s work. As an ‘evaluation entrepreneur’, the judges were impressed by this nominee’s capacity to constantly scan the horizon for where evaluation is headed, and to create fresh approaches and techniques. The judges were impressed by her pioneering work on no less than five innovative approaches.

The judges were impressed by this nominee’s other contributions to evaluation knowledge. She is a contributor to evaluation textbooks, an author of refereed journal articles including publications in the American Evaluation Journal and New Directions in Evaluation, and a prolific trainer who has reached more than 1,000 participants, including many AES members.

Jess has been an AES member since 1997 and a Board member since 2014 in the role of treasurer; and was involved in the aes18 International Evaluation Conference as convenor. The judges acknowledged this nominee’s significant contribution to the AES.

ABOUT US   |    CONTACT US   |    LEGALS
Search site
© 2023 Australian Evaluation Society Limited 
ABN 13 886 280 969 ACN 606 044 624
Address: 425 Smith Street, Fitzroy, Victoria, 3065, Australia

We acknowledge the Australian Aboriginal and Torres Strait Islander peoples of this nation. We acknowledge the Traditional Custodians of the lands in which we conduct our business. We pay our respects to ancestors and Elders, past and present. We are committed to honouring Australian Aboriginal and Torres Strait Islander peoples’ unique cultural and spiritual relationships to the land, waters and seas and their rich contribution to society.