AES Awards

It was clear that the evaluation made a strong contribution to capacity building in Timor Leste. The project is an exemplar of a community development evaluation that other evaluators could use to inform and improve their own practice.
The evaluator worked closely with the local communities using methods that were respectful, collaborative and inclusive. The methods were derived from leading edge Appreciative Inquiry approaches and Participatory Rural Appraisal techniques. Multiple data sources were used and brought together in an excellent example of triangulation. The report was clear and concise. The limitations of the evaluation were clearly articulated as caveats early in the report. The evaluation demonstrated a transparent process, with all of the conclusions drawn firmly from the available evidence.

The Best Evaluation Study Award for 2011 was awarded for the Evaluation of the Victorian Bushfire Case Management Service,
sponsored by Department of Human Services, Victoria and conducted by a team from Urbis including; Claire Grealy, Duncan Rintoul, Jessie Connell, Ginette Anile, Christine Healy, Dr Gail Winkworth & Dr Kristen Murray.

The nomination was for an evaluation of a service provided in an emergency. The evaluation was an example of the highest quality of evaluation practice in its use of, and application of, evaluation theory, principles, methods and practice.The evaluation was conducted while the service was in operation, requiring high sensitivity, confidentiality and speed. The evaluation design was innovative, and well-crafted using rapid response principles well adapted to the needs of the program. Multiple methods and data sources were used to provide triangulation and to test the data. The quality assurance mechanisms were carefully considered and developed to support the evaluation’s objectives. The evaluation reports and literature review have made a significant contribution to the evaluation of crisis case management. The evaluation developed models to improve evaluation practice and scholarship and added to the evidence base of what works in emergency case management. The recommend-ations in the three reports were immediately applicable and operationalised with minimal burden to the large workforce.

The AES Awards for Excellence in Evaluation support the objects of the AES and contribute to the Strategic Directions of the AES. The awards recognise exemplary evaluation practice, evaluation systems or evaluation capacity building in Australasia (Australia, New Zealand, Papua New Guinea and Pacifica). Awarded annually, the awards provide significant peer recognition for leading evaluators, leading evaluations and evaluation best practice. The award winners represent best-in-class for each Award category.

The AES encourages all Award winners to play an active role in promoting excellence in evaluation. Award winners will be asked to consider ways in which their knowledge and experience may be shared with others.

The call for nominations for the 2012 AES Awards for Excellence in Evaluation will be announced soon

The Awards for Excellence in Evaluation are announced and presented at the AES International Conference each year. All nominees for awards are invited to attend the conference or nominate someone to attend on their behalf.

Details of the recipients appear in the Evaluation Journal of Australia and are available on the AES web site (see below).

To recognise, promote and encourage exemplary evaluations, evaluators and evaluation practice in Australasia which adhere to the following principles:

The principles of professionalism and ethical conduct mainly apply to the role of the evaluator and/or the evaluation team. The principle of excellence is mainly concerned with evaluation practice and products.

Awards Assessment Criteria

Nominations will be assessed against the objectives of the category for which they are nominated and the assessment criteria that apply to that award.

The assessment criteria for ‘professionalism’ and ‘ethical conduct’ are defined as:

professionalism – demonstrated quality and adherence to professional standards especially in dealing with challenges presented in their work;
ethical conduct – demonstrated high standards of ethical conduct as individuals (including a demonstrated understanding of the vulnerability or disempowerment of disadvantaged groups) and adherence to the AES Guidelines for the Ethical Conduct of Evaluations and the AES Code of Ethics.

Excellence in evaluation is assessed using some or all of the six assessment criteria outlined below:

quality – of the evaluation or in the application of evaluation theory, principles, methods and practice – this involves appropriateness, fitness for purpose and technical quality;
leading edge – clearly an example of innovation and good practice in this class of evaluation (the specific award category);
contribution to knowledge – to improving/refining one or more of evaluation theory, scholarship, principles, methods and/or evaluation practice;
effectiveness – in achieving client/project objectives, in its response to context, in partnering and/or cross training with the client/s or in making a difference/delivering better client outcomes through the application of evaluation theory, principles, methods and practice;
equitable – inclusiveness through the advancement of the prospects, demonstrating fairness, social justice, equity and involvement of key stakeholders, including the effective engagement of special interest groups (culturally and linguistically diverse, Indigenous, youth, etc.); and
sustainable – providing continuity and sustainability of evaluation and/or the integration of evaluation as an ongoing component of client, agency or sectoral policy, programs, projects or services.

Eligibility

A nominee must have been a member of the AES for a minimum period of 12 months prior to the nomination closing date:

except that a nominee for the Emerging New Talent Award must be a member at the time of nomination but is exempt from the 12-month minimum membership requirement.
in the case of a group nomination, at least one nominee must meet the 12-month minimum membership requirement.

A completed nomination form - nominations must be made on the nomination form.

The nomination itself - addressing each of the assessment criteria for an award, including:

a summary statement about the contribution the nominated work or individual has made to the objectives of the category.
a statement addressing each of the assessment criteria for an award. All criteria must be addressed.
claims or statements made must be substantiated with evidence. Evidence can be provided in attachments to the nomination but must be clearly referenced in the nomination criteria.
This document is limited to 5, 7 or 10 pages (as variously stipulated below for the different awards).

A nomination may also include supporting documents as a clearly-marked attachment or appendix (these are optional) as follows:

Evidence to support the nomination. This could be a report, print outs of web pages, support / reference letters, media stories/press releases. The evidence should be relevant to the nomination. For example, if the nomination focuses on the quality of methods, then the evidence provided should be about the methods used and how they their effectiveness.
The Award judges are asked to read large volumes of material. Nominators are asked to consider carefully what evidence is pertinent to the award and limit the material provided as much as possible

There are seven AES Awards for Excellence in Evaluation which may be awarded each year. They are divided into two main categories:

Five awards that recognise organisations, project teams, evaluation projects or programs (nominated by the evaluators themselves)

This award recognises emerging new evaluators who have demonstrated a significant contribution to the field of evaluation, or to a major evaluation project in the short period of time they have been working in the evaluation area. Evidence should be provided about the specific contributions made by the nominee.

Assessment Criteria

quality
leading edge
contribution to knowledge
effectiveness
professionalism
ethical conduct

Previous Winners

2007 - Bradley Shrimpton
2009 - Jessica Kenway
2010 - James Curtis

Outstanding Contribution to Evaluation (ET&S) Award
(Nominations must not exceed 10 pages. Relevant attachments may be included.)

AES member, Anona Armstrong, initiated this award in the name of her company ‘Evaluation Training and Services.’ The award recognises an AES member for his/her outstanding career contribution to the objectives of the AES, to evaluation theory and to the evaluation profession in Australasia.

Assessment Criteria

quality
leading edge
contribution to knowledge
equitable
sustainable
professionalism
ethical conduct

Previous winners

Stephen Kemmis
Bryan Lenne
Jane Hall
Jerome Winston
Colin Sharp
Darrel Caulley
Dr John Owen
Rosemary Ince
Yoland Wadsworth
Ian Trotman
Assoc Prof Ralph Straton
Sue Funnell
Anthea Rutter and Dr Zita Unger
Dr Patricia Rogers
Maggie Jacob-Hoff
Assoc Prof Rosalind Hurworth
Penny Hawkins
Professor Rick Cummings

Best Evaluation Publication (Caulley Tulloch) Award
(Nominations must not exceed 5 pages)

Initiated by AES member Darrel Caulley this award recognises the best journal article, conference paper, research paper, undergraduate or postgraduate thesis or other peer reviewed publication in evaluation.

Assessment Criteria

quality
leading edge
contribution to knowledge
effectiveness
equitable
sustainable
professionalism
ethical conduct

Previous winners

1995 - Yoland Wadsworth, Merinda Epstein & Maggie McGuiness
1996 - Patricia Rogers & Gary Hough
2001 - Scott Bayley
2003 - Bron McDonald, Patricia Rogers and Bruce Kefford
2005 - Glenys Jones and her team, Parks and Wildlife Service, Tasmania
2009 - Coffey International Development, Jennifer Rush and AusAID

Best Evaluation Study Award
(Nominations must not exceed 7 pages. Relevant attachments must include an Executive Summary.)

This award recognises an individual or team that has conducted a specific evaluation study which has made or has the potential to make, a significant contribution to the practice or use of evaluation in Australasia.

Assessment Criteria

quality
leading edge
contribution to knowledge
effectiveness
equitable
sustainable
professionalism
ethical conduct

Previous winners

1994 - Evaluating Taxpayer Audit Program of the New Zealand Department of Inland Revenue (Team lead by Alan Pinder)

1996 - Technical Quality Evaluation Team, N Z Department of Inland Revenue (Team included Prue Oxley, Heather Turner, Valmai Copeland, Fiona Hoult, Colin Usherwood and Robyn Pullar)

1997 - Team from Australian Bureau of Transport and Communications Economics (Team included Joe Motha, Bogey Musldlak, Seu Cheng, Catarina Williams)

1998 – two awards - Team from NZ Ministry of Justice (Team included Alison Chetwin, Steve Dunstan, Miriama Scott, Jennifer Leigh, Mark McCallum) and
Peter Bycroft and Ellen Vasilauskas, Corporate Diagnostics Pty Ltd

1999 - Pamela Williams, KPMG

2000 - Robert Curnow, Community Changes

2001 - Dr Rick Cummings, Murdoch University and Kath Stephenson, Estill and Associates

2002 - ARTD Consultants (Chris Milne, Marie Delaney, Klas Johansson and Marita Merlene)

2003 - Review and Evaluation Unit of the Queensland Police Service (Team included Dr Robert Lake, Angela Richardson, Dr Diana Beere, Ruth Beach and Joe Nucifora)

2005 - Wendy Searle, Tania Slater, Trish Knaggs, Janet November and Christopher Clark, Ministry of Justice, New Zealand

2006 - Virginia Lum Mow, VL Educational Research and Development; and Retraining Unit, NSW Department of Education and Training

2007 - The Consortium for the Strategy 2000-2004 Evaluation at CIRCLE, RMIT (Team included Associate Professor Patricia Rogers, Ms Sue Funnell, Mr John Scougall, Ms Keryn Hassall, Mr Peter Tyler, Associate Professor Gerald Elsworth, Ms Sue Kimberley, and Ms Kaye Stevens)

2008 - Simon Smith, Julie McGeary, the Victorian Department of Primary Industries & Dr. Martin Andrew, Lili Pechey, Dr. Don Burnside, Dr. Todd Richie from URS Australia

2009 - Rosemarie Tweedie, Mary Carey, Kim Stewart and the Baptist Community Services (NSW and ACT)

2010 - Mathea Roorda, Heather Nunns, Dr Ieti Lima, Amton Mwarksurmes, Senorita Laukau, Kateata Binoka, Talonga Pita, Alison Gray, Charlotte Bedford for Evalue Research and Sankar Ramasamy, Department of Labour, Wellington

2011 - Claire Grealy, Duncan Rintoul, Jessie Connell, Ginette Anile, Christine Healy, Dr Gail Winkworth & Dr Krsten Murray from Urbis

Best Evaluation Policy and Systems Award
(Nominations must not exceed 10 pages. Relevant attachments must include a two page summary of the policy/system/ framework/methodology.)

This award recognises the work of an individual or team in developing the best evaluation policy, integrated evaluation system, evaluation program, framework and/or implementation of evaluation policies, systems, programs or frameworks.

It is expected that the award will recognise work undertaken in partnership with clients or users who have championed the evaluation and can attest to increased value for their business or endeavour arising from the evaluation process or specific evaluation findings.

Assessment Criteria

quality
leading edge
contribution to knowledge
effectiveness
equitable
sustainable
professionalism
ethical conduct

Previous winners

1994 - Evaluation Management Team, NSW Public Works in conjunction with staff from Canberra University

1995 - Terrence Measham, Director, Powerhouse Museum & Carol Scott, Co-ordinator of the Powerhouse Museum's Evaluation & Visitors Research Unit

1996 - DASFLEET, Australian Department of Administrative Services and Corporate Diagnostics Pty Ltd

1998 - Team from Olympics Roads and Transport Authority for the Royal Easter Show Transport Evaluation

1999 - Australian Taxation Office/Corporate Diagnostics Pty Ltd
and

1999 - Evaluation and Auditing Services Ltd (formerly the Evaluation Department, Queen Mary Hospital, Wellington)

2000 - Jennifer Leigh, Ministry of Social Policy, Wellington

2001 - Dr Zita Unger, Evaluation Solutions

2003 - Julie Rolfe, Victorian Premier’s Drug Prevention Council and John Pilla, BearingPoint Australia

2004 - Anna L Johnson and the Strategic Review Evaluation and Research Branch, Queensland Department of Communities

2005 - Bruce Davidson, Noosa council and Ellen Vasiliauskas, d-sipher

2006 - Dr Jessica Dart & Dr Rick Davies

2007 - Ms Karen Goltz, Health Promotion/Public Health, Department of Human Services; Prof Yoland Wadsworth, Dr Ani Wierenga and Ms Gai Wilson c/ Youth Research Centre, Melbourne University; and The Victorian Department of Human Services, North and West Metropolitan Region.

2009 - The Centre for Health Policy, School of Population Health, University of Melbourne and the Department of Health and Ageing

2010 - ARTD Consultants, Klas Johansson, Janet Kidson and Joanne Battersby from Ageing, Disability and Home Care, NSW Dept of Human Services

This award encourages evaluations and evaluation processes that contribute to and have made a demonstrable impact on community development.

For the purposes of this award, community development is considered to be a process of collective effort in which individuals and/or groups act together to effect change in their own life or within their community and the decisions affecting them.

The award recognises evaluation work that addresses the needs of specific interest groups and/or those often not included or not effectively engaged in evaluation studies (e.g. culturally and linguistically diverse, Indigenous, youth, women). It also covers evaluations of overseas aid and/or development initiatives.

Assessment Criteria

quality
leading edge
contribution to knowledge
equitable
sustainable
professionalism
ethical conduct

Previous winners

2005 – Brad Shrimpton, Centre for Program Evaluation and Mandy McKenzie, Domestic Violence Resource Centre, Melbourne

This award recognises evaluation practice and evaluation capacity building with Indigenous peoples in Australasia (Australia, New Zealand, Papua New Guinea and Pacifica).

For the purposes of the award, ‘Indigenous peoples’ refers to the ‘first peoples’ in the place where the work is being conducted. The Indigenous peoples of Australasia include the Aboriginal and Torres Strait Islander peoples of Australia, the Maori peoples of New Zealand, people from Papua New Guinea and Pacifica.

support positive outcomes for the Indigenous communities;
promote sound, respectful and appropriate practice of evaluation;
engage Aboriginal and Torres Strait Islander, Maori and Pacific peoples in evaluation practice;
enhance ethical evaluation processes; and
strengthen the evaluation capacity of Aboriginal, and Torres Strait Islander, Maori and Pacific peoples.

Capacity building may include work in building evaluation capacity, building a culture of evaluation, and/or an awareness and interest in

Assessment Criteria

quality
leading edge
contribution to knowledge
effectiveness
equitable
sustainable
professionalism
ethical conduct

2008 - Department of Finance and Deregulation, Office of Evaluation and Audit (Indigenous Programs) with Anne Markiewicz, Director, Anne Markiewicz and Associates Pty Ltd

David Roberts (Chair)
April Bennett
Brian English
Ellen Vasiliauskas
Yoland Wadsworth

2011 Award Winners were announced at annual Excellence in Evaluation Awards Dinner in Sydney on September 1, 2011

Of the 10 nominations two received awards one for work in Community Development and one for Best Evaluation Study.

The Community Development Award was awarded for “The End of Project Evaluation: Bobonaro Early Childhood Care and Development Project”, sponsored by World Vision and conducted by Dr John Donnelly.