Welcome to the Welington Branch online hub
Seminar: Recent developments in the use of evaluability assessment for more effective and useful evaluation practice
Date: Tuesday, 16 January 2018
Dr Ruth Jepson, Reader in Public Health Evaluation, Lead for Public Health Evaluation, Scottish Collaboration for Public Health Research and Policy, University of Edinburgh
Dr John McAteer, Senior Research Fellow. Scottish Collaboration for Public Health Research and Policy, University of Edinburgh
Evaluability assessment is a practice that has developed over 50 years, and has been written about and applied widely. At its simplest, it tests whether a programme is suitable for useful evaluation. Until recently, they have not been used in the UK to any great extent in the health arena. Dr Jepson and Dr McAteer will discuss how it is being used in Scotland for testing the evaluability of public health policies at a national level. They will include examples they have been involved with including:
- Free School Meals
- Enhanced Family Nurse Partnership
- Teenage Parenting Strategy
The Scottish Government is particularly keen on having evaluabilty assessments undertaken to ensure that the subsequent evaluations are appropriate and good value for money. As members of Scottish Government are part of the process, it enables them to better understand their programme theory, and the outcomes that are most important for them to have evaluated. However the process is not always unproblematic, and each evaluability assessment brings its own learning.
The presenters are keen to learn lessons from how evaluability assessments are being used in NZ, and what learning they can take back to the UK and visa versa.
Seminar: Making sense of evaluation guidance: relevance to your professional practice
Date: 16 March 2017
Venue: EvaluationConsult offices, 114 The Terrace, Wellington
Presenter: David Turner, Independent Evaluation Practitioner
Practitioners of programme evaluation have considerable guidance to consider in the form of codes of practice, ethics guidelines, evaluation standards, and statements of evaluator competencies. There is so much material, much of which presents the same ideas in different ways, that it is hard to know what to pay attention to, and how to use it. Some of the guidance is brief but some is not—the US Program Evaluation Standards run to over 300 pages.
Evaluation practitioners have the greatest use for guidance on practice, but they are not the only audience. Some of the materials are directed to those who commission evaluation work and organisations that employ evaluators, as well as to practitioners themselves. Evaluation practitioners can draw on this material to guide their clients in understanding their own responsibilities for useful and ethical evaluations.
This seminar is intended to give practitioners a better understanding of the guidance that is available for their use, and how it is relevant to them. David will describe different types of guidance and how they overlap, discussing their underlying purposes and principles. He will draw on examples from New Zealand and Australia as well as other countries such as the US and the UK. He will discuss similarities between them as well as differences in scope and purpose.
Participants will then be asked to talk in small groups about their awareness of ethics guidance in different forms, the extent to which such guidance has influenced their professional work, and how they may want to use it in future.
Expanding the Influence of Evaluation on the Policy Process by Dr John Owen
THE AES welcomes policy makers, programme managers and evaluators to a free lunch time seminar for a brief interactive discussion on expanding the influence of evaluation in the policy process. For those who missed out on Dr Owen's one day workshop on Rapid Response Evaluation, the seminar offers an excellent opportunity to gain an overview of emerging international thinking on the inter-relationship between policy formulation and evaluation.
About the seminar
Andrew Kibblewhite, Chief Executive of the Department of the Prime Minister and Cabinet and Head of the Policy Profession, has recently emphasised the importance of evaluation in quality evidence-informed policy advice. The Policy project, led by DPMC in collaboration with public service policy leaders, aims to improve the performance of the policy function and quality of policy advice across government. For more information see www.dpmc.govt.nz/policyproject. Dr Owen will build on this analysis to discuss shortcomings in the role of evaluation within the policy cycle, due to the reliance on conventional impact evaluation. He suggests that other approaches to evaluation should be added as a way of increasing the influence of evidence in the policy process.
About the Presenter
Dr John M Owen is interested in providing useful evaluation based knowledge to policy and program decision makers. He has led more than 40 evaluative projects over the past two decades, initially as Director of the Centre for Program Evaluation at Melbourne University, and more recently as Director of Program Review Consultants. He has worked in conjunction with a range of clients such as the Asia Education Foundation, and Emergency Management Australia as well as with all levels of government.
He is a regular contributor to journals and at international evaluation conferences in Australia and overseas. His book: Program Evaluation: Forms and Approaches provides an integrated framework to evaluation theory and practice that has had favourable reviews worldwide.
Date: Thursday 30 July 2015
Time: 12pm–1pm. Please come early to allow time to sign in at Reception. Cellphones and other electronic devices will need to be handed in at Reception
Venue: Ministry of Foreign Affairs & Trade Manatū Aorere, 195 Lambton Quay, Level 12, Te Whare o Aorere.
There is no food and beverages inside the Whare. Te Whare o Aorere is a special space so please respect this simple rule.
For a copy of John's presentation click here
Workshop: Rapid response evaluation – relevant, timely, policy-embedded evaluation
Presented by: Dr John Owen and Dr Jenny Neale
Date and time: Wednesday 29 July 2015, 9am - 5pm (registration from 8.30am)
AES invite you to a Rapid Response Evaluation workshop
Evaluation has a critical role in evidence-based policy. The Policy Project, led by DPMC in collaboration with public service policy leaders, aims to improve the performance of the policy function and quality of policy advice across government. Andrew Kibblewhite Head of the Policy Profession in his speech at the launch of the International Year of Evaluation 2015 challenged that evaluation must be relevant, timely and embedded in policy process rather than a search for perfection disconnected from decision-making.
In responding to this challenge, the AES has designed a one day workshop that equips participants to undertake evaluation work that requires the use of rapid response approaches. The rapid response workshop will provide participants with the skills and tools to provide timely information that is relevant to the identified needs of policy and program decision-makers in ways that encourages its use.
For more information click here.
Wellington Evaluation Group Lunchtime Seminar - Thursday 26 March
Evaluation Ethics and Perspectives on Professional Practice
Presenter: Dr David Turner
Venue: Inland Revenue, office, Asteron Centre, 55 Featherston Street (opposite the Railway Station)
Date and Time: Thursday, 26 March 2015 12noon to 1:15pm
Registration: No need to register, just turn up on the day
Research ethics are often treated as an issue of correct procedures, such as informed consent. In this seminar, participants will be asked to focus instead on the perspectives that underlie their approaches to ethics in evaluation practice. The session will begin with an overview of research carried out among members of the American Evaluation Association and Australasian Evaluation Society. Seminar participants will be asked to consider a sample of statements used in that research to capture different perspectives on ethics, and will then be asked to discuss what their reactions suggest about the values and principles that inform their work.
About the presenter: David has worked in the NZ public sector for 20 years after similar experience in the US. He worked in different roles in various policy areas such as labour, immigration, employment relations, justice, and housing. He is currently an independent evaluation practitioner. He has chaired the AES ethics committee, has lectured on evaluation, and retains an interest in ethics and other issues of professional practice.
Official Launch of the cross-agency initiative for the International Year of Evaluation
The 2015 International Year of Evaluation (IYE) is a global initiative to improve policy and programme outcomes by applying evidence based approaches to design and delivery.
As part of the International Year of Evaluation, twenty four public sector research and evaluation teams have joined up to deliver a practical series of "How Do I" workshops, seminars, panel and roundtable discussions. These will be led by government agencies at various times of the year.
Finance Minister Bill English will launch the cross-agency event for the International Year of Evaluation. He will also participate in the panel discussion on the benefits of evidence-based policy along with Dr Robin Peace (Associate Professor, Research and Evaluation, Massey University), Andrew Kibblewhite (Chief Executive, Department of the Prime Minister and Cabinet) and Chris Whelan (Executive Director of Universities NZ).
The official launch is being organised by the Steering Committee Secretariat at MBIE, and co-sponsored by MBIE, Department of the Prime Minister and Cabinet and the New Zealand Productivity Commission.
To register for the event, please click here.
For more information on:
- Cross-agency event for International Year of Evaluation
- Calendar of Events
Year of Evaluation Seminar
Professor Robert Picciotto presentation on Ten challenges for evaluation worldwide. Download
How do you get 20 evaluators on their knees slapping the floor…
…ask Sally Duckworth Litmus to facilitate a session on participatory games. In August, 20 evaluators gathered to learn through play how to incorporate participatory games into their evaluation practice.
We are pleased to share so you can add a few new games to your toolkit.
Sally has offered to repeat this session when next in Auckland.
Using Q Methodology in a benchmarking evaluation
Phillipa Gaines' presentation on using Q methodology in a benchmarking evaluation for a New Zealand District Health Board adult mental health service. Download
New Zealand AES Review Insights 2014
Support for AES reincorporation as an Australian Public Company Limited by Guarantee
- AES needs to register as a company in NZ as the current legal liability of NZ committee members is ambiguous.
- In the AES Constitution, NZ and AU should be referenced throughout the document and beyond the preamble there will be no particular clauses relating to NZ.
- Registering as a NZ company means AES is the 'NZ and Australasian' association.
- With reincorporation create an 'AES NZ chapter/branch' which will hold an account.
How NZ members view the AES:
- Professional body: AES is professional, relevant, welcoming and pursuant of excellence but not culturally competent, responsive to member needs, and lacklustre. Lack of presence in Auckland.
- Professional development and capacity: AES offers good but not outstanding training, and value for money questioned.
- Advisory/ evaluation advocate: AES could be a stronger advocate of evaluation and dynamic thought leader.
How to strengthen AES's value to NZ members
- Professional body – ideas:
- More affordable membership options for under graduates, new graduates, emerging evaluators
- Professional development and capacity – ideas:
- More affordable local or online professional development opportunities
- Develop careers pathways for evaluators that enables them to remain engaged as they move across roles/ employment situations
- AES award winners encouraged to share the value of AES and being a member
- Mentorship programmes with Fellows and experience evaluators
- More networking opportunities
- Workshops related to competencies.
Some strengthening of AES's offer to NZ members ... more work to do
- Introduced the two monthly book club
- Ensured a monthly lunchtime seminars or networking programme
- Strengthening linkages to other professional associations.
Strengthening relationship with ANZEA ... more relationship building to do
- Discussions to explore how AES and ANZEA can work together more effectively to support evaluators and evaluation in NZ
- Commitment to collaborate with ANZEA wherever possible, such as shared workshops.
- Supporting ANZEA in developing Evaluation Standards for NZ
- Seeking to coordinate the sharing of ANZEA and AES key note conference speakers across the Tasman pre or post conferences.
Thank you to all who contributed.
Renewing democratic evaluation: The case for a progressive evaluation model
A presentation by Robert Picciotto, King's College, London
February 2014, Wellington
"There are moments in history when people all over the world seem to rise up to say that something is wrong" Joseph Stiglitz
In late February 2014, the Ministry of Foreign Affairs and the Wellington branch of the AES had the very great pleasure of hosting Professor Robert Picciotto, King's College, London (pictured left). Around 40 evaluators listened to Professor Picciotto's presentation on the need to renew Democratic Evaluation and the case for progressive evaluation.
Professor Picciotto offers a very thought provoking argument which encourages evaluators to reflect on their role, practice and contribution in a world of growing inequity and social instability.
The Wellington AES Committee wish to thank Professor Picciotto for agreeing to share his presentation with the wider AES community. We also thanks the Ministry of Foreign Affairs for hosting and chairing the presentation.
New Zealand Presentation of the Best Evaluation Policy and Systems Award
He Oranga Poutama: What we have learned? A report on the developmental evaluation of He Oranga Poutama.
(Pictured (L-R): Liz Smith, AES Vice President, Veronica Thompson, Sport New Zealand, Kate McKegg, Nan Wehipeihana, and Kataraina Pipi, the Kinnect Group and Dr Jenny Neale, Awards Committee Chair)
The AES was joined by over 25 people to celebrate the success of Sport New Zealand and Kinnect Group in winning the Best Evaluation Policy and Systems Award. A presentation followed on the developmental evaluation approach used to develop a grounded understanding of what as Māori participation looks like, in sport and recreation contexts. The framework that emerged - Te Whetu Rēhua – articulates five key principles and concepts that collectively define as Māori participation in the He Oranga Poutama programme context.
Further information on this evaluation can be found here http://www.sportnz.org.nz/en-nz/communities-and-clubs/He-Oranga-Poutama/.
AES members across Australia and New Zealand engage with each other through networking, special interest groups, professional learning events, and many interact through their local regional group and local events. For new members the Wellington Regional Branch offers a variety of activities throughout the year, including lunchtime seminars, workshops on a range of evaluation topics and breakfast get-togethers. Non-members are welcome to attend many of these events and activities.
This regional online hub provides access to committee information, Wellington Branch specific news and events and an online forum. Non-members have limited access to the full range of features of the regional online hubs. Find out about the benefits of membership and how to join.
Please take the time to browse the new site, including your regional online hub, and become familiar with its features.