1. Evaluative reasoning – Getting to defensible evaluative judgments
Mentors: Mathea Roorda and Kim Grey
This group will look at how to set up the building blocks for defensible evaluative reasoning, including framing evaluation questions to fit the organisational and sector context, developing evaluation criteria, transparent and systematic analysis, and evaluative conclusions. Mentees will gain an appreciation of the differences between evaluation and research, the confidence to reflect on and learn from their own and others' practice, and to think about what is required to conduct and deliver a quality evaluation product.
Co-mentor: Dr Mathea Roorda
Mathea has worked as an evaluation practitioner since 1998, in government agencies, as a self-employed evaluator and most recently as a team member in a large consultancy. Since 2008 she has met regularly with four evaluation colleagues to systematically reflect on and support each other's practice. She has written journal articles and blogs (on topics such as developing a logic for a complicated policy, the challenges to implementing good practice guidelines for evaluation with Maori and developing evaluation criteria). She also co-authored a book chapter on the experience of being a Pakeha evaluator.
Mathea has tutored students in the MA Practice of Evaluation and Foundations of Evaluation subjects at Melbourne University. In 2020, she completed a PhD on the topic of developing defensible criteria for public sector evaluations (University of Melbourne). Prior to this she completed a Post Graduate diploma in social sector evaluation research (Massey University, NZ). In 2010 she was part of a team that won an AES Award for the Best Evaluation Study. Over in her career, Mathea has actively engaged with the AES in a voluntary capacity, as a Board member and Awards and Recognitions Working Group member.
Her mentoring style is personable; she loves to reflect on her practice and learn from others.
Co-mentor: Kim Grey
Kim has lectured and conducted projects in realist evaluation with Charles Darwin University and has a Masters in Evaluation by research from University of Melbourne.
As a manager of internal evaluation in Indigenous Affairs across most of the last 10 years, Kim has led teams in social policy and employment programs. She has led and advised on many complex evaluations and seen a lot of organisational change.
Kim mentoring style focuses on ‘meeting people where they at’ and supporting mentees to adopt a reflective approach to situations.
2. Managing evaluation in complexity: linking theory and practice in our choices
Mentor: Judy Oakden | Associate Mentor: Kara Scally-Irvine
This group will explore how to conduct evaluation in ways that are more complexity informed. Using generative conversations, this group will build mentees confidence in understanding the consequences of choices they make in their evaluation practice whether taking a complexity informed approach or not. Topics open for consideration include:
- Talking past one another: Why is understanding different worldviews and how we create knowledge important?
- Whose perspectives count, and how do we include diverse perspectives? And being more explicit about who we are not including.
- Reflection and reflexivity: Why is it important? How can we use it more?
- How to use standard evaluation tools more effectively when taking a complexity informed approach
Mentor: Judy Oakden
Judy has conducted and provided strategic advice on evaluation and research projects to build evaluation capacity for the last 15 years. Prior to this, she Consulted with central and local government agencies, NGO's and businesses, in Aotearoa, New Zealand and internationally, for 30 years as a market researcher.
She is an active member of the Australian Evaluation Society (AES), Aotearoa New Zealand Evaluation Association (ANZEA), and the American Evaluation Association (AEA) and the Kinnect Group, and is the owner of Pragmatica Limited.
As a mentor, Judy aims to encourage generative conversations where mentees can share their experiences and questions. She looks to support mentees become more confident in understanding the consequences of the choices they make in their evaluation practice when either taking or not taking a complexity informed approach.
Judy was a mentor in the AES Group Mentoring Program 2022.
Associate Mentor: Kara Scally-Irvine
Dr Scally-Irvine has been working in monitoring, research, and evaluation for over 20 years. More recently she has been working as an evaluation consultant for the past 12 years, engaging with a broad range of sectors and settings during this time.
With a mind for systems thinking and mixed methods, Kara has an interdisciplinary PhD in international development and biodiversity conservation.
She is an active member of both AES and ANZEA, as well as a member of AEA.
Her approach to mentoring is inquiring, supportive and friendly. Kara aims to support mentees to explore the value of open thinking and the importance of applying reflective inquiry and how to be ok in the ‘grey’.
3. Emerging talents
Mentor: Professor Rick Cummings | Associate Mentor: Cat Street
This group will look at building awareness and tapping into the strong knowledge and practice base that already lies within each mentee. During the mentoring program, mentees will gain confidence in their abilities as evaluators. The opportunity to connect with a network of evaluators will support peer-to-peer learning and provide valuable connections for the future.
Mentor: Professor Rick Cummings
Professor Cummings has a PhD in educational evaluation and has taught in this field for nearly 20 years. He has held senior evaluation positions in the WA government and at Murdoch University. He has conducted over 40 evaluations including for the World Bank, published a number of peer-reviewed articles, and supervised a number of Master’s and PhD students. Rick has also been an editor of the EJA, President of the AES and a Fellows since 2013.
Rick has been a mentor in the AES Group Mentoring programs in 2021 and 2022. He has a passion for contributing to the careers of evaluators.
Rick provides structure to the mentoring sessions yet encourages mentees to have agency within the group. Rick provides the space for mentees to voice their areas of interest, encouraging individual preparation and very contribution at the group sessions.
Rick has been a mentor in the AES Group Mentoring programs in 2021 and 2022.
Associate Mentor: Cat Street
Cat has worked in research and evaluation related roles in the Northern Territory and nationally for the past ten years, predominantly in First Nations public policy/program settings. Her began in remote First Nations health promotion evaluation strategy and capacity building. From there she transitioned into the higher education sector, where she led a developmental evaluation for a research project seeking to strengthen pathways into higher education for remote First Nations communities. Cat has also worked in housing and homelessness, NT and national schooling education research and evaluation, and in evaluation consulting for a majority Aboriginal owned professional services company. Recently she has been engaged by organisations in the NT and WA to develop evaluation frameworks for various community-based initiatives as a self-employed evaluator.
Cat participated in the AES mentoring program in 2022 as a mentee. Her mentoring style is participatory, empowering and inclusive. She places high value of different perspectives and encourages mentees to actively provide feedback.
Mentor: Duncan Rintoul | Associate Mentor: Alex Lorigan
Led by a highly experienced evaluator with a passion for evaluation capacity building, Youngbloods is focused on a specific demographic: Gen Z, evaluators in their 20’s.
This group will focus on evaluation capacity building for young evaluators. Youngbloods will support evaluators early in their career who are professionally isolated. Those who are the sole evaluator in an organisation, are in a small team or need guidance will benefit from participating in this group.
Mentor: Duncan Rintoul
Dr Rintoul is passionate about ECB (evaluation capacity building) and has delivered evaluation training over many years. With a PhD in survey design, Duncan believes he knows a thing or two about it.
He has worked in the social sciences for nearly 25 years, in a variety of settings: consulting (Urbis, Rooftop Social), academia (University of Wollongong) and government (most notably NSW Department of Education). Duncan’s areas of focus are mostly in four policy domains: education, health, justice, and the environment.
Duncan has been very involved in the AES, including on the board (2012-15) and running training workshops.
His mentoring style is flexible, responsive, collaborative and friendly. He likes to create mentoring groups with good vibes and high hopes. He encourages (and expects) mentees to be open about what's going on for them, to be thirsty for good ideas and willing to risk a bit and be generous in the way they respond to each other.
Duncan was a mentor in the AES Group Mentoring Program 2022 and an Associate Mentor in 2021.
Associate Mentor: Alex Lorigan
Alex started working at ARTD Consultants after finishing university in 2017.
With no prior evaluation experience, Alex found herself working in projects in the mental health and disability sectors. Since then, she has conducted numerous smaller evaluations for NGOs and larger evaluations for government. She has also been involved in developing organisational performance frameworks and service design projects.
Alex is a Gen Z evaluator. She undertakes qualitative data collection and analysis, and regularly designs surveys and conducts quantitative analysis. Her work also involves onboarding, training and supporting people with disability or mental health concerns to work alongside the ARTD team in various capacities as part of the evaluation team.
Alex’s mentoring style is gentle, informal, and flexible, adapting to what people want to focus on. Mentees working with Alex can expect to feel welcome, safe to share and reassured in their experiences. She expects mentees to come with a good attitude and willingness to participate and share and work collaboratively with others in the group.
5. Evaluation Capacity Building united: 2023 edition
Mentor: Liam Downing | Associate Mentor: Kate Randall
This group will support mentees to build an applied knowledge of evaluation capacity building that is contextually applicable in their current roles, allowing them to consider career progression. Mentees will be able to apply what is discussed in each session in the workplace, and to critically reflect on these applied lessons in a safe space.
Mentor: Liam Downing
Liam brings with him 15+ years of experience in evaluation across consultancy, higher education, and government, many of those years leading evaluation both within his workplaces and more extensively.
He currently leads a team of evaluators in NSW Education in an embedded and responsive context doing utilisation-focused developmental evaluation (a la Patton) and realist evaluation (a la Pawson and Tilley) on significant pieces of work aimed at improving student outcomes through shifting teaching quality.
This work includes a dedicated evaluation capacity building (ECB) function, and supporting educators to build their capability, opportunity, and motivation to do good evaluation.
Liam’s mentoring style is responsive and backed by significant resources and learning materials. He applies a structure from one session to the next but is adaptable to the situation and informal. He likes to ensure that mentees experience connecting theory to practice and is responsive to participants' needs at a point in time. Liam has a suite of resources and a broad curriculum of learning.
Liam was an Associate Mentor in the AES Group Mentoring Program 2022.
Associate Mentor: Kate Randall
Kate’s experience has been focused on the social/community sector with various NGOs and she has worked with government more recently. She first participated in evaluation as a program manager. Enlightened by the experience of working with external evaluators, in 2016 she decided to specialise in the field of evaluation. Kate has worked as an internal evaluation manager, an independent evaluator, and most recently an evaluation consultant.
Her work is now solely focused on evaluation capacity building and includes Monitoring, Evaluation and Learning (MEL) training and other ECB supports.
Kate’s mentoring style is down to earth and encouraging. She believes mentees possess valuable experience and knowledge and provides a safe mentoring environment to share and expand their knowledge. With her skills, Kate can respectfully and gently provide alternative perspectives, helping to get to the heart of issues.
6. Careers for evaluators
Mentor: Charlie Tulloch | Associate Mentor: Lydia Phillips
Evaluation is gaining increasing recognition in different fields and sectors. This group is for those who want to explore where a career in evaluation can take them. It will provide an opportunity for evaluators to think creatively about how they position themselves in the jobs market and how their career pathway could promote greater understanding and engagement with the evaluation field across sectors and industries. This group may also explore pathways into self-employment and establishing a consultancy business.
Mentor: Charlie Tulloch
Charlie’s experience includes 15 years as an evaluation/ policy consultant in the private sector working in small, medium, and large consulting firms. Prior to this he worked with the Victorian government for 5 years.
Charlie has tutored for the University of Melbourne's Master of Evaluation program in the subject Impact Evaluation and has been an online facilitator for Oxford University's Evidence in Public Policy course in 2020. He has trained over 300 emerging evaluators in the Introduction to Evaluation training program.
He holds a Master of Public Policy and Management, a Masters of Evaluation, and a Bachelor of Arts.
Charlie now runs his own consultancy business as an evaluation and policy consultant.
Charlie’s mentoring style is that of a facilitator of learning, working with mentees to direct their own learning by bringing topics for discussion to the group. Through connection, reflection and networking Charlie supports peer-to-peer learning. He also openly shares his experiences and reflections of things that affect mentees in their day-to-day work.
Associate Mentor: Lydia Phillips
Lydia is an independent consultant providing social policy and evaluation services to for-purpose organisations. She has worked across the government, corporate, community and academic sectors and navigated career changes including as a commercial litigation lawyer, academic researcher and principal policy officer in state government.
Lydia holds a Graduate Certificate in Evaluation (1st Class Hons) and she is currently completing a Master of Evaluation. She sits on the Victorian committee of the AES.
Lydia has also trained and worked as an organisational coach supporting people in executive, management and service delivery roles across the corporate, government and community sectors. She is passionate about professional development and keen to explore with this group how evaluation can be further promoted in a range of fields.
As an Associate Mentor in this group, Lydia will focus on building group engagement and facilitating opportunities for group discussion at and between sessions. She will draw on organisational coaching practices to encourage reflective peer learning.