Workshop: Weighing the Evidence: Understanding Quantitative Methods and Impact Evaluation

Date and time: Wednesday 5 May and Wednesday 12 May 2027, 9.30am to 1.00pm AEST (registration from 9.15am). Registrants are to attend both sessions. (full day workshop - 2 sessions)

Venue: Via Zoom. Details will be emailed to registrants just prior to the workshop start time

Facilitator: Dr. Samantha Abbato

Register online by: 4 May 2027, unless sold out. Spaces limited to 25 participants

Fees (GST inclusive): Members $400.00, AES Organisational member staff $525.00, Non-members $609.00, Student member $200.00, Student non-member $300.00* *Students must send proof of their full-time student status toThis email address is being protected from spambots. You need JavaScript enabled to view it. 

Workshop Overview

Have you ever skipped past the tables, p-values and confidence intervals in an evaluation report and gone straight to the conclusions, hoping the author “knew what they were doing”? Or been asked to make a decision based on evaluation findings without being sure how much weight the evidence could bear? The purpose of this workshop is to build participants’ confidence in understanding and interpreting quantitative methods and findings, and in critically assessing the methodological quality and strength of evaluation evidence, whether quantitative, qualitative or mixed methods. The goal is for participants to apply the learnings of the workshop to increase rigour and make better-informed judgements, whether they are evaluators, commissioners of evaluation or people using evaluation findings to inform decisions.
Most evaluations rely on numbers in some form (outcome measures, survey results, administrative data, before-and-after comparisons), usually combined with qualitative evidence in a mixed-methods design. With the Australian Centre for Evaluation (ACE) championing rigorous impact evaluation across the Australian Government, including randomised trials and quasi-experimental designs, evaluators and commissioners are increasingly expected to understand, commission and critically assess these designs. Yet quantitative findings are often reported and read without a clear understanding of what they can and cannot show. Decisions made based on evaluation conclusions and recommendations assume that the evidence base is sound. Evaluation misuse happens when decisions are made based on flawed or weak evidence, or when the strength of a claim, particularly a claim that a program caused an outcome, is overstated. It is critical, not only for evaluators but for commissioners and all users of evaluation findings, to be able to ask the right questions of the methods behind the findings and to judge how much confidence those findings deserve.
The workshop will be interactive and non-technical, involving the sharing of experiences as well as hands-on exercises using real evaluation reports, tables and charts. No statistical background or software is required. Practical strategies, “how to” guides and checklists will be provided, discussed and put to the test without the technical or research jargon.

Workshop Content

Participants will learn the essentials of quantitative methods for evaluation and how to read, interpret and question quantitative findings, including the range of quantitative designs from descriptive and before-and-after studies through to quasi-experimental and experimental (randomised) designs. They will gain practical skills in judging the methodological quality of evaluation evidence and the strength of the claims made from it, across quantitative, qualitative and mixed methods. The workshop focuses on appraising and using evidence rather than on how to collect or analyse qualitative data, and is designed to complement introductory workshops on qualitative methods and data collection.

Content includes:

(1) Quantitative essentials and interpreting findings: types of data, measures and indicators; descriptive statistics (counts, percentages, averages and rates); sample size and representativeness; statistical versus practical significance; p-values, confidence intervals and effect sizes in plain language; and common traps such as small numbers, missing data and misleading charts;

(2) Quantitative evaluation designs, from descriptive to experimental: descriptive and monitoring designs, before-and-after studies, quasi-experimental designs (such as matched comparison groups, difference-in-differences and regression discontinuity) and randomised controlled trials; the counterfactual and why it matters; bias, confounding and alternative explanations; and attribution versus contribution;

(3) Assessing experimental and quasi-experimental impact evaluations: practical questions for commissioners and users to judge the quality of a trial or quasi-experimental design and the findings drawn from it, including how comparison groups were formed and whether they are genuinely comparable, whether the sample was large enough to detect a meaningful effect, drop-out and missing data, whether outcomes and analyses were set out in advance, and whether results are likely to hold in other settings and populations;

(4) Assessing methodological quality and strength of evidence: practical checklists and questions for appraising quantitative, qualitative and mixed-methods evidence, including the credibility of qualitative findings and how well methods are integrated; and “fit-for-purpose” rigour rather than a single hierarchy of evidence;

(5) Commissioning and using evidence, reviewing draft reports, key questions to ask evaluators, recognising and reporting limitations, and weighing evidence of different strengths when making decisions.

Real-life evaluation examples will be used throughout the activities. In addition, participants will be provided opportunities to apply the learned skills to evaluation reports and data from their own work.

Workshop Outcomes

This workshop is for evaluators, evaluation commissioners  and other professionals who would like to:

- Understand the essentials of quantitative methods used in evaluation and confidently read and interpret common quantitative findings, tables and charts;

- Explain in plain language key concepts such as sample size, statistical and practical significance, confidence intervals and effect sizes;

- Distinguish between quantitative designs, from descriptive and before-and-after to quasi-experimental and experimental, and assess what each can and cannot conclude about program outcomes and causation;

- Critically appraise the design and findings of randomised and quasi-experimental impact evaluations, and judge how much confidence their conclusions deserve;

- Use checklists of key practical questions to appraise the methodological quality and credibility of quantitative, qualitative and mixed-methods evidence;

- Identify weaknesses in method, how to acknowledge limitations and strategies to strengthen rigour;

- Build quality expectations into commissioning, and confidently apply or challenge evaluation findings on the basis of the strength of the evidence.

PL competencies

This workshop aligns with competencies in the AES Evaluator’s Professional Learning Competency Framework. The identified domains are

• Domain 1 – Evaluative attitude and professional practice

• Domain 2 – Evaluation theory

• Domain 3 – Culture, stakeholders and context

• Domain 4 – Research methods and systematic inquiry

Who should attend?

This workshop is designed for evaluation commissioners, program managers, policy makers and others who use evaluation findings to inform decisions, including public servants commissioning or overseeing impact evaluations, as well as evaluators who want to strengthen their quantitative and impact evaluation skills and their ability to judge the quality of evidence. It will particularly benefit those who are less confident with numbers, including evaluators whose experience is mainly qualitative. No prior statistical training is required.

Workshop start times

• VIC, NSW, ACT, TAS, QLD: 9.30am

• SA, NT: 9.00am

• WA: 7.30am

• New Zealand: 11.30am

• For other time zones please go to https://www.timeanddate.com/worldclock/converter.html

About the facilitator

Samantha Abbato is an evaluation consultant and director of Visual Insights, a Pictures and Stories approach to evaluation. As an independent evaluation consultant for the past 14 years working with more than 50 NGO and government organisations in Queensland, New South Wales and the Northern Territory, she regularly applies a mixed-methods (qualitative and quantitative) approaches. Her evaluation work is based on an extensive quantitative and qualitative academic background that includes a PhD in epidemiology (UC Berkeley), an MPH in biostatistics and four years of applied academic training in the qualitative methods of medical anthropology (UC Berkeley) applied to a thesis and publication in Aboriginal and Torres Strait Islander health. She is a specialist in health and community sector evaluation with extensive experience in qualitative and quantitative evaluation approaches, working with Aboriginal and Torres Strait Islander communities and a range of culturally and linguistically diverse communities, including refugees. 

Sam has held university lecturing positions including as both a lecturer in epidemiology and qualitative methods for the School of Public Health, University of Queensland (1997-2000). Sam continues to publish peer-reviewed evaluation and research papers in collaboration with clients in quantitative (including statistical analysis), qualitative and mixed methods. Sam was the recipient of the 2026 AES Awards for Excellence in Evaluation – Enhancing the Social good and received the 2015 AES Evaluation Publication Award (Caulley Tulloch Award). Drawing on her own shift from a purely academic approach to a “use focus” to achieve outcomes goals in partnership with organisations, she is able to share effective tools and strategies that are responsive to organisations’ needs whilst maintaining the integrity of method and evidence.

It is rare to find someone who is able to navigate and translate across both the academic and grass-roots worlds.”

Brad, Brisbane NGO client