Workshop: Experiments and quasi-experiments: research designs for undertaking outcome evaluations

Date and time: Friday 22 August and Friday 29 August 2025, 10.00 am to 1.30pm AEST (registration from 9.45am). Half day workshop - 1 session

Venue: Via Zoom. Details will be emailed to registrants just prior to the workshop start time

Facilitator: Scott Bayley

Register online by: Thursday, 21 August 2025, unless sold out prior. Spaces limited to 25 participants

Fees (GST inclusive): Members $325.00, AES Organisational member staff $460.00, Non-members $535.00, Student member $155.00, Student non-member $250.00* (GST inclusive) *Students must send proof of their full-time student status to This email address is being protected from spambots. You need JavaScript enabled to view it. 

Workshop Overview

This is the second in a series of three workshops on outcome evaluation - each workshop builds upon the content of its predecessor. The first introductory level workshop introduced participants to theories of causality and four key criteria for evaluating program outcomes. This second workshop (one day, Intermediate level) will familiarise participants with the typical challenges to formulating credible conclusions about program outcomes, preparing  suitable outcome evaluation (OE) questions, and understanding the strengths / weaknesses of different types of research designs and their appropriate application for evaluating program outcomes.

Workshop Content

Workshop #2 will cover the following content areas:

1. Overview of the evaluation process and logic

2. Implications of different OE approaches for:

   1. managing common challenges when evaluating social programs

   2. identifying the evaluation’s purpose and audience

   3. describing the evaluand, including the nature of expected change

   4. types of evaluation questions, descriptive, normative, economic, cause-effect

   5. designing the valuing process including choosing criteria and setting standards

   6. considering ethical issues. Common objections to the use of experimental designs (discussion)

3.  Research designs

   1. What is a research design? A plan for measurement, control, and comparison.

   2. Terminology / notation

4. Types of designs (Experiments, Quasi-experimental and Passive Observational studies, group and single subject versions of each)

5. Types of evaluation questions and their suitable designs

   1. Internal / external validity (focus on temporal order, managing confounding, selection bias and regression towards the mean)

   2. Strengths / weaknesses of Experiments / Quasi-experiments and Passive Observational designs for OE (incl the role of theory based methods, the back box problem)

   3. Conditions favourable to the use of experiments & quasi-experiments

6.  Issues to consider when planning an OE

7. Case study discussion

Take home readings: copies of selected papers will be provided.

Workshop Outcomes

The second workshop will introduce participants to different types of research designs, appropriate OE questions, and the typical threats to reaching defensible conclusions about program outcomes. After completing this workshop participants will:

• understand the defining features, strengths, and weaknesses of experimental and quasi-experimental designs and the limitations of passive observational methods
• be able to determine what types of outcome evaluation questions can be answered by different research designs
• be able to recognise the basic threats to internal and external validity posed by different approaches to outcome evaluation
• have a full appreciation of the ethical challenges of experimentation as well as some common myths.

PL competencies 

• Domain 1 - Evaluative Attitude and Professional Practice 

• Domain 2 - Evaluation Theory 

• Domain 4 - Research Methods and Systematic Inquiry 

• Domain 7 – Evaluation Activities

Evaluators' Professional Learning Competency Framework  

Who should attend?

Intermediate – Intermediate sessions assume the audience has a working knowledge of the topic or concept but provides accessible language and avoids technical details to ensure engagement from non-expert audiences. Participants are encouraged to complete workshop #1 prior to enrolling in workshop #2.

This workshop is the second of a three-part series of planned workshops to run in 2025.

The first workshop is half day and is a foundational level. The second workshop (one day, Intermediate level) will undertake a more in-depth analysis of research designs, appropriate outcome evaluation questions, and the typical threats to reaching defensible conclusions about program outcomes. The third workshop (one day Intermediate / Advanced) will examine the technical aspects of conducting and analysing some common experimental and quasi-experimental designs, and contrast these with theory-based passive observational studies.

Workshop start times

• VIC, NSW, ACT, TAS, QLD: 10.00am

• SA, NT: 9.30am

• WA: 8.00am

• New Zealand: 12.00pm

For other time zones please go to https://www.timeanddate.com/worldclock/converter.html

About the facilitator

Scott Bayley manages his own evaluation consultancy business and holds a MA in Public Policy majoring in evaluation and social measurement. He has over 30 years of experience in evaluation and is a Fellow of the Australian Evaluation Society. Prior to having his own consultancy Scott held senior positions with Oxford Policy Management, the Australian Department of Foreign Affairs and Trade, AusAID, and the Asian Development Bank. Scott has also served as an advisor to several UN agencies. Scott has published over 70 evaluation reports across a number of fields.