Selected Publications and Presentations

This page brings together selected publications, presentations, workshops and seminars. It highlights work on evaluation, systems thinking, theory of change and strategy, with references and links for further reading.

Reworks: a robust system efficiency measure

Renger, R., Keogh, B., Hawkins, A., Foltysova, J. and Souvannasacd, E. (2018) ‘Reworks: a robust system efficiency measure’, Evaluation Journal of Australasia, 18(3), pp. 183–191. Available at: https://journals.sagepub.com/doi/10.1177/1035719X18796611

This peer-reviewed article introduces “reworks” as a system-level efficiency measure grounded in Systems Evaluation Theory. It shows how evaluators can assess system efficiency in ways that go beyond narrow program outputs and better reflect the performance of complex systems.

Applying Systems Evaluation Theory (SET)

Renger, R., Keogh, B. and Atkinson, L. (2019) ‘Applying Systems Evaluation Theory (SET)’, paper presented at the Australian Evaluation Society International Conference, Sydney, September 2019. Available at: https://www.aes.asn.au/annual-conference-papers/2019-aes-international-conference-sydney-presentations

This presentation introduced Systems Evaluation Theory and showed how it can be used to define system boundaries, identify systemic inefficiencies and generate stronger evaluation insights for complex interventions.

Changing the Theory of Change

Keogh, B. and Scally-Irvine, K. (2023) Changing the Theory of Change? Integrating change theories from other disciplines to standard ToC practice so we can better inform responses to complex situations and a climate-changing world, pre-conference workshop, Australian Evaluation Society International Conference, Brisbane, October 2023. Available at: https://conference2023.aes.asn.au/images/aes23_workshop_program2.pdf

This workshop examined how theories of change can be strengthened by drawing on wider traditions of change theory, especially in complex and climate-affected contexts where linear models are often too limited.

The Dynamics of Influence: Evaluating Social Power

Keogh, B. and Maloney, J. (2025) ‘The Dynamics of Influence: Evaluating Social Power’, paper presented at the Australian Evaluation Society International Conference, Canberra, September 2025. Available at: https://www.aes.asn.au/images/images/stories/files/2025_Conference_Canberra/aes25_Keogh.pdf

This presentation explores practical ways to analyse social power in evaluation, with a focus on mapping influence, understanding power relations and assessing whether those relations are changing over time.

Systems Evaluation Theory - Practice and Implementation

Renger, R., Keogh, B. and Atkinson, L. (2024) Systems Evaluation Theory – Practice & Implementation, online workshop, Australian Evaluation Society, February 2026. Available at: https://www.aes.asn.au/evaluation-learning/professional-learning-events/february-workshops/workshop-systems-evaluation-theory-practice-implementation-1-8-may-2024

This professional learning workshop focused on translating Systems Evaluation Theory into practice and helping evaluators apply systems concepts in concrete evaluation settings.

Demystifying Critical Systems Heuristics

Elliott, J., Hawkins, A., Keogh, B. and Scally-Irvine, K. (2024) Demystifying Critical Systems Heuristics, AES Systems Special Interest Group seminar, February 2024. Available at: https://www.aes.asn.au/evaluation-learning/professional-learning-events/special-interest-group-sig/seminar-demystifying-critical-system-heuristics-2-february-2024

This seminar introduced Critical Systems Heuristics as a practical way for evaluators to question assumptions, test boundary choices and examine who is included or excluded in evaluation framing.

Panarchy – How do complex systems change and persist over time?

Scally-Irvine, K., Elliott, J., Hawkins, A. and Keogh, B. (2025) Panarchy – How do complex systems change and persist over time?, AES Systems Special Interest Group seminar, June 2025. Available at: https://www.aes.asn.au/evaluation-learning/professional-learning-events/special-interest-group-sig/aes-systems-sig-panarchy-how-do-complex-systems-change-and-persist-over-time-online-13-june-2025

This seminar explored panarchy as a way of understanding cycles of stability, disruption and renewal in complex systems, and considered the implications for systems-informed evaluation.