Developing a framework for a program theory-based approach to evaluating policy processes and outcomes: Health in all policies in South Australia

Background: The importance of evaluating policy processes to achieve health equity is well recognised but such evaluation encounters methodological, theoretical and political challenges. This paper describes how a program theory-based evaluation framework can be developed and tested, using the example of an evaluation of the South Australian Health in All Policies (HiAP) initiative. Methods: A framework of the theorised components and relationships of the HiAP initiative was produced to guide evaluation. The framework was the product of a collaborative, iterative process underpinned by a policy-research partnership and drew on social and political science theory and relevant policy literature. Results: The process engaged key stakeholders to capture both HiAP specific and broader bureaucratic knowledge and was informed by a number of social and political science theories. The framework provides a basis for exploring the interactions between framework components and how they shape policy-making and public policy. It also enables an assessment of HiAP’s success in integrating health and equity considerations in policies, thereby laying a foundation for predicting the impacts of resulting policies. Conclusion: The use of a program theory-based evaluation framework developed through a consultative process and informed by social and political science theory has accommodated the complexity of public policy-making. The framework allows for examination of HiAP processes and impacts, and for the tracking of contribution towards distal outcomes through the explicit articulation of the underpinning program theory.