Fall 2020 - 59235 - PA 397C - Advanced Empirical Methods for Policy Analysis

Purpose This course is designed to teach the elements of program evaluation for nonprofit and social impact programs. Program Evaluation is the systematic use of empirical information to assess and improve the efficacy of public or non-profit programs and policies. The purpose of this course is to present students with an overview of approaches used to understand and assess nonprofit and social impact programs and students in this course will become familiar with the concepts, methods, and applications of evaluation more generally. This course will introduce you to the rationale and framework of evaluation, the development of plans for evaluations, and the data collection tools for implementing evaluation. The course will draw on your knowledge of organizational design, management, statistics, and data analysis. Course Description Philanthropic and public resources are limited, and decisions regarding how to use these scarce resources must be informed by an understanding of how well programs (nonprofit and public) and policies produce their desired outcomes. However, measuring causal effects can be a challenge. This course is designed to provide a broad – yet rigorous – overview of the rationale, frameworks, and tools available to evaluate the causal effects of nonprofit and social impact programs and policies. Many governments and organizations have built the logic of evaluation into their work through systems of performance management and monitoring and evaluation is increasingly required by funders and policy makers concerned with accountability and efficient use of public or philanthropic resources. All nonprofit and social impact programs, however clearly stated, have goals and serve citizens, clients, or recipients. Program evaluation aims to determine whether these programs achieve their intended goals or contribute to those they serve and how to improve their effectiveness. We will build intuition around the different quantitative and qualitative methods commonly used in practice so that students learn how to interpret evaluation results, read evaluation research critically, and understand the pros/cons of each method. We will draw on illustrations and case studies from a variety of substantive policy areas and different nonprofit programs. In addition, we will explore the role of evaluators, funders, program staff, and stakeholders (powerful and less so) in planning, implementing, and responding to program evaluation. The key learning objectives are to: Understand the purposes and logic of program evaluation Apply principles of research design to evaluation questions Explore qualitative and quantitative data collection methods for evaluation Understand the processes of evaluation and best practices for participation of stakeholders As an academic practice, program evaluation is informed by, and contributes to, social science research by testing ideas and generating knowledge. Moreover, there are several different ways to assess the extent to which programs are being carried out, operating, achieving goals, and producing desirable changes and benefits. Class Texts Rossi, Peter H., Mark W. Lipsey, and Gary T. Henry. “Evaluation: A Systematic Approach.” 8th Edition. Thousand Oaks, CA: Sage Publications. Angrist, Joshua D. and Jorn-Steffen Pischke. 2015. “Mastering Metrics: The Path from Cause to Effect.” Princeton University Press. Princeton, New Jersey. In addition to this required textbook, we will read case studies and journal articles that apply the methodology discussed in the textbook. All such articles will be posted on Canvas. You are expected to do assigned readings before class; this is especially important on days when we will discuss journal articles and case studies. Other readings are available on Canvas or via web links provided in the syllabus.