A portal dedicated to impact: the IPSEE, or the Inventory of Problems, Solutions and Evidence on Effects.
Impact assessment: a new insight
This year we have devoted a number of articles to monitoring and evaluation, analysis tools and finally impact evaluation, which are further explored in a separate section of the Guide. We complement our coverage of the topic by presenting the contents of theIPSEE, or the Inventory of Problems, Solutions and Evidence on Effects. TheIPSEE is a platform dedicated to the topic and maintained byASVAPP, the Association for the Development of Public Policy Evaluation and Analysis. More specifically, it is the first Italian website dedicated to the effects of public policies. It offers a review of the problems that public policies should address, the solutions adopted and, most importantly, the existing empirical evidence on their effects. Its mission is illustrated through a clear analogy: public policies, like drugs, are created to “cure” problems. But do they work? Do they produce the desired effects? The same analogy, albeit in more circumscribed and specific terms, can be applied to European projects.The IPSEE platform offers three main contents: a glossary of terms used in the field of impact evaluation, an overview of the methods used in this field, and an archive of “cases” of public policy, with an analysis (supported by scientific evidence) of its impact.
A glossary on impact
IPSEE’s glossary offers a review of terms used in the field of impact assessment.A few examples, among the terms already mentioned in our Guide and particularly in the dedicated section, other concepts are more specific and specialized, but equally suggestive to those involved in projects. A few examples: Cost-benefit analysis, Control group design, Deadweigth, Difference in differences, Effect, Lock-in effect, Effectiveness, Factual / Counterfactual, Impact, Outcome, Pre post design, Selection bias.
An overview of methods for impact assessment
In the same section of the platform, a guide to the main methods used in public policy impact evaluation is offered, which can be borrowed in the project area, or at least inspire the evaluation approach. The first part provides an overview: what is impact evaluation, outcome and treatment variables, the counterfactual paradigm, and basic strategies for estimating the effect of a policy. A second part describes, briefly and in more formal terms to those presented in our Guide, the different methods used: Experimental evaluation, Cross-sectional analysis, Statistical matching, Interrupted time series and Difference in differences, to name a few.
Impact evaluation: examples and policies
The largest part of the platform is devoted to public policyimpact analysis. The approach followed is simple and informative:
- a (normally interesting and challenging) question on a public policy issue, summarized in a few lines, categorized (and searchable) by subject area, type of activity, country, and evaluation model followed;
- The definition of the objective of the policy in question, its main aspects and the results that emerged from the evaluation (also in a few lines);
- academic references to the issue and a two-page summary of the problem addressed by the policy, the solution proposed by the policy, the results of the evaluation, and the method followed.
The areas and tools covered are very varied. The platform offers nearly 200 specific fact sheets on the analysis of public policy “cases” and the evaluation of their effects. A similar repository, also maintained by ASVAPP, is available under the Valut-AZIONE heading of the CAPIRe project, which has the mission of promoting the culture and use of policy evaluation of the Legislative Assemblies of the Autonomous Regions and Provinces.We propose below some examples, in various policy areas. Many of the analyses presented on the platform can provide useful reflective material for good project design: which is (apart from the more technical aspects of evaluation methodologies) the ultimate goal of our Guide. At the same time, we have updated the chapter on monitoring and evaluation to include some new insights from ASVAPP.
- Does a little sports activity at school improve children’s condition?
- Do vocational training and apprenticeships help the integration of unemployed youth?
- Do specific offerings and activities in schools increase young people’s museum attendance?
- Do web-based communication strategies help reduce virus infections?
- Does redistribution of refugees to local organisations help inclusion?
- Does agile work increase worker productivity and well-being?
- Are there long-term benefits for those who attend fewer classes in school?
- Does playing chess make it easier to learn math?
- Does temporary work experience promote the reintegration of ex-offenders?
- Does teaching entrepreneurship work with children?
- Is the teaching quality of e-learning equal to that of face-to-face courses?
- Can a discharged elderly care service reduce the risk of re-hospitalization?
