Youth unemployment and active labor market policies in Europe

Abstract

Since the economic crisis in 2008, European youth unemployment rates have been persistently high at around 20% on average. The majority of European countries spends significant resources each year on active labor market programs (ALMP) with the aim of improving the integration prospects of struggling youths. Among the most common programs used are training courses, job search assistance and monitoring, subsidized employment, and public work programs. For policy makers, it is of upmost importance to know which of these programs work and which are able to achieve the intended goals - may it be the integration into the first labor market or further education. Based on a detailed assessment of the particularities of the youth labor market situation, we discuss the pros and cons of different ALMP types. We then provide a comprehensive survey of the recent evidence on the effectiveness of these ALMP for youth in Europe, highlighting factors that seem to promote or impede their effectiveness in practice. Overall, the findings with respect to employment outcomes are only partly promising. While job search assistance (with and without monitoring) results in overwhelmingly positive effects, we find more mixed effects for training and wage subsidies, whereas the effects for public work programs are clearly negative. The evidence on the impact of ALMP on furthering education participation as well as employment quality is scarce, requiring additional research and allowing only limited conclusions so far.

JEL codes: J13, J68, J64

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1

DOI 10.1186/s40173-015-0056-3

ORIGINAL ARTICLE Open Access

http://crossmark.crossref.org/dialog/?doi=10.1186/s40173-015-0056-3-x&domain=pdf

Web End = Youth unemployment and active labor market policies in Europe

Marco Caliendo1,3* and Ricarda Schmidl2

*Correspondence: mailto: [email protected]

Web End [email protected] 1University of Potsdam, IZA Bonn, DIW Berlin, IAB Nuremberg, Potsdam, Germany3University of Potsdam, Chair of Empirical Economics, August-Bebel-Str. 89, 14482 Potsdam, GermanyFull list of author information is available at the end of the article

Abstract

Since the economic crisis in 2008, European youth unemployment rates have been persistently high at around 20% on average. The majority of European countries spends significant resources each year on active labor market programs (ALMP) with the aim of improving the integration prospects of struggling youths. Among the most common programs used are training courses, job search assistance and monitoring, subsidized employment, and public work programs. For policy makers, it is of upmost importance to know which of these programs work and which are able to achieve the intended goals may it be the integration into the first labor market or further education. Based on a detailed assessment of the particularities of the youth labor market situation, we discuss the pros and cons of different ALMP types. We then provide a comprehensive survey of the recent evidence on the effectiveness of these ALMP for youth in Europe, highlighting factors that seem to promote or impede their effectiveness in practice. Overall, the findings with respect to employment outcomes are only partly promising. While job search assistance (with and without monitoring) results in overwhelmingly positive effects, we find more mixed effects for training and wage subsidies, whereas the effects for public work programs are clearly negative. The evidence on the impact of ALMP on furthering education participation as well as employment quality is scarce, requiring additional research and allowing only limited conclusions so far.

JEL codes: J13, J68, J64

Keywords: Youth unemployment, Active labor market policies, Evaluation, Training, Job search

1 Introduction

Young individuals entering the labor market are generally considered to be an at-risk population. They face a higher risk of unemployment than older workers, are more likely to switch between states of joblessness, training and working, and are more likely to enter temporary or precarious types of employment (see, e.g., Quintini et al. 2007). One reason for the lower labor market attachment of youth is their initially low labor market experience. During the school-to-work transition period, labor market entrants tend to learn about their abilities and preferences by job-shopping (Topel and Ward 1992), resulting in higher rates of turn-over and more frequent periods of non-employment. At the same time, firms commonly face higher costs of investment and lower costs of termination when employing young workers, making the youth labor market situation more sensitive to demand-side fluctuations, which was recently demonstrated in the aftermath of

2016 Caliendo and Schmidl. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License http://creativecommons.org/licenses/by/4.0/

Web End =(http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 2 of 30

the financial crisis 2007/2008 (Bell and Blanchflower 2010; Choudhry et al. 2012; Verick 2011). Between 2008 and 2009 youth unemployment rates increased by about five percentage points to a 20% average, then until 2013 they further climbed to 24%; during the same time period, adult unemployment rate changed from 6% to 10%.1

High and persistent levels of youth unemployment give rise to concern, as the negative consequences of extended spells of unemployment early in the career are well documented. On the one hand, joblessness is directly associated with psychological distress and financial hardship for the affected youth (Goldsmith et al. 1997). On the other hand, early unemployment spells may have negative effects on later-life outcomes such as lower wages (Arulampalam 2001; Gregg and Tominey 2005; Kahn 2010; Skans 2004), lower labor market attachment (Gregg 2001), lower well-being (Daly and Delaney 2013), and a higher propensity to engage in criminal activities (Bell et al. 2014; Fougre et al. 2009). The immediate monetary costs of youth unemployment include direct costs for unemployment benefits and social assistance, as well as indirect costs of foregone tax payments and social security contributions. Calculations by Eurofund (2011) suggest that the immediate public costs accruing from youth neither in education nor in employment (NEET) in the EU-26 amounted to 120 billion euros (1% of GDP) in 2008, and to 153 billion euros (1.2% of GDP) in 2011, not accounting for discounted future costs and foregone payments due to the lowered labor market attachment in the long run. In light of demographic change and increasing old-age dependency ratios, the burden of these costs are expected to further increase. Raising employment levels is seen as the most effective strategy with which countries can prepare for population ageing (European Commission 2008), making the youth unemployment problem all the more urgent.

To smooth the school-to-work transition process, to prevent extended spells of unemployment or the complete withdrawal from the labor market, and to promote entry into stable employment relationships, policy makers in many countries resort to active labor market programs (ALMP). Prominent national initiatives include the New Deal for Young People (NDYP) in the UK, Jugend mit Perspektive (JUMP) in Germany and the Youth Unemployment Program (YUP) in Denmark. More recently, the Youth Guarantee (YG) adopted by the European Union in 2013 called all member states to set up ALMP programs to ensure that unemployed youth were offered high quality employment or education opportunities within four months of entering unemployment (European Commission 2014).

The types of programs most commonly used can be divided into four categories, namely, labor market training, job search assistance and monitoring, wage subsidies and public sector work programs. Given the relatively high rates of youth unemployment, it is not surprising that large shares of the active youth population participate in ALMP, as can be seen in Fig. 1. The average youth unemployment rate in the presented countries was at 23% in 2012, and, on average, 9% of the active youth participated in ALMP. Unfortunately, however, the quantitative importance attributed to the use of ALMP to fighting youth unemployment stands in stark contrast to the low level of knowledge we have regarding their effectiveness. Grubb (1999) provides a descriptive summary of the early evidence (up to 2000) on the effectiveness of specific ALMP for disadvantaged youth in the US. He concludes that training and education measures are largely unsuccessful, but points to promising combinations of training with a close labor market link and additional support schemes. Reproducing the negative findings on training programs for

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 3 of 30

Fig. 1 Youth unemployment rates and stock of ALMP participants relative to the active working population in selected European countries, 2012(a). Source: Eurostat, 2012. Note: Dark gray bars depict youth unemployment rates, light gray bars the ALMP participation among all active youth. (a) Only countries that provide information on ALMP participant numbers for the youth population. ALMP participants numbers refer to the stock of participants in ALMP types 2 to 7 (Training, Employment Incentives, Supported Employment and Rehabilitation, Direct Job Creation and Start-up Incentives) are included

OECD countries, Martin and Grubb (2001) suggest that wage subsidies are among the most promising programs for youths. Heckman et al. (1999) study the evidence of ALMP in the US and Europe, reaching the conclusion that none of the program types substantially benefit unemployed youth. A meta-analysis of European ALMP evaluation studies up to the year 2000 is provided by Kluve et al. (2002) who find that youth tend to benefit less from ALMP participation than adults. Similar results emerge in the more recent and comprehensive meta-analyses by Kluve (2010) and Card et al. (2010) showing that programs targeted at youths, and in particular those involving training, are less effective in increasing employment levels than programs targeted at the general population of unemployed.

The particularity of the youth labor market situation and the results from the meta-analyses suggest that assessments of the effectiveness of ALMP for adults are most likely not valid for youth. So far, no consensus exists on the effectiveness of the different ALMP programs for this age group. With this study we aim to fill this gap. Aggregating the evidence of a substantial number of recent studies assessing the effectiveness of ALMP for youth, we provide a comprehensive re-assessment of the questions revolving around which policies work for whom and under which conditions. Unlike the recent meta-analyses we thereby pay special attention to the specificity of the youth labor market situation, highlighting the particularities that might promote or mitigate the effects of ALMP programs for young individuals. In line with the previous literature, we focus on the micro-evidence of the effectiveness of ALMP, distinguishing between the four types of programs that are most commonly employed to promote a direct labor market entry: labor market training, job search assistance and monitoring, wage subsidies and public sector work programs. Besides focusing on the effect of ALMP on employment take-up,

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 4 of 30

we also examine the evidence available on education participation, and the findings on the quality of the accepted employment whenever available. The paper is organized as follows. Section 2 gives a quick overview of the labor market situation of youth in Europe and discusses the role of ALMP. Section 3 presents the collected evidence on the effects of the different policies, before Section 4 concludes.

2 The labor market for youth in Europe

2.1 A quick glance at the European situation

Table 1 depicts various indicators describing the labor market situation of youth for different European countries in 2013. Column (1) shows that there exists substantial variation across countries in the unemployment rates2 for youth below 25 years, ranging from 8% in Germany to over 40% in Italy, Spain and Greece. Similarly, the share of youth in long-term unemployment in column (3) ranges from 5% to 7% in Finland and Sweden to over 50% in Italy, Greece and the Slovak Republic. As these cross-country differences are linked to differences in economic performance or the institutional set-up, youth-specific labor market patterns are highlighted when relating youth labor market outcomes to

Table 1 Absolute and relative labour market indicators for youth in selected European countries, 2013

Unemployment Unemployment > 1 year Inactive all NEET(a) ALMP(b)

Country youth youth/adult youth youth/adult youth youth/adult youth youth youth/adult

(1) (2) (3) (4) (5) (6) (7) (8) (9)

Austria 9.2 2.0 14.8 0.6 40.7 3.6 10.6 1.03 1.5

Belgium 23.7 3.2 30.8 0.6 69.0 4.7 18.7 0.75 0.8

Czech Republic 19.0 3.1 32.7 0.7 68.5 6.3 14.2 - -

Denmark 13.0 2.1 10.1 0.3 38.3 3.1 13.4 0.28 0.3

Estonia 18.8 2.3 34.8 0.8 60.2 4.9 16.8 0.06 0.6

Finland 19.6 3.0 5.3 0.2 48.2 3.6 15.5 0.32 0.5

France 24.0 2.8 27.3 0.6 62.7 5.4 19.4 0.91 2.3

Germany 7.8 1.6 23.0 0.5 49.2 4.0 10.3 1.29 2.5

Greece 58.3 2.2 52.0 0.8 71.6 4.4 33.1 - -

Hungary 27.2 3.0 33.0 0.6 72.8 4.3 26.1 1.09 1.8

Ireland 26.8 2.2 41.2 0.6 60.3 3.1 22.0 0.27 0.9

Italy 40.0 3.5 53.3 0.9 72.8 3.2 33.7 0.58 1.3

Luxembourg 15.8 3.0 23.0 0.8 74.1 5.9 8.2 0.52 0.3

Netherlands 11.0 1.9 17.0 0.4 30.0 2.4 9.5 0.09 0.1

Norway 9.1 3.1 11.0 0.4 43.0 3.2 10.9 0.34 0.4

Poland 27.3 3.0 31.7 0.7 66.7 4.3 20.2 0.16 0.4

Portugal 38.1 2.5 36.3 0.6 65.0 5.6 22.0 0.34 1.8

Slovak Republic 33.7 2.6 61.3 0.9 69.2 5.4 21.0 0.20 1.0

Slovenia 21.7 2.2 39.4 0.8 66.2 7.1 13.7 0.09 0.6

Spain 55.5 2.3 39.4 0.8 62.2 4.8 32.4 0.27 0.5

Sweden 23.5 3.9 6.9 0.3 45.5 5.0 12.9 0.20 0.2

Switzerland 8.5 2.1 16.9 0.5 32.4 3.2 10.2 - -

United Kingdom 20.7 3.6 28.8 0.7 41.6 2.9 19.1 - -

Source: Eurostat, 2013Youth unemployment rates are based on individuals below 25 years. The youth-adult ratio is based on yearly unemployment rates of youth (1524 years) and adult (2554 years) unemployment levels in 2013

(a)NEET (Neither in Employment nor Education) rates among all inactive youth are depicted for the 20 to 24 year-olds only

(b)Participation shares in ALMP refer to the year 2012 and are calculated as the number of ALMP participants over the stock of unemployed and may be larger than 1 due to multiple participation. The youth-adult ALMP participation ratio divides the share of ALMP participants among youth by the share of ALMP participants among adults

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 5 of 30

those of more senior workers. In particular, this reveals that the youth labor market situation across countries shares several common features. The youth/adult unemployment ratio in column (2) shows that youth are, on average, two to three times more likely to be unemployed than adult workers with the largest differences observed in Sweden, the UK, and Italy. At the same time, the probability of becoming long-term unemployed is commonly substantially lower for youths than for adults. The relative shares of long-term unemployed between youth and adults depicted in column (4) show that youth long-term unemployment risk is, on average, only two thirds the value of adult long-term unemployment.

Part of the differences in the youth-adult (long-term) unemployment risk can be explained by the higher job mobility and the higher likelihood to enter inactivity among youth. As depicted in columns (5) and (6) of Table 1, the inactivity rate3 among youth ranges between below 30% (Netherlands, Switzerland and Iceland) and 70% (Greece, Hungary, Luxemburg and Italy). On average, they are hence about four times more likely to be inactive than adults. While part of this higher inactivity is driven by entry into formal education note that countries with an extensive apprenticeship-based secondary education system in which youth are counted as working, such as Germany, Austria, Denmark and Switzerland, exhibit a much lower inactivity rate youth are generally also less attached to the labor market, and hence more likely to withdraw from labor force. An indicator capturing the size of this risk group is the share of youth neither in education nor in employment (NEET). Column (7) shows that about one third of countries exhibit NEET shares above 20% among the age group of 20 to 24-year-olds, with the Southern European countries (Italy, Portugal, Spain and Greece), some Eastern European countries (Poland, Slovak Republic and Hungary) and Ireland being affected most heavily.

The labor market transition of youth is generally highly correlated with the levels of educational attainment (Quintini et al. 2007). Averaged European unemployment rates from 2013 suggest that youth who have obtained at most a schooling degree at the lower secondary level are 1.7-times more likely to be unemployed than youth who have at most obtained an upper secondary degree, and they about two times more likely to be unemployed than youth with a tertiary education degree. In the Southern European countries that were hit particularly hard during the economic crisis Greece, Italy, Portugal and Spain the unemployment rates of lower secondary and upper secondary school graduates only differ by a factor of 1.1 in favor of the higher educated. In the other European countries, the respective unemployment rates differed by a factor of 1.8 (Eurostat 2013).4

Differences in economic conditions, education and labor market policies may result in systematic cross-country differences in the chances of youth to enter the labor market. As outlined above, youth from countries that were hit hardest by the economic crisis (Southern European countries, Ireland) and that are hence plagued with structurally low labor demand, are more likely to be (long-term) unemployed, are more likely to enter inactivity, and to experience lower returns to education. Several studies underline the close relation between labor market institutions, e.g., hiring policies, minimum wages, and youth unemployment levels (Addison and Teixeira 2003; Bertola et al. 2007; Jimeno and Rodrguez-Palenzuela 2003). The close link between the schooling system and labor market entry of youth is documented most intensely. On the one hand, the general schooling system may affect the tendency of youth to leave school early, i.e. to finishing their

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 6 of 30

education with a degree at the lower secondary or lower level, with the consequence of experiencing more difficulties entering the labor market.

On the other hand, the structure of post-mandatory, professional education options affects the youth labor market transition. In countries with established vocational education tracks or an extensive apprenticeship system, youth participating in vocational education tend to experience a faster entry into the labor market than youth entering the labor market after participating in mostly general education (Hanushek et al. 2011; Neuman and Ziderman 1999; Riphahn and Zibrowius 2015; Ryan 2001; Winkelmann 1996).5 The reason for this faster entry due to vocational education is attributable to an improved short-run alignment of the supply and demand for skills. In particular, skill or qualification mismatch is likely to reduce the quality and stability of initially accepted jobs (Wolbers 2003), with potentially long-lasting negative effects of later-life labor market outcomes (Liu et al. 2012). At the same time, the increasing phenomenon of over-education (Quintini 2011) may result in decreased chances of low-skilled workers finding employment due to crowding-out (Borghans et al. 2000; Dolado et al. 2000). According to Quintini (2011), the Netherlands, Greece, Spain, Sweden, Luxembourg and Portugal are amongst the countries with highest levels of over-qualification.

2.2 The role of active labor market policies

The primary objective of active labor market programs for youth is to integrate unemployed youth into the labor market, stabilize their career entry, and/or to promote the take-up of vocational training as an intermediate step to labor market entry. Column (8) in Table 1 shows that countries exhibit substantial differences with respect to participation in ALMP. While in Slovenia, Estonia and the Netherlands the share of youth in ALMP is below 10% of the unemployed youth population, it exceeds 100% in Austria, Germany and Hungary (meaning that individuals participate in more than one program on average). Column (9) relates the share of unemployed youth participating in ALMP to the share of unemployed adults participating in ALMP. An indicator larger than 1 shows that youth are over-represented in ALMP compared to adults (e.g., in Austria, France and Germany), while an indicator below 1 indicates that they are under-represented (e.g., in Denmark, Estonia and Finland).

The previous Section points to three areas of intervention depending on the source of the labor market problem: the stimulation of labor demand, the avoidance of long-term unemployment, and the elimination of educational mismatch. In case educational mismatch is at the root of the unemployment problem, ALMP training programs serve to align the skill-level of the unemployed youth to that of labor demand. As youth are more prone to engage in further formal education, it is sensible to distinguish short- and long-term training measures. While short-run programs tend to be of remedial nature, aiming to overcome minor and specific skill deficits, long-run training programs aim to overcome more structural skill deficits such as gaps in general education. As the latter types of programs stand in direct competition to formal education, their benefits need to be evaluated relative to regular schooling or training options and are thus likely to be less strong than programs for adults who would otherwise not engage in any education. Also, depending on the types of professional education options available, ALMP training may not be similarly valued as formal education by employers, and participation in them may entail negative stigmatization effects. A smaller part of training ALMP are preparatory training

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 7 of 30

programs that promote the take-up of regular formal education, such as the continuation of general schooling, or participation in apprenticeship-based vocational education.

Since the returns to upgrading skills may be low if the overall demand for youth labor is low as outlined above a second issue to be addressed by ALMP programs is the stimulation of youth labor demand. There are several reasons why the demand for youth labor may be low. First, youth are commonly the ones most affected by economic downturn, as firms may be less willing or able to let go of workers with longer tenure. Second, even under normal economic conditions, employers may prefer hiring more experienced workers, in particular, if previous work credentials or colleague referrals allow employers to discriminate better between low and high ability workers (compare Montgomery, 1991, for a theoretical analysis of employee referrals). Third, in the presence of job-specific human capital, firms may be less willing to hire youth and invest in costly training if there is a high probability that youth leave the firm without redeeming this initial investment (see Wolter and Ryan, 2011, for an extensive summary of firms training involvement). Hence, if low work- or job-specific experience is a barrier to initial labor market entry, the provision of financial incentives for employers to hire and train young people may constitute an effective tool to improve labor market integration. As youth gain more experience and firms are better able to observe their ability, it is intended that youth are offered regular work contracts following the initial subsidy period. Furthermore, Cockx and Picchio (2013) suggest that stigmatization rather than the depreciation of human capital may be the source of state-dependence in long-term unemployment among youth. Following this line of reasoning, wage subsidies promoting the take-up of real employment, albeit subsidized, may help youth signal their employability.

A third issue to be addressed by ALMP is the avoidance of long-term unemployment spells, which were found to have particularly detrimental long-term effects for youth (see Section 1). Job search programs and tight monitoring schemes aim to achieve a fast activation of youth early in the unemployment spell. As youth are more likely to enter inactivity as a response to longer spells in unemployment, these measures may also prevent complete withdrawal of youth from the labor market in the long-run. A potential downside of these approaches is that they may result in a direct withdrawal from the labor market when monitoring and sanction are imposed too fiercely. Studies for adult unemployed find that monitoring and sanctioning increase take-up of employment, but may also result in withdrawal from the labor market (see, e.g., Arni et al. 2013; van den Berg et al. 2013). Because of the higher risk of youth of entering inactivity, higher intensity activation schemes may result in stronger drop-out responses among youths compared to adults. In a similar vein, but targeted at more disadvantaged youth, public employment schemes are used to keep youth in the labor market by offering a work-like environment including low levels of payment. While these types of programs are usually ineffective for adults, they could provide a stepping-stone for youth when combined with training for regular jobs.

A general issue arising from the previous Section is that youth tend to experience higher labor market mobility than adults, the consequences of which on labor market outcomes are ambiguous. On the one hand, job changes may allow an upward movement in the career- and wage-ladder and allow youth to become aware of their skills and preferences (Topel and Ward 1992). On the other hand, it has been found that job-changes promote further job-changes (occurrence dependence) and that early job instability significantly

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 8 of 30

reduces later wages (see, e.g., Doiron and Grgens 2008; Neumark and Wascher 2006). Consequently, ALMP programs should also be measured by their ability to improve the job match quality and stability of accepted employment. If participation in ALMP helps youth to learn about their preferences, career or schooling opportunities, or allow to signal their abilities to employers, this is likely to improve choices and stability of subsequent employment. Against this background it is particularly interesting to assess the long-run effects of, e.g., training measures or financial incentives to employers, as the benefits of these measures may lie in an increased long-run stability of employment.

3 Active labor market policies and their effects

In this Section, we summarize the findings of 37 evaluation studies assessing the impact of ALMP measures for youth in Europe (see Table 3 in the Appendix for a full list of studies). We include evaluation studies that consider programs specifically targeted at unemployed youth, as well as evaluation studies assessing the performance of general ALMP programs including a subgroup-specific analysis for youth. Tables 4 to 7 in the Appendix provide a brief overview of the studies included in the analysis, classified by the type of program under investigation. In line with the most common types of ALMP encountered in practice, four broad types of programs are distinguished: labor market training, job search assistance and monitoring, wage subsidies, and public sector work programs.

We restrict our overview to studies that convincingly address the problem of non-random selection into treatment, e.g., by conducting randomized field experiments, exploiting exogenous variation in program access, controlling for a large set of informative control variables, or using other state-of-the-art econometric evaluation techniques. The bulk of evaluation studies use the conditional independence or unconfoundedness assumption for identification, thus assuming that controlling for observed characteristics is sufficient to capture selection into the different treatment options. The ALMP evaluation literature offers some guidelines regarding the most informative characteristics to be included to control for non-random selection (e.g., Caliendo et al. 2014; Lechner and Wunsch 2013), and we preselected studies adhering to these guidelines. However, it is clear that by the short previous labor market history of youth and the stronger variability in terms of initial labor market attachment, there is a risk that unobserved heterogeneity plays an even stronger role for the youth than for the adult working population.

Next to considering the employment effect of the ALMP programs, we also assess, where available, program impact estimates on education participation and the quality of accepted employment. On the one hand, participation in education constitutes a relevant outcome of interest as youth may wish to continue further formal education once exposed to the treatment rather than entering the labor market directly. Under the assumption that higher levels of human capital result in a better long-term labor market prospects, this may also be a desirable outcome of ALMP programs for youth. On the other hand, as youth tend to consider participation in formal education as an alternative to remaining unemployed, short-run positive employment effects of policies may arise due to the control group entering education rather than remaining unemployed. As it is not clear how the returns to labor market policy programs compare to that of formal education participation, this substitution effect should be kept in mind when evaluating the effect of ALMP for youth.

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 9 of 30

In the next four subsections we briefly describe the programs under consideration, the evidence we collected on their effects, and summarize the main findings; subsection 3.5 condenses the findings across programs and outcome variables and concludes.

3.1 Labor market training

Program Description Labor market training courses are one of the main interventions used to integrate unemployed individuals into employment. The training content can be very heterogenous, ranging from purely classroom-based training to purely on-thejob training within firms, with varying durations between a couple of weeks to over one year. The purpose of these labor market training programs is to extend or to adapt the labor market relevant skill set of participants, so that a faster and more stable integration into employment can be achieved. Training programs within firms serve the additional purpose of providing youth with some level of work experience and send productivity signals to employers, making them potentially more likely to hire participants later on. This additional foot-in-the-door effect of within-company training may be particularly relevant for youth with limited signals of prior labor market participation. It may hence be expected that, all else equal, within-firm training has a better employment effect than formal classroom training. At the same time, by being more practically oriented, firm-based training tends to be offered predominantly to disadvantaged youth with stronger barriers towards direct labor market integration. This differential selection into classroom-based and school-based training programs suggests that the evaluation results for these two programs are not readily comparable.

Evaluation Results In total, we collected evidence from 19 studies in 8 countries. Overall, the employment effects of purely classroom-based labor market training are mixed, showing both positive and zero employment effects. Abstracting from negative locking-in effects during participation, only one of the studies (Norway) finds a negative longer-term employment effect. Three studies also assessing the effect of classroom-based training on unemployment or education show that a positive employment effect often coincides with a zero effect on the unemployment levels (see Table 4 for details) or a negative effect on education. This confirms the hypothesis that classroom-based training may result in a crowding out of formal education or serve as a substitute for it. The close link between ALMP training and formal training participation is also supported by a study from Denmark (1) assessing the effects of announcing labor market training to the unemployed. Here it is found that the threat of expected labor market training results in an increase in education take-up but no increase in the employment probability. It has to be noted that a large share of the evidence of classroom based training comes from Germany (five out of nine studies), but omitting these studies from the aggregation would not change the general conclusion.

The evidence of the employment effects of mixed school- and work-based training is somewhat more scarce and also more mixed: four studies find either positive, zero or negative effects. The negative results are found in the Swedish (3) and the Norwegian study, both examining a period during the early 1990s where both economies experienced a tremendous downturn.6 Hence, the evidence is likely to be affected by these economic circumstances. The French (1) study finds zero (negative)7 effects for the higher educated and positive effects for youth with low previous education, thus suggesting

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 10 of 30

that work-based training is more beneficial for the more disadvantaged and may entail stigmatization for youth with higher schooling.

Studies analyzing the effectiveness of labor market training relative to other types of ALMP tend to find that training performs better or similar to public sector job creation but worse than practical work experience, as provided, e.g., by wage subsidies. The Swedish (5) study suggests that practical training works better than school-based training. Four studies assess the effect of classroom training on the match quality of subsequent jobs, as, e.g., reflected by the wage level or duration of employment spell. These studies mostly find a positive effect, but by the limited amount of evidence, further studies are needed to support this finding.

Summary The overall effectiveness of ALMP training is heterogenous. Purely school-based training has either positive or no effects on employment but negative effects on participation in formal education. The effects of firm-based training or mixtures of firm-based and classroom training are more negative in terms of employment outcomes, but they seem to entail less of a trade-off in terms of education outcomes. While there is much less evidence for firm-based training overall, the negative evidence from Sweden and Norway may not be representative for the overall effectiveness as both pertain to periods with very negative labor market conditions, a situation in which training is likely to be less effective. However, two other studies (Germany and Finland) also show zero overall effects; the only positive evidence comes from France (for youth with low levels of previous education). A potential explanation for the lower effectiveness of mixed training is that the most disadvantaged youth for whom practical training programs are intended may require more intensive or additional support. Negative employment effects for youth with higher levels of previous education could be due to the low reputation of practical training, so that participation in such programs exhibits stigmatization effects.

3.2 Job search assistance and monitoring

Program Description Job search assistance measures comprise counseling and mentoring activities by caseworkers of the public employment services or external providers, including the provision of vacancy information as well as short-term training or coaching programs assisting youth in their application process. A primary focus of these programs is to increase the commitment and motivation of job search among youth. In most countries, this comes with higher intensity support and is also linked to higher levels of monitoring. This, in combination with implicit or explicit threats of benefit sanctioning, is expected to increase compliance with job search requirements and thus increase the exit rate out of unemployment. Previous research points to the double-edged consequences of monitoring and sanctioning schemes, as increased unemployment exit rates in the short-run tend to come at the cost of lower quality job matches and decreased employment stability in the long-run. For youth, the effectiveness of sanctioning is a-priori uncertain, as youth may be more likely to resort to their parents for financial support. Consequently, the threat of or even actual benefit withdrawal may push youth towards non-activity rather than into employment relationships.

Evaluation Results Based on evidence from 16 studies in 8 countries (see Table 5 in the Appendix), we find that counseling and monitoring results in significantly positive

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 11 of 30

employment effects in the majority of evaluation studies . We distinguish between studies of combinations of counseling and monitoring (eight studies), counseling without monitoring (four studies), and monitoring or/and sanctioning only (four studies). We find that the programs involving counseling tend to yield predominantly positive effects, irrespective of whether they involve monitoring. Out of the studies combining job search assistance and monitoring, the studies from Denmark (3), Portugal and Sweden (8) are the only ones that result in a zero or even a negative effect. The Danish study assesses the effects of job search just prior to the start of the economic crisis in 2008 and suggests that the combination of unfavorable economic conditions and a very high baseline activation in the control group may have resulted in a zero effect and an increased exit rate into inactivity among the treated. In Sweden (8) zero effects are found for long-term unemployed youth under difficult economic conditions, suggesting that job search activities may not be sufficient for this difficult target group. The Portuguese study, in contrast, explores a pilot project implemented during fairly beneficial economic conditions. It hence seems more plausible that the results are attributable to other factors, such as the low connectedness of the public employment services with local economic actors (Centeno 2001).

Of the four studies assessing job search assistance alone, three studies explore the effects of changing from public to private providers of assistance. As private providers have higher monetary incentives to place the unemployed in stable employment, these studies aim to capture the effect of more intensive counseling, assuming that the provider change itself does not have an additional employment effect. The studies from Germany (3) and France (4) find positive employment effects of the provider change, while the Swedish study does not find an effect, suggesting that this is due to the high baseline activation for youth at the public employment services. Exploiting random differences in the share of youth changing from public to private providers, the French study shows that a higher counseling intensity at private providers entails significant displacement effects for youth not benefitting from the intensified counseling, in particular in areas with problematic labor market conditions. The overall labor market effect of the intensified counseling in France was zero.

Monitoring alone yields either zero results (in Hungary and Sweden (4)), or short-run positive results (in Germany (8), UK (4)) and long-run negative results due to increased long-run exits from the labor force. The studies for Hungary and Sweden evaluate the effect of an increased (announced) monitoring intensity and do not find any affect on employment rates. While the program set-up differs in many details, both interventions are reported to exhibit rather low actual risks of sanctioning for non-compliance in job search or for turning down job offers, which may hence have resulted in a low level of compliance. The study in the UK, in contrast, considers a policy reform increasing the actual risk of sanctioning, finding that after an initially positive employment effect, the medium-run effects on employment are negative, as youth tend to resort to alternative sources of social benefits, exempting them from any job search activities. Furthermore, negative effects on wage levels are found, the absolute size of which are larger for youth than for adults. The study for Germany (8) assesses the effects of temporary benefit withdrawal and finds substantial positive short-run employment effects that are increasing with the intensity of the sanction. While the effect on employment quality is not available, the authors suggest that the complete withdrawal of benefits may have had substantial negative effects on employment quality or labor force participation.

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 12 of 30

Summary In summary, the evaluation evidence suggests that job search assistance programs exhibit a similar effectiveness for youth as they do for the overall working population (Kluve 2010). Positive long-run effects of job search assistance suggest that counseling may not only help youth find employment faster but also helps them enter more stable employment of higher quality. But further research is needed to corroborate the quality aspects of job search assistance. The study from France (4) suggests that positive long-run results may arise due to displacement of non-treated youth. This would clearly lower the overall benefit of the intensified job search. At the same time, there is some evidence that job search assistance exhibits decreasing marginal returns for the treated. Several countries having developed intensified activation specifically for youth e.g., Denmark, Sweden and Germany - report zero or even negative effects of additional activation (see for a German study on an intensified activation mix Wolff and Nivorozhkin 2012). At the same time, the effectiveness of counseling and monitoring seems to be low under bad economic conditions and for long-term unemployed youth.

The small evidence available regarding the effect of monitoring and sanctioning suggests that youth respond positively to a sanction (threat of sanctioning) in terms of employment outcomes in the short-run, but they may also leave the labor force. Accompanying qualitative evidence of the consequences of sanctions suggests that the unemployed subject to sanctions are commonly those exhibiting more profound difficulties, such as family-related problems, mental illness or drug addictions. Sanctions further tend to aggravate this problem, so it does not seem advisable to use sanctions as a method to activate the most disadvantaged.

3.3 Wage subsidies

Program Description Subsidized wages or income support schemes are primarily aimed at providing a financial incentive for employers to hire youth with lower relative initial productivity. As employers may expect costs of initial training investment, or may need to pay wages exceeding the expected initial productivity, wage subsidies are intended to compensate employers for these incurred costs. For example, in the presence of minimum wage regulations, wage subsidies may bridge the gap between a productivity equivalent payment and the minimum wage. At the same time, if youth are unwilling to work for the low wages offered to them by employers, wage subsidies may increase the wage level and thus the incentives of youth to work. Commonly, wage subsidies are paid for a limited time period (between several months up to one or two years) in the hope that by the time the subsidy expires, youth have sufficiently increased their skill set to be hired under regular working contracts by the same or a different firm. Some programs combine the payment of a wage subsidy with vocational training arrangements in an apprenticeship-type working contract. The level of the subsidy varies; in Germany and the UK it amounted to 40% to 60% of the wage costs. Additionally in the UK, a lump-sum payment was paid for additional training expenses.

Evaluation Results We have identified 8 studies in 6 countries examining wage subsidies (see Table 6 in the Appendix for details). Studies for Belgium, Germany, Norway and Sweden consider the effectiveness of subsidized wage contracts in which training is not a prominent element and find that wage subsidies either increase employment levels or have a zero employment effect. As the programs set up in Norway, Sweden and Germany

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 13 of 30

were very similar, the zero employment effects found for Sweden and Norway may be linked to the very bad labor market conditions mentioned above. The set-up of the Belgium subsidy was somewhat different in that it was only for part-time working women who earned less than the full-time minimum wage. The set-up included an in-built dis-incentive for employers to increase the wages of subsidy-recipients, and it was paid for an unlimited duration as long as the part-time workers were willing to accept fulltime employment if it became available. Despite the risk that the program would become a part-time trap, the program showed substantial positive effects on the take-up of regular employment, which were most likely attributable to the work-experience during the subsidy. The beneficial labor market conditions during the observation period may have also contributed positively to this stepping-stone effect of the subsidy.

Three studies from France (2) and the UK (1 and 3) analyze the effects of wage subsidies in combination with training. In France, low-skilled youth benefitted less from this practical work than from regular short-term work contracts but benefitted similarly compared to public work schemes, which could again be explained by potential stigmatization of practical training schemes. The UK (1) study finds, in contrast, that recipients of wage subsidies exhibit a significantly positive wage growth in the long-run. In a comparative study for the UK (3), it is found that subsidized wages in combination with training performs best in terms of employment compared to all other ALMP programs.

Summary While most of the studies find pronounced positive employment and wage effects of wage subsidies, an obvious caveat of wage subsidy programs is the high likelihood of deadweight effects arising from employers hiring youth with a wage subsidy even though they would have hired the youth without. This is generally difficult to measure and none of the empirical studies addresses this problem. A further issue to be taken into account is the problem of worker substitution, resulting in a zero-sum game, when workers eligible for the subsidy are hired at the expense of un-eligible but substitutable workers (e.g., unemployed below 25 years vs. older than 25 years). Riley and Young (2001) and Van Reenen (2004) assess the magnitude of substitution effects in context of the NDYP in the UK and find little to no evidence of substitution of different age groups. However, as not all youth participating in the NDYP took part in the wage subsidy program, this evidence may not be transferable to more intensive uses of subsidies. Unfortunately, we are unaware of any study addressing the magnitude of the substitution effect of wage subsidies directly.

3.4 Public sector work programs

Program Description Public Sector Work programs are state-funded temporary employment opportunities in the public sector that usually involve production of socially valuable goods or services. The programs are mainly aimed at creating employment opportunities and giving youth some work experience. While they may comprise parallel participation in practical training courses, they are often intended to familiarize inexperienced, disadvantaged youth with a routine work environment. Participants usually receive a low level of remuneration during participation. While this may increase the willingness of youth to participate in the program, this may also have the unintended effect of reducing the search effort for real jobs. Also, as public work programs do not establish contact to real firms or employers, they do not offer a direct way into the labor market.

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 14 of 30

Evaluation Results The evaluation evidence for these programs is largely based on evidence from Germany and France, where we have identified five and two studies, respectively (plus one from the UK, see Table 7 in the Appendix). However, here, a rather homogenous picture arises in that work programs most commonly have zero effect on the employment outcomes. While this may result from a very negative selection of youth into these programs, comparative studies also suggest that public work programs tend to perform worse in terms of employment outcomes than other ALMP options.

Summary The results for the young unemployed participating in public work programs are rather discouraging. Very similar to the results for the adult working population, we find overwhelmingly zero or negative effects. Hence, it is very questionable whether these programs are an adequate solution even in challenging economic situations.

3.5 Summary of results

Table 2 summarizes the findings from Sections 3.1 to 3.4 and provides a brief description of the identified effects for the four types of programs along three outcome dimensions. As we have already mentioned previously, the most common outcome examined is the integration into the first labor market, i.e., into regular employment, and (nearly) all studies provide evidence on that. Since this is not the only interesting outcome, we also summarize the effects on job quality and education where available. The first two outcome dimensions are usually also examined for the adult population, the last one is of interest in the given context of youth unemployment.

Table 2 not only lists the number of studies, but also the number of estimated effects. These numbers can differ because some studies contain effects for different subgroups (e.g., with/without a formal school degree) and or regions (e.g., East and West Germany). If we concentrate on the employment outcome first, we see that the most positive effects are found for job search assistance (12 positive out of 20), followed by wage subsidies (4 positive out of 8). The evidence for labor market training is mixed (11 positive, 12 insignificant and 6 negative), whereas the effect for public work programs is clearly negative (5 negative, 8 insignificant, 1 positive). With respect to education participation and employment quality, evidence is rather scarce. However, the cumulated evidence on training allows for the tentative conclusion that training has a negative effect on formal education participation (7 out of 11) and a positive effect on employment quality (5 out of 9). For the other types of programs, the evidence is too scarce to make a meaningful interpretation.

Differentiating training programs by purely classroom based training, training containing practical elements, and training within the firm, we find that purely classroom based training tends to perform somewhat better (9 positive, 9 zero, 2 negative) than mixed classroom and practical training schemes. As outlined in the text, this differential finding may be attributable to difference in characteristics of the participants or stigmatization effects related to participation in practical training. In contrast, when differentiating job search assistance evidence by studies looking at programs that include counseling versus programs that only include monitoring or sanctioning, the qualitative finding on employment outcomes does not change. Overall, it hence seems that job search assistance and wage subsidies perform similarly well in terms of improving employment outcomes of their participants. Classroom based training often performs similarly well, but risks losing effectiveness if it stands in competition to regular formal education. Unfortunately,

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 15 of 30

Table 2 Overview - Number of studies finding positive, zero or negative effects of ALMP for different outcomes

Impact Employment Education Quality

Labor market + 11 2 5

Training 0 12 2 3

(Studies: 19/ - 6 7 1

Effects: 47) (n/a) (1) (11) (12)

Job search + 12 0 0 Assistance 0 8 3 2

(Studies: 16/ - 1 0 1

Effects: 27) (n/a) (0) (12) (12)

Wage subsidies + 4 1 1

(Studies: 8/ 0 4 1 1

Effects: 13) - 0 1 0

(n/a) (1) (5) (6)

Public work + 1 1 1 Programmes 0 8 2 0

(Studies: 8/ - 5 1 1

Effects: 20) (n/a) (0) (6) (7)

Note: A detailed list of the used studies can be found in Tables 4 to 7 in the Appendix. The number of effects includes effects for different sub-groups within a study

however, a further comparison in terms of education outcomes and employment stability

is not possible due to the lack of evidence.

Unfortunately, most of the evaluation studies under consideration do not address the issue of costs-effectiveness of the respective programs. It is interesting, however, to look at approximate calculations8 based on the program specific expenditure information drawn from the Eurostat Database. This suggests that the average per-capita expenditure is highest for wage subsidies (about 3,200 euros), followed by training programs (2,500 euros), job creation schemes (2,200) and job search services as the least expensive (1,200 euros). The relatively high costs of job creation schemes stand in contrast to their effectiveness, and are likely to be driven by having the disadvantaged youths as focus group. Based on the calculations by Eurofund (2011, p. 62ff.) of the country-specific costs of NEET arising due to yearly foregone earnings in 2008, the per-capita costs incurred by these ALMP programs can be compared with the costs incurred by not having youths in the labor force. This can provide a more tangible assessment of the importance of the costs of maintaining ALMP. This exercise shows that the per-capita cost of wage subsidies amount to about 40% of foregone earnings, 30% of foregone earnings in case of training programs, about 35% for job creation schemes and 10% for job search programs.9 The costs of wage subsidies are quite sizeable, but given their above average effectiveness, they stand out relative to similarly costly job creation schemes that are often found to result in zero or negative employment effects. In contrast, the low relative costs of job search measures in combination with their relatively high effectiveness suggest that they are commonly cost-effective.

4 Conclusions

Reducing youth unemployment is a key challenge in many European countries, especially in the aftermath of the most recent economic slowdown. Youth are a population at risk

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 16 of 30

and exhibit a much higher risk of becoming unemployed than the adult population. The youth-adult unemployment ratio ranged from 1.6 to 3.6 in 2013, but even in relatively calm economic circumstances, as in the pre-crisis years 2005 to 2007, the youth-adult unemployment ratio was between 2 and 3 on average. Keeping in mind that unemployment at early stages can have severe long-term consequences on labor market outcomes as well as on other socio-economic factors, policy-makers use active labor market policies as a potential solution and spend significant resources on them. Most commonly, four broad types of programs are distinguished: labor market training, job search assistance and monitoring, wage subsidies, and public sector work programs.

We review the existing evidence on the effectiveness of these different ALMP programs from evaluation studies in Europe addressing the potential problem of selection bias in a convincing way. What becomes immediately apparent is that the overwhelming majority of studies only assess the effects on employment take-up. While this is understandable given the main objective (and also comparable to the adult population), it might be too shortsighted for the youth population. Here, additional outcome dimensions such as the take-up up of higher formal education or the job quality should also be considered.

The presented evidence shows differences in the results across but also within different ALMP types. Job search assistance with and without monitoring results in overwhelmingly positive effects. These programs are usually not very cost intensive and seem to work in many different circumstances. However, it also seems that too much job search activation can result in zero or even negative employment effects, suggesting that they do not provide the optimal solution for all youth. Similarly, the question of whether job search and monitoring also lead to better jobs has not been addressed sufficiently by the literature. The few studies considering employment quality find rather mixed effects, suggesting, however, that harsh monitoring and sanctioning schemes result in negative employment outcomes.

Labor market training programs are most commonly used for youths and have been studied most extensively. The overall results are somewhat mixed: for less than half of the programs/sub-groups, positive effects are found; and for the majority, we find insignificant or even negative effects. The negative findings are predominantly driven by the performance of combinations of classroom and practical training, as well as classroom-based training programs under adverse economic conditions in which an upgrading of skills may not be sufficient to improve labor market integration chances in the short- to medium-run. Excluding these studies, the evidence on classroom-based training is similarly positive compared to that of job search assistance. A particularity of classroom-based training ALMP is that it may reduce the take up of formal education. This crowding out of formal training is clearly an issue to be addressed in further research.

The hopes one might have had with public work programs are not met. Very similar to the results for the adult population, only one of the reviewed studies shows positive effects. Instead of providing a bridge to regular employment, they seem to entail locking-in or even stigmatization effects. It is thus questionable whether they should be used in the current (or any) situation. In contrast, the effects of wage subsidies on employment take-up are always positive (or insignificant), suggesting that subsidized real work experience often provides a stepping stone in regular employment. Wage subsidies evaluated under bad economic conditions were found to perform less well, suggesting that they are no

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 17 of 30

panacea for stimulating labor demand. Unfortunately, little is known about the effects on job quality or stability.

Overall, the aggregate evidence of the effectiveness of ALMP is somewhat discouraging, suggesting that some but not all elements of ALMP programs can be a solution for the youth unemployment problem. The evidence also raises the question of whether the money spent on ALMP for the young unemployed is well invested or whether the money should be used to tackle potential problems earlier. While investing in early education is definitively favorable (Heckman 2006), these investments have a long-term horizon and will not be a solution for all youth at risk.

In particular, it remains an important challenge for policy-makers to devise activation schemes that benefit youth with very low skills and motivation and/or socially disadvantaged backgrounds. While job search and monitoring programs seem to be a very promising start for many youth, neither of the programs under study were targeted to the benefit of the most disadvantaged. Similarly, training programs with practical elements that tend to be targeted at youth with low initial schooling are found not to work very well. Promising strategies for disadvantaged youth may comprise intensive mentoring programs (see Rodriguez-Planas 2012 for evidence on the U.S.) but also programs that take into account social or health-related factors that may be at the source of a difficult labor market entry (currently ongoing projects include Crpon et al. 2015; van den Berg et al. 2015).

Similarly, more policy emphasis should be given to promoting the take-up of stable employment relationships. Helping youth to learn about their preference by personalized coaching, information about career opportunities, or structured labor market entry program could help improve the career choices made by youth. Saniter and Siedler (2014) show that the introduction of occupational counseling centers in Germany resulted in smoother labor market entry among affected school leavers. Similarly, it is expected that training programs might also bring the largest return in terms of human capital investment in the long-run. Unfortunately, most of the studies summarized here were not able to identify long-term effects. The observation period of most examined studies ranges from several months to two years (with very few exceptions), thus hardly qualifying as long-term. The review hence also highlights the shortcomings in current research with respect to the long-term consequences of policy measures. Further research should put a stronger focus on education and job quality outcomes in the long-term perspective.

Endnotes

1Based on unemployment rates for youths (aged 15 and 24) and adults (aged 25 and54) in 2008 and 2013 in the EU-27; provided by Eurostat.

2The European Labor Force Survey defines unemployment as without work during the reference week, currently available for work and either actively seeking work in the last four weeks, or expecting to start a job within the next three months. The unemployment period is defined as the duration of job search or the time since the last job was held.

3Inactivity is defined as not working and not searching for work and/or unavailable for work. The concept of inactivity is hence not related to receipt of unemployment benefits, which would affect youth disproportionately as they had less time to build up unemployment benefit claims.

4The country-average is based on Belgium, Czech Republic, Denmark, Germany, Estonia, Ireland, Greece, Spain, France, Italy, Luxemburg, Hungary, the Netherlands, Austria, Poland, Portugal, Slovenia, Finland, Sweden and the UK.

5European countries with the most developed vocational education and apprenticeship systems are, e.g., Germany, Austria, Switzerland, Denmark and partially

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 18 of 30

also the Netherlands. See Eichhorst et al. (2015) for a more detailed definition and assessment of the role of vocational education in industrialized countries.

6The Swedish unemployment rates more than tripled during the time period of investigation (from 3% to over 9%). In Norway, the unemployment rate during the period of investigation was at an all-time high at 5%.

7Depending on the specification; unfortunately the authors do not provide guidance of which of the two estimates is more reliable.

8According to the share of young participants in the respective program types, total expenditure information for youths were calculated for each country and program-type assuming that the program cost for youth and adult participants are similar. Price-differences across countries are accounted for by dividing country-specific costs by the purchasing power parities for government services. To increase reliability we only included countries that provided expenditures on at least two types of programs, and averaged expenditure information available between 2010 and 2013 whenever possible, otherwise the latest information available was used. Countries included are BE, DE, EE, ES, FR, LV, LT, AT, PL, PT, SI, SK, FI, SE.

9The increase in relative costs of job creation schemes suggests that countries with low foregone earnings invest relatively more on job creation schemes than on other measures.

AppendixTable 3 Evaluation literature by Countries

Country Author

Austria Winter-Ebmer (2006)

Belgium Cocky et al. (2013)

Denmark (1) Graversen and van Ours (2008) Denmark (2) Jensen et al. (2003)

Denmark (3) Maibom et al. (2014)

Denmark (4) van den Berg et al. (2012) Finland Hmlinen and Ollikainen (2004) France (1) Bonnal et al. (1997)

France (2) Brodaty et al. (2011)

France (3) Cavaco et al. (2004)

France (4) Crpon et al. (2013)

France (5) Fougre et al. (2009)

Germany (1) Achatz et al. (2012)

Germany (2) Bernhard and Kruppe (2012) Germany (3) Bernhard and Wolff (2008) Germany (4) Caliendo et al. (2008)

Germany (5) Caliendo et al. (2011)

Germany (6) Hartig et al. (2008)

Germany (7) Hohmeyer and Wolff (2007) Germany (8) van den Berg et al. (2014) Germany (9) Wolff and Jozwiak (2007) Germany (10) Wolff et al. (2010)

Hungary Micklewright and Nagy (2010) Norway Hardoy (2005)

Portugal Centeno et al. (2009)

Sweden (1) Bennmarker et al. (2013) Sweden (2) Carling and Larrson (2005) Sweden (3) Costa Dias et al. (2013)

Sweden (4) Engstrm et al. (2012)

Sweden (5) Forslund and Skans (2006) Sweden (6) Larrson (2003)

Sweden (7) Andrn and Gustafsson (2004) Sweden (8) Hgglund (2014)

UK (1) Bell et al. (1999)

UK (2) Blundell et al. (2004)

UK (3) Dorsett (2006)

UK (4) Petrongolo (2009)

UK (5) van den Berg et al. (2014)

CaliendoandSchmidlIZAJournalofLaborPolicy(2016) 5:1 Page19of30

Table 4 Evaluation studies on labor market trainingAuthor Country Treatment Sample Horizon Estimation Employment Unemployment Education Quality/Wages

Winter-Ebmer (2006) Austria School-based, full-time retraining courses and coaching

Unemployed

26, lay-offs at

Steel Factory

Five years after treatment

Tobit IV 0 n/a n/a +

Up to 10 months after introduction

Competing risk duration model

0 n/a + 0

Jensen et al. (2003) Denmark (1) Introduction of vocational education program with 50% reduction of UB during participation

Unemployed 16 to 24 years, long-term unemployed, low education levels.

LMT: +

YPT: 0

LMT 0 YPT: 0

LMT -YPT: 0

LMT +

YPT: 0

Hmlinen and Ollikainen (2004) Finland Labor Market Training school-based, various durations Youth Practical Training: paid training within a firm

First-time unemployed youth, between 16 and 30 years

5 years after entry Propensity ScoreMatching

Directly after participation, 612 months and > 12 months after participation

Multiproportional hazard model with unobserved heterogeneity

Bonnal et al. (1997) France (1) Workplace training on a temporary contract

Young men who were less than 26 years, low or high levels of education

Low educ: +

high educ: 0/-

Low educ: -high educ: 0/+

n/a + (probability

of employment over 1 year)

Brodaty et al. (2011) France (2) General or job-oriented practical training

Low-skilled unemployed below 27 years

Up to 6 month after treatment

Propensity Score Matching with PS derived from a competing-risk duration model

0 (relative to JCS) - relative to fixed-term contracts

n/a n/a n/a

Cavaco et al. (2004) France (3) Retraining and job seeking assistance for 6 months

Recently displaced workers with min 2 years experience in the firm, < 25 years

Up to 3 years after program entry

Generalized Tobit Model with multiple selectivity criteria

+ n/a n/a n/a

CaliendoandSchmidlIZAJournalofLaborPolicy(2016) 5:1 Page20of30

Table 4 Evaluation studies on labor market training (Continued) Achatz et al. (2012) Germany (1) Short coaching or classroom training

1830 year old recipients of means-tested UB benefits entering treatment within 9 weeks of start of receipt

Up to 30 months Propensity ScoreMatching +

n/a n/a n/a

Bernhard and Kruppe (2012) Germany (2) Medium- to long-term labor market training based on vouchers

Unemployed between 15 to 24

Up to 28 month after program entry

Propensity Score Matching +

0 n/a n/a

Caliendo et al. (2011) Germany (5) Classroom based training programs

Unemployment entrants up to 25 years, East and West separately, high/low previous education (lE,hE,lW,hW)

Up to 5 years after treatment entry

Propensity Score Weighting

n/a

lE: 0 hE: +

lW: + hW: +

n/a lE: - hE: -lW: - hW: -

Hartig et al. (2008) Germany (6) Short coaching or classroom training

Unemployed between 15 to 25 years, receiving means-tested benefits, by East/West/ Male/Female (Wm, Wf, Em, Ef)

n/a n/a

Up to 25 months after treatment entry

Propensity Score Matching

Wm: +

Wf: 0 Em: 0 Ef: 0

Wm: 0 Wf: 0 Em: 0 Ef: 0

Wolff and Jozwiak (2007) Germany (9) Short classroom training

Unemployed 1524 receiving means-tested unemployment benefits

Up to 20 months after program start

Propensity Score Matching

Men: +

women: 0

Men: 0 women: +

n/a n/a

Hardoy (2005) Norway Mix of classroom and firm-based training, max 6 months with potential subsequent program participation

Unemployment entrants between 16 to 25

2 years after unemployment entry

Structural discrete choice model with selection

- + (also ALMP

participation)

- n/a

CaliendoandSchmidlIZAJournalofLaborPolicy(2016) 5:1 Page21of30

Table 4 Evaluation studies on labor market training (Continued) Hardoy (2005) Norway Classroom training for work-related skills, upto 5 months

Unemployment entrants between 16 to 25

2 years after unemployment entry

Structural discrete choice model with selection

- + (also ALMP

participation)

- s n/a

n/a - n/a n/a

Carling and Larrson (2005) Sweden (2) Youth Guarantee(workplace training, classroom trainig, mix of the two) within 100 days of unemployment entry

20 to 24 year old unemployment entrants

18 months years after unemployment entry

DIDID, using variation in time, age and municipality dimensions

Costa Dias et al. (2013) Sweden (3) Practical labor market training (mostly work)

Unemployment men 20 to 24 years

1 and 2 years after unemployment registrations

Propensity Score Matching with IV-correction exploiting age-eligibility cut-offs

- + + (upon request) n/a

Forslund and Skans (2006) Sweden (5) Youth practice vs. classroom training

Unemployed youth between 20 and 24

Up to two years after program entry

Propensity Score Matching, Regression

+ 0 n/a +

Larrson (2003) Sweden (6) Labor market training of different durations

First-time unemployed youth 20 to 24 years

One and two years after program start

Propensity Score Matching

0 n/a 0 -

Selection model with random coefficient

n/a n/a n/a T1: 0

T2: +

T3: +/0

Andrn and Gustafsson (2004) Sweden (7) Labor market training Unemployed entering training 2025, different cohorts: T1(84/85), T2(87/88), T3(90/91)

Average of three years after treatment entry

n/a n/a

Dorsett (2006) UK (3) Full-time education

(FTET) relative to other NDYP-options

Unemployed male UB benefit claimants for longer than 6 months, 18 to 24 years

Up to 1.5 years Propensity Score

Matching

-/0 (relative to JCS, WS, and extended job search)

+ (rel. to WS),

-/0 (relative to JCS and extended job search)

CaliendoandSchmidlIZAJournalofLaborPolicy(2016) 5:1 Page22of30

Table 5 Evaluation studies on job search assistance and monitoringAuthor Country Treatment Sample Horizon Estimation Employment Unemployment Education Quality/Wages

Graversen and van Ours (2008) Denmark (1) Intensive contact with caseworkers and job search program with potential subsequent training participation

Unemployment entrants < 30 years

10 to 15 weeks after unemployment entry (before training entry)

Randomized Controlled Trial +

n/a n/a n/a

Maibom et al. (2014) Denmark (3) Information letter, and intensified caseworker meeting, and mentoring, training participation.

Short- or long-term unemployed youth below 30 years, with or without qualifying education

Up to 3 years after entry into program

Randomized Controlled Trial

Uneducated: -educated: 0

n/a Uneducated: 0 educated: 0

n/a

van den Berg et al. (2012) Denmark (4) Meeting with the caseworker (counseling or monitoring)

Unemployed men and women with UB receipt, below 30 years

(Instantaneous effect)

Timing-of-events + n/a n/a n/a

Crpon et al. (2013) France (4) Job search assistance by by private providers

University graduates (under 30 years), at least 6 months in unemployment

8 month Randomized

Controlled Trial +

(partially due to displacement)

n/a n/a n/a

Fougre et al. (2009) France (5) Vacancies information by the public employment service

Registered unemployed job seekers

Max. 1.5 years Structural partial job search model +

n/a n/a n/a

Bernhard and Wolff (2008) Germany (3) Contracting out placement services to private providers

Recipients of social welfare, 15 - 24, East and West Germany, males and females (Wm, Wf, Em, Ef)

Up to 20 month Propensity Score

Matching

Em: + Ef: +

Wm: 0 Wf: 0

Em: 0 Ef: +

Wm: + Wf: +

n/a n/a

van den Berg et al. (2015) Germany (8) Partial or complete withdrawal of social benefits (sanctions) for up to 3 months

Unemployed social welfare recipients 18 to 24 years

Immediate effect of sanction

Timing-of-events + - (only for single

households)

n/a -

CaliendoandSchmidlIZAJournalofLaborPolicy(2016) 5:1 Page23of30

Table 5 Evaluation studies on job search assistance and monitoring (Continued) Micklewright and Nagy (2010) Hungary Increased job searchmonitoring for 4months

New UI claimants with 75179 days of UI entitlement

Up to 3 months Randomized Controlled Trial

0 n/a n/a n/a

Centeno et al. (2009) Portugal Early intervention, including job search assistance, counseling, monitoring and training, threat of sanctions

Youths below the age of 25 who were unemployed less than 6 months

12 and 24 months after participation

Difference-in-Difference Matching

0 n/a 0 n/a

Bennmarker et al. (2013) Sweden (1) Assignment to private placement providers, up to 6 months

Unemployed less than 25 years, with unemployment duration > 3 months

Up to 12 months after assignment

Randomized Controlled Trial

0 n/a n/a 0

0 n/a n/a n/a

Engstrm et al. (2012) Sweden (4) Increased threat of monitoring of job search activity

Unemployed less than 30 years

4 to 8 months Randomized Controlled Trial

Hgglund (2014) Sweden (8) Increased intensity of job search activities and monitoring

Regular and long-term unemployed between 18 to 24 years

Up to 3 years Randomized

Controlled Trial

regular: +

long-term: 0

regular: 0 long-term: 0

n/a regular: +

long-term: 0

Blundell et al. (2004) UK (2) Intensified job search assistance/mentoring and wage subsidies, threat of sanctions

Unemployed, 18 to 24 years, longer than 6 months unemployed

First 4 months after program entry

DID using age and regional variation in eligibility

Men: + (partially

due to wage subsidy) women: 0

n/a n/a n/a

3 months and 6 months

DID approach Short-run: +

long-run: - +

n/a -

Petrongolo (2009) UK (4) Reduction of amount and duration of UB and introduction of job search requirements (diary)

Male unemployment entrants 16 to 24

CaliendoandSchmidlIZAJournalofLaborPolicy(2016) 5:1 Page24of30

Table 5 Evaluation studies on job search assistance and monitoring (Continued) Dorsett (2006) UK (3) Extended job searchrelative to otherNDYP options

Unemployed male UB benefit claimants for longer than 6 months, 18 to 24 years

Up to 1.5 years

Propensity Score Matching +

(rel. to all other options)

0 (rel. to WS);- (rel. to training and JCS)

n/a n/a

van den Berg et al. (2014) UK (5) Enrollment in NDYP -

Gateway period (job search)

Unemployed for at least 6 months, 1824 years

One month Kernel hazard regression exploiting random timing of introduction

+ n/a n/a n/a

CaliendoandSchmidlIZAJournalofLaborPolicy(2016) 5:1 Page25of30

Table 6 Evaluation studies on wage subsidiesAuthor Country Treatment Sample Horizon Estimation Employment Unemployment Education Quality/Wages

Cockx et al. (2013) Belgium Receipt of UB during part-time work

Timing-of-events + n/a n/a n/a

Women aged 18 to 25 years, unemployed for at least 9 months

Up to 12 months after the end of the program

Caliendo et al. (2011) Germany (5) Wage subsidies up to one (two) years

Up to 5 years after treatment entry

n/a - n/a

Unemployment entrants up to 25 years, East and West separately

Propensity Score Weighting +

Brodaty et al. (2011) France (2) General or job-oriented practical training

Low-skilled unemployed below 27 years

Up to 6 month after treatment

PSM with PS derived from a competing-risk duration model

0 (relative to JCS) - relative to fixed-term contracts

n/a n/a n/a

Hardoy (2005) Norway Subsidized employment in the public and in the private sector

Unemployment entrants between 16 to 25

2 years after unemployment entry

Structural discrete choice model with selection

0 - (also ALMP participation) +

n/a

Larrson (2003) Sweden (6) Youth practice(subsidized work) up to 6 months and

Youth 20 to 24 years, first-time unemployed for at least 4 months

One and two years after program start

Selection on Observables Assumption (Propensity Score Matching)

0 n/a 0 0

Bell et al. (1999) UK (1) Wage subsidies Unemployed UB benefit claimants for longer than 6 months, 18 to 24 years

Up to 1 year Trend-adjusted DID n/a n/a n/a +

First 4 months after program entry

DID using age and regional variation in eligibility

Men: +

women: 0

n/a n/a n/a

Blundell et al. (2004) UK (2) Intensified job search assistance/ mentoring and wage subsidies, threat of sanctions

Unemployed, 18 to 24 years, longer than 6 months unemployed

Dorsett (2006) UK (3) Wage subsidies relative to other NDYP-options

Unemployed male UB benefit claimants for longer than 6 months, 18 to 24 years

Up to 1.5 years Propensity Score

Matching +

(rel. to all other options)

- (rel. to all other options)

n/a n/a

CaliendoandSchmidlIZAJournalofLaborPolicy(2016) 5:1 Page26of30

Table 7 Evaluation studies on public work programsAuthor Country Treatment Sample Horizon Estimation Employment Unemployment Education Quality/Wages

(Bonnal et al. (1997) France (1) Public sector working schemes

Young men who were less than 26 years, low or high levels of education

1 month, 612 months and > 12 months after participation

Multiproportional hazard model with mutiple spells and unobserved heterogeneity

n/a (proba of employment over 1 year) low educ: +

high educ: -

Low educ: 0 high educ: -

(proba of unemployment over 1 year) low educ - high educ 0

Brodaty et al. (2011) France (2) Job creation schemes, part-time, minimum wage receipt

Long-term unemployed, low-skilled below 25 years

Up to 6 months after treatment

Propensity Score Matching with PS derived from a competing-risk duration model

0 (relative to general or practical training) - relative to fixed-term contracts

n/a n/a n/a

Achatz et al. (2012) Germany (1) Public-Work-Program (One-Euro Jobs), up to 6 months, 30 hours per week

1830 year old recipients of means-tested UB benefits entering treatment within 9 weeks of start of receipt, male/female

Up to 30 months Propensity ScoreMatching

Men: -women: 0 single-mothers with children: -

n/a n/a n/a

Caliendo et al. (2008) Germany (4) Job Creation Schemes(JCS)

Unemployed less than 25 years

Up to 35 months Propensity Score

Matching

0 n/a n/a n/a

Caliendo et al. (2011) Germany (5) Job creation schemes Unemployment entrants up to 25 years, East and West separately

Up to 5 years after treatment entry

Propensity Score Weighting

East: -West: 0

n/a East: -

West: 0

n/a

Hohmeyer and Wolff (2007) Germany (7) Public-Work-Program

(One-Euro Jobs)

Social welfare recipients below the age of 25

Up to 25 months Propensity Score

Matching

West: 0 East: 0

West: +

East: 0

n/a n/a

CaliendoandSchmidlIZAJournalofLaborPolicy(2016) 5:1 Page27of30

Table 7 Evaluation studies on public work programs (Continued) Wolff et al. (2010) Germany(10)

Public-Work-Program (One-Euro Jobs)

Unemployed between 15 to 24 years (male, female in East and West, Em/Ef/Wm/Wf)

Up to 28 month

Propensity Score Matching

Em/Ef/Wm/Wf: 0 Ef: +

Em/Wm/Wf: 0

Em: +

Ef/Wm/Wf: 0

n/a

Dorsett (2006) UK (3) Job creation schemes relative to other NDYP options

Unemployed male UB benefit claimants for longer than 6 months, 18 to 24 years

Up to 1.5 years Propensity Score

Matching

- (rel. to all other options) +

(rel. to all other options)

n/a n/a

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 28 of 30

Competing interests

The IZA Journal of Labor Policy is committed to the IZA Guiding Principles of Research Integrity. The authors declare that they have observed these principles.

Acknowledgements

The authors thank Gerard van den Berg, one anonymous referee, and the editor for helpful comments and Juliane Hennecke and Claudia Stier for valuable research assistance.

Author details

1University of Potsdam, IZA Bonn, DIW Berlin, IAB Nuremberg, Potsdam, Germany. 2University of Mannheim, IZA Bonn, Mannheim, Germany. 3University of Potsdam, Chair of Empirical Economics, August-Bebel-Str. 89, 14482 Potsdam, Germany.

Received: 2 June 2015 Accepted: 18 November 2015

References

Achatz J, Fehr S, Schels B, Wolff J (2012) Ein-Euro-Jobs, betriebliche und schulische Trainingsmanahmen: Wovon junge

Arbeitslose im SGB II am meisten profitieren. IAB-Kurzbericht 6, IAB NurembergAddison J, Teixeira P (2003) The economics of employment protection. J Lab Res 24(1):85128Andrn T, Gustafsson B (2004) Income effects from labor market training programs in Sweden during the 1980s and

1990s. Int J Manpow 25(8):688713

Arni P, Lalive R, Van Ours JC (2013) How effective are unemployment benefit sanctions? Looking beyond unemployment exit. J Appl Econ 28(7):11531178Arulampalam W (2001) Is unemployment really scarring? Effects of unemployment experiences on wages. Econ J

111(475):F585F606

Bell B, Bindler A, Machin S (2014) Crime scars: recessions and the making of career criminals. CEP Discussion Papers dp1284, Centre for Economic Performance, LSE, LondonBell B, Blundell R, Van Reenen J (1999) Getting the unemployed back to work: the role of targeted wage subsidies. Int Tax

Public Financ 6(3):339360

Bell DN, Blanchflower DG (2010) Youth unemployment: Dj Vu?. IZA Discussion Papers 4705, Institute for the Study of

Labor (IZA), Bonn

Bennmarker H, Grnqvist E, ckert B (2013) Effects of outsourcing employment services: evidence from a randomized experiment. J Public Econ 98:6884Bernhard S, Kruppe T (2012) Effectiveness of further vocational training in Germany. Empirical findings for persons receiving means-tested unemployment benefits. Schmollers Jahrbuch: Zeitschrift fr Wirtschafts- und Sozialwissenschaften 132(4):501526Bernhard S, Wolff J (2008) Contracting out placement services in Germany. Is assignment to private providers effective for needy job-seekers? Discussion Paper 5, IAB NurembergBertola G, Blau FD, Kahn LM (2007) Labor market institutions and demographic employment patterns. J Popul Econ

20(4):833867

Blundell R, Costa Dias M, Meghir C, Van Reenen J (2004) Evaluating the employment impact of a mandatory job search program. J Eur Econ Assoc 2(4):569606Bonnal L, Fougre D, Srandon A (1997) Evaluating the impact of French employment policies on individual labour market histories. Rev Econ Stud 64(4):683713Borghans L, de Grip A, Network E (2000) The overeducated worker?: The economics of skill utilization, Elgar Monographs.

Edward Elgar Publishing, Cheltenham

Brodaty T, Crpon B, Fougre D (2011) Using matching estimators to evaluate alternative youth employment programs: evidence from France, 19861988. In: Lechner M, Pfeiffer F (eds). Econometric Evaluation of Labour Market Policies, no. 13 in ZEW Economic Studies. Physica-Verlag, HeidelbergCaliendo M, Hujer R, Thomsen S (2008) Identifying effect heterogeneity to improve the efficiency of job creation schemes in Germany. Appl Econ 40(9):11011122Caliendo M, Knn S, Schmidl R (2011) Fighting youth unemployment: the effects of active labor market policies. IZA

Discussion Papers 6222, Institute for the Study of Labor (IZA), Bonn

Caliendo M, Mahlstedt R, Mitnik O (2014) Unobservable, but unimportant? The influence of personality traits (and other usually unobserved variables) for the evaluation of labor market policies. IZA Discussion Paper 8337, Institute for the Study of Labor (IZA), BonnCard D, Kluve J, Weber A (2010) Active labour market policy evaluations: a meta-analysis. Econ J 120(548):452477

Carling K, Larrson L (2005) Does early intervention help the unemployed youth? Labour Econ 12(3):301319 Cavaco S, Fougre D, Pouget J (2004) Conventions de conversion et retour lemploi. conomie et Prvision

164(3):93111

Centeno L (2001) Inserjovem and Reage initiatives : a preventive action, Final report. ec peer review programme.

European Commission, Brussels

Centeno L, Centeno M, Novo A (2009) Evaluating job-search programs for old and young individuals: heterogeneous impact on unemployment duration. Labour Econ 16(1):1225Choudhry MT, Marelli E, Signorelli M (2012) Youth unemployment rate and impact of financial crises. Int J Manpow

33(1):7695

Cockx B, Goebel C, Robin S (2013) Can income support for part-time workers serve as a stepping-stone to regular jobs?

An application to young long-term unemployed women. Empir Econ 44(1):189229Cockx B, Picchio M (2013) Scarring effects of remaining unemployed for long-term unemployed school-leavers. J R Stat

Soc Ser A (Statistics in Society) 176(4):951980

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 29 of 30

Costa Dias M, Ichimura H, van den Berg GJ (2013) Treatment evaluation with selective participation and ineligibles. J Am

Stat Assoc 108(502):441455

Crpon B, Duflo E, Gurgand M, Rathelot R, Zamora P (2013) Do labor market policies have displacement effects? Evidence from a clustered randomized experiment. Q J Econ 128(2):531580Crpon B, Pernaudet J, Romanello L (2015) Increasing health investments among unemployed young people: Evidence from a French experiment. Working Paper, ParisDaly M, Delaney L (2013) The scarring effect of unemployment throughout adulthood on psychological distress at age

50: Estimates controlling for early adulthood distress and childhood psychological factors. Soc Sci Med 80(0):1923 Doiron D, Grgens T (2008) State dependence in youth labor market experiences, and the evaluation of policy interventions. J Econ 145(12):8197. The use of econometrics in informing public policy makersDolado JJ, Felgueroso F, Jimeno JF (2000) Youth labour markets in Spain: Education, training, and crowding-out. Eur Econ

Rev 44(4):943956

Dorsett R (2006) The new deal for young people: effect on the labor market status of young men. Labour Econ 13:405422 Eichhorst W, Rodrguez-Planas N, Schmidl R, Zimmermann KF (2015) A road map to vocational education and training in industrialized countries. ILR Review 68(2):314337Engstrm P, Hesselius P, Holmlund B (2012) Vacancy referrals, job search, and the duration of unemployment: a randomized experiment. LABOUR 26(4):419435Eurofund (2011) NEETs: Young people not in employment, education or training: Characteristics, costs and policy responses in Europe. Discussion paper, The European Foundation for the Improvement of Living and Working Conditions, LuxemburgEuropean Commission (2008) Demography Report 2008: Meeting social needs in an ageing society. Discussion Paper

SEC(2008) 2911, Directorate-General for Employment, Social Affairs and Equal Opportunitie, BrusselsEuropean Commission (2014) Draft Joint Employment Report from the Comission and the Council accompanying the

Commission on the Annual Growth Survey 2015. COM(2014) 906, European Commission, BrusselsEurostat (2013) European Union Labour Force Survey. Available at http://ec.europa.eu/eurostat/data/database

Web End =http://ec.europa.eu/eurostat/data/database, accessed on 03/2015Forslund A, Skans ON (2006) Swedish youth labor market policies revisited. Discussion Paper 6, IFAU - Institute for

Evaluation of Labour Market and Education Policy, Uppsala

Fougre D, Kramarz F, Pouget J (2009) Youth unemployment and crime in France. J Eur Econ Assoc 7(5):909938 Fougre D, Pradel J, Roger M (2009) Does the public employment service affect search effort and outcomes? Eur Econ

Rev 53(7):846869

Goldsmith A, Veum J, Darity W (1997) Unemployment, joblessness, psychological well-being and self-esteem: Theory and evidence. J Socio-Econ 26:133158Graversen BK, van Ours JC (2008) How to help unemployed find jobs quickly: Experimental evidence from a mandatory activation program. J Public Econ 92(1011):20202035Gregg P (2001) The impact of youth unemployment on adult unemployment in the NCDS. Econ J 111(475):626653 Gregg P, Tominey E (2005) The wage scar from male youth unemployment. Labour Econ 12(4):487509Grubb M (1999) Lessons from education and training for youth: five precepts. In: Preparing Youth for the 21st Century:

The transition from education to the labour market. OECD Publishing, Paris

Hanushek EA, Woessmann L, Zhang L (2011) General education, vocational education, and labor-market outcomes over the life-cycle. IZA Discussion Papers 6083, Institute for the Study of Labor, (IZA), BonnHardoy I (2005) Impact of multiple labour market programmes on multiple outcomes: the case of Norwegian youth programmes. Labour 19(3):425467Hartig M, Jozwiak E, Wolff J (2008) Trainingsmanahmen: Fr welche unter 25-jhrigen Arbeitslosengeld-II-Empfnger erhhen sie die Beschftigungschancen? Forschungsbericht 6, IAB NurembergHeckman JJ (2006) Skill formation and the economics of investing in disadvantaged children. Science

312(5782):19001902

Heckman, JJ, LaLonde RJ, Smith JA (1999) The economics and econometrics of active labor market programs. Handb

Labor Econ 3:18652097

Hgglund P (2014) Experimental evidence from active placement efforts among unemployed in Sweden. Eval Rev

38(3):191216

Hmlinen K, Ollikainen V (2004) Differential effects of active labour market programmes in the early stages ofyoung peoples unemployment. Research Reports 115, Government Institute for Economic Research Finland (VATT), HelsinkiHohmeyer K, Wolff J (2007) A fistful of Euros: does one-euro-job participation lead means-tested benefit recipients into regular jobs and out of unemployment benefit II receipt? Discussion Paper 32, IAB NurembergJensen P, Rosholm M, Svarer M (2003) The response of youth unemployment to benefits, incentives, and sanctions. Eur J

Polit Econ 19:301316

Jimeno JF, Rodrguez-Palenzuela D (2003) Youth unemployment in the OECD: demographic shifts, labour market institutions and macroeconomic shocks. Economics Working Papers 019, European Network of Economic Policy Research Institutes, BrusselsKahn LB (2010) The long-term labor market consequences of graduating from college in a bad economy. Labour Econ

17(2):303316

Kluve J (2010) The effectiveness of European active labor market programs. Labour Econ 17(6):904918Kluve J, Schmidt CM, van Ours JCv, Vandenbussche H (2002) Can training and employment subsidies combat European unemployment? Econ Policy 17(35):409448Larrson L (2003) Evaluation of Swedish youth labor market programs. J Human Res 38(4):891927Lechner M, Wunsch C (2013) Sensitivity of matching-based program evaluations to the availability of control variables.

Labour Econ 21:111121

Liu K, Salvanes KG, Srensen E (2012) Good skills in bad times: cyclical skill mismatch and the long-term effects of graduating in a recession. IZA Discussion Papers 6820, Institute for the Study of Labor (IZA), Bonn

Caliendo and Schmidl IZA Journal of Labor Policy (2016) 5:1 Page 30 of 30

Maibom J, Rosholm M, Svarer M (2014) Can active labour market policies combat youth unemployment Nordic

Economic Policy Rev 1:215262

Martin JP, Grubb D (2001) What works and for whom: a review of OECD countries experiences with active labour market policies. Working Paper Series 2001:14, IFAU - Institute for Evaluation of Labour Market and Education Policy, Uppsala Micklewright J, Nagy G (2010) The effect of monitoring unemployment insurance recipients on unemployment duration: evidence from a field experiment. Labour Econ 17(1):180187Montgomery JD (1991) Social networks and labor-market outcomes: toward an economic analysis. Am Econ Rev

81(5):14081418

Neuman S, Ziderman A (1999) Vocational education in Israel: wage effects of the VocEd-occupation match. J Hum Resour

34(2):407420

Neumark D, Wascher W (2006) Minimum wages and employment: a review of evidence from the new minimum wage research. Working Paper 12663, National Bureau of Economic Research, CambridgePetrongolo B (2009) The long-term effects of job search requirements: evidence from the UK JSA reform. J Public Econ

93(11):12341253

Quintini G (2011) Right for the job: over-qualified or under-skilled?. OECD Social, Employment and Migration Working

Papers 120, OECD, Paris

Quintini G, Martin JP, Martin S (2007) The changing nature of the school-to-work transition process in OECD countries.

IZA Discussion Papers 2582, Institute for the Study of Labor (IZA), Bonn

Riley R, Young G (2001) Does welfare-to-work policy increase employment?: Evidence from the UK New Deal for Young

People. NIESR Discussion Papers, National Institute of Economic and Social ResearchRiphahn RT, Zibrowius M (2015) Apprenticeship, vocational training, and early labor market outcomesevidence from

East and West Germany. Educ Econ (ahead-of-print):125

Rodriguez-Planas N (2012) Longer-term impacts of mentoring, educational services, and learning incentives: Evidence from a randomized trial in the United States. Am Econ J Appl Econ 4(4):121139Ryan P (2001) The school-to-work transition: a cross-national perspective. J Econ Lit 39(1):3492Saniter N, Siedler T (2014) The effects of occupational knowledge: job information centers, educational choices, and labor market outcomes. IZA Discussion Papers 8100, Institute for the Study of Labor, (IZA), BonnSkans ON (2004) Scarring effects of the first labour market experience: A sibling based analysis. Working Paper Series

2004:14, IFAU - Institute for Evaluation of Labour Market and Education Policy, UppsalaTopel RH, Ward MP (1992) Job mobility and the careers of young men. Q J Econ 107(2):43979van den Berg G, Uhlendorff A, Wolff J (2015) Under heavy pressure: intense monitoring and accumulation of sanctions for young welfare recipients in Germany. mimeovan den Berg GJ, Blasco S, Crpon B, Skandalis D, Uhlendorff A (2015) Do search clubs help young job seekers in deprived neighborhoods? Evidence from a randomized experiment. mimeovan den Berg GJ, Bozio A, Costa Dias M (2014) Policy discontinuity and duration outcomes. IZA Discussion Papers 8450,

Institute for the Study of Labor (IZA), Bonnvan den Berg GJ, Hofmann B, Uhlendorff A (2013) The role of sickness in the evaluation of job search assistance and sanctions. mimeovan den Berg GJ, Kjaersgaard L, Rosholm M (2012) To meet or not to meet (your case worker) - that is the question. IZA

Discussion Papers 6476, Institute for the Study of Labor (IZA), Bonnvan den Berg GJ, Uhlendorff A, Wolff J (2014) Sanctions for young welfare recipients. Nord Econ Policy Rev 1:177208 Van Reenen J (2004) Active labor market policies and the British new deal for the young unemployed in context. In:

Seeking a Premier Economy: The Economic Effects of British Economic Reforms, 1980-2000, NBER Chapters. National Bureau of Economic Research, Inc. pp 461496Verick S (2011) Who is hit hardest during a financial crisis? The vulnerability of young men and women to unemployment in an econonomic downturn. In: Islam I, Verick S (eds). From the Great Recession to Labour Market Recovery: Issues, Evidence and Policy Options. ILO/Palgrave Macmillan, BasingstokeWinkelmann R (1996) Employment prospects and skill acquisition of apprenticeship-trained workers in Germany. Ind

Labor Relat Rev 49(4):658672

Winter-Ebmer R (2006) Coping with a structural crisis: evaluating an innovative redundancy-retraining project. Int J

Manpow 27(8):700721

Wolbers MH (2003) Job mismatches and their labour-market effects among school-leavers in Europe. Eur Sociol Rev

19(3):249266

Wolff J, Jozwiak E (2007) Does short-term training activate means-tested unemployment benefit recipients in Germany?

Discussion Paper 29, IAB Nuremberg

Wolff J, Nivorozhkin A (2012) Give them a break! Did activation of young welfare recipients overshoot in Germany? (A regression discontinuity analysis). Annual Conference 2012: New Approaches and Challenges for the Labor Market of the 21st Century 62046, Verein fr Socialpolitik / German Economic Association, GoettingenWolff J, Popp S, Zabel C (2010) Ein-euro-jobs fr hilfebedrftige Jugendliche. Hohe Verbreitung, geringe

Integrationswirkung. WSI-Mitteilungen 63(1):1118

Wolter SC, Ryan P (2011) Apprenticeship. In: Hanushek E, Machin S, Woessmann L (eds). Handbook of the Economics of

Education Vol. 3. pp 521576