Notes on Outcomes in Labour Market Programs

A review of research that examines outcomes in labour market programs

EI related services

Handouyahia, 2016

• Handouyahia (see Handouyahia et al., n.d.) uses admin data matched to CRA they find that EBSMs have positive effects on employment and earnings

Findings

• EBSMs caused gains in employment earnings, incidence of employment (+2.4% first year to +4.4% by fifth year) and reduction of EI benefits after completing
• SD had most pronounced effect (between $204 and $4,059 higher)
• SD represents the largest incremental gains

Highlights

“The study examines the impacts of Employment Benefits such as EAS, SD and TWS by comparing individuals who received these interventions to those who did not participate.” (Handouyahia et al :9)

“two groups with the same probability of participation will show up in the participant and non-participant samples in equal proportions” (Handouyahia et al :9)

“The use of standard personal characteristics, pre-treatment outcomes, transitions between different labour market states, regional information and labour market history is very important for decreasing the likelihood of biased estimator of the true treatment effect.” (Handouyahia et al :9)

“For example, factors such as ability, health, education and motivation to seek employment were not directly measured except to the extent they were captured in prior income and labour market attachment patterns.” (Handouyahia et al :10)

“on earnings and the probability of employment for participants” (Handouyahia et al :11)

“SD is effective at increasing the employment earnings and incidence of employment for active claimants. Importantly, for active claimants, participation in SD led to the largest incremental gains in employment earnings among all EBSMs” (Handouyahia et al :12)

Ontario Works

• Adams et. al (see Adams, Chow, and Rose 2018) examine Ontario Works employment programming
• how effective are OW ALMPs on two outcomes:
  • case duration (benefit ‘spell’ measured in months)
  • probability of returning to OW within 1 or 2 years of exit
• this client group has weaker LM attachment and so not perfectly aligned to existing literature on ALMPs
• accordingly existing canadian work suggests this group is differentially affected by ALMPs

Findings

• placement services extend OW spells by 16 months
• structured job search not having an effect on ‘return’
• training programs (literacy, skills/vocational) does reduce ‘return’ by 07% year 1 and 1.1% year 2
• among all interventions there’s a trade-off between short and long term effectiveness
  • placement means longer spells but reduces P(return) by 7% in year 1 and 10% year 2
  • structured job search reduces spell by 1.5 - 3 months but doesn’t affect P(return)
  • training program alone is more effective in the short-term than assignment to boht training program and structured job-search (implying the interventions aren’t additive)

Highlights

“a significant differential impact on both spell durations (time on social assistance) and one and two-year return rates (return to social assistance)” (Adams et al 2018:1)

“Examining the program effects on spell duration gives an idea of the short-term efficacy of the programs: how effective they are at getting recipients employed.” (Adams et al 2018:2)

“outcomes depend on the type of assistance provided.” (Adams et al 2018:2)

“while a combination of such workshops and training or employment assistance through direct job placements increase the spell length, direct job placements have a statistically significant and economically meaningful effect on reducing the oneand two-year return rates.” (Adams et al 2018:3)

“For example, the government could increase assignment to programs that lead to shorter spell durations if its goal was to get recipients off social assistance more quickly. Alternatively, it could increase assignment to programs that lead to lower probabilities of return to social assistance if its aim was focused more on longer-term labour-market attachment.” (Adams et al 2018:3)

“the positive impact of programs such as jobtraining tends to be larger for the long-term unemployed (this would include Ontario Works recipients) and that many treatments that do not show short-term evidence of a statistically significant impact show positive mediumterm impacts.” (Adams et al 2018:3)

“all three lead to statistically significant increases in employment compared to a control group of untreated EI claimants, with the effects becoming more pronounced over the medium term” (Adams et al 2018:4)

“this study is the first to examine their impact on recipients of social assistance.” (Adams et al 2018:4)

“Many OW beneficiaries may have been ineligible to claim EI benefits following a job separation, and a common reason for joining OW is the exhaustion of EI benefits. The work of Brown and Koettl (2015) and Card, Kluve, and Weber (2010, 2015) suggests that this group could be differentially affected by these programs, so it follows that any policy aimed at improving the labour-market outcomes of social assistance beneficiaries should be based on evidence that is pertinent to them.” (Adams et al 2018:4)

“The return rates for each program give the proportion of recipients that return to social assistance within, respectively, one year of leaving and two years of leaving.” (Adams et al 2018:8)

“independent job-search program or placement services are, on average, more likely to return to social assistance within one year or two years after exit. By contrast, those assigned to structured job search, training programs, or both have a lower propensity of returning to unemployment after leaving Ontario Work” (Adams et al 2018:8)

“Being assigned to structured job search will, on average, reduce spell duration by approximately 1.7 months compared to recipients assigned to independent job search.” (Adams et al 2018:9)

“However, being assigned to both these program categories (structured job search and a training program) adds 4.6 months to a spell - a significantly longer period than after assignment to only one of the programs.” (Adams et al 2018:9)

“The average recipient is better off being assigned only to a training program rather than also being assigned to structured job search, or vice versa.” (Adams et al 2018:9)

“being given a direct job or volunteer placement has a negative impact on spell duration relative tto independent job search, adding 16.6 months to the average spell on social assistance. By this measure, direct placements are the least effective of the treatments studied in the short term.” (Adams et al 2018:10)

“The results show that being assigned to structured job search does not have a statistically significant effect on the probability of return within one year or two years.” (Adams et al 2018:10)

“a training program has a small statistically significant effect, reducing the return probabilities by 0.7 percent within one year and by 1.1 percent in two years.” (Adams et al 2018:10)

“It should be noted that assignment to either structured job search or a training program alone is more effective (by the measures of efficacy examined here) than assignment to both these programs; in this case, more is not better.” (Adams et al 2018:10)

“it is possible to improve outcomes for recipients of social assistance in Ontario by emphasizing programs that are more effective in meeting objectives.” (Adams et al 2018:10)

Adams, Jason, Ken Chow, and David Rose. 2018. “Assessing Active Labour-Market Programs: How Effective Is Ontario Works?” SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3276081.

Handouyahia, Andy, Stéphanie Roberge, Yves Gingras, Tony Haddad, and Georges Awad. n.d. “Estimating the Impact of Active Labour Market Programs Using Administrative Data and Matching Methods,” 7.