Notes on Allocating Resources

A review of research related to allocation of resources in employment programs

Targeting vs Profiling

(Behncke, Frölich, and Lechner 2009) distinguish between two types of statisical approaches to resource allocation in ALMPs:

• targeting which selects for each client the program with the highest expected outcome (based in the impact effect of each type of intervention), and;
  
• profiling which based on a set of client characteristics, estimates the risk of becoming long term unemployed where those at the highest risk of being unemployed are assigned to the most intensive services

The authors suggest that targeting is preferable to profiling when:

1. there are a variety of types of programs/interventions offered, and/or;
   
2. the risk of long-term unemployed is not highly correlated with program impacts

The authors detail the challenges of maximizing the allocation of ALMPs in practice - namely:

• moral hazard of employment agencies (contracting challenge, more on this here)
  • caseworkers have their own goals for clients (e.g. improving personal situation, long-term stability rather than rapid re-integration)
    
• caseworker prediction (informational challenge)
  • small sample sizes to build intuition on what works (no view of the system)
  • no systematic evidence of outcomes/success (e.g. clients maye return but to a difference provider/caseworker)

Findings

The authors review a pilot study in Switzerland where PES caseworkers were randomized into 2 groups, one who had access to statistical targeting tools and one where no prediction support was offered. The design was robust:

• randomization at the caseworker level
  
• 1 year study from 2005 to 2006
  
• over 18k jobseekers in the prediction group

The conclusions were as follows:

• the statistical predictions offered had no impact on program assignment caseworkers made their own predictions and ignored the model
  
• incentives are required when caseworker discretion is at odds with the model

Highlights

“The evaluation results show that caseworkers did not change their behaviour in any significant way due to having access to the additional information” (Behncke et al :51)

“the profiling systems often estimate the risk of becoming long-term unemployed if not taking part in ALMP” - “statistical targeting systems, which select for each client the programme with the highest expected outcome” (Behncke et al :53)

“Small sample sizes are even more reduced by a limited capacity of caseworkers to follow-up on their clients after they are de-registered from the employment office.”- “there is evidence suggesting that caseworkers did not succeed in estimating employment probabilities between the various programmes or followed other aims than employment maximisation of their clients: Bell and Orr (2002) found that caseworkers did not systematically select those into treatment who would benefit most from it” (Behncke et al :54)

“initiated a pilot study” - “where caseworkers were randomized either into the in or the out group.” (Behncke et al :57)

“This is a big advantage vis-à-vis simple profiling models as it takes into account that the optimal time when a labour market programme should start may also vary across individuals.” (Behncke et al :61)

“every second week by incorporating new information on time varying covariates” - “the number of months in stable employment in the next 12 months,” - “the conclusion is that the availability of the SAPS predictions had no impact on the actual choice of ALMP.” (Behncke et al :65)

“first, caseworkers make their own predictions” - “ignored the econometric predictions either because of confidence in the superiority of their own judgements or because of unwillingness to comply with an external tool that could pose a potential threat to their future autonomy and discretion if introduced nationwide” (Behncke et al :66)

“The evaluation results showed that caseworkers largely ignored the statistical support system” (Behncke et al :69)

“Our evaluation results indicate that caseworker discretion may conflict with the provision of statistical targeting. In order to evaluate the impact of these systems, specific incentives may be required for caseworkers to comply. Bonus payments for caseworkers who complied with the experiment may be one example. These positive incentives should compensate for potential negative effects due to being unfamiliar with the system or being uncertain about its merits during the pilot phase” (Behncke et al :69)

Profiling as allocation method

(Berger, Black, and Smith 2001)

This research considers how to evaluate statistical profiling methods in allocation mechanisms for government programs.

More notes to come

Behncke, Stefanie, Markus Frölich, and Michael Lechner. 2009. “Targeting Labour Market Programmes — Results from a Randomized Experiment.” Swiss Journal of Economics and Statistics 145 (3): 221–68. https://doi.org/10.1007/BF03399281.

Berger, Mark C., Dan Black, and Jeffrey A. Smith. 2001. “Evaluating Profiling as a Means of Allocating Government Services.” In Econometric Evaluation of Labour Market Policies, edited by Michael Lechner and Friedhelm Pfeiffer, 13:59–84. Heidelberg: Physica-Verlag HD. https://doi.org/10.1007/978-3-642-57615-7_4.