The Initial Screening Order Problem

Code

• github.com/cc-jalvarez/initial-screening-order-problem
  OfficialIn papernone★ 2

Abstract

We investigate the role of the initial screening order (ISO) in candidate screening. The ISO refers to the order in which the screener searches the candidate pool when selecting k candidates. Today, it is common for the ISO to be the product of an information access system, such as an online platform or a database query. The ISO has been largely overlooked in the literature, despite its impact on the optimality and fairness of the selected k candidates, especially under a human screener. We define two problem formulations describing the search behavior of the screener given an ISO: the best-k, where it selects the top k candidates; and the good-k, where it selects the first good-enough k candidates. To study the impact of the ISO, we introduce a human-like screener and compare it to its algorithmic counterpart, where the human-like screener is conceived to be inconsistent over time. Our analysis, in particular, shows that the ISO, under a human-like screener solving for the good-k problem, hinders individual fairness despite meeting group fairness, and hampers the optimality of the selected k candidates. This is due to position bias, where a candidate's evaluation is affected by its position within the ISO. We report extensive simulated experiments exploring the parameters of the best-k and good-k problems for both screeners. Our simulation framework is flexible enough to account for multiple candidate screening tasks, being an alternative to running real-world procedures.

Tasks

Reproductions