One Month until NYC Local Law 144 Enforcement: Are You Ready?

New York City Local Law 144 is set to be enforced from 5 July 2023 and will have important implications for AI and automation in talent management. It will require that employers or employment agencies using automated employment decision tools (AEDTs) to evaluate candidates for employment or employees for promotion in New York City to obtain an independent, impartial bias audit annually.

What is New York City Bias Audit Law?

Introduced by the New York City Council in 2020 and enacted in 2021, the law was set to come into effect on 1 January 2023. However, after a significant consultation process on the rules proposed by the Department of Consumer Protection (DCWP), the enforcement date was initially postponed to 15 April 2023 before being pushed back a final time to 5 July 2023, when the rules adopted by the DCWP will be enforced. In this blog post, we outline the four key steps to prepare for compliance.

1. Identify if your tools fall under the regulations

Local Law 144 applies to any computational process, derived from machine learning, statistical modelling, data analytics, or artificial intelligence, that results in a simplified output, such as a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions.

Under the rules, to substantially assist or replace discretionary decision making means i) to rely solely on a simplified output without considering other factors; or ii) to use a simplified output as one of a set of criteria where it is weighted more than any other factor; or iii) to use a simplified output to overrule conclusions derived from other factors including human decision-making.

Machine learning, statistical modelling, data analytics, or artificial intelligence are defined as a group of mathematical, computer-based techniques that generate a prediction such as an assessment of a candidate’s fit or likelihood of success or that generate a classification such as the assignment of an individual to a group based on skill sets or aptitude, for example. A computer also, at least in part, identifies the inputs, the relative importance placed on those inputs, and any other parameters for the models in order to improve the accuracy of the prediction or classification.

However, even if tools are not used as the most important or sole criteria, they could still have significant knock-on effects if they were biased, so it’s a good idea to obtain a bias audit even if the criteria aren’t quite met.

2. Identify an independent auditor to conduct your audit

Bias audits must be conducted by an independent entity; an impartial individual or group that was not involved in using, developing, or distributing an AEDT, that has no employment relationship with the employer or employment agency, and that has no direct or material indirect financial interest in the employer or employment agency using the AEDT or the vendor that developed or distributed the AEDT.

Audits must be based on the metrics put forward by the DCWP, which are different depending on the output of the system. For classification systems, the metric is based on the proportion of individuals in each group being allocated to the positive condition, known as the selection rate:

For regression systems that result in continuous outcomes, data must first be binarized depending on whether scores are above or below the median score for that outcome across the data set, known as the scoring rate:

3. Prepare the required data for the audit

Audits must calculate the above impact ratios based on sex/gender and race/ethnicity for both standalone and intersectional groups. The required sex/gender categories are male and female and optionally other, and the required race/ethnicity categories are Hispanic or Latino, White, Black or African American, Native Hawaiian or Pacific Islander, Asian, Native American or Alaska Native, and two or more races. It is therefore vital that these categories are captured in the data provided.

Ideally, the data provided will be based on real-life applicants who have been evaluated by the AEDT, known as historical data. This data can be from one or more employers or employment agencies that use the AEDT, but an individual employer or employment agency can rely on a bias audit of an AEDT that uses the historical data of other employers or employment if they provided their own historical data or if they have never used the AEDT. However, if historical data is unavailable to conduct the audit, test data can also be collected providing there is a justification for doing so.

4. Establish notification procedures

The requirements of Local Law 144 do not stop at just commissioning an independent audit; employers and employment agencies must also clearly and conspicuously make a Summary of Results publicly available on the employment section of their website. This must include the date of the audit, the number of applicants or candidates in each category, selection and scoring rates, impact ratios for all categories, and the distribution date of the tool. This must remain posted for at least 6 months after the tool is no longer used.

Employers and employment agencies must also notify candidates or employees of the following at least 10 business days before the AEDT is used to evaluate them: i) that an AEDT will be used; ii) the job qualifications and characteristics that the AEDT will use in its evaluation; iii)  the type of data collected for the AEDT and its source; iv) instructions on how to request an alternative procedure or accommodation; v) its AEDT data retention policy. Instructions for how to make a written request for such information should also be provided, and requests should be met within 30 days unless doing so would violate local, state, or federal law or interfere with a law enforcement investigation.

For candidates, notice can be provided on the employment section of the employer or employment agency’s website in a clear and conspicuous manner; in a job posting; or via mail or email. For employees, notice can be provided in a written policy or procedure that is provided to employees, in a job posting, or via mail or email.

It is important that these notification procedures are established at least 10 business days before the 5 July deadline in order to be able to use the tool from this date.

It’s not too late!

Remember, this law applies to any employer or employment agency promoting employees or hiring candidates that reside in New York City, regardless of where the employer or employment agency is located themselves. Local Law 144 is, therefore, set to have far reaching implications for employers around the world, and not obtaining an audit could see you unable to hire or promote within New York City using an AEDT, potentially limiting your talent pool.

Although the deadline for compliance with Local Law 144 is fast approaching, there is still time to comply. At Holistic AI, we combine expertise in computer science, law, and business psychology to holistically understand a system and its capabilities while conducting an audit. Schedule a demo to find out how we can help you.