Navigating New York City's Local Law 144: What HR Leaders Nationwide Need to Know

New York City Local Law 144, also known as the bias audit law, requires annual, independent, impartial bias audits of automated employment decision tools used to evaluate candidates for employment or employees for promotion in New York City. Enforced from 5 July 2023, the law was the first of its kind and has paved the way for the proposal of other, similar bias audit laws in the United States.

Despite being a Local Law, the bias audit law does have an extraterritorial reach, affecting employers around the world using automated tools in New York City. In this blog post, we outline how the law affects employer inside and outside of New York City.

How does New York City Local Law define automated employment decision tools?

According to Local Law 144, an automated employment decision tool is:

“any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.”

Rules for the enforcement of the law published by New York City’s Department of Consumer and Worker Protection (DCWP) further clarify some of these terms, where a simplified output includes a score, recommendation, or classification, and to substantially assist or replace discretionary decision making means to use the tool i) as the only factor to make a decision, ii) to use the tool to override the decisions from other factors, or iii) to weight the influence of the tool higher than any other factor.

Moreover, machine learning, artificial intelligence, predictive analytics and statistical modelling are defined as a group of mathematical, computer-based techniques used to generate a continuous or binary prediction where a computer at least in part identifies the inputs and their importance to improve model performance.

What is a bias audit according to Local Law 144?

A bias audit is an evaluation of the outcomes of a tool by an independent, impartial auditor. This is someone who does not have an employment relationship with or direct or indirection financial interesting the employer using the AEDT or vendor providing the tool. Independent auditors also cannot have been involved in the development and deployment of the tool throughout the bias audit process.

Evaluations carried out by independent auditors must follow the metrics specified by the DCWP, where the metric for categorical systems calculates the hiring rate of individuals in each group based on the proportion of applicants designated to the positive condition and then divides this rate by the rate of the group with the highest rate.

For continuous systems, the scoring rate must first be calculated by binarizing scores based on whether they are above or below the median for that dataset. The proportion of individuals in each group that are designated to the positive condition is then used to represent the scoring rate. This must then be divided by the scoring rate of the category with the highest rate. This must be carried out for the protected characteristics of sex and race/ethnicity, as well as the intersection between the two.

As part of the bias audit, a summary of the results must be published on the website of the employer that contains the impact ratios calculated with these metrics, the number of applicants in each category and the number that have unknown protected characteristics, the source and type of data used for the audit, and whether any groups were removed from the analysis due to a small sample size. Specifically, auditors do not have to calculate the impact ratio for groups representing less than 2% of the data, but the summary of results must still contain the group size of this category and the scoring or selection rate.

Who does NYC Local Law 144 affect?

The original law text of Local Law 144 required that bias audits be conducted by employers using automated employment decision tools to evaluate candidates or employees residing in New York City. This remained unchanged by the three versions of the DCWP rules. However, FAQs on the law published by the DCWP provided some clarity on what “in the city”. This shifted the focus away from where candidates or employees reside towards the location of the position.

According to the FAQs, in the city means:

• The position is located in office located in New York City at least some of the time; or
• The position is remote but is linked to an office in New York City; or
• The employment agency using the AEDT is in NYC or it is outside of NYC but one of the above is true

Does Local Law 144 affect employers outside of New York City?

Given the DCWP’s rules, the scope of the law narrowed slightly. Nevertheless, the law still applies to employers based outside of New York City if they have a NYC office. In other words, no matter where an employer is head quartered, if they have an NYC office and are using an AEDT to evaluate candidates for positions associated with this office, even if the positions are remote, then they must comply with Local Law 144.

Given the fact that the DCWP FAQs allow vendors of AEDTs to initiate a bias audit on behalf of their clients – the employers using the tool – using data aggregated across multiple users of the tool, this can extend the extraterritorial scope of the law. Indeed, if a vendor is located elsewhere but their clients use the tool in NYC, then they might commission an independent bias audit on behalf of their client as part of their package. This could see vendors in New Zealand, for example — the other side of the world — primitively complying with NYC Local Law 144. As such, Local Law 144 has a truly global reach despite it only being a local law.

Compliance is key to maintaining trust

Conducting an independent, impartial bias audit of an AEDT, even if just outside of the scope of the law, can be a driver of trust in employers and vendors of AEDTs. Bias audits create greater accountability for and transparency around the use of AEDTs, and can help candidates to make more informed decisions when applying for positions.

Non-compliance with Local Law 144 can see penalties of up to $1500 per violation, in addition to reputational damage and the financial implications that come with this. Compliance is key to upholding trust and embracing tools with confidence.

Schedule a demo with our expert team to find out how Holistic AI’s Bias Audit Solution can help you comply with NYC Local Law 144.