What is New York City Local Law 144?

New York City Local Law 144 requires annual independent, impartial bias audits of automated employment decision tools, but is colloquially known by names such as the NYC Bias Audit Law, NYC Bias Audit Mandate, and AEDT Audit Law. However, NYC Bias Audit Compliance is not as simple as just commissioning a bias audit; the law also imposes transparency and notification requirements. Enforced from 5 July 2023, Local Law 144 is the first law of its kind, but has already influenced similar proposals in New York State and New Jersey.

What is an automated employment decision tool under Local Law 144?

An automated employment decision tool (AEDT) is a computation process that is derived from machine learning, statistical modelling, data analytics, or artificial intelligence that is used to issue a simplified output, such as a score, classification or recommendation. This simplified output is used to substantially assist or replace discretionary decision making.

Here, machine learning, statistical modelling, data analytics, or artificial intelligence describes a group of mathematical, computer-based techniques used to generate a prediction or classification, where a computer is used at least in part to identify the inputs, relative importance of such inputs, and other model parameters to improve the accuracy of the model.

Further, for an AEDT to substantially assist or replace discretionary decision making, it must be the only factor considered to make decisions, must be the most important factor, or must be able to override decisions made based on other factors, including human decision making.

Importantly, the output of tools used to translate or transcribe text is not considered a simplified output, so tools used to transcribe an interview, for example, would not be within the scope of Local Law 144.

Who does NYC Local Law 144 apply to?

NYC Local Law 144 applies to employers and employment agencies using AEDTs to evaluate candidates for employment or employees for promotion that reside in New York City. This means that employers and employment agencies are liable for the decisions made by tools even if they are provided by third-party vendors. However, many vendors are supporting employers and employment agencies using their tools with commissioning the audit, and some are doing it on their behalf.

What is an NYC Bias Audit?

A bias audit is an impartial evaluation by an auditor that assesses whether an AEDT results in disparate impact against individuals based on race/ethnicity and/or sex/gender, where disparate impact refers to disproportionately negative outcomes for a particular group.

The protected categories that must be included in the audit are male, female, and optionally other for gender, and Hispanic or Latino, White, Black or African American, Native Hawaiian or Pacific Islander, Asian, Native American or Alaska Native, and two or more races for race/ethnicity.

The audit must be carried out using specified metrics depending on whether the AEDT is a regression system that results in a continuous score or a classification system that has a binary output.

For classification systems, the metric is used to calculate an impact ratio that compares the selection rate for a category to the selection rate of the highest scoring category:

Where the scoring rate refers to the proportion of people in each group designated to the positive condition.

For regression systems, outcomes must first be binarised to calculate the scoring rate, where individuals are designated to pass/fail based on whether their score is above or below the median score for the dataset used to complete the audit. The impact ratio can then be calculated using the scoring rate in a similar way to with classification systems:

These impact ratios must be calculated based on standalone groups (e.g. male, female) and intersectional groups (e.g., black male, black female).

What data is required to conduct a NYC Local Law 144 bias audit?

Data provided to conduct a bias audit must reflect the required categories that impact ratios must be calculated for. Where possible, this data should be historical data collected from the real-life use of the tool. However, where this data is insufficient to conduct a bias audit or the tool has not yet been used in practice, test data can be used instead, providing this is disclosed.

Additionally, an employer or employment agency can rely on a bias audit of an AEDT that was conducted using the historical data of other employers or employment agencies if they also contributed to this aggregated data or if they have never used the tool themselves.

Auditors can exclude groups with a small sample size that represent less than 2% of the audit data from the analysis, but still must calculate the scoring rate or selection rate of the group.

Who can conduct bias audits under Local Law 144?

Bias audits must be conducted by an independent, impartial entity from the employer or employment agency using the AEDT. Auditors must be able to exercise objective and impartial judgement, so cannot be anyone that is or was involved in using, developing, or distributing the AEDT, has an employment relationship with the employer/employment agency or vendor of the tool during the course of the bias audit, or has a financial direct or material indirect interest in the employment agency, employer, or vendor of the AEDT during the bias audit.

In short, bias audits must be conducted by third-party entities that are not affiliated with the employer or employment agency using the tool or vendor that developed and provides the AEDT.

What are the transparency requirements of Local Law 144?

Employers or employment agencies using an AEDT must provide a summary of the results of the bias audit on their website before using the tool.

This summary must include:

• The source and explanation of the data used to conduct the bias audit
• The number of applicants in each category
• The number of individuals assessed by the AEDT that were not included in the calculations due to missing demographic data
• The distribution date of the tool and date of the audit
• Whether any categories were excluded from the analysis due to small sample size
• The impact ratios for standalone and intersectional groups

This summary must be updated annually with the most recent bias audit and kept online for 6 months after the tool is retired.

What are the notification requirements of Local Law 144?

Employers or employment agncies within the scope of Local Law 144 must inform candidates and employees that an AEDT will be used to evaluate them at least ten working days before the tool is used. The notice must contain all of the following:

This notice can be provided to candidates through the employment section of the website in a clear and conspicuous manner, in a job posting, or through mail or e-mail. Similarly, notice can be given to employees in a written policy or procedure, in a job posting, or via mail or e-mail.

Who enforces New York City Local Law 144?

New York City Local Law 144 is enforced by New York City’s Department of Consumer and Worker Protection (DCWP). Any suspected violations should be reported to the DCWP by contacting 331 or visiting the DCWP website. Complaints must detail the job posting or position, the name and type of AEDT being used, any notices provided, and an explanation of the violation.

Penalties for non-compliance with NYC Local Law 144 range from $500 to $1500 per default, meaning fines could quickly add up. Moreover, the reputational damage for failing to get an independent bias audit, publish a summary of results, or provide notice could be even more costly.

Prepare for a bias audit with Holistic AI

With Local Law 144 already showing signs of snowballing, employers using AEDTs that are not quite covered by the scope of the law yet are likely to be in the near future. Obtaining a bias audit will become a necessity to ensure ongoing compliance, remain competitive, and promote trust in the fairness of AEDTs.

Schedule a demo with our expert team to find out how Holistic AI’s independent AI Bias Audits can help you get ahead of these and other upcoming AI laws.