New York City Government Proposes Changes to the Mandatory Bias Audit Law: Here’s Everything You Need to Know

On 19 September, the New York City (NYC) Department of Consumer and Worker Protection (DCWP) published proposed amendments to Local Law 144, which mandates independent bias audits of ‘automated employment decision tools’ used by employers or employment agencies.

The law was enacted by the NYC Council in November 2021. It implicates any U.S. company which uses AI tools to screen, hire or promote candidates or employees residing in NYC. Read our FAQs on the law to learn more.

The DCWP’s proposed amendments have not yet been enacted and are open for consultation until 24 October, when there will be a public hearing on the matter. Information about the amendments and how to submit comments and participate in the hearing can be found here.

The proposed amendments and consultation demonstrate that the NYC authorities are prioritising the implementation and enforcement of this legislation.

1^st January 2023 remains the date the law takes effect. (Update: Local Law 144 has now been pushed back to 5 July 2023). The use of an automated employment decision tool is prohibited after this date, unless the tool has been independently audited for bias.

How will the proposed NYC Bias Audit law amendments affect your business?

The DCWP’s amendments provide practical guidance for businesses and new compliance requirements. The key changes being proposed are:

• Clarification that an ‘independent auditor’ must be a group or person that is not involved in using or developing an automated employment decision tool.
• More detailed definition of which automated tools require bias audits.
• Confirmation that bias audits must, at a minimum, include calculations of the ‘selection rate’ and ‘impact ratio’ for different races/ethnicities and sexes. Intersectional approaches are encouraged.
• Employers must publish the ‘summary of results’ of their bias audit, including the selection rate and impact ratio for different races/ethnicities and sexes.
• Employers do not have to provide an alternative selection method, even if a candidate requests to not be assessed by an automated tool.

Who can be an independent auditor?

The legislation requires that bias audits must be impartial evaluations conducted by an independent auditor.

The proposed amendments further clarify that an ‘independent auditor’ must be a group or person that is not involved in using or developing an automated employment decision tool.

This means that a tech vendor that develops and supplies an automated tool to an employer cannot conduct the bias audit themselves.

Despite the employer or employment agency being ultimately liable, vendors who supply them will likely benefit from commissioning bias audits of their automated tools. Employers may also insist that vendors commission bias audits prior to purchasing a tool, given their new legal obligations.

Which automated employment decision tools require bias audits?

The legislation defines ‘automated employment decision tools’ as those which “substantially assist or replace discretionary decision making” about hiring candidates or promoting employees.

The proposed amendments further clarify that this only covers situations where:

• decisions are made solely based on the automated tool’s outputs; or
• the automated tool’s output is the most heavily weighted criterion in the decision-making process; or
• the automated tool’s output is used to overrule or modify human-decision making.

An automated tool’s outputs are defined as a prediction or classification, which can be expressed as a score, tag, categorisation or ranking.

What are the minimum requirements for a bias audit?

The legislation already specifies that a bias audit must include the testing of an automated tool to determine whether it adversely affects one category (i.e., race/ethnicity or sex) over another.

The proposed amendments provide additional information on exactly how this should be done.

The bias audit must include a calculation of the ‘selection rate’ and ‘impact ratio’ for different races/ethnicities and sexes.

The ‘selection rate’ is the proportion of individuals in a category who are selected to move forward in the hiring or promotion process.

To determine whether a system results in disparate impact, auditors must also calculate the impact ratio of a system, which can be measured in two ways:

• the selection rate for one category divided by the selection rate for the most selected category; or
• the average score of all individuals in a category divided by the average score of all individuals in the highest scoring category.

In the bias audit examples described by the DCWP in the proposed amendments, the calculations are also made for intersectional groupings (i.e., selection rates and impact ratios are provided for different races/ethnicities according to their sex). This intersectionality enables deeper insight into the outcomes of different sub-groups.

Despite this, the proposed amendments do not appear to mandate calculations on intersectional groupings.

However, as these rules are explicitly labelled as the minimum requirements for a bias audit, businesses who go above and beyond, and take a more granular and in-depth approach, will be viewed more favourably by the enforcement authorities, courts and other stakeholders.

What needs to be published in the bias audit ‘summary of results’?

‍Under the new proposals, after a bias audit is conducted, employers and employment agencies will have to publish:

• the ‘summary of results’, which must include the ‘selection rates' and ‘impact ratios’ for each sex and race/ethnicity; and
• the date of the bias audit; and
• the date the automated tool was first used.

This information must be visibly posted and easily accessible on the organisation’s website for at least 6 months after the relevant tool was last used. If the tool continues to be used, the summary of results must continue to be publicly available.

The requirement to publish such granular detail about the employment outcomes for different demographic categories exposes employers to damaging reputational and legal risks.

How to provide ten days’ notice to candidates and employees?

The legislation also requires employers or employment agencies to give at least ten business days’ notice to candidates or employees prior to using an automated tool to make decisions about hiring or promotion. No exceptions or derogations are given to this requirement.

The legislation already stipulates that the notice must include the job qualifications and characteristics being assessed by the automated tool.

The proposed amendments further clarify how this notice should be given to candidates or employees. It gives employers three options:

• include the notice on the website’s careers section (for candidates) or in a written company policy (for employees); or
• include the notice in a job posting; or
• provide notice via U.S. mail, e-mail or in person (for employees only).

In the 10-day period, individuals can request the use of an alternative selection method or another accommodation. The proposed amendments clarify that the notice must include information about how to make this request.

However, the amendments also explicitly state that employers and employment agencies are not required to provide an alternative selection process or accommodation.

It is therefore up to employers whether to change their processes for candidates and employees who do not wish to be assessed by an automated tool.