With the increasing use of Artificial Intelligence (AI) in hiring processes, New York City’s Local Law 144 has become a pivotal piece of legislation for employers. Enacted to ensure fairness and reduce bias in automated employment decision tools (AEDTs), this law mandates annual audits to assess these tools for discriminatory practices. As businesses enter the second year of compliance, understanding the nuances of the auditing process and what changes may occur is crucial for maintaining legal and ethical hiring practices. This guide provides a comprehensive overview of what employers need to know for year two and beyond.

Understanding Local Law 144: Background Recap

Local Law 144, implemented to address potential biases in AEDTs used for hiring and promotions, requires that these tools be independently audited annually. The law aims to ensure that all candidates have a fair chance during the hiring process by preventing AI systems from perpetuating historical biases. These audits must evaluate the tools against set criteria to identify any discriminatory patterns affecting gender, race, and other protected classes.

What Changes in the Second Year?

• Enhanced Audit Quality Expectations: In the first year, many employers were primarily focused on understanding the basics of the law and achieving compliance. As we move into the second year and subsequent ones, the quality and depth of the audits will come under greater scrutiny. Regulators and stakeholders will likely look for more thorough assessments and detailed reporting that demonstrates not just compliance but a commitment to ongoing improvement.

• Significance of Accumulated Data: Employers will have accumulated more data on the functioning of their AI tools over multiple hiring cycles by the second year. This expanded dataset provides a richer basis for analysis, allowing for more precise audits. However, it also means that any inherent biases in the AI algorithms could have a more pronounced impact, making rigorous audits even more essential.

• Updates in Audit Standards and Methodologies: As the understanding of AI’s role in hiring deepens, the standards and methodologies for audits may evolve. Auditors and regulatory bodies might update the criteria for evaluation or the statistical methods used to detect biases. Employers need to stay informed about these changes to ensure their compliance efforts remain current.

• Implementation of Intersectional Analysis: The initial audits may have focused on broad categories of race and gender. With the call for more granular insights, the second-year audits might require intersectional analysis that considers overlapping identities (e.g., race and gender together), providing a more detailed view of how AI tools affect diverse applicant groups.

Consistent Elements in Year Two and Beyond?

• Annual Audit Requirement: The fundamental requirement for annual audits remains unchanged. Employers must continue to ensure that their AI hiring tools are audited every year, with no gaps in compliance.

• Independence of Audits: The need for audits to be conducted by independent third parties also continues. This ensures that the evaluations are impartial and meet the rigorous standards set forth by the law.

• Persistent Focus on Bias and Discrimination: The central focus of these audits on detecting bias and preventing discrimination will not shift. This aspect remains at the heart of Local Law 144, reflecting ongoing concerns about fairness in AI-driven employment decisions.

Best Practices for Ensuring Compliance in Year Two

• Collaborate with Expert Auditors: Partner with auditors who have a proven track record in evaluating AI systems for bias. Their expertise will be invaluable in navigating the complexities of second-year audits.

• Maintain Robust Data Documentation: Ensure that all data used by your AI tools is accurately recorded and stored. This data will be crucial for audits and any necessary adjustments to the AI models based on audit findings.

• Continuous Training and Update: Keep your HR and compliance teams well-informed about any changes in audit requirements or relevant laws. Regular training will help them understand the implications of audit outcomes and how to apply these insights in practice.

• Commit to Transparency: Be transparent with candidates and employees about the use of AI in hiring decisions. This not only helps in building trust but also aligns with broader ethical standards and regulatory expectations.

Conclusion: Mastering AI Audit Requirements

As employers prepare for the second year of compliance under Local Law 144, understanding both the changes and constants in the auditing process is crucial. By focusing on thorough preparation, ongoing education, and engagement with skilled auditors, businesses can navigate these requirements successfully. This not only ensures compliance but also strengthens the fairness and integrity of hiring practices in an increasingly AI-driven world.

Need Help?

Want to know if your company needs a New York City Bias Audit done? Contact BABL AI and one of their Audit Experts will be able to answer all your questions related to this regulation and more.