Skip to content

We ensure that algorithms are safe, fair, and prioritizing human flourishing.

Who We Are

We are BABL AI, a boutique consulting and audit firm focused on responsible AI. We believe that algorithms should be developed, deployed, and governed in ways that prioritize human flourishing.

We unlock the value of responsible AI for clients by combining leading research expertise and extensive practitioner experience in AI and organizational ethics to drive impactful change at the frontier of technology and emerging standards. Our team consists of leading experts and practitioners on AI, Ethics, Law, and Machine Learning

What We Do

BABL AI has been at the forefront of developing proven methodologies, standards, and best practices since 2018. Our services are focused on four key areas.

Algorithm Process Audits

We independently verify and impartially evaluate your organization’s bias testing through our criteria-based process audit based on clearly defined audit criteria and certified audit standards.

Algorithm Risk, Impact, and Bias Assessments

We identify and evaluate your organization’s algorithms’ risks and impacts through an in-depth assessment involving our proprietary methodology, stakeholder interviews, and direct technical testing.

Responsible AI Governance Gap Analysis

We assess your current Responsible AI practices and governance in relation to current (and emerging) standards, regulations, and best practices.

Corporate Training in Responsible AI

We develop and deliver training and compliance courses in Responsible AI to fit your corporate upskilling needs.

Pre-audit
Scoping phase for the audit
BABL AI walks the prospective auditee through a series of questions to determine if sufficient testing of tools has been conducted to qualify for an audit. In cases where more preparation is required, we provide guidance for the prospective auditee. Once these criteria are met, then the auditee is on-boarded onto a client portal for progression to the actual audit phase.
Pre-audit
Audit
Core phase of the process audit
The auditee submits documentary evidence to the client portal, which our trained and certified auditors review to satisfy audit criteria. During the review process, BABL AI auditors might ask for more supporting documentation or interact with the auditee’s internal and external stakeholders, such as employees or other third-parties, to verify the truth of statements made in the submitted documentation.

At the end of the audit phase, the auditors reach an overall audit opinion that determines the result of the audit. This opinion can be a “Pass,” “Minor Remediations,” or “Fail” result.
Audit
Post-audit
Public Summary Drafting
In this final phase, BABL AI drafts a public report for each AEDT, if mandated by the regulatory body, and presents the final deliverable, including the audit opinion, to the auditee.
Post-audit

Pre Audit

Scoping phase for the audit

As HR and recruitment technology has started to incorporate AI and machine learning, various stakeholders are demanding assurance that vendors and employers have done their due diligence to mitigate potential bias and other ethical, privacy, and compliance risks associated with the use of their proprietary AI.

Starting January 2023, employers and employment agencies are required by New York City’s Local Law No. 144 of 2021 to have available “independent bias audits” for automated 4 tools used to substantially assist or replace decisions in hiring or promotion – i.e., automated employment decision tools (AEDTs). Read our one-pager for 5 key points about the “bias audit,” or watch our free webinar to learn more about the details of the law.

BABL AI’s Process Audit is a streamlined and targeted solution for our clients that satisfies the requirement of a bias audit, defined as “an impartial evaluation by an independent auditor.” Developed from our experience in performing direct testing for algorithmic bias, and emerging best practices and standards in responsible AI, this set of transparent, binary audit criteria allows employers and vendors using AEDTs to conduct their own bias testing that trained and certified BABL AI independent auditors can verify.

In addition to NYC Bias Law Audits, we are working closely with our clients and partners to provide similar cost-effective and efficient solutions for upcoming regulation including the Proposed Amendments to California’s Employment Legislation Regarding Automated-Decision Systems and EU AI Artificial Intelligence Act.

Audit

Core phase of the process audit

The auditee submits documentary evidence to the client portal, which our trained and certified auditors review to satisfy audit criteria. During the review process, BABL AI auditors might ask for more supporting documentation or interact with the auditee’s internal and external stakeholders, such as employees or other third-parties, to verify the truth of statements made in the submitted documentation.

At the end of the audit phase, the auditors reach an overall audit opinion that determines the result of the audit. This opinion can be a “Pass,” “Minor Remediations,” or “Fail” result.

Post Audit

Public Summary Drafting

In this final phase, BABL AI drafts a public report for each AEDT, if mandated by the regulatory body, and presents the final deliverable, including the audit opinion, to the auditee.

News

Stay up to date with the latest BABL AI stories, people, insights, research, and more.
Article
BABL AI Testifies on NYC Bias Law

The City of New York passed Local Law 144, which requires yearly “bias audits” of automated employment decision tools (AEDTs). The NYC Department of Consumer …

Youtube
Using AI Ethically in Online Testing

How can test publishers and EdTech vendors use …

Conference
BABL Sponsors ForHumanity Conference

BABL is a proud associate sponsor of ForHumanity’s …

Podcast
Are you a robot? Podcast

Our own Shea Brown I had a great …

BABL AI Testifies on NYC Bias Law

The City of New York passed Local Law 144, which requires yearly “bias audits” of automated employment decision tools (AEDTs). The NYC Department of Consumer and …

Read More →

Using AI Ethically in Online Testing

How can test publishers and EdTech vendors use AI in their products “while still upholding fairness and avoiding discrimination and bias?” Shea Brown participates in …

Read More →

BABL Sponsors ForHumanity Conference

BABL is a proud associate sponsor of ForHumanity’s AT->GO conference, where the community is coming together to discuss the future of AI Governance. Our CEO …

Read More →

Are you a robot? Podcast

Our own Shea Brown I had a great time talking with Demetrios Brinkmann about auditing AI and machine learning systems on the Are You A …

Read More →

Clients & Partners