“The questions we get asked most, answered plainly.”
If you are trying to work out what BABL AI does, whether it applies to you, and how each part of the firm could support you, start here. If your question is not covered, a short conversation is the fastest way to get a clear answer.
Audit & Assurance
An AI audit is a structured evaluation of an AI system against defined criteria, performed by a party independent of the people who built or operate the system, and documented so that a third party can rely on the conclusion. The criteria may come from regulation (such as New York City Local Law 144), from a recognized framework (such as the NIST AI Risk Management Framework or ISO/IEC 42001), or from the organization’s own stated policies and public claims. What makes it an audit is not the testing itself but the evidentiary discipline around it: an agreed scope, suitable criteria, documented procedures, retained evidence, and a written conclusion the auditor is professionally accountable for.
This distinguishes an audit from red teaming, a model evaluation, or an internal risk review. Those produce useful information. An audit produces a conclusion supported by evidence, issued under an assurance standard such as ISAE 3000, that someone outside your organization can act on.
Most engagements are driven by one of five things: a legal or regulatory obligation, a customer or procurement requirement, diligence pressure from investors, insurers, or the board, exposure to discrimination and consumer protection liability, or an internal need to confirm that governance controls described on paper actually operate in production.
The common thread is that someone outside the team that built the system needs to be able to trust a statement about it. Internal testing rarely satisfies that need. A regulator assessing conformity, an enterprise buyer completing a vendor review, or a court examining a challenged decision all weigh independent evidence differently from a self-report.
Self-assessment carries little evidentiary weight with the audiences that matter: regulators, courts, enterprise procurement teams, and insurers discount conclusions the subject reached about itself. Additionally, some regimes require independence outright rather than as a matter of preference, including New York City Local Law 144, which requires that bias audits of automated employment decision tools be conducted by an independent auditor.
Internal assessment remains valuable and we expect clients to do it. It complements independent assurance rather than substituting for it.
We can provide assurance over any subject matter for which criteria can be made suitable, and for which sufficient evidence is available. In practice this includes bias and disparate impact in system outcomes; accuracy, calibration, and robustness of model performance; data provenance, quality, and permissible use; completeness of technical documentation against a required schema; whether governance and oversight controls operate as described; whether human review is substantive rather than nominal; the accuracy of public or contractual claims about what a system does; and conformity with specific legal provisions.
We cannot provide assurance that a system is safe, fair, or trustworthy in the abstract. Those terms become auditable only once translated into measurable criteria with stated thresholds, which is a substantial part of the scoping work at the start of an engagement.
We work across a variety of AI systems, including classical machine learning systems such as scoring and classification models, generative and large language model systems including retrieval-augmented and agentic architectures, computer vision and biometric systems, and ranking and recommendation systems. We work with developers and deployers of AI systems.
We have started working with New York City Local Law 144. However, we can design engagements around whatever criteria apply to you: the EU AI Act, Illinois HB 3773, the Uniform Guidelines on Employee Selection Procedures, Colorado SB 26-189.
As long as suitable criteria can be developed with sufficient evidence, we can cover it. Talk to us about your needs!
There are three main dimensions along which our most common bias audits can be distinguished:
Criteria: Either BABL AI’s peer-reviewed NYC Local Law 144 criteria or our more flexible General Bias Audit criteria.
Engagement Structure: direct (we conduct testing) or attestation (you conduct testing, we verify it).
Assurance Level: reasonable (includes governance and risk assessment criteria as well, more in depth) or limited (just technical criteria).
If there are no relevant standards, we can build the audit criteria around the specific assertions you want to be audited. For example, if you need defensible third-party evidence about your data governance practices but no relevant standards exist, we can formalize the claims you want to make into auditable statements. From there, we can develop suitable criteria to evaluate your evidence against. The end result will still be an audit report that generates credibility as third-party evidence. The lack of existing standards is not a significant barrier to assurance.
An average attestation bias audit engagement typically runs around 1-2 months. The timeline depends on the number of models in scope, the maturity of your existing documentation, and how quickly evidence requests are answered.
Annual re-auditing is the common cadence, and some regimes require it explicitly (like New York City Local Law 144). Between annual cycles, re-audits should be triggered by changes to the system that could be reasonably expected to materially change the system’s behavior: significant retraining, a new feature, updated decision thresholds, or deployment to a new population or jurisdiction. An audit conclusion speaks to a specific version of a system as of a specific date, and a system that materially deviates from the one we examined should be re-audited.
Independence is maintained through structural separation rather than assurance alone. We do not design, build, or remediate a system and then form an opinion on it. Fees are fixed in the engagement letter and are never contingent on our findings. We hold no financial interest in clients and perform conflict checks at engagement acceptance. Conclusions are subject to engagement quality review by senior advisors before the report is issued. Where a client wants both advisory and assurance work, the engagements are separated, staffed separately, and the relationship is disclosed in the report.
These requirements follow from the ethical requirements underlying ISAE 3000. They also protect you, because a report from an auditor with a stake in the outcome is worth less to the third parties you need to persuade.
The access required depends on the type of engagement. For an attestation style engagement where we verify your testing, we do not need access to your system. We review your narrative responses to our worksheet, documentation you provide to us as needed, and share the code used to generate the test results on a meeting with us so we can verify the numbers were generated accurately.
For a direct engagement where we conduct testing, system access is oftentimes still not a requirement. For example for NYC Local Law 144 we only need a dataset containing the candidates in scope of the audit and the outcomes they received. Clients often share this with us in the form of csv or json files, with no access to your system required.
Education & Certification
Our flagship certification is the AI Audit & Assurance Professional Certification, which is designed for professionals who wish to demonstrate their competence in AI auditing and assurance. Candidates can prepare for the certification by taking our training program, which consists of five core courses. These courses cover AI and machine learning, algorithmic risk and impact assessments, AI governance and risk management, AI testing and evaluation, and algorithm auditing and assurance. Training is optional and separate from the certification process.
We also offer role-based training, including “AI Governance for Business Professionals” and “AI Governance for Legal Professionals,” as well as a free introductory course for individuals interested in exploring the field of AI audit and assurance and its certification pathway.
As our educational offerings continue to evolve, please visit the Education section of the BABL AI website for the latest information on available courses, certifications, specialized training, and upcoming programs.
Our courses are designed for professionals who need to understand, govern, evaluate, or audit AI in their work. This includes internal auditors; AI, responsible AI, and risk and compliance professionals; business leaders and managers; legal professionals working with AI; and individuals preparing for a career in AI audit and assurance.
We offer different programs for different roles and levels of experience. Our role-based programs focus on the AI governance knowledge most relevant to business and legal professionals. Our AI Audit & Assurance Professional training program is designed for individuals looking to develop the knowledge and skills necessary for AI auditing and assurance.
You do not need to be a technical specialist or already an auditor to get started. Any specific prerequisites or recommended experience are listed on the program or course page.
Yes. Most of our courses are designed to be accessible to professionals without a technical background. You don’t need to be a data scientist, software engineer, or machine learning specialist to get started.
For instance, our AI Audit & Assurance Professional training program covers the essential AI and machine learning concepts to help you understand how AI systems work, ask the right questions, evaluate risks and controls, and interpret technical and testing evidence from an auditor’s perspective. Our goal is not to turn you into an AI engineer but rather to provide you with the technical knowledge necessary to evaluate AI systems responsibly and effectively.
However, some specialized or advanced training may require prior technical knowledge. Any prerequisites or recommended experience will be clearly stated on the individual course page.
Yes. BABL AI may offer discounts on select training courses and programs for eligible groups or during promotional periods. Contact us to check whether you are eligible!
The available discounts, eligibility requirements, and promotional offers may change over time. Please note that discounts for training do not necessarily apply to certification fees, as training and certification are separate.
We also encourage organizations enrolling multiple team members to contact us to discuss group enrollment or custom training options.
It depends on the program. Completing a BABL AI training course or program is different from earning a professional certification. Where applicable, learners may receive a certificate of completion for successfully completing training, but course completion alone does not make someone a certified AI audit and assurance professional.
Our flagship professional credential is the Certified AI Audit & Assurance Professional. It is earned through a separate assessment process. Taking BABL AI training is optional and is not a requirement for earning the credential. The certification will be available starting latest beginning of 2027.
The Certified AI Audit & Assurance Professional credential is recognized at Tier 1 by the International Association of Algorithmic Auditors (IAAA). We are also developing the certification program in alignment with ISO/IEC 17024, the international standard for organizations that certify people, as a step toward future accreditation.
Recognition and credential details vary across our other training programs. Please refer to the individual program page for the most current information.
Our programs are built from our experience doing the work. BABL AI professionals work directly on AI audit, assurance, governance, risk, and testing engagements, and we bring lessons from that real-world experience into our training.
That means our courses go beyond explaining frameworks and concepts. We focus on how they are applied in practice: how to assess AI risks and impacts, evaluate governance and controls, examine evidence, interpret testing results, and reach defensible conclusions.
Our work with organizations across industries also helps us keep the training grounded in the challenges professionals encounter in practice. The goal is to give learners knowledge and skills they can apply in real AI governance, audit, and assurance work and not simply prepare them to complete a course.
The AI Audit & Assurance Professional training program is self-paced. The time it takes to complete the program depends on your background and experience, as well as how much time you dedicate to the material each week.
The program consists of five core courses that cover AI and machine learning; algorithmic risk and impact assessments; AI governance and risk management; AI testing and evaluation; and algorithm auditing and assurance.
Since you can work through the training on your schedule, you can complete it more intensively or spread your learning out over a longer period to accommodate your professional commitments.
If you are new to the field or not yet sure which learning path is right for you, we recommend starting with our free introductory course, Finding Your Place in AI Ethics Consulting.
The course introduces the AI governance, audit, and responsible AI landscape and helps you understand the different roles and career paths available in the field. It is a good starting point before deciding whether you want to pursue professional certification, build AI governance skills for your current role, or continue with more specialized training.
If you already know what you want to achieve, you can also explore our individual program pages to choose the training that best matches your role and goals.
Yes. Organizations can enroll multiple team members in our training programs. This is a good option for teams that want to develop a shared understanding of AI governance, risk, auditing, and assurance throughout their organization.
We also offer custom training tailored to larger groups or organizations with specific learning needs. Contact us to discuss the best option for your organization.
Although our courses are self-paced, you will not be learning on your own. Students have access to multiple channels for asking questions, discussing the material, and receiving support throughout their learning journey.
Depending on the program, support may include live Q&A and office-hour sessions with BABL AI experts; access to a private Slack community, where you can connect with instructors and fellow learners; and the opportunity to schedule one-on-one support calls with our Training Programs Manager for additional guidance.
Support options may vary by program, so please refer to the program page for details.
Our training and certification programs can help you develop knowledge and skills relevant to AI auditing and assurance. However, they do not guarantee employment or a specific career outcome.
The AI Audit & Assurance Professional program is designed around practical competencies, such as understanding AI systems, assessing risks and impacts, evaluating governance and controls, interpreting testing evidence, and applying audit and assurance principles. Earning the professional credential demonstrates that you have met the certification requirements.
For individuals new to the field, the program provides a solid foundation for pursuing opportunities in AI audit, assurance, governance, risk, and responsible AI. However, certification does not substitute for hands-on professional experience, and the qualifications that employers look for vary by role and organization.
Although there are no formal education or experience prerequisites, all certification candidates must complete a short application.
You can. BABL AI training is optional and not required to earn the Certified AI Audit & Assurance Professional credential.
If you have the necessary knowledge and experience, you can pursue the certification independently.
All candidates must meet the same certification requirements, regardless of how they prepare. Details will be shared on our website soon.
Earning the Certified AI Audit & Assurance Professional credential demonstrates that you have met BABL AI’s certification requirements for knowledge and skills. However, earning the credential does not mean you are automatically prepared to independently lead every type of AI audit.
Conducting an AI audit requires professional judgment and practical experience. Some engagements may also require specialized technical, legal, regulatory, or industry expertise. The necessary level of experience and expertise depends on the scope and complexity of the audit.
This certification provides a strong foundation and certifies that you can participate as a team-member in an audit team for a system-level attestation style audit. Other scopes (lead auditor or a direct testing engagement) are not certified through the credential.
Still have a question?
Tell us the situation and we will point you to the right person, whether that is an audit, a course, an advisory call, or something we have not listed here.