Managing the risks associated with the use of artificial intelligence (AI) and machine learning (ML) has been an urgent topic in recent years. The potential for these algorithms to discriminate, limit access to important life opportunities, and otherwise harm individuals and organizations has motivated the need for companies to implement deliberate AI Risk Management Frameworks.

Recent activity includes:

• The release of NIST’s AI Risk Management Framework 1.0, which outlines four core functions for a robust framework; Govern, Map, Measure, and Manage.
• Article 9 of the proposed EU AI Act, which outlines general risk management requirements for “high-risk” AI systems.
• ISO’s recently released ISO/IEC 23894:2023, Guidance on risk management.
• Articles 34 and 35 of the Digital Services Act (DSA) require large online platforms to assess and manage “systemic risks”, including those posed by algorithms such as recommender systems or targeted advertisements.

What is Colorado’s Senate Bill 21–169?

The increased use of external data and predictive algorithms in the insurance industry has given rise to worries about unfair discrimination, and the need for insurers to manage the unique risks that AI/ML may entail. This is why Senate Bill 21–169 was enacted by the General Assembly of the State of Colorado. The law recognizes the increasing use of what it calls “external consumer data and information sources” (ECDIS), as well as algorithms and predictive models using external consumer data, in insurance rating, underwriting, claims, and other business practices. These tools have the potential to benefit insurers and consumers, however, the accuracy and reliability of external consumer data can vary greatly, and some algorithms and predictive models may lack a sufficient rationale for use in insurance practices. The use of these tools could potentially have a negative impact on the availability, affordability, and utilization of such insurance.

To address these issues, Colorado’s Department of Regulatory Agencies Division has released draft regulations for SB21–169, which focuses on underwriting practices in life insurance, that require insurers to adopt a governance and risk management framework. The framework is designed to control the use of external consumer data, algorithms, and predictive models to prevent any unfair discrimination based on race, color, national or ethnic origin, religion, sex, sexual orientation, disability, gender identity, or gender expression.

What are the Governance and Risk Management Framework Requirements?

While the framework proposed by NIST allows for a lot of flexibility depending on a company’s size and risk profile, the governance and risk management framework proposed for SB21-169 is quite prescriptive, and must include the following components:

1. Documented governing principles outlining the values and objectives of the insurer that ensure that ECDIS, algorithms, and predictive models using ECDIS are designed, developed, used, and monitored transparently and accountably and do not lead to unfair discrimination.
2. Board of directors and senior management responsibility and accountability for setting and monitoring the overall strategy, and providing direction for governance on the use of ECDIS, algorithms, and predictive models. This includes establishing clear lines of communication and regular reporting to senior management on the performance and potential risks of ECDIS, algorithms, and predictive models.
3. Cross-functional algorithm and predictive model governance committee composed of representatives from key functional areas including legal, compliance, risk management, product development, underwriting, actuarial, data science, marketing, and customer service, as applicable.
4. Clearly assigned and documented roles and responsibilities of key personnel involved in the design, development, use, and oversight of ECDIS, algorithms, and predictive models using ECDIS.
5. Established written policies and processes for the design, development, testing, deployment, use, and ongoing monitoring of ECDIS and algorithms and predictive models that use ECDIS and to ensure that they are documented, tested, and validated.
6. Development and implementation of an ongoing supervision and training program for relevant personnel on the responsible and compliant use of ECDIS, algorithms, and predictive models including issues related to bias and potential unfair discrimination.
7. Implementation of controls to prevent unauthorized access to algorithms or predictive models.
8. Processes and protocols in place for addressing consumer complaints and inquiries about the use of ECDIS, algorithms, and predictive models in a manner that provides consumers with sufficiently clear information necessary for consumers to take meaningful action in the event of an adverse decision.
9. Plan for responding to and recovering from any unintended consequences.
10. Engage outside experts for performing audits when internal resources are insufficient.

Additionally, if an insurer uses third-party vendors and other external resources with respect to ECDIS and predictive models, the insurer is responsible for ensuring regulatory requirements are met.

What are the Documentation Requirements?

Life insurers must also maintain comprehensive documentation for their use of all ECDIS and algorithms and/or predictive models that use ECDIS, including those supplied by third parties. Documentation must include an up-to-date inventory of all ECDIS, algorithms, and predictive models in use, including a detailed description of each, results, and timing of annual reviews of the inventory. Insurers must also maintain a system for tracking and managing changes, a description of testing conducted to detect unfair discrimination, a description of the input and output of the algorithm and/or predictive model, and a description of any limitations of the algorithm and/or predictive model.

Insurers must also conduct regular reviews and updates to the documentation to ensure its continued accuracy and relevance, and all documentation must be easily accessible to appropriate insurer personnel and available upon request by the Division.

What are the Reporting Requirements?

Beyond making documentation easily accessible, the draft regulations have a number of reporting requirements.

• Insurers currently using ECDIS and algorithms/predictive models with ECDIS must submit a progress report to the Division within six months of the effective date of the regulation (TBD), outlining current compliance with risk management and documentation requirements, areas under development, any difficulties, and expected completion date.
• Insurers must also submit a final report demonstrating compliance within one year of the effective date of the regulation, including details of their completed compliance with the risk management and documentation requirements.
• Insurers must submit a report every two years following the report required above, containing an up-to-date inventory of all ECDIS and algorithms/predictive models, results and timing of reviews, any material changes to the governance and risk management framework, and any risks detected and steps taken to mitigate them.
• Insurers not using ECDIS or algorithms/predictive models must submit an attestation within one month of the regulation’s effective date and annually thereafter, signed by an officer indicating that the insurer does not use ECDIS or algorithms/predictive models.
• Insurers not using ECDIS or algorithms/predictive models but planning to use them in the future must first submit the progress report specified above and then comply with the full reporting requirements upon adoption.

Conclusion

The new AI risk management framework required by Senate Bill 21–169 is a particular example of a more general approach to AI risk management that companies can and should use when using or developing AI/ML business solutions. For insurers, and the vendors they may rely on for AI/ML solutions, considering these requirements holistically with other global standards like the NIST AI Risk Management Framework and the requirements of the EU AI Act will ensure good coverage in a rapidly changing regulatory environment.

Need help? Contact us for a free consultation.