The U.S. Government Accountability Office (GAO) recently released a report examining the fast-evolving commercial sector of generative artificial intelligence (AI). The report highlights practices used by developers to ensure responsible deployment, alongside key challenges that compromise model reliability and security.
Best Practices
The GAO report identifies several best practices employed by commercial developers in the responsible development of generative AI. Comprehensive testing, multidisciplinary evaluations, and safety protocols are emphasized in the report.
- Benchmark Testing: Developers rely on standardized benchmark tests to assess model accuracy and reliability across various domains. These tests evaluate model performance in areas like general reasoning, mathematical problem-solving, and coding capabilities, providing a quantitative foundation for further improvement.
- Multidisciplinary Review: AI developers frequently involve experts from multiple fields, such as cybersecurity, ethics, and legal studies, in evaluating the potential risks of their models before release. This collaborative approach helps detect vulnerabilities, particularly in relation to sensitive or harmful content, and informs modifications for safer deployment.
- Red Teaming for Security: Red teaming—emulating potential attacks—is central to AI risk management. This strategy is applied across AI models to identify flaws that malicious users might exploit. The report notes that developers now conduct red teaming focused on threats like unauthorized replication and cybersecurity breaches. Red teaming practices have proven essential in mitigating risks, though developers acknowledge they may not address every potential vulnerability.
- Post-Deployment Monitoring: Once AI models are live, developers continue to monitor them for misuse. This includes tracking user interactions that could indicate attempts to exploit or manipulate model outputs, such as spreading misinformation or generating explicit content. Developers often use such data to restrict access for users who violate safety policies.
- Data Policies for Privacy and Safety: Companies have established data policies to guide the ethical collection and use of information for model training, reducing reliance on personal data where possible. Privacy and safety standards are applied to training datasets to minimize biases and protect users’ personal information.
Ongoing Limitations and Vulnerabilities
Despite improvements, generative AI systems still face major limitations. Developers admit that models can produce inaccurate or biased answers, a problem often labeled as “hallucination” or “confabulation.” Because these responses appear convincing, they can lead to unintentional misinformation. GAO also highlights threats such as prompt injection attacks and jailbreaking, which allow users to bypass safeguards and generate harmful or restricted content. These vulnerabilities require continuous monitoring and fast mitigation to keep systems secure.
Transparency remains a central issue. Developers usually disclose only high-level information about training data. That lack of detail has raised concerns about privacy, copyright, and the presence of personal information in large datasets. Although companies run privacy checks and attempt to filter sensitive data, experts caution that it is extremely difficult to remove all personal information from massive training sets.
Future GAO Work
GAO plans additional reports on generative AI. Future assessments will focus on the societal and environmental impacts of AI and how federal agencies should respond as commercial models grow more capable and widespread. The ongoing review reflects the government’s intent to manage risk while encouraging responsible development that aligns with public safety and ethical expectations.
Need Help?
You’re probably concerned or have questions about how to navigate the U.S. or global AI regulatory landscape. Therefore, don’t hesitate to reach out to BABL AI. Hence, their Audit Experts can offer valuable insight and ensure you’re informed and compliant.
