A new study from researchers at the University of Minnesota and University of Michigan law schools suggests that next-generation legal AI tools can significantly enhance the quality and speed of legal work, offering the strongest empirical evidence yet that generative AI could meaningfully augment—not just automate—lawyering.
Published in March, the study is the first randomized controlled trial to assess the impact of two emerging AI innovations: retrieval-augmented generation (RAG) and reasoning models. Researchers tested these tools by assigning 127 upper-level law students a series of realistic legal tasks. Participants used either Vincent AI, a RAG-powered legal assistant developed by vLex, OpenAI’s o1-preview reasoning model, or no AI at all.
The results were striking. Across five of six tasks, both AI tools delivered large and statistically significant productivity gains, improving efficiency by as much as 140 percent. More notably, the quality of legal work also improved—an outcome that contrasts with earlier studies showing AI’s impact on legal writing to be mixed or minimal.
Vincent AI improved performance largely by boosting clarity, organization, and professionalism, while minimizing factual hallucinations. Its grounding in legal source material meant that students using Vincent AI made roughly the same number of factual errors as those working without any AI support. Meanwhile, the o1-preview model improved analytical depth, with participants demonstrating more coherent and nuanced legal reasoning. However, the tool did generate more hallucinations than Vincent.
The study also found that AI’s benefits were concentrated in litigation-related tasks such as drafting memos and client letters, rather than transactional work like contracts. Survey data collected afterward revealed that students grew more confident using AI throughout the trial and indicated they would be more likely to use similar tools in future professional settings.
Despite some lingering concerns about hallucinations and overreliance, the researchers conclude that these AI tools, especially when used in tandem, could reshape legal practice in meaningful ways—streamlining routine work and freeing up lawyers to focus on complex judgment calls.
In a profession long defined by precedent and precision, the findings signal a potential turning point. With the right safeguards, generative AI may become an indispensable part of the modern lawyer’s toolkit.
Need Help?
If you have questions or concerns about how to navigate the global AI regulatory landscape, don’t hesitate to reach out to BABL AI. Their Audit Experts can offer valuable insight, and ensure you’re informed and compliant.
