Ninety percent of professionals who participated in a month-long pilot program giving them access to generative AI tools reported a rise in productivity by the end of the trial, with a quarter indicating a "medium to high level of productivity increase," according to the study.
The study was led by Colleen V. Chien, co-director of the Berkeley Center for Law and Technology, as well as Munger Tolles Olson LLP partner Miriam Kim, who is also a fellow at the center. It included an initial survey of 202 legal aid professionals, followed by a field study of 91 people who received free access to paid generative AI tools for at least one month. Chien and Kim believe it to be the first field study of lawyers using generative AI.
The preliminary draft of their paper "Generative AI and Legal Aid: Results from a Field Study and 100 Use Cases to Bridge the Access to Justice Gap" was published on March 14, and the final is set to be published in the Loyola of Los Angeles Law Review.
"There is a huge justice gap that we both have personally witnessed," Chien said in an emailed statement to Law360 on Wednesday. "Generative AI has the potential to level the playing field, in a number of realms, but it won't do so if the technologies are not accessed or applied by those on the bottom as well as on the top. As well-resourced law firms rush to adopt these powerful new technologies, we thought it was important to focus on what legal aid attorneys were doing."
Chien had noted in an earlier statement that in addition to most participants reporting increased productivity, 75% said they planned to continue the use of these tools. She acknowledged this was "despite concerns like AI bias and hallucinations," which are common with the nascent technology.
"We wanted to show, through use cases, how legal aid professionals had managed to [surmount] the problems, manage the risks and save time. What was striking to us is that the concerns following the pilot were still there, but despite them, people wanted to keep using the tools and ... said that they saved them time," Chien said Wednesday.
In managing those risks, the participants generally used AI tools for tasks that can be considered "lower risk," such as document summarization, preliminary legal research, translations, brainstorming and first drafts, according to the study.
The generative AI tools provided to participants were OpenAI's GPT-4, Casetext's AI assistant Co-Counsel and Gavel.io's eponymous document automation tool. The participants were also invited to use the free versions of ChatGPT competitors including Google's Gemini, Microsoft's Bing Chat and Anthropic's Claude.
At the start of the study, women, who made up a majority of the survey pool, were one-third as likely as their male counterparts to already be using AI tools, and half of the women surveyed were also unsure of or not planning to use AI, compared to 28% of men responding in the same way.
"By the end of the pilot, the gender gap closed, with women experiencing positive outcomes using AI as much as men. This shift shows the importance of ensuring that usage of generative AI does not tilt along traditional gender lines," Kim said in a Tuesday statement.
Additionally, a randomized subset of participants received additional training and support for these tools, dubbed "concierge services." This group was more likely to report satisfaction and productivity increases, with 88% reporting positive impacts from using generative AI tools, compared to 62% of the group with standard access.
In addition to urging AI tool providers to commit to making products freely accessible to legal aid providers, Chien and Kim suggested the creation of a national help desk program that could provide the concierge-type assistance to legal aid professionals on a broader scale.
They also called on tech providers and legal aid providers to work together to create more AI-powered tools.
"Legal aid lawyers are at the forefront of the justice gap, so they are well-qualified to help design solutions that will increase their efficiency, reach and impact," the study said.
It is estimated that 92% of low-income Americans' civil legal needs go unmet, according to the Legal Services Corporation. And while legal aid organizations are a key source of support for this community, they can only serve about 30% of those who come to them, the study noted.
Alongside the paper, Chien and Kim released a database of 100 "helpful use cases" that participants in the trial provided. For example, Michael Semanchick, executive director at the Innocence Center, reported using Co-Counsel to summarize appellate court opinions and trial records, while a pro bono program director at Legal Aid of North Carolina used Gavel to convert a template for durable power of attorney into a fillable form for volunteer attorneys.
Unidentified participants also used GPT-4 to brainstorm discovery requests, draft a memorandum of understanding between two non-profits and generate talking points, among other use cases.
Update: This story has been updated to clarify a quote.
Have a story idea for Access to Justice? Reach us at accesstojustice@law360.com.
For a reprint of this article, please contact reprints@law360.com.
