Customizing LLMs for domain-specific tasks

The expansion of large language models (LLMs) in recent times has brought about a revolutionary change in machine learning processes and has introduced fresh perspectives on the potential of AI, according to Predibase.

Based on survey data from organizations experimenting with LLMs, researchers have found that enterprises are looking for ways to customize and deploy open-source LLMs without giving commercial vendors access to proprietary data, and they are exploring other use cases beyond generative AI capabilities.

“It is now open season for LLMs. Thanks to the widespread recognition of OpenAI’s ChatGPT, businesses are in an arms race to gain a competitive edge using the latest AI capabilities. Still, they require more customized LLMs to meet domain-specific use cases,” said Piero Molino, CEO of Predibase.

“This report highlights the need for the industry to focus on the real opportunities and challenges as opposed to blindly following the hype,” Molino added.

Enterprise adoption of LLMs

Less than a quarter of enterprises are comfortable using commercial LLMs. 33% cite concerns about sharing sensitive or proprietary data with commercial LLM vendors, leading to increased interest in privately hosted, open-source alternatives.

Open-source LLMs are gaining momentum. Nearly 77% of respondents either don’t use or don’t plan to use commercial LLMs beyond prototypes in production, citing concerns about privacy, cost, and lack of customization, leading to an uptick in open-source alternatives. Meta, for example, has moved away from building closed-source LLMs like LLaMA-1, replacing it with LLaMA-2, available as open-source and free for commercial and research applications.

While generative AI use cases remain popular, enterprises see the potential of other applications to provide business value. Information extraction is the second most popular use case (selected by 32.6% of respondents).

This involves leveraging LLMs to convert unstructured data like PDF documents or customer emails into structured tables for aggregate analytics. Next was Q&A and Search (15.2% of respondents), the brain in chatbots that provides accurate and relevant responses to user queries in real-time.

Customized LLMs

Organizations are turning to customized LLMs to achieve more accurate and tailored results. Most teams plan to customize their LLMs by fine-tuning (32.4%) or reinforcement learning with human feedback (27%). The roadblocks team face with fine-tuning continue to be a lack of data (21%) and the overall complexity of the process like managing infrastructure (46%).

“We see clear potential to improve the outcomes of our conservation efforts using customized open-source LLMs to help our teams generate insights and learnings from our large corpus of project reports,” said Dave Thau, Global Data and Technology Lead Scientist, World Wildlife Fund.

“The trick, of course, will rest not in building these outcomes but in ensuring that they deliver consistent, secure, responsible outcomes. With an increasing desire to customize and deploy open-source models, enterprises will need to invest in operational tooling and infrastructure capable of keeping up with the rapid pace of innovation in the open-source community,” Shimmin concluded.