Search engines and intelligent assistants both serve the critical function of providing information and answers to users. However, digital workplace applications architects frequently question whether GenAI assistants can replace enterprise search. While these technologies are different, they are also complementary. One does not replace the other. The hype surrounding GenAI is blurring the distinction between search engines and intelligent assistants. They are two halves of a composite solution, and it is important to understand the role of each in the digital workplace.
Distinct Roles of Search Engines and Intelligent Assistants
The purpose of enterprise search is to make an organization’s information resources findable and accessible. Search platforms also manage access control by respecting the permissions and privileges assigned to content. The core mechanism enabling search is the index, which captures the content and its structure, allowing information to be matched to queries. This content can be enriched with additional metadata to enhance indexing capabilities beyond simple keyword matching. The index is updated regularly and automatically to ensure the availability of the most current information.
AI assistants, on the other hand, use the linguistic and reasoning capabilities of large language models (LLMs) to interpret natural language requests and either perform a task or formulate a natural language response. Intelligent assistants are becoming a common feature of major platforms. In most cases, the capabilities of such vendor-specific GenAI assistants are restricted to their native platforms. Incorporating information hosted outside of that platform is often possible but problematic. External content lacks the contextual information available from content hosted within the platform and is often disadvantaged or overlooked when the assistant responds to a request.
Foundation Models Rely on Search for Enterprise Context
The LLMs powering most intelligent assistants are foundation models like GPT or Gemini. These models have a significant limitation: they are unaware of any information not included in their training data. This means that any private, proprietary, or recent information is unavailable to the model.
LLMs can be “tuned” to perform specific tasks or incorporate additional information beyond the initial training data. While useful and often necessary, tuning is difficult and expensive, yet the model becomes obsolete as soon as tuning is complete.
A search engine can overcome this limitation by providing the LLM with the additional information needed to fulfil a request or complete a task in real time. This method, known as retrieval-augmented generation (RAG), is a key enabler for intelligent assistants. An assistant built with RAG uses a search engine to locate and retrieve enterprise resources, which are then passed to an LLM as part of a prompt, along with usage instructions. This allows knowledge resources from across the enterprise to be used as raw material for GenAI, forming the basis for a generated response.
Grounding Intelligent Assistants in Enterprise Context
Using search to provide information to an intelligent assistant grounds the latter in an enterprise context. This approach can significantly reduce the likelihood of counterfactual responses or “hallucinations,” although it cannot eliminate them entirely. The quality of the information retrieved for RAG largely determines the output quality, making content management and governance essential. Optimizing the search infrastructure by incorporating both lexical and semantic search increases the likelihood that only relevant information is passed to the LLM.
Additional techniques to ground GenAI in the enterprise are emerging, including knowledge graphs, model ensembles, and advanced prompt engineering. Despite these advancements, hallucinations and incorrect responses will sometimes be generated by any intelligent assistant. Therefore, any content generated by an LLM and RAG solution must still be verified and validated. Human oversight is required to ensure that the generated content is accurate and fit for use.
Author bio: Darin Stewart, VP Analyst at Gartner
Read next: Gartner: Comparing and evaluating large language models
Image: Freepik