6.4 Retrievers: Individual, Ensemble & Filtered Retrieval

6.4 Retrievers: Individual, Ensemble & Filtered Retrieval

Quick Answer: A retriever is the use-case-specific way to search a Data 360 search index and return the most relevant results to a prompt template, agent, or flow. You build and activate retrievers in AI Models (formerly Einstein Studio). Individual retrievers query one index with filters, output fields, and a number of results. Ensemble retrievers combine several individual retrievers and rerank their results. Dynamic filters take values at run time. Retrievers respect data access, so users and agents only get what they're allowed to see.

Why Retrievers Sit Between Indexes and Prompts

A search index is often general-purpose, such as an index of the entire knowledge base. A retriever specializes it for one job, for example:

• Only articles in a product category
• Only the most recent content (such as the last 90 days)
• Only a geographic region
• Only certain fields, such as Description but not Resolution (using DataSource__c)

At run time the retriever gives the LLM accurate, current, pertinent information, which improves relevance and reduces hallucination.

Retriever Types

Configuring an Individual Retriever

In AI Models → Retrievers:

1. Select the search index (the data space and DMO it covers).
2. Define filter conditions (up to ten) to narrow results.
3. Choose output fields, meaning what data returns to the prompt. By default, all fields defined in the retriever configuration are added.
4. Set the number of results, such as 15 knowledge chunks or 10 similar case chunks.
5. Optionally enable citations at the individual retriever level so callers such as prompt templates can show sources.
6. Save (new version) and activate the version to make it available in Prompt Builder.

Filter conditions

• Combine conditions with AND, OR, or custom logic such as 1 AND (2 OR 3).
• Like: in the retriever definition, don't add a trailing %, because one is appended internally. When supplying a value from Prompt Builder or Flow for a dynamic Like filter, you must append the trailing %.
• Data type guidance: text up to 255 characters (254 with Like), Boolean True or False, dates such as "Dec 31, 2024", no ArrayOfTexts, and no single quotes in values.
• Dynamic filter conditions use placeholders such as {!$ItemName}. Placeholder names must start with a letter, use alphanumeric characters and non-consecutive underscores, have no spaces, not end with an underscore, be at most 40 characters, and be unique across pre-filter fields.

Using Retrievers in Prompt Builder

1. In the Prompt Template Workspace, type @ or click Insert Resource → Retrievers → Configure Retrievers.
2. Select an individual, ensemble, or web retriever (it must be active).
3. In Template Settings, configure:
   • Search Text – merge fields from prompt inputs or free text. It's limited to globals and prompt inputs, 255 characters, and can't use related lists, flows, or Apex.
   • Output Fields – select only the fields the prompt needs.
   • Number of Results – the maximum results returned.
   • Pre-filter – appears only if the retriever has at least one dynamic filter condition. Bind it to a run-time value, such as the case's country or product ID.
4. Apply and Insert, save, and preview. The Preview Resolution shows retriever data in JSON.

If you don't set output fields or number of results, Prompt Builder uses the values from the retriever's configuration.

Permissions: Prompt Template Manager to build templates, the Data Cloud permission set to create and manage retrievers, and Data Cloud User to run prompts with Einstein search.

Using Retrievers with Agents

• A Data Library creates its own retriever, and Answer Questions with Knowledge uses it.
• A custom retriever data library can point to any active retriever, including an ensemble, so an agent can draw on multiple sources.
• Dynamic pre-filters aren't supported with the Answer Questions with Knowledge template. Use static pre-filters, separate libraries per category, or a custom Apex action that passes filter values.
• For custom logic, a prompt template action with its own retriever and dynamic pre-filter bound to a variable-driven input gives precise control.

Testing Retrievers: The Retriever Playground

In AI Models, the retriever playground lets you configure pre-filters, choose test data, adjust result parameters, and review intermediate outputs and metrics, such as Source Recall Score and Context Recall Score, before the retriever goes into an agent. Salesforce calls it a cost-effective way to test and debug retrievers. Testing still consumes Data Queries credits.

Limitations:

• Pre-filter conditions for individual retrievers are tested separately, not inside an ensemble.
• Ensemble retrievers that contain custom SQL retrievers can't be tested in the playground.

Security and Governance

• Retrievers adhere to the data governance policies for your data.
• Row-level access checks ensure the user or AI agent calling the retriever can access the source records. If the caller can't access a record, it doesn't appear in results.
• For Service agents, the agent user's access applies. For Employee agents, the logged-in user's access applies.

Measuring Retrieval Quality

Knowledge/RAG Quality Data and Metrics (enabled in Einstein Audit, Analytics, and Monitoring Setup) scores run-time retrievals for individual and ensemble retrievers:

Deployment Note

Retrievers and search indexes aren't included in change sets or Metadata API deployments of prompt templates. Recreate them in the target org first, or move Data 360 configuration with data kits. Packaged templates that depend on retrievers need those retrievers provided through data kits.

Exam Traps

• Only one active version per retriever. Activating a new version deactivates the old one.
• Dynamic-filter retrievers can't join an ensemble.
• The Pre-filter field appears only for retrievers with a dynamic filter.
• Retrievers enforce access. Missing results can be a permissions issue, not a relevance issue.