Adaptive Instructed-Retriever: Frontier-Quality Search at 2x Lower Latency

Effective enterprise data agents require search that is both accurate and fast. Earlier this year we released Instructed-Retriever-1, a retrieval model that can incorporate enterprise data schemas and custom instructions while using parallel test-time scaling to improve retrieval accuracy with low latency. This single-step search approach works well for a large share of user requests. However, more complex, multi-hop questions can still benefit from sequential search, where the model iteratively gathers evidence and refines its queries over multiple steps – at the cost of additional latency.

That's why we're now introducing Adaptive Instructed-Retriever, which combines the speed of parallel retrieval with the performance of sequential search while maintaining strict cost and latency guarantees. The goal is simple: spend additional search steps only when they are useful. This is the same retrieval-efficiency problem faced by Genie Code, Databricks' data agent: it must find the right tables, notebooks, dashboards, and documents in a large, changing workspace without wasting turns on brute-force exploration. Adaptive Instructed-Retriever is designed for this retrieval layer, returning quickly when the evidence is clear and using sequential search only when a harder request requires it. As we show in this post, the trained model matches the quality of leading third-party models at 2x lower latency and improves substantially over single-step search on our retrieval benchmarks.

To build Adaptive Instructed-Retriever, we impose a fixed upper bound on the number of sequential steps and train the agent to adaptively decide how much computation each user request requires. When sufficient evidence has already been found, the agent stops early and returns the relevant evidence; when additional search is likely to improve retrieval quality, it can continue searching up to the step limit.

Training Adaptive Instructed-Retriever

To achieve a better balance between retrieval quality and the latency cost of sequential scaling, we trained Adaptive Instructed-Retriever – a small custom model that supports both parallel single-step retrieval and sequential search while operating at substantially lower latency than leading third-party models. We evaluate the model on a mix of proprietary enterprise retrieval and public retrieval benchmarks, including tasks that benefit from multi-hop reasoning. Across these benchmarks, Adaptive Instructed-Retriever achieves performance comparable to leading third-party and open-source models while delivering 2x lower latency.

To prepare the training data, we rely on synthetic enterprise retrieval environments and an agentic data synthesis process similar to Instructed-Retriever-1 and published in the KARL report. We reuse the existing training data for Instructed-Retriever-1 to preserve the model’s ability to do fast parallel single-step search, and additionally introduce synthetic multi-hop questions that benefit more from multiple agentic search steps.

Starting from the base model, we use online reinforcement learning (ORL) to teach the model to take additional search steps only when they are likely to improve final performance. Specifically, we optimize the model end-to-end using CISPO (Clipped Importance Sampling Policy Optimization), with a reward design that balances trajectory quality against search cost: the model is rewarded for high-performing trajectories while being penalized for additional search steps that do not yield corresponding performance gains.

We train the model using AI Runtime (AIR). AIR is also available to Databricks customers, making this approach practical for developing specialized models for their own domains and workloads. The training recipe is intentionally lightweight: we start from a pretrained base model and use a modest amount of synthetic data to specialize its search behavior. As Figure 2 demonstrates, our lightweight approach generalizes well to new search tasks and domains.

Figure 2 compares the Adaptive Instructed-Retriever to two leading third-party models (Claude Sonnet 5 and GPT-5.6 Luna) and one open-source model (DeepSeek-V4-Flash). We plot retrieval quality against average end-to-end latency for every model. The light-colored bar shows the retrieval score from a single search step while the dark-colored bar shows the result from multi-step search, measured against the left axis (higher is better); the hatched bar shows end-to-end latency against the right axis (lower is better). Adaptive Instructed-Retriever matches the performance of the leading third-party and open-source models while answering in just 5.8 seconds, more than 2x faster than either Claude Sonnet 5, DeepSeek-V4-Flash, or GPT-5.6 Luna.

Online Reinforcement Learning for Customized Quality-Latency Tradeoffs

Training Adaptive Instructed-Retriever allows us to choose the quality-latency trade-off by adjusting the magnitude of the step penalty used during ORL. Therefore, we can train a family of checkpoints that powers Adaptive Instructed-Retriever, and pick whichever one fits the production workload.

Figure 3 shows how, by sweeping the magnitude of the penalty during ORL training, we obtain a family of checkpoints, each landing at a different point on the quality-latency plane — score on the y-axis (higher is better) against end-to-end latency on the x-axis (plotted so that faster is to the right). The red curve connects these operating points into a frontier: a lighter step penalty lets the model take more steps and reach higher scores, while a heavier penalty reduces its latency.

The full frontier of the trained Adaptive Instructed-Retriever models outperforms the alternatives. Compared with the untrained Instructed-Retriever base model, every checkpoint delivers higher quality at similar or lower latency, suggesting that the gains come from learning when to search—and when not to. At the top of the frontier, our model reaches scores comparable to other leading models while being more than 2x faster. Because the trained frontier includes checkpoints with different tradeoffs, we can choose the one that best fits each workload and budget – favoring speed for interactive use or quality for harder offline retrieval.

Adaptive Instructed-Retriever Formulates More Efficient and Targeted Search

We show some examples comparing the search policies of the alternatives, to those of our trained Adaptive Instructed-Retriever model. These examples illustrate how Adaptive Instructed-Retriever searches efficiently on straightforward questions and, on harder questions, can use its remaining search budget more effectively.

Verify a negative answer without exhaustive search

Adaptive Instructed-Retriever reaches the same reward one step earlier than Sonnet and two steps earlier than Luna.

Question: Does Company X explicitly report restructuring costs as an FY2022 income-statement line item?

Use the next round to change strategy

Adaptive Instructed-Retriever learns to adjust search strategy to improve recall: it moves from broad discovery to targeted account searches. It achieves the highest reward while tying Sonnet for the fewest steps.

Question: Which customers are using or have considered using LiteLLM Proxy?

Conclusion

Adaptive Instructed-Retriever extends the previously published Instructed-Retriever-1 from parallel single-step retrieval to adaptive multi-step search, enabling the model to spend additional search steps on complex queries where iterative reasoning and evidence gathering can substantially improve retrieval quality, while returning early on simpler queries to minimize latency. This adaptive approach delivers significantly stronger retrieval performance than single-step search while achieving performance comparable to leading third-party models at 2x lower latency. By varying the step penalty during Online Reinforcement Learning, we can explicitly optimize the trade-off between retrieval quality and inference cost, producing checkpoints at different points along the quality–latency frontier and selecting the operating point that best fits a production workload.

Our contribution provides a practical retrieval building block for data agents at Databricks operating over large, constantly changing workspaces. Experiences such as Genie Code, Genie One and Genie Agents must find the right tables, notebooks, dashboards, and documents without spending excessive time and effort on deep exploration. Adaptive Instructed-Retriever offers a bounded policy for this setting: it is fast on common lookups, while being thorough on difficult discovery tasks, and controllable within a product latency budget. More broadly, these results illustrate the competitiveness of small specialized models\ even for tasks that require multi-step reasoning. Databricks AIR allows our customers to tailor specialized models for their own domains and performance requirements.