Spring AI Production Blueprint: ChatClient + RAG + Tools + Security + Observability in One Service

July 11, 2026 · Spring AI for Enterprise Java (part 12)

▶ Watch on YouTube & subscribe to The Stack Underflow

Most Spring AI content — including the first eleven episodes of this series — teaches one seam at a time. A ChatClient in a test method. A QuestionAnswerAdvisor wired against an in-memory vector store in a throwaway main(). A @Tool method called from a unit test with no controller in sight. Each of those pieces is genuinely useful to learn in isolation, and each of them lies to you a little: production code doesn’t get to isolate anything.

A real enterprise service has a controller that real clients hit, a service class that has to own the ChatClient for the lifetime of the application, an advisor chain where the order of operations changes behavior, a model provider that has to be swappable when procurement changes its mind, secrets that can never be a literal string in a .java file, and an evaluation step that runs before a response is trusted enough to ship. None of the isolated demos show you what happens when all of that has to coexist in one request.

This tutorial is the assembly instructions. It takes every seam from this series — ChatClient (Episode 3), RAG (Episode 6), tool calling (Episode 7), advisors and observability (Episode 9), evaluation (Episode 10), and security (Episode 11) — and wires them into one @Service and one @RestController, verified against Spring AI 2.0.0 GA.

The one-sentence version: every cross-cutting concern — logging, memory, RAG, tools — is an advisor in one ordered chain built by ChatClient.Builder, not an if-statement hand-rolled inside your service method.

The shape of a production Spring AI service

The architecture is a layered stack, and the rule for every layer is the same: keep your code thin, and let Spring AI own the orchestration.

CLIENT
  │  HTTP POST /api/assistant/ask
  ▼
PolicyAssistantController   (▸ your code — one line: delegate)
  │
  ▼
PolicyAssistantService      (▸ your code — owns the ChatClient)
  │
  ▼
┌─────────────── ADVISOR CHAIN (ordered) ───────────────┐
│  1. SimpleLoggerAdvisor    (observability)             │
│  2. MessageChatMemoryAdvisor (conversation state)       │
│  3. QuestionAnswerAdvisor  (RAG — grounds the answer)   │
│  4. tools (@Tool methods — live facts)                  │
└─────────────────────────────────────────────────────┘
  │
  ▼
◆ MODEL   (OpenAI / Anthropic / Ollama — swappable underneath)

The controller’s entire job is delegation:

@RestController
@RequestMapping("/api/assistant")
public class PolicyAssistantController {

    private final PolicyAssistantService service;

    public PolicyAssistantController(PolicyAssistantService service) {
        this.service = service;
    }

    @PostMapping("/ask")
    public String ask(@RequestBody String question) {
        return service.ask(question);
    }
}

No prompt construction, no advisor logic, no RAG here. The controller doesn’t even know Spring AI exists. That’s the point — if you can’t describe your controller’s job in one sentence, you’ve let the AI seam leak upward.

The service owns the ChatClient — and the advisor chain is built once

The PolicyAssistantService is where the real seam lives. One field, one builder call, assembled at construction time:

@Service
public class PolicyAssistantService {

    private final ChatClient chatClient;

    public PolicyAssistantService(ChatModel chatModel,
                                   ChatMemory chatMemory,
                                   VectorStore vectorStore) {
        this.chatClient = ChatClient.builder(chatModel)
            .defaultAdvisors(
                new SimpleLoggerAdvisor(),
                MessageChatMemoryAdvisor.builder(chatMemory).build(),
                QuestionAnswerAdvisor.builder(vectorStore).build()
            )
            .defaultTools(new PolicyLookupTools())
            .build();
    }

    public String ask(String question) {
        return chatClient.prompt()
            .user(question)
            .call()
            .content();
    }
}

ChatClient.Builder.defaultAdvisors(...) and .defaultTools(Object...) do the assembly work before the client is ever used. defaultTools accepts a @Tool-annotated POJO directly — you don’t have to wrap PolicyLookupTools in a ToolCallback yourself, Spring AI reflects the annotations for you.

The order of defaultAdvisors(...) matters, and it isn’t cosmetic: each advisor wraps the request on the way in and the response on the way out, like middleware. Logging first means it sees the raw request and the final response — nothing upstream of it is hidden. Memory before RAG means the retrieved context gets folded into a conversation that already has history. Get the order backward and your logs miss context, or your RAG grounding gets stale memory instead of the live retrieval.

RAG: grounding answers in your own documents

QuestionAnswerAdvisor is what keeps the model from guessing. It intercepts the user’s question, retrieves relevant chunks from a VectorStore, and stuffs them into the prompt before the model ever sees it:

Advisor ragAdvisor = QuestionAnswerAdvisor.builder(vectorStore).build();

If you’re following older Spring AI course material, note this carefully: Spring AI 1.0.3-era code constructed this advisor directly — new QuestionAnswerAdvisor(vectorStore). In 2.0.0, the constructor path is gone from the documented API in favor of the static factory, QuestionAnswerAdvisor.builder(vectorStore).build(). Same behavior, different construction — but code copied from an older tutorial will not compile against 2.0.0 without this change.

For anything past a single-advisor RAG pipeline — multiple query transformers, a document joiner, post-processing before the answer is generated — Spring AI 2.0.0 also ships RetrievalAugmentationAdvisor, a newer, more modular advisor that composes a documentRetriever, documentJoiner, and queryAugmenter as separate beans instead of one opaque advisor. QuestionAnswerAdvisor is still the right starting point for straightforward retrieval; reach for RetrievalAugmentationAdvisor when you need to compose the retrieval pipeline itself.

Tool calling: when the model needs a live fact

RAG grounds the model in static documents. Tools let the model reach into your live system — the important distinction being that your code decides what’s allowed, the model only requests:

@Component
public class PolicyLookupTools {

    @Tool(description = "Look up the current status of an insurance policy by ID")
    public String getPolicyStatus(String policyId) {
        // your service call — the model never touches this directly
        return policyRepository.findStatus(policyId);
    }
}

When the model decides it needs live data mid-conversation, it emits a tool call; Spring AI executes your annotated Java method, and the result flows back into the same conversation before the final answer is generated. The model never gets network access, database credentials, or anything else — it gets exactly the surface area your @Tool method exposes.

Externalized config and observability: the two cross-cutting rails

Security and observability don’t sit inline in the request path — they wrap the whole service as configuration and infrastructure, not as advisor logic you write by hand.

Secrets are externalized through environment variables, never hardcoded:

# application.properties — correct, current in 2.0.0
spring.ai.openai.api-key=${OPENAI_API_KEY}

The nested, options-level override some older snippets use — setting the key under spring.ai.openai.chat.options.api-key — is @Deprecated(since = "2.0.0", forRemoval = true). The top-level spring.ai.openai.api-key is the one supported path forward; if you see the nested form in a tutorial or a coworker’s code, flag it for cleanup.

Observability is a property switch, not a manual logging call:

ChatObservationProperties (bound under spring.ai.chat.observations) feeds Micrometer, so every call through the advisor chain becomes a traced span with token counts and latency — without you writing a metrics client anywhere in your service.

The evaluation gate

Before a RAG-grounded answer ships, RelevancyEvaluator checks that it’s actually supported by what was retrieved, rather than trusting that the model behaved:

Evaluator relevancyEvaluator = RelevancyEvaluator.builder()
    .chatClientBuilder(chatClient.mutate())
    .build();

EvaluationResponse result = relevancyEvaluator.evaluate(
    new EvaluationRequest(question, retrievedDocuments, answer)
);

if (!result.isPass()) {
    // block, log, or fall back — do not ship an ungrounded answer
}

Treat this the way you’d treat a CI test gate, not a nice-to-have: an evaluator that never runs is a hope, not a guarantee.

How to apply this right now

1. Pin the platform first. Spring AI 2.0.0 requires Java 21 to build (Spring Boot 4.0’s runtime floor is Java 17, but the build toolchain needs 21) and Spring Boot 4.0 / Spring Framework 7.0. Get this wrong and nothing downstream matters.
2. Add the starter with the current coordinate. org.springframework.ai:spring-ai-starter-model-openai:2.0.0 — not the pre-1.0 spring-ai-openai-spring-boot-starter name you’ll still find in stale blog posts.
3. Keep the controller to one line. If your controller builds prompts, calls advisors, or touches the ChatClient directly, move that logic into the service.
4. Build the advisor chain once, in the service constructor, with ChatClient.builder(chatModel).defaultAdvisors(...).defaultTools(...).build() — not scattered .advisors(...) calls per request unless you genuinely need per-request overrides.
5. Order advisors deliberately: logging first (so it sees everything), memory before RAG (so retrieval works with real conversation context), tools last in the chain definition (the model decides when to invoke them).
6. Externalize every secret via spring.ai.openai.api-key=${OPENAI_API_KEY} and profile-scoped configuration — never a literal key in source.
7. Turn on an evaluator before you turn on traffic. Wire RelevancyEvaluator (or FactCheckingEvaluator for stricter fact-grounding) as a gate, not an afterthought.

Common misconceptions

“The advisor chain order doesn’t matter — advisors are independent.” They aren’t. Each advisor wraps the request/response pair like middleware; logging placed after memory won’t see the raw incoming request, and RAG placed before memory won’t have conversation context to work with. Order is part of the contract.

“Tool calling means the model executes code.” It doesn’t. The model only requests a tool call with structured arguments. Your @Tool-annotated Java method executes, on your infrastructure, under your access controls. The model never gets a shell, a database connection, or the internet — only whatever your method returns.

“QuestionAnswerAdvisor is deprecated in favor of RetrievalAugmentationAdvisor.” It isn’t deprecated — the construction path changed (constructor to builder), but the advisor itself remains the straightforward, documented option for simple RAG. RetrievalAugmentationAdvisor is a separate, more modular tool for composing multi-step retrieval pipelines, not a replacement.

“Observability is something you add later, once you have a performance problem.” In a production Spring AI service, observability is a property flip (spring.ai.chat.observations.*) wired through the same advisor chain you already built. There’s no good reason to ship without it from day one — the cost of adding it later is re-architecting logging you should have had at launch.

Frequently asked questions

Do I need all four advisors — logging, memory, RAG, and tools — in every Spring AI service? No. This blueprint shows the full enterprise shape because this is the capstone episode, but a simple stateless Q&A endpoint might only need SimpleLoggerAdvisor. Add memory when you need multi-turn conversation, RAG when you need grounding in your own documents, and tools when the model needs live data. Each advisor is additive — start minimal and add advisors as requirements demand them.

Why does the controller stay so thin — isn’t that just extra boilerplate? The thin controller isn’t boilerplate, it’s a boundary. Keeping AI orchestration entirely inside the service means you can unit-test PolicyAssistantService without spinning up the web layer, and it means the ChatClient configuration lives in exactly one place instead of leaking into every endpoint that happens to touch the model.

What’s the actual difference between QuestionAnswerAdvisor and giving the model a tool that queries the same documents? QuestionAnswerAdvisor retrieves and injects context automatically, before the model generates a response — the model always sees the retrieved documents whether it “asks” for them or not. A tool is invoked at the model’s discretion mid-conversation. Use RAG for grounding that should always apply; use a tool when retrieval should only happen when the model decides it’s relevant.

Is spring.ai.chat.observations.log-prompt=false the safe default, or should I turn it on? Leave it false in production. Prompts frequently contain user-submitted data, and logging full prompt text by default is a straightforward way to leak PII into your log aggregator. Turn it on selectively — a scoped profile, a debugging session — never as the standing production default.

Does the evaluation gate run on every request, adding latency to the response path? That’s an architectural choice, not a Spring AI constraint. Some teams run RelevancyEvaluator synchronously before returning a response (adds latency, blocks bad answers immediately); others run it asynchronously against a sample of traffic as a quality monitor. For anything regulated or high-stakes, synchronous is worth the latency cost.

What happens when Spring AI ships a new version and this blueprint changes? It will change — this series is pinned to 2.0.0, and the QuestionAnswerAdvisor builder shape you see here already moved once from the 1.0.3 course code. Update notes for future Spring AI releases, including a fresh pass on this exact blueprint, live at thestackunderflow.com/spring-ai.

Where this fits in the series

This is the finale of Spring AI for Enterprise Java — Episode 12 of 12. It assembles everything from the series: ChatClient (Episode 3), provider swapping (Episode 4), RAG (Episode 6), tool calling (Episode 7), advisors and observability (Episode 9), evaluation (Episode 10), and secrets/security (Episode 11, Security & Compliance).

Runnable code for this episode — the full PolicyAssistantController, PolicyAssistantService, and PolicyLookupTools classes wired together — lives in the spring-ai-enterprise-examples repository, module 12-production-blueprint.

Spring AI moves fast, and this series will keep moving with it: subscribe to @TheStackUnderflow on YouTube for the update episode when the next Spring AI release lands, and browse all tutorials to catch any episode in this series you haven’t watched yet.