Spring AI ChatClient Explained: prompt → model → response, and reading token usage

July 11, 2026 · Spring AI for Enterprise Java (part 3)

▶ Watch on YouTube & subscribe to The Stack Underflow

Your team ships a feature that calls an LLM. It works in the demo. Three weeks later finance asks how much the feature costs per day, and nobody can answer — because the code that calls the model only ever asked for the text back. No model name logged. No token count captured. The only way to find out is to reconstruct usage from a provider dashboard that has no idea which of your seventeen endpoints made which call.

This is not a monitoring problem you bolt on later. It’s a problem you solve — or fail to solve — the moment you write the line that calls the model. In Spring AI, that line runs through exactly one object: ChatClient. Everything the rest of this series builds — RAG, tool calling, memory, evaluation, observability — sits on top of the same fluent chain you’re about to learn, and that chain already hands you the cost data if you know where to look.

If you’ve used Spring’s RestClient or WebClient, the shape will feel immediately familiar: build a request, chain some options, execute, unwrap a response. Spring AI didn’t invent a new mental model for talking to LLMs — it reused one you already trust.

The one-sentence version: ChatClient is a fluent request builder — prompt() → system()/user() → call() → content()/chatResponse() — and the same call that gets you the model’s answer also hands back the model name and exact token usage, for free, if you ask for the response envelope instead of just the text.

The fluent chain, whole

Every ChatClient call starts the same way: get a client, open a prompt, ask a question, send it, read the answer.

ChatClient chatClient = ChatClient.create(chatModel);

String answer = chatClient.prompt()
    .user("Why is the sky blue?")
    .call()
    .content();

Five links, left to right: create() builds the client around whatever ChatModel Spring Boot autowired for you, .prompt() opens the request, .user(...) is the actual ask, .call() is the one line that leaves your process and talks to the model, and .content() unwraps the answer as a plain String.

In a real Spring Boot service you won’t call ChatClient.create(...) yourself most of the time — you’ll inject a ChatClient.Builder that autoconfiguration already wired up:

@Service
public class AnswerService {

    private final ChatClient chatClient;

    public AnswerService(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    public String answer(String question) {
        return chatClient.prompt()
            .user(question)
            .call()
            .content();
    }
}

That’s the whole surface area for a first call. Everything from here is about what you can insert into that chain, and what you can pull back out of it.

Crossing the seam

.call() is the one moment in this chain that matters architecturally. Everything before it — prompt(), system(), user() — is your code, building a request object. .call() is where that request crosses a boundary into the model provider, and Spring AI is the abstraction that sits on that boundary.

┌─────────────────────────────┐
│   ▸ YOUR CODE (cyan)         │   .prompt().system().user()
│                               │
│  - - - - SPRING AI - - - -   │   .call()  ← the seam
│                               │
│   ◆ MODEL (clay)              │   OpenAI / Anthropic / Ollama / ...
└─────────────────────────────┘

Nothing about .prompt(), .system(), or .user() knows or cares which provider answers the call. That’s deliberate: it’s what lets you point the exact same ChatClient chain at a different model with zero rewrite (the subject of the next episode). For now, just notice that .call() is the line worth pausing on — it’s the only part of this chain that leaves your JVM.

System vs. user messages

A prompt is rarely just one string. Most real calls carry two: a system message that steers behavior — the role, the tone, the guardrails — and a user message that’s the actual question. Add .system(...) before .user(...) and nothing else about the chain changes:

String answer = chatClient.prompt()
    .system("You are a helpful assistant that responds like a pirate.")
    .user("Why is the sky blue?")
    .call()
    .content();

This matters more than it looks. In production, the system message is where you encode the constraints that make an LLM feature safe to ship — not the user message, which you generally don’t fully control.

content() vs. chatResponse()

.call() returns a ChatClient.CallResponseSpec, and from there you have a choice. .content() gives you just the text. .chatResponse() gives you the whole envelope:

// Just the words — fine for prototyping
String text = chatClient.prompt()
    .user("Why is the sky blue?")
    .call()
    .content();

// The whole envelope — what you want in production
ChatResponse response = chatClient.prompt()
    .user("Why is the sky blue?")
    .call()
    .chatResponse();

String sameText = response.getResult().getOutput().getText();

response.getResult() returns a Generation; .getOutput() returns the AssistantMessage; .getText() gives you the same string .content() would have handed you directly. So .chatResponse() is strictly more information for the same call — the only reason to reach for .content() is that you don’t need anything else yet.

Opening ChatResponseMetadata

The part the quick-start tutorials skip: ChatResponse also carries metadata about the call itself. Call .getMetadata() and you get a ChatResponseMetadata object with a model name and a usage report sitting right there.

ChatResponse response = chatClient.prompt()
    .user("Why is the sky blue?")
    .call()
    .chatResponse();

ChatResponseMetadata metadata = response.getMetadata();
String modelUsed = metadata.getModel();

Usage usage = metadata.getUsage();
Integer promptTokens = usage.getPromptTokens();
Integer completionTokens = usage.getCompletionTokens();
Integer totalTokens = usage.getTotalTokens();

Three numbers, zero extra code, zero extra API call. getModel() tells you which model actually answered — useful the moment you have more than one configured. getUsage() gives you getPromptTokens(), getCompletionTokens(), and getTotalTokens() — your cost meter, riding along on the same response you were already reading.

Wire that Usage object into a Micrometer counter or a log line and you have per-call cost tracking before you’ve shipped a single dedicated feature for it. That’s the difference between cost governance you designed in and cost governance you bolt on after finance asks the question you couldn’t answer.

How to apply this right now

1. Confirm your build target. Spring AI 2.0.0 needs Java 21 to build (Boot 4’s runtime floor is lower, but the build toolchain isn’t) and Spring Boot 4.0 / Spring Framework 7.0. Check your pom.xml/build.gradle toolchain before anything else.
2. Add the starter for your provider. For OpenAI: org.springframework.ai:spring-ai-starter-model-openai:2.0.0. Swap the suffix for anthropic, ollama, vertex-ai, or bedrock-converse for other providers — Spring Boot autoconfigures the ChatModel bean from there.
3. Set the model via the flattened property. In 2.0.0 the chat model property is spring.ai.openai.chat.model=gpt-4o (or your provider’s equivalent) — a single flat key, not a nested options block.
4. Inject ChatClient.Builder, not ChatClient. Build the client in your own @Service/@Component so you control where defaultSystem(...), advisors, and tools get attached later in this series.
5. Never call .content() when cost or governance matters. Default to .chatResponse() and read ChatResponseMetadata even if you discard the metadata for now — it’s one extra method call away from a cost dashboard.
6. Put your guardrails in .system(...), not .user(...). Treat the system message as the artifact your security review inspects.

Common misconceptions

“ChatClient is a chatbot SDK.” It’s a fluent HTTP-shaped request builder, structurally similar to RestClient. It has no concept of a UI, a conversation widget, or turn-taking beyond what you build on top of it with memory advisors (a later episode).

“You need extra code to track token usage.” You don’t. Usage.getPromptTokens(), .getCompletionTokens(), and .getTotalTokens() come back on every .chatResponse() call with no configuration. The only “extra code” is choosing .chatResponse() over .content().

“.content() and .chatResponse() hit the model differently, so .chatResponse() is more expensive.” No — .call() has already crossed the seam and gotten the full response back before either method runs. .content() and .chatResponse() just differ in how much of that already-returned response you unwrap. There’s no extra round trip.

“The system message is optional decoration.” It’s the primary place enterprise guardrail and persona policy lives. Skipping it doesn’t make your call simpler — it just means the model is operating with Spring AI’s or the provider’s defaults instead of yours.

Frequently asked questions

Do I have to call .system() before .user()? Yes — the fluent chain is order-sensitive by convention even though the underlying request is just a list of messages. Put .system(...) first so it reads the way the request will actually be interpreted: role and rules established, then the ask.

What’s the difference between ChatClient.create(chatModel) and injecting ChatClient.Builder? ChatClient.create(ChatModel) is a static factory for wiring things up manually or in a quick test. In a Spring Boot application, autoconfiguration already exposes a ChatClient.Builder bean pre-wired to your configured ChatModel — inject that builder and call .build() in your own service instead of calling create() yourself.

Is Usage the same across every model provider? The three accessors covered here — getPromptTokens(), getCompletionTokens(), getTotalTokens() — are the stable, provider-agnostic surface. Some providers expose additional fields on the Usage/DefaultUsage implementation (for example, prompt-cache token counts), but treat those as provider-specific extras, not something every model guarantees.

Why does .getResult().getOutput().getText() look so nested? ChatResponse can technically hold multiple Generation results (multi-candidate responses); getResult() returns the primary one. Generation.getOutput() is the AssistantMessage, and .getText() reads the message body. .content() exists specifically so you don’t have to write this chain out by hand for the common case.

Does reading ChatResponseMetadata cost extra tokens or a second API call? No. The metadata is part of the same HTTP response your model provider already sent back for the call you made. Reading it costs you nothing beyond calling .chatResponse() instead of .content().

Is this chain still correct if my team is migrating from Spring AI 1.0.3? Yes — ChatClient.create(), .prompt(), .system(), .user(), .call(), .content(), .chatResponse(), and the getResult().getOutput().getText() chain are identical in shape between 1.0.3 and 2.0.0. It’s one of the few surfaces in this series with no breaking change to flag.

Where this fits in the series

This is Episode 3 of Spring AI for Enterprise Java. The previous episode covered why Spring Boot teams adopt Spring AI — the pattern-reuse case for choosing it over hand-rolled provider SDKs. The next episode takes this exact ChatClient chain and swaps the model provider underneath it with zero rewrite: model portability in Spring AI.

Runnable code for this episode lives in the 03-chatclient-explained module of the spring-ai-enterprise-examples repo.

Spring AI moves fast — this series is pinned to 2.0.0; update notes live at thestackunderflow.com/spring-ai. If you want the video version of this walkthrough as soon as it’s live, subscribe to @TheStackUnderflow on YouTube.