A LangGraph agent with retrieval as a tool
On a frozen set of dishes, the agent's first pick was a grape the dish calls for in all 4 dishes it ran; searching the reviews with the dish text alone managed 3 of those 4, and 3 of all 10. Each pairing cost $1.25 at API prices and took 78 seconds, because the critique node rejected all 12 drafts it saw and used up both retries every time.
The agent reasons from the dish to a wine style, searches the Wine Enthusiast review
index through a LangChain StructuredTool with the budget as a filter,
recommends with citations, and sends itself back when its critique finds a problem.
Parse, strategize, retrieve, recommend, critique, with a conditional edge from critique back to strategize.
Candidates inside the $45 budget in the recorded run, because the budget is a filter on retrieval.
For the local retrieval step in a traced run, against 1.2 to 3.0 seconds for each LLM node.
On every recommended bottle, pointing at a retrieved review. The full candidate list prints with the answer.
eval/dishes.json). The agent ran on the first 4The problem
The sibling project, Wine Sommelier RAG, answers requests that already contain their search terms: "a bold red under $25" is close to a query. "Grilled ribeye with peppercorn sauce" says nothing about tannin or dark fruit, so a retriever has nothing to match on.
This agent reasons about the dish first, picks a target style from a short list of
pairing principles in src/pairing.py (acid cuts fat, tannin binds protein,
match weight, sweetness tames heat, regional dishes with regional wines), writes a search
query for that style, and only then calls retrieval. Afterwards it checks its own answer.
Course by course
Step through the recorded ribeye run from examples/sample_run.md. Each
node's width in the strip is its time in the traced ribeye run in the README (same
$45 budget, shorter wording of the dish): 9.0 seconds in all, of which the local search
took 0.3. The retrieve segment is drawn wider than its share so its label fits. On a phone the strip shows node numbers only.
The retrieve step ranks 40 candidates by 0.8 × cosine similarity + 0.2 × rating, with the rating scaled so 80 points is 0 and 100 is 1. On that blend a 95-point wine beats a 92-point one only if its similarity is no more than 0.0375 lower, so the rating breaks near-ties and does not override the match.
The answer
The agent printed the six candidates it chose from as well as its two picks, so the choice can be checked against the alternatives.
The scored run
eval/dishes.json lists 10 dishes, each with a budget and the grapes that
classic pairing rules accept for it (Cabernet, Syrah or Malbec for the ribeye; Muscadet,
Chablis, Champagne or Riesling for oysters). It was committed before anything ran. A
pick counts when its Wine Enthusiast variety is on the list. The baseline is the agent's
own search tool given the dish text as the query, no reasoning step, top hit taken.
python -m eval.run_eval runs both.
Accepted grapes are in the accent color. The baseline goes wrong where the dish words sound like a wine: "lamb" pulls up Lambrusco, twice. The key is strict, and two of its misses are arguable: the Torcolato is a sweet wine, which suits chocolate, and Riesling with roast chicken is a common choice. The agent ran on 4 dishes only, because each pairing cost about $1.25 (below), so 4 of 4 against 3 of 4 is a small sample.
Error analysis
The critique node passes a pairing only when its LLM review lists no issues. In the 4
scored runs it listed issues in all 12 rounds, so every run went parse, then three rounds
of strategize, retrieve, recommend and critique, and stopped at the attempt limit with
the issues still open. Grouping the first issue of each round, and every issue in each
run's last critique (eval/critique_issues.json, coded by hand):
The age complaint can't be fixed by retrying. No vintage in the
review file is newer than 2017, the newest of the 24 retrieved bottles was 2016, and after the first rejection
strategize starts asking for "a recent vintage", which sends the search after
wines the index does not have. The critique prompt has no notion that the index is a
snapshot, and no rule says which issues are worth a retry. The July recorded run above
passed on its first critique; on 2026-11-05 none did.
Cost and latency
Measured on 2026-11-05 on an Apple M4 by running the agent through
eval/run_eval.py, which adds --output-format json to the
agent's own Claude CLI calls and keeps the reported cost and duration. The cost is the
API-equivalent figure the CLI reports; it runs on a Claude subscription, so nothing was
billed. The CLI's default model was Opus 5.5.
eval/calls.csv). Each call read about 13,300 cached tokens and wrote 12,100 to 15,300 more: the CLI's own system prompt and context. The agent's prompt and the reply are a few hundred tokenssrc/llm.py, which sends only the agent's prompt; there is no API key here to measure itTwo things set the bill: the retry loop that never exits early (10 calls where 4 would do), and the CLI's context on every call. A pairing that passed its first critique would make 4 calls.
Observability
Because retrieve is a named LangChain tool, it shows up in a trace as a tool
call. --trace times each node, prints a trace tree, and saves the run as JSON
under traces/. The traced ribeye run behind the strip above, as recorded in
the README with its bullet glyphs dropped:
AGENT TRACE (node · time · detail) parse 1.2s ribeye · rich body strategize 2.3s style → full-bodied tannic red retrieve 0.3s 6 candidate wines (tool: search_wines) recommend 3.0s drafted pairing with citations critique 2.2s sound
Nearly all the latency is the four LLM calls. When critique sends the agent back to strategize, the second pass appears in the same trace.
Limitations
The scored run covers 4 dishes for the agent and 10 for the baseline, against an answer key of accepted grapes that I wrote from standard pairing rules. Pairing has few single right answers, so a grape match is a floor, not a judgment of the pairing; no sommelier or blind tasting checked the picks. The walkthrough above is one recorded run from July, and its timings come from a separate traced run of a similar ribeye request.
assets/make_trace_figure.py can draw a saved trace as a timeline, but no
rendered timeline from a live run is in the repository. The pairing principles are a
short list, so fortified wines, very hot food and dessert courses get thinner treatment.
Retrieval filters on price only; a color preference reaches the agent through the query
and the critique, with no hard filter behind it. The index holds the 30,000
highest-scored reviews by default.
How it was built
StateGraph with five nodes and a conditional retry edge (src/graph.py)src/tools.py wraps retrieval as a LangChain StructuredTool with a typed schema--all for the full 129,971ANTHROPIC_API_KEY switches to the APIsrc/trace.py: per-node timing, a tree view, JSON output