Local vision, grounded, free

Cellar Scanner

Photograph a wine label — get a grounded profile and pairings, cited to real reviews from a 30,000-bottle index. Plus a locally fine-tuned model that guesses the grape from a blind tasting note, no label needed.

By Jeremy Lee

Why

Ask a chatbot to identify a wine from a photo and it will happily invent a producer, a score, a tasting note it never read. Cellar Scanner reads the label for real — local OCR + Claude, no vision API key — retrieves genuinely similar wines, and refuses to say anything the retrieved reviews don't support.

📷

No vision API key

Local OCR (tesseract) + Claude reads the label — refuses non-wine photos, flags low-confidence reads.

🍷

30k real reviews

Stratified across every represented grape variety, not just the highest-rated bottles.

🎯

Grounded & cited

Every claim checked against retrieved reviews before it ships. Ungrounded claims are dropped, not shown.

🧠

Locally fine-tuned

A LoRA-tuned model guesses grape variety from a tasting note alone — runs on a laptop, $0 marginal cost.

Blind Taste Mode

Paste or speak a tasting note with the label hidden. A Qwen2.5-1.5B model — fine-tuned locally on 41,389 masked Wine Enthusiast notes across 40 varieties — guesses the grape before the reveal. Benchmarked honestly against the untrained base model and Claude:

System	Top-1	Top-3	Cost
Base model (no fine-tune)	8.7%	14.3%	$0
+ LoRA fine-tune	27.7%	27.7%	$0
Claude (teacher, zero-shot)	23.0%	30.5%	Max subscription

🏆 The fine-tuned local model's Top-1 accuracy beats Claude's zero-shot Top-1 — a specialized 1.5B model trained on this exact task, outperforming a generalist frontier model prompted cold, at $0 marginal cost and 7x lower latency.

📝 The grape-name is regex-masked out of every training and reference note first — otherwise many reviews just name the grape and the task collapses to string matching, not blind tasting. The masking rate is measured and reported, not assumed.

How it works

Photograph the label — local OCR extracts the text, Claude structures it into a wine card and flags anything illegible for you to correct.
The card queries a 30k-review Chroma index for genuinely similar wines.
Claude writes a grounded profile and 2-3 pairings, citing the exact reviews it drew from — nothing invented.
Re-scan the same bottle later and get an instant cached answer — no repeat generation.

🔒 Everything runs locally except one claude -p call per action — no metered API, no vision API key. See the full writeup for the honest engineering story, including three real bugs found and fixed along the way.

View on GitHub Read the writeup