A 1.5B model fine-tuned on a laptop sorts receipt items at 90.5%. Claude zero-shot: 80.0%.

Error analysis

Every miss, sorted into five kinds

All 65 misses (21 for the LoRA, 44 for Claude) from the saved predictions, grouped by a rule on the true and predicted labels. Four of the five groups are the same disputed boundaries, and both systems fall into all four; they disagree on which side of each boundary is right. The training data explains why. Of the 4,309 training rows, line items that look like gasoline are labeled fuel 370 times, transport 231 times and travel 67 times. Items from hardware-store merchants are household 118 times and other 109 times. A model can learn the majority habit and still miss every row that went the other way.

Hardware is where the two systems split in opposite directions. The LoRA sends 7 household items (paint, caulk, hinges, trash bags) to other, following the half of its training data that did the same; Claude sends 20 other items (cables, screws, paint, laundromat time) to household. On 5 of the LoRA's 7 gasoline misses, Claude gave the same wrong answer. The kind of error a fine-tune removes here is a mismatch with the label scheme; it does not remove ambiguity the scheme itself contains.

Cost and latency

Loaded once, the adapter answers in 0.19 s at no per-call cost

The benchmark timed a fresh mlx_lm.generate process per item, so its 4.5 s per call is mostly model loading. A service loads the model once. With Qwen2.5-1.5B-Instruct (4-bit) and the adapter held in memory on an Apple M4 with 16 GB, the first 100 test items took a median 0.19 s each, 0.225 s at p95, about 5.2 items per second one at a time and 12.8 per second in batches of 16. All 100 answers match the benchmark's stored LoRA outputs. Loading the model and adapter takes 1.0 s.

For Claude I sent the first 25 of the same items through claude -p with the rerun's exact prompt and flags. Median 3.6 s, p95 9.0 s, and every answer matched the 2026-10-28 rerun. The CLI reports an API-equivalent cost of $0.0030 per item. On a Max plan nothing is billed per call; at API prices 1,000 items would cost $3.03. Each call sends about 745 input tokens, almost all of them written to the prompt cache, and gets back about 6.

Design

The model phrases numbers; code computes them

Two language models touch a receipt, and neither does arithmetic. Each step below is labeled with what runs it.

1. 01codetesseract reads the photo locally.
2. 02claude -pOne call per receipt rejects photos that are not receipts and fills a Pydantic Receipt. The raw OCR text is then discarded.
3. 03codeprivacy.sweep_receipt masks card-like digit runs to the last four and replaces emails and phone numbers.
4. 04codereconcile adds line items and tax and compares the sum with the printed total. A mismatch is flagged for a one-tap correction, never adjusted to agree.
5. 05LoRA 1.5BEach line item gets one of 12 categories.
6. Later, a spending question
7. 06pandasanalysis.answer_query computes the figure from the stored receipts.
8. 07claude -pWrites one sentence around the computed numbers.
9. 08codeassert_numbers_present checks every number survived to within a cent. If one was dropped or changed, the app shows a template sentence instead.

Privacy

Four layers of PII handling

The Receipt and LineItem schemas have no field for a cardholder name, card number, address or loyalty number, so extraction never asks for them. The raw OCR text is discarded once a validated Receipt exists. The regex sweep in step 03 catches what slips through and reports how many redactions it made. On a hosted deployment, receipts stay in memory for the session; privacy.is_space_environment() is the one check both paths use before writing anything to disk.