I don’t trust the model

Dima · October 2026

I use a language model to write product listings from an image: a title, thirteen tags, a description and alt text. It writes well. I still treat everything it returns the way I would treat a form field arriving from the internet: as input to be checked before it is used.

This is what surrounds the model. Most of it is not artificial intelligence.

1. The limits are data, not sentences in a prompt

Etsy accepts titles of at most 140 characters and thirteen tags of up to 20 characters each, with no repeats. The easy thing is to write that in the prompt and hope.

Instead, the limits live in a block of data at the top of a document:

verified_date: "2026-07-11"
title:
  max_chars: 140
  forbidden_chars: ["$", "^", "`", ".", "°"]
  max_allcaps_words: 3
tags:
  count: 13
  max_chars_per_tag: 20
  allow_duplicates: false
alt_text:
  max_chars: 250

That block is the single source of truth. The code contains no hand-written numbers: it reads the block on every request. The same block is given to the model and to the validator, so both work from the same figures. If a limit changes, one file is edited and nothing else needs touching.

2. The limits expire

A platform’s rules change without notice, and what the model remembers of them is the least reliable source there is.

So the block carries verified_date. After 30 days the application warns that the limits are stale. There is an action to verify them again: the model searches the official sources and proposes the changes as a diff against the block. It is never applied on its own; I confirm it.

In the document, every limit has its source and the date it was read. What I could not confirm is marked as unverified.

3. The server validates, and separates errors from warnings

The model answers in JSON. The server parses it and runs it through a validator with nothing intelligent in it: it counts characters and compares.

There are two levels:

Errors: what the platform would reject. A title that is too long, a forbidden character, anything other than exactly thirteen tags, a tag over 20 characters, repeated tags, HTML in the description.
Warnings: my own style rules. A title outside 80 to 120 characters, a phrase I do not want, alt text that reads like a list of keywords. They do not block; they are shown.

The difference matters. An error means the result cannot be used. A warning means I want to look at it.

The validator has 20 tests of its own. It is the part of the system that has to be exact, so it is the part that is tested.

Fig. 1 — Thirteen tags through the validator, twice. The tags are an example written for this figure; the three checks are the real ones and run in the page: exactly 13 tags, at most 20 characters each, no duplicates. The first attempt fails two of them, the failures go back to the model, and the second attempt passes.

4. One retry, with the specific error

If validation fails, the server asks once more. It does not say “try again”: it sends back the previous answer and the exact list of failures.

for attempt in (1, 2):
    raw = await ask_model(prompt)
    listing = parse(raw)
    result = validate(listing, limits)
    if result["ok"]:
        break
    if attempt == 1:
        prompt = base_prompt + previous(raw) + what_to_fix(result["errors"])

Why only one? Because if the model fails twice with the error in front of it, a third attempt is a coin toss, and the cause is usually somewhere else: rules that contradict each other. The message I see in that case sends me to review the rules file, not to press the button again.

After the second failure the result is returned anyway, with the invalid fields marked in red. It is not hidden or silently repaired. I decide.

5. What must be identical is not written by the model

Every description carries a five-line block about the paper and the printing. It has to be the same in every listing, letter for letter.

Asking a model to copy a text verbatim works almost every time. That “almost” is the problem.

So the model does not copy it: it writes a marker, [[PRACTICAL_BLOCK]], and the server replaces it with the real text. The block comes out byte-identical because code puts it there. The general rule: ask the model to point at where something goes, not to reproduce it.

6. A rule that stops generation

Before writing anything, the model assesses whether the image is, or looks like, a reproduction of a work still under copyright: a character, a brand, a living artist. If it looks like one, it returns the reason and that is the end of it. No text is generated, there is no retry and the card shows in red.

It is the only rule that comes before all the others, because it is the only one whose failure cannot be fixed by editing a title: on a marketplace, an intellectual-property complaint can close the shop.

It has an obvious limit: that judgement is also a model output, and it can be wrong. That is why the rule is conservative (looking like one is enough) and why I see the image anyway. It reduces the risk; it does not remove it.

7. The small things

Print files weigh more than 100 MB. The model receives a reduced JPEG.
The only tool it can use is reading that file. Nothing else.
Every request has a time limit. If it runs out, I see a clear message, never a stack trace.
The queue processes listings one at a time, and the tokens used are recorded for each.

8. What I do not check

The validator checks form, not truth. It does not know whether the description says what is in the image, or whether the title is what a buyer would type. I read that myself.

That is why what comes out of the system is a draft with green and red marks, not a published listing. Publishing is a manual step.

9. The pattern

It applies to any job where a model’s output ends up somewhere with rules:

Write the constraints as data, with a date and a source.
Give the same data to the model and to the validator.
Validate in code everything that can be counted.
Retry once, with the exact error.
If it fails again, make it visible.
Whatever has to be exact, let code do it.