How Swatcher builds a paint palette

Swatcher has had palette generation from the start. Describe a room, get some colours back, matched to real paints. With the rebuild I took the chance to turn it inside out.

The old way was the obvious one. Ask a model for some hex codes, then find the nearest paint to each. It mostly worked. But nearest can be a long way off, and the model’s reasoning was written for a colour that doesn’t exist. Nobody can walk into a shop and buy #C8B8A2.

So now there’s one rule: every colour Swatcher suggests is a real paint. The models never get to make one up. One reads what you wrote, the catalogue does the colour work, and another picks from a short list. That’s really it. Here’s how it hangs together.

Two model calls, and neither one is ever the source of a colour: the reader only quotes, and the composer can only choose ids from the shortlists.

#Reading the Brief

The first step is boring on purpose. A small, quick model (Claude Haiku) reads the brief and pulls out what it names. It’s told plainly that it doesn’t choose colours.

It works out what kind of ask it is: a whole room, one thing like a front door, or just the name of a paint typed into the wrong box. Then it picks out anchors, meaning anything with a colour attached. A paint you want to keep. Oak floors. A red door. “Nothing like Hague Blue.” It also notes which way the room faces, whether you want something a bit lighter than a paint you know, and whether you’ve stuck to one brand.

What it never does is guess what colour a paint is. It doesn’t know Colourtrend’s Valkriss. The catalogue does.

Each name goes through the same fuzzy matching as the search box, and it only counts as found if one paint clearly wins. If two paints share a name, you get asked which one you meant. If the whole brief was just a paint’s name, you get sent to that paint’s page instead of a palette of one.

Photos go the same way. A second reader turns the photo into the same kind of brief, but the colours come from measuring the image’s pixels, not from the model eyeballing it. On a test set of 17 photos that landed closer to the original, and two runs on the same photo agreed about three times as often.

#Shortlists

This is the bit with no AI in it at all, and it’s where most of the colour work happens. Plain PHP, about 100ms, same answer every time.

Everything fans out from one starting colour. That’s a paint you said you want, or something already in the room, or the mood the reader picked up. From there it uses the same code behind “what goes with this” on every paint page. That sorts real paints by the job they’d do in a room: the walls, a quieter colour alongside, a white for the woodwork, and an accent.

I went with jobs rather than colour theory because that’s the question people actually ask. Nobody wants the split-complementary. They want to know which white goes on the skirting.

Each job gets up to eight paints, then the brief gets applied. A brand limit cuts the catalogue down before anything else, so “only Dulux” can’t leak. Something you’ve ruled out takes its lookalikes from every other brand with it. A paint you already own goes to the top of every list. “A bit lighter” means the same hue a step lighter, and if a range doesn’t have one, it comes back empty rather than handing you something darker.

One-colour asks get their own shape: the nearest paints, a step lighter and a step darker. An earlier version reached for accents there, and a brown for the hall came back as two greens. Fair enough on paper. Not what was asked for.

#Picking

Now a model gets to have some taste. The composer sees the brief, the fixed colours with their real values, and each job’s shortlist. It picks one paint per job, names the palette, and writes a line about each choice.

This is the part a lookup can’t do. A north-facing room wants warmer, brighter walls. A red front door should be answered by the accent, not ignored. “Something warm” should get a warm accent, not a cool contrast.

The trick is in the schema. The only paint IDs it will accept are the ones on this brief’s shortlists, as an enum:

'choices' => $schema->array()
    ->items($schema->object(fn ($schema) => [
        'role' => $schema->string()->enum(array_column(PaletteRole::cases(), 'value'))->required(),
        'paint' => $schema->string()->enum($this->paintIds)->required(),
        'reason' => $schema->string()->required(),
    ]))

It runs in strict mode, so there’s no other way for it to name a paint. That’s not asking nicely in the prompt. It just can’t produce an ID that isn’t on the list.

I still check afterwards, since a real ID can land in the wrong job. Anything picked from outside its own list gets dropped, nothing is used twice, and if it forgets a paint you said you own, that goes back in.

The composer started on Claude Sonnet and now runs on GPT-6 Sol with reasoning off. On 18 real briefs side by side it picked as well, in about 4.5 seconds against 7.4. Thinking didn’t earn its keep here. Once the shortlists had done the narrowing, it didn’t change the picks, only the wait.

#The Wait

The first version of this took about 20 seconds. That’s a long time to stare at a spinner, and the usual fix is a list of made-up stages on a timer. “Choosing your accent…” People notice.

The shortlists are ready well before the composer is done, so the form asks for them first. While you wait, you see what it heard: the light, a brand, a paint you named with its colour beside it. Under that is a row of real paints for each job, one lifting at a time. The status only changes when something actually finishes.

Then I made it faster. Most of the time was the composer thinking. Turning that down got it to about 14 seconds, and the move to Sol brought it under 10. In my last test run, 19 palettes took between 6.5 and 11 seconds, about 9 on average, and photos took about the same as written briefs. You’re looking at shortlists after the first couple of seconds, which helps.

#The Cost

I ran 19 real generations to see what one actually costs: 12 written briefs and 7 photos, with the token counts logged on every call.

So somewhere between half a cent and a cent. A one-colour ask is cheaper because there are only two jobs to fill. A photo costs more because its colours put more paints on the shortlists, so the composer reads more.

Almost all of that is the composer: over 90% of every bill. The reader on Haiku costs about $0.0003 for a written brief and $0.0006 for a photo. Sol is priced like Sonnet ($2 per million tokens in, $10 out), so the switch made it faster without making it dearer.

Keeping the catalogue in charge helps here too. The composer sees eight paints per job, not the thirteen thousand or so in the catalogue, so a prompt stays around 2,000 tokens.

#What I Learned

Let the model read and judge, and let the data handle the facts. It’s good at understanding “the door is red and I hate beige”. It’s bad at knowing what colour a specific paint is.

Make the wrong answer impossible, then check anyway. “Only choose from this list” in a prompt is a request. An enum is a wall.

Never hand a model an empty list. This one caught me out. A very pale colour has hardly any accents, and “no orange” next to a blue can empty the accent list by itself. Asked to fill a job with nothing in it, the composer made up an ID. Now an empty job gets honest stand-ins, or it’s left out.

Try things side by side on real briefs before swapping anything. Smaller models were quicker as the reader, and quietly worse. They decided almost everything should be a bit lighter.

Give it a spin on Swatcher and let me know what you think.