Why coding agents build the same website, and what a design skill changed

We build websites with coding agents in the loop, and we have the same complaint as everyone who does. Ask an agent for a site and you get the house style: the same layout set in the same few typefaces, with marketing copy to match. So we had Claude Code build a website-design skill that replaces the single prompt with a gated workflow, then measured it. The same three tasks went to the same model with and without the skill, one run each, against 27 written checks. This piece is the numbers, and the limits of what they show.

The headline is 24 of 27 checks with the skill against 10 without. The honest number is narrower. Twelve of the checks test the skill's own paperwork, which a plain model can only pass by accident. On the 15 checks about what the user receives, the skill scored 15 and the plain model scored 9, and no check passed only without the skill.

Why do coding agents build the same website every time?

Because an unguided model samples the centre of its training data, and both vendors say that centre is one look. Anthropic attributes the default (Inter, purple gradients on white, minimal animation) to distributional convergence: safe choices dominate the pages the model learned from. OpenAI describes the same failure as a fallback to high-frequency patterns when a prompt is underspecified. Neither vendor publishes a measurement of that distribution, so both are explanations rather than evidence.

The research around them predates the tools and points the same way.

The last row is the one you can check yourself. Add the two tiers and more than half of a sample of launch pages carried at least two of the patterns the scan looked for.

Why doesn't a list of banned fonts fix it?

Because the list moves the default instead of removing it. Anthropic writes that even with explicit instructions to avoid certain patterns, the model can fall back on other common choices such as Space Grotesk. That is a stated tendency rather than a measured result, and it matches what practitioners report. Paul Bakaus, who built the Impeccable design toolkit, describes a banned list as pushing the model into a different part of its latent space, where the escape route becomes the next template. In an interview he adds that widely shared skills become a source of sameness in their own right. Anthropic's own frontend-design skill dropped its named font bans in a June 2026 rewrite, and its September 2026 revision lists five default looks as patterns to recognise, with the brief's own words winning over all of them.

This finding shaped the whole skill. It keeps a list of the current default looks, because the list is useful today, and it adds two named tests meant to outlive the list. The swap test replaces the business with a competitor in the same category and lists the decisions that would have to change; if only the name and logo change, the design is a template. The category test is the one to remember: if someone could guess the design from the category alone (a bakery, an HR tool, a law firm), or from the category plus the usual avoidance, the direction fails. The skill runs the category test on its candidate concepts and both tests on the approved direction before any code is written. It is the same principle as in our note on what makes a prompt work in production: a test the model can run beats an adjective it can interpret.

What does the skill do that a prompt does not?

It splits one request into seven stages with a person's decision between them. The order came from a shared ChatGPT deep-research conversation on the same complaint, dated 14 September 2026, which concluded that creative direction, content, implementation and visual review should run as separate steps, and which said plainly that it had established techniques rather than a validated ranking. The skill's stages are:

1. Brief. Built from the owner's real material, with a source for every fact and every missing fact marked unknown. Questions go only to gaps that would change the design, and never to fonts or colours.
2. References. Three to five, each viewed in a browser and annotated, with at least one from outside the competitor category and one artefact of the subject itself, such as a label, a form or a map.
3. Directions, then the person chooses. Six to eight one-line concepts, filtered by the category test, then a random draw of three, developed to differ in hierarchy, composition and imagery, with a labelled standard option alongside and one recommended with a reason.
4. Art direction. The approved concept, composition, visual system, signature element with fallbacks and motion rules, written down so a later session does not repeat the rejected directions.
5. Representative slice, then the person approves. Navigation, the opening section and one substantial content section, with their mobile behaviour, built with real copy and visible placeholders for missing facts, then screenshotted at desktop and mobile widths.
6. Full build. The approved system extended to every page in the brief, with the copy track running alongside under a claim ledger.
7. Critique, bounded. One inspection round of screenshots, three located findings, one fix batch, one confirmation round, then the remaining trade-offs go to the person.

Three things changed on the way from that playbook to the skill. The playbook specified six design documents and four prompt files. The skill keeps four documents, adds one for unasked questions, and routes small changes around the whole workflow, because OpenAI's September 2026 guidance warns that stacked skills with long descriptions get truncated and can contradict each other, and that demanding a pile of documents before every change wastes context. The playbook warned against banning fonts and gradients globally, and the skill turned that warning into the two tests. And every instruction that rests on research now carries its source and a confidence label inside the skill, so a user who pushes back gets a citation.

Mechanically, it is a Claude Code skill: a folder the model loads when a request matches its description. It holds 12 files and 115,435 bytes. The main instruction file is 208 lines and about 3,128 words, and the description that decides whether it loads is 1,010 characters against a limit of 1,024.

Five decisions worth copying

1. A random draw decides which concepts get developed. The model writes six to eight one-line concepts, deletes the ones that fail the category test, numbers the survivors, and draws three at random with a one-line Python call. Asked for its best three, a model returns its most probable three, and the draw takes that ranking away from it. It adapts a seeded dealing mechanism from the Impeccable toolkit, whose author reports that across sixteen creative framings, 30 of 35 answers proposed the same concept. One line of research attributes this collapse to typicality bias in preference data and recovers 1.6 to 2.1 times more diversity in creative writing by asking for several candidates with their probabilities; the draw goes further by removing the model's ranking entirely. The draw itself has never been tested on its own.

python3 -c "import random; print(sorted(random.sample(range(1, N + 1), 3)))"

2. Alternatives come before critique. The evidence here is the strongest in the skill's file, and it is about people. In Dow et al., novice designers who produced several prototypes in parallel before any critique beat serial refiners on live click-through (445 against 398 clicks per million impressions), on blind expert ratings and on measured diversity, and they took the critique better: 8 of 17 serial participants called it negative, against none of the 16 parallel ones. Whether the benefit transfers to model-generated directions depends on a person doing the choosing, which is what the skill's gate asks for.

3. The build stops at the second section. The slice covers navigation, the opening section and one substantial content section, at desktop and mobile widths. A striking opening can hide a system that goes flat as soon as it has to explain something. The slice also has to carry one signature element that says something true about the subject, described concretely enough to build. The example the skill uses comes from a Codrops designer spotlight from May 2025: a site where stickers reset with a fizz, like a poured drink.

4. Copy runs as its own track with a claim ledger. Every number, customer, testimonial, certification and capability gets a source and a status, and unverified rows never reach the page. Part of the reason is legal. Advertisers need a reasonable basis before a claim runs, a requirement the US Federal Trade Commission set out in 1983. UK marketers must hold documentary evidence before publication under the CAP Code, endorsements must reflect the endorser's honest experience, and fake reviews, generated ones included, have been banned in the United States since 21 October 2024. The checker enforces the mechanical half: 64 banned words, 24 filler phrases, seven false-antithesis patterns, 11 claim flags, seven style flags, and an em-dash ceiling of 1.5 per thousand words. The word list only flags, for the same reason Kobak et al. could only put a floor of 13.5% on the share of 2024 biomedical abstracts written with model help: excess vocabulary is a fingerprint, and a fingerprint is where review starts.

5. The critique loop has a stop. One inspection round, one fix batch, one confirmation round, then the remaining trade-offs go to the person. Each finding is located (the problem, where it is, the fix) with before and after screenshots, the same discipline we use when we audit digital UX. Screenshot comparison detects change rather than quality. Practitioners who gave their agent a browser report that a structured instruction to shoot the page, list the differences and fix the top few works where vaguer requests do not, and the Impeccable toolkit prescribes the same bounded rounds.

How did we test it?

With Anthropic's skill-creator loop: write the skill, run the same tasks with and without it, grade against written assertions, aggregate, then read the outputs. Two agents, both Claude Opus 5 in Claude Code, each ran all three tasks in sequence. One was told to follow the skill. The other was told to use no skill and to stay out of the skills folder. Nobody was available to answer questions during either run, which is the skill's headless path.

A script graded the mechanical facts: typefaces against the default list, the prose checker, bracketed placeholders, screenshot sizes, question payloads and report length. The rest was judged by reading the outputs, with the evidence recorded for each check. After grading, each check was labelled process if it tests the skill's own artefacts or formats and outcome if it tests what the user receives. The split was applied to both runs equally.

Two disclosures belong here. An earlier round ran the same three tasks inline, with no separate agent and no baseline, and self-graded 27 of 27. That round was useful for the outputs it produced and worthless as a measurement, which is why the second round exists. And the same model family wrote the skill, the tasks, the checks and the grades, under our direction. Evidence is recorded for every check so a person can audit them, and that is the only defence we offer.

What were the results?

The skill passed 24 of 27 checks and the plain model passed 10. Per task: 9 of 11 against 2 of 11 on the bakery site, 6 of 7 against 3 of 7 on the vague SaaS brief, and 9 of 9 against 5 of 9 on the copy edit. The chart shows the pass rate per task, one run each, with the plain model in grey.

The pooled rate counts every check once. The mean rate averages the three task rates, which is why the two differ, and the spread beside it is the spread across tasks, since every cell holds one run.

Bakery site: 9 of 11 against 2 of 11

The skill run wrote a brief sourcing 14 facts to the owner, listed eight candidate concepts, dropped two that failed the category test, drew three of the six survivors at random, developed them with a standard option alongside, and recommended one called "Tomorrow's bake", where the page opens each evening with the next day's bake. It built only the header, the opening band and a "This week" section: 197 visible words and nine bracketed gaps such as the WhatsApp number and the pre-order cut-off. It set the page in Dela Gothic One and Radio Canada, neither on the default list, shot it at 1440 and 390 pixels wide, found three located faults, fixed them and shot it again.

The plain run built the entire site: a bar showing what is baking today, a card per bake, a pre-order builder that composes a WhatsApp message, a dark section about the oven, hours and directions. It checked its work at two widths with the clock set to three different times, and wrote an 823-word report listing ten things it needed from the owner. It lost points for a typeface on the default list and for adding a postcode the owner never gave, which it did flag for checking.

Vague SaaS brief: 6 of 7 against 3 of 7

This is the sharpest difference in the set. The skill run wrote a brief, asked four questions, each with a line on what the answer would change, and built nothing. The plain run assumed a product, named it Keel, and built a complete Next.js page with eight sections for it, in three typefaces from the default list. It was careful about it: it marked 15 elements in the code as claims to confirm and wrote a section headed "What I did not invent". A reader would still see a finished page for a product that does not exist.

Copy edit: 9 of 9 against 5 of 9

Both runs removed the unsupported "AI-powered" claim and replaced "hundreds of leading restaurants across Southeast Asia" with the true count of 62. Both wrote clean prose. The skill run produced a claim ledger of 14 rows, seven passages in a Current, Problem, Rewrite, Evidence format, and left four gaps in brackets. The plain run wrote fuller, more finished copy and invented three specifics on the way, including what the Monday summary contains, all of which it listed for checking. Its headline led with a benefit instead of saying what the company does.

The original copy, measured: 90 words and four em dashes, a density of 44.4 per thousand words against the skill's ceiling of 1.5. The checker reported 15 hard findings at the time of the run. After it gained four patterns during the project, the same copy scores 16.

Where does the skill's margin come from?

Mostly from doing things the plain model was never asked to do. Twelve of the 27 checks test the skill's own paperwork, such as whether a brief exists or whether a critique lists three located findings, and a skill will always win those. The 15 outcome checks are the honest comparison: true copy, no invented detail, no default typefaces, a hero that says what the business does.

In the grid, squares are outcome checks and circles are process checks. Lime marks the 14 checks that only the skill passed, and no check passed only without it.

All three failures under the skill traced to gaps in the skill itself. It never said where questions should go when nobody can answer, and its 200-word cap on the final report fought its own instruction to list every assumption. Both are fixed: unasked questions now go into a questions document as a ready-to-send payload, and the open-facts list sits outside the word cap.

What did the skill cost?

17.3% more tokens and 7.6% more wall time across the three tasks: 407,617 tokens against 347,578, 2,605 seconds against 2,420, and 71 tool calls against 57. Per task the overhead is uneven. On the bakery task the skill run spent about 2,438 seconds producing a brief, three directions and one critiqued section, while the plain run spent about 1,106 seconds producing a whole site. On the vague brief the skill run spent about 265 seconds and stopped, while the plain run spent about 1,063 seconds building a page for a guessed product. The per-task figures are estimates split out of each agent's total, and two of them are unreliable because the agents logged their own start times late.

The research phase is where the money went, and at first it went badly. A dynamic research workflow spawned 106 agents, of which 40 finished and 66 died, spending 5,168,295 subagent tokens in about 15 minutes. Seven separate research agents ran alongside it. One finished, at 205,658 tokens. The other six died with the workflow when the session hit its usage limit. That workflow alone spent 12.7 times the tokens of the whole skill run.

The recovery is the useful part. The dead agents had already written most of their notes to disk, and the workflow's own journal kept every completed search and fetch, so the work was recoverable by reading files, with nothing re-run. The rule we set afterwards held for everything that followed: two Opus subagents per part of a task, never a wide fan-out. It is the same lesson as in our piece on building production agents, where multi-agent systems cost about 15 times the tokens of a chat and earn it only on parallelisable work.

One more practical detail. The shared ChatGPT conversation that started the project could not be read by the normal page-fetch tool, which returned only the page title. Reading it needed a real browser that renders the page first. A link and a readable source are different things, and an agent doing research has to know which one it has.

Does the skill load at the right moment?

In simulation, yes: 24 of 24 routing decisions in a first round and 40 of 40 in a second. A skill that never loads is worth nothing, and one that loads too eagerly hijacks unrelated work. The description is the only thing the model sees before deciding. Two judges, each an Opus 5 agent, were given 12 skill descriptions and a batch of user messages and asked which skill Claude Code would load first, or none.

Because the first round was perfect, the usual tuning loop never ran and the description was never rewritten. The second round exists to make the result harder to believe by accident. A sample of its near-misses:

The caveat is heavy. This is a simulation of the routing decision by the same model family, on queries written by the skill's author. It shows that the description is unambiguous. It does not show how the live product behaves.

What broke along the way?

Six tooling defects, each worth knowing about before you run the same loop.

• The evaluation viewer rendered blank. Anthropic's skill-creator writes run data into an inline script without escaping it, so the first output file containing a closing script tag ends the script early. Both review pages showed no runs, and bakery page markup leaked into the page body. The fix was a wrapper that escapes angle brackets and ampersands before they reach the page.
• The screenshot script crashed on pages that never go quiet. It waited for network idle and threw when that never arrived. It now falls back to the load event and reports the timeout as a finding.
• The prose checker missed whole families of puffery, including "leading restaurants", "never miss a filing" and "forever". It also cannot see copy held in scripts or attributes, which is why the screenshot script now saves the rendered text for checking.
• Skill frontmatter accepts only six keys. A "when to use" field is one of the keys it rejects, so the triggering guidance had to fold into the description, which is capped at 1,024 characters.
• The aggregation script over-counted runs. It reported three runs per configuration where there was one run per task, which would have overstated the sample to any reader of the benchmark file.
• A shared chat link is unreadable to a plain fetch tool, as described above.

What does this study not show?

• One run per cell. There is no run-to-run variance here. The spread quoted beside the mean is the spread across three different tasks.
• The author graded its own work. The same model family wrote the skill, the tasks, the checks and the grades. Evidence was recorded for every check so a person can audit them.
• Twelve of 27 checks test the skill's own paperwork, and a skill will always win those. The outcome subset is the honest comparison.
• Nobody could answer questions during the runs, so the gates that ask a person for a decision were never exercised as designed.
• No aesthetic judgment was measured. No human rated distinctiveness, no page was tested with visitors, and no conversion data exists. Nobody, this study included, has shown that any design skill produces more distinctive or better-converting websites. Every claim here is about process, truthfulness and defaults.
• The list of overused typefaces is a 2026 snapshot drawn from practitioner catalogues and vendor skills, and part of it is inferred from how prompt examples spread, with no measurement behind it.

The experiment that would settle it is described in both the source playbook and the skill's evidence file, and neither has run it. Take several real briefs. Hold the model, the stack and the content constant. Produce each site three ways: an ordinary prompt, one design skill, and the full workflow. Show whole pages rather than heroes to reviewers who do not know which method produced which, and have them score brand specificity, hierarchy, copy usefulness, coherence and mobile quality, with editing time and functional failures recorded alongside. With small samples it stays directional, which is still more than anyone has now.

What should you take from this?

If you use coding agents to build websites, five habits carry over whether or not you use this skill.

• Give the agent real material and let it stop. The plain model was truthful about every fact it was given and invented a product when it had none. Missing facts should end as bracketed gaps or questions, never as plausible filler.
• Ask for alternatives before critique, and draw rather than rank. A model's best three are its most probable three.
• Keep a claim ledger. Every number, customer and capability on the page gets a source and a status, and the unverified rows stay off the page. The regulators expect the evidence to exist before publication.
• Judge a skill on outcome checks. A skill can score 24 of 27 while its real margin is 15 against 9. Label the checks before you read the headline rate.
• Cap the fan-out. A 106-agent workflow spent 5.2 million tokens and lost 66 agents. The work was recovered from files on disk, and two subagents per part of a task was enough for everything after.