Bilingual site translation pipeline: putting a quality gate on LLM translations

Self-use system · open method 2026.07 – present In production

I used to think keeping this site's English mirror in sync meant either translating by hand or gambling that AI would get it right in one pass. Now I edit the Chinese source, run one command, and an incremental translation step plus a byte-for-byte quality gate blocks garbled structure and dropped numbers — the English mirror just catches up on its own.

The problem

This site is written in Chinese, and I wanted an English mirror of it — pages, blog, resume data, the whole set. Maintaining that by hand wasn't realistic: the Chinese source changes often, and having the English chase after it would mean running a second site. Just handing whole pages to an LLM for translation turns out to have three pitfalls:

  • Structure gets broken — tags, attributes, SVG paths; the model "helpfully optimizes" the page while it's at it
  • Wording gets upgraded — the Chinese source honestly says "AI-assisted development," and the translation casually upgrades it into bragging that erases the AI's involvement entirely. A translation that makes me look more capable than the original is an incident on this site, not a pleasant surprise
  • Details quietly disappear — numbers, links, qualifiers dropped, and nobody notices

So the real problem isn't "how to translate" — it's how to keep machine translation from overstepping when nobody's watching.

The approach: whole-page translation + deterministic rewriting + quality gates

  • The translation unit is the whole file: HTML, Markdown, and JSON sources each get their own system prompt; whole page in, whole page out, no fragment-and-reassemble
  • Incremental translation: a manifest records the content hash of each Chinese source at translation time, and anything unchanged is never re-translated — editing one page only costs the price of that one page
  • Deterministic tasks stay off the model's plate: mechanical transforms like path rewriting, canonical tags, and language markers are all applied by code after the translation lands, before it's written; the model handles language only
  • Four quality gates, all must pass before anything is written to disk: byte-for-byte tag-skeleton diffing, wording red lines, number-loss checks, and a leftover-Chinese-text check
  • Pure Node standard library, zero npm dependencies — same principle as the rest of the site; one script of roughly a thousand lines is the whole thing
Chinese source Single source of truth Incremental diff Skip if hash unchanged LLM translate Whole page in/out Quality gate Skeleton · wording · numbers en/ mirror Pure output Failed: retranslate w/ feedback Error list fed back to model Still fails: reject No write, no record Publish gate Stale mirrors blocked Translations that fail the gate are never written or recorded; a rerun retries automatically; deterministic rewrites like paths and links stay off the model's plate
The model sits in just one cell in the middle; everything around it is code that doesn't trust it.

The hardest fight: not against mistranslation, against bragging

Before building this, I assumed the quality gate would mainly catch translation errors. After building it, I found the biggest enemy was the model's "enthusiasm." The Chinese source says "built by directing Claude Code in natural language," and partway through translating, the model would upgrade that into an unqualified claim that erased the AI's role entirely — thinking it was polishing my copy, when really it was lying on my behalf.

So the wording gate is a hard-coded, non-negotiable rule: English bragging phrases that erase AI assistance — claims of solo, unassisted development; claims of manual, line-by-line coding; claims of building everything up on one's own — go on a banned list and get blocked on sight; wherever the Chinese source carries an "AI-assisted" qualifier, the translation must keep the corresponding AI-assisted phrasing — dropping it also gets blocked. The number gate takes every number in the Chinese source and looks for it in the translation; if it can't find it, it blocks the output (it understands unit conversions like 万/亿 to million/billion, to avoid false positives).

This isn't a hypothetical worry. On the first translation pass across 20 files, the quality gate caught 7 violations: some tried to claim development happened without any AI assistance, some dropped the "AI-assisted" qualifier, one lost the number "1400+" entirely. Once blocked, the error list gets fed back to the model verbatim for a retranslation pass; if it still fails after two rounds, the whole file is rejected outright — nothing written, nothing recorded in the manifest, and the whole thing restarts on the next run.

In a system where the LLM does the work, the most important code isn't the few lines that call the model — it's the few lines that catch it when it's wrong. Truthfulness can't be outsourced to the model; it can only be enforced by the gate.

Production hardening

  • Publish gate: the publish script checks the manifest first — if a Chinese source changed but hasn't been retranslated, publishing simply stops, so the English mirror never silently falls behind the Chinese
  • Routine health checks: already-generated English output is folded into the site-wide check, re-run through the full quality gate every time, plus a global banned-phrase scan and an internal-link existence check
  • Channel fault tolerance: when generating long files, a few minutes without a byte on the wire will get the connection killed by proxies and middleboxes. Switched to SSE streaming output; if the main channel drops, it falls back to a curl backup channel automatically, retrying with backoff on disconnects
  • Guardrails against truncation: oversized source files are rejected outright to prevent truncation; any output that gets cut off is treated as a failure, and a half file is never written to disk

Results and reflections

The workflow now: edit the Chinese, run one command, and the English mirror catches up on its own; if I forget to run it, the publish gate stops me. This case study has no client — the deliverable is the site itself, and the very page you're reading right now also gets run through this pipeline into English.

The honest boundaries need spelling out too: two archived tutorial posts contain ASCII flowcharts and batch-processing content that the model refused to translate across several rounds in a row. I ended up granting those two an explicit exemption, logged as a to-do, rather than pretending they passed the gate. Also, the quality gate only protects "structure and facts" — it can't guarantee the translation reads naturally; prose quality still needs human spot-checking. The limits of a gate have to be stated plainly, or the sense of safety it gives will exceed the safety it actually provides.