The Translation Research Is Public. Why Do We Find So Little of It Available in the Market?

There is a question I keep coming back to: why is the AI translation pipeline available in the market today still so far behind what the research already described?

The papers are all public, they are benchmarked, and most of them can be reproduced by anyone with a GPU and patience. I know this because that is what I have been doing for the last year: going into the research to see what good translation quality actually looks like, and building what I found into the production line of my own translation company, piece by piece.

So in this article I take the research finding by finding, and for each one I answer the two questions that actually matter to anyone buying or building translation: how do I run this in production, and where can you buy it elsewhere, if you want to run it too.

In my own reading I settled on seven specific techniques, the ones I judged would make the difference if the goal was a translation engine that could stand next to what the best human teams achieve:

1. a verification loop on the draft,
2. a reference-free score on every segment,
3. error annotation by a language model,
4. a round-trip check on the worst segments,
5. context mining before translating,
6. semantic retrieval over the translation memory,
7. and review effort pointed by the scores.

Seven has a long pedigree as a number, and I would love to tell you I chose it for the mythology, but the truth is less grandiose: I stopped at seven because that is where the research that mattered to my engine ran out.

I decided to build around them, and somewhere along the way I realised that almost nobody else was building on them, or they were building on one or two and stopping there. The full engine will have all seven. Today, four are running in production and three are under study.

I understand that building these takes time, and that running them in production costs compute on top. But I am not trying to build an engine that translates vast volumes of text quickly and cheaply, because those engines already exist, and the world does not need another one.

What I am building is an engine that translates at a very high level of quality, and that can tell me, segment by segment, which parts of its own work need a human and which parts do not, so that the translator's time, the scarce resource, goes only where it is needed, and the final translation stands as good as the all-human version while needing a fraction of the human time.

So here they are, one at a time.

1. The research says: do not trust the first draft. In 2022, researchers at Unbabel published quality-aware decoding: instead of accepting the model's first translation, you generate several candidates and let a quality metric pick the best one, and in their experiments this reduced severe errors by up to 40% across language pairs (Fernandes et al., 2022).

In my pipeline one model translates, and a second model reads that translation, compares it with the source text, and identifies errors. The translation only moves forward when the critic runs out of objections or the loop hits its ceiling.

So although the research picks the best of several drafts, my loop interrogates one draft until it survives. The principle is the same, generate and verify instead of generate and hope, but my verifier has to point out what is wrong, which means every objection becomes a piece of evidence I can analyse afterwards.

Can you find this on the market? Almost nowhere, off the shelf. Some platforms route between several models and pick one per job, but routing chooses a translator; it does not make one translator answer to another. The verification loop is the piece of the 2022 research that never made it into the products.

2. The research says: you can score a translation without a reference. That same year the same group released COMETKiwi, a model that scores the quality of a translation against its source with no reference translation needed, which means quality measurement stopped requiring a human to have translated the text first. By 2023 its successor xCOMET could point at the specific errors themselves.

In my engine, a dedicated quality model reads every single segment against its source, on our own hardware, in batches, taking no human time at all, and the grade it produces decides what happens to that segment next.

What the research paper does not give you is the calibration. The scores are operating points, not probabilities: thresholds chosen because we measured what the quality actually looks like above and below those thresholds.

We had to measure where the model goes blind, because it does go blind. We found that a segment where the source and the target are equal gets top scores, and also that very short segments can have inflated scores. So those exceptions are handled by rules, because we know we cannot trust the number in those cases.

Here, to be fair, the market did its job. Phrase ships a real quality score built on MQM by people who spent careers on evaluation, Translated trains its own quality estimation, and Unbabel, who wrote the papers, runs it in production. This is the one finding that genuinely shipped, and if your platform still gives you no per-segment score in 2026, someone other than you has decided you do not need it.

3. The research says: a model can name the errors, not just count them. In 2023, Microsoft's GEMBA-MQM showed that a well-instructed general-purpose model could annotate translation errors in MQM, the industry's standard taxonomy of error types and severities, at state-of-the-art levels when ranking whole systems (Kocmi and Federmann, 2023).

I am not doing this yet. Earlier tests were a bit hit or miss, but I am studying it as I write this, and I am going to give it another go. I already know where it stands in my pipeline: the record of what my revisers change. Every one of those changes is classified today by my own scheme, and MQM would turn that private classification into the industry's vocabulary, the one enterprise buyers already speak, to make it easier to assess and compare.

I also already know where it stops, because the same research is clear that a model naming errors is far more trustworthy about a whole system than about a single segment, and single segments are exactly where my pipeline makes its decisions. So in my engine it will report on aggregates, and it will not be allowed to decide the fate of a segment.

The platforms are moving here now, and credit where it is due: automated MQM annotation is arriving in commercial tools, and if it holds up it will give the industry a shared language for machine output that it badly needs.

4. The research says: a translation can read perfectly and still be wrong. A 2023 paper rehabilitated an idea the field had buried: round-trip translation, translating the output back into the source language to check the meaning survived. It was dismissed for good reasons in the statistical era, and it works again with modern models (Zhuo et al., 2023).

In my engine, the segments the score distrusts get translated back into the source language, and the two meanings are compared. I run this because the dangerous errors are those that are fluent enough to be missed: the sentence reads beautifully, but says something else, and a reviser moving quickly forgives a sentence that reads well.

So when the two meanings diverge, the segment is flagged for a human to check. I do not run this on everything; I run it where the earlier layers already raised doubts on the quality of the translation, and that is what makes it affordable.

I have not found this in any commercial platform's public documentation. It sits in the papers, and the products never picked it up.

5. The research says: prepare before you translate. The MAPS paper showed that making a model work like a translator preparing a text, mining keywords, topics and similar past translations before translating, then generating candidates and selecting by quality estimation, resolved a majority of the model's hallucination errors (He et al., 2023).

I partially do this: we do create an ad-hoc glossary before each project starts, but that is as far as we go, so far. So I am studying it and in reality half of it is already running: the quality model that does the selecting. What remains is the preparation step in front of it, and then the measurement, on real jobs.

Nobody sells it. The candidate-and-select pattern, in any form, is the part of the research the market has most consistently left on the shelf.

6. The research says: your translation memory knows more than your search can find. Through 2023 and 2024, work by Moslem and colleagues showed that retrieving translation memory matches by meaning, with embeddings, beats the character-based fuzzy matching every CAT tool still runs on.

This is the one I want most, and the study has already started, because the asset it unlocks is the one we have sitting on our servers: decades of human translation pairs, searchable today only by spelling. Somewhere in that memory is the sentence that says the same thing in different words, and my tools cannot find it yet. The design is drafted: the memory embedded and searched by meaning, the matches feeding the engine as worked examples rather than pretending to be exact matches.

On the market, nothing. Every CAT tool still matches by edit distance, and the research showing meaning beats spelling is two years old.

7. The research says: point the human where the risk is. The final stop after all of the above, and through the adaptive translation literature of the last two years, is that once you can score machine output reliably, spreading human review evenly across a text stops making sense, because most of the text no longer needs it, and you know exactly where to look.

Review effort in my pipeline concentrates where the signals show distrust in the translation. One specific characteristic of my system is that the human revisers still work blind, that is, they never see the grades, so their judgement is not biased by what the machine decided, and we can be sure that every word they change is changed because they believed it had to be.

Every one of those changes is recorded and classified, which means the process that measures the machine also measures itself.

Pointing review at flagged segments is starting to appear commercially, and it is the natural next step for anyone with a quality score. What I have not seen elsewhere is the blind part, and I would argue the blind part is what makes the measurement honest.

The state of the art at a glance. For each technique: the year the research demonstrated it, what the market sells, and where my engine stands.

Four of those seven run in my production line today, on files that go out to paying clients, and the other three are under study, and I already know what the first experiment for each of them will be.

The market is the combined output of an industry's worth of engineering teams. My engine is one person, directing an AI that writes the code, inside a translation company that had to keep delivering every single day while the engine grew around it. Things do not appear overnight when the team is one man; they appear in sequence, and the sequence is public.

The pieces, though, are not the point, and I am not trying to keep a running tally against what other people are doing. What matters is what the pieces make together, and the quality of the translation that goes out the door.

A segment in my pipeline is translated by one model and interrogated by a second, then graded against its source, and then, if the grade raises doubts, translated back into the source language to check that the meaning survived. Only then does it reach the human layer, with the reviser's time going where the signals point instead of being spread evenly across text that does not need it.

The important part is that the revisers see no grades, so their judgement is not biased by what the machine decided, and every word they change goes into the record, with the original grade attached.

That record is what gives the grades their real use, because the grades do not just route individual segments; they sort every job into tiers, from the segments every signal cleared down to the ones that collected doubts along the way. When the record shows that the revisers barely touch a tier, that tier starts to arrive locked: closed in the translation tool, delivered as final, needing no reviser time at all. The lock depends on that evidence, job after job; the day the record turns against a tier, the lock comes off.

The locked tier currently runs at 0.90% real errors, and the best tier still waiting outside the lock runs at 1.26%.

So why did the platforms stop where they stopped?

Part of the answer is honest economics: every technique they skipped multiplies the inference cost per segment, and a platform serving a hundred language pairs at scale feels that multiplication in a way a single company serving its own clients does not.

But there is a second reason, and I think it is the bigger one. Every one of those techniques only earns its place in the workflow when someone who knows the domain, the language pairs, the context, sets the thresholds, watches the failures, and answers for what goes out the door, and to be honest that person almost never sits inside a software company, but inside their clients' offices.

A platform can sell you the score; it cannot sit with your revisers and decide when to trust the score and when to ignore it. That is an owner's job. In my engine the owner is me: the AI writes the code, and I decide what it builds and answer for what ships.

Ask your translation vendor whether they measure how many machine words a human changes before delivery. Mine is the pipeline built around that number. If theirs do not measure it, everything else they told you about quality is an opinion.