July 14, 2026
Two words. Six models. Three different translation decisions. Here's what happened when I ran 8 test sentences through the latest AI models available on MachineTranslation.com, and what the outputs actually reveal about when a model fails silently.
Two words. "That's sick." Six models. Three distinct translation decisions.
In Japanese, one model rendered it as それはやばい, preserving the ambiguity. Another dropped the subject pronoun entirely and wrote the bare form やばい, the most colloquial version possible. A third wrote それはすごい, which resolves the ambiguity entirely in favor of "amazing," stripping out the double meaning that made the phrase interesting in the first place. In Vietnamese, one model wrote Đỉnh vậy (slang, correct for youth register). Another wrote Thật tuyệt vời, grammatically sound, but closer to saying "that is truly wondrous" when someone texts you a meme.
All of these are defensible. None of them are the same thing. And if you are running translations through a single model, you will never see the choice that was made on your behalf.
That is the central question this article is trying to answer. Not which AI model is best for translation in 2026 (the answer depends on language pair, register, domain, and sentence structure), but rather: how would you know when your model made the wrong call? Especially in domains like medical terminology, legal contracts, or professional formality, where the wrong call looks exactly like a correct translation until a specialist reads it.
Here's what I did. I ran 8 test sentences through two rounds of models on MachineTranslation.com: first a baseline of 5 widely-used models, then an expanded pool that adds several of the newest premium-tier model variants now available to paying users. Normally, comparing outputs from models like this would mean separate paid subscriptions with each provider individually. On MachineTranslation.com, they sit in the same comparison view. The language pairs were English→Japanese and English→Vietnamese. The sentence categories were chosen specifically to stress-test areas where model disagreement is most consequential: idiomatic phrasing, legal terminology, medical terminology, formality-sensitive context, and deliberate semantic ambiguity.
A note on evaluation methodology: machine translation research has long grappled with how to score outputs automatically. Metrics like BLEU and COMET as measured in WMT shared tasks give useful aggregate signal, but they are poor at detecting register errors, idiom failures, and terminological precision, exactly the categories that matter most here. What you are about to read is output analysis, not a BLEU race.

Not every sentence surfaces a meaningful disagreement. Two of the eight tests, "Let's touch base once the dust settles on this deal" (→ Japanese) and "We're burning the midnight oil to hit this launch date" (→ Vietnamese), produced high agreement across both rounds. The idioms were correctly understood as idioms. Nobody translated "touch base" as touching a physical base. Nobody translated "the dust settles" as sediment falling.
This is worth pausing on: idiomatic English business language is reasonably well-handled by current frontier models when the idiom is common enough. The consensus for "touch base" held stable across both rounds; variation was purely stylistic, around whether to use この件 (this matter), この取引 (this deal), or この案件 (this case) for the source phrase.

The "burning the midnight oil" test is where things got more interesting. Every model except one correctly understood the idiom. MiniMax M2.7, introduced in Round 2, translated "burning" literally, producing đốt đuốc, meaning "burning torches." The consensus correctly excluded it.

Japanese is a formality-layered language. The choice of verb ending, pronoun, and referential noun form is not stylistic. It signals the relationship between speaker and recipient. Sending a message to your CEO using casual verb forms is not a translation error in the grammatical sense. It is a social error, and a significant one.
The test sentence: "Can you send that over? Thanks, appreciate it." Context: messaging a CEO.

The consensus shifted when the model pool expanded. The mechanism is straightforward: adding models that correctly read the formality context shifted the majority, and the consensus followed. Here is the full spectrum of what the models produced in Round 2:

The Vietnamese register test surfaced a different kind of formality error, one that is subtler. The source sentence: "Hey, quick question — are you free to hop on a call?" Context: messaging a client. Qwen in Round 1 included "mình," a first-person pronoun that implies casual peer-level friendship. Every other model avoided first-person self-reference entirely, which is the safer professional choice. Crucially, Qwen corrected this in Round 2 and dropped "mình." The consensus in both rounds excluded it.

The vendor/client indemnification sentence is a standard clause in commercial agreements: "The vendor shall indemnify the client against any third-party claims arising from breach of this agreement."
In Round 1, all five models correctly identified the legal register and used loanwords ベンダー (vendor) and クライアント (client), standard in Japanese commercial contracts with English origins. One exception: GPT-4.1-NANO used 販売者 (seller) and 顧客 (customer), legitimate Japanese words that lack the formal legal specificity of the loanword equivalents.
The more important finding emerged in Round 2. The key distinction is between two words:
These are legally different concepts. "Indemnify" specifically means to shield a party from third-party liability. 免責 is the correct legal term. All Round 1 models used 補償. In Round 2, DeepSeek V4-Pro was the only model to produce 免責, and it did so in Round 2 only, not Round 1.

In a contract, 補償 and 免責 are not synonyms. One means "we'll reimburse you." The other means "you are shielded from the claim in the first place." If a translation platform uses 補償 where 免責 is correct, a lawyer reading the Japanese version won't see the same agreement the English version describes.
This is the most consequential finding in the dataset. The test sentence: "This medication is contraindicated in patients with a history of hepatic impairment." Source: clinical product documentation. Target: Vietnamese.
The critical term is "hepatic impairment." There are two Vietnamese candidates:
These are clinically distinct. A contraindication warning based on "hepatic impairment" applies to anyone with reduced liver function, not only those who experienced complete organ failure. Using suy gan narrows the indicated population incorrectly. A patient with moderate hepatic impairment who reads suy gan might not recognize themselves as the contraindicated population.

Some source sentences are ambiguous by design. "That's sick" in contemporary English slang means something excellent, but it also still carries its literal meaning (that is sickening/disgusting), and the tension between those two meanings is part of how the phrase communicates. The challenge for a translation model is: do you preserve the ambiguity, collapse it to the most likely meaning, or pick one and commit?


The models in this test are not all equivalent. They span different architectures, training regimes, and deployment contexts. The Round 1 baseline includes models that are widely accessible. The expanded Round 2 pool adds several of the newest premium-tier model variants now available to paying users on MachineTranslation.com, the same tier of model that would otherwise mean a separate paid account with each individual provider.

The expanded pool in Round 2 includes models that are more advanced and available to paying users on MachineTranslation.com. The effect of expanding the pool is not uniform. In some tests (formality/Japanese), it shifted the consensus meaningfully. In others (medical terminology/Vietnamese), the split deepened.

The test data points to two distinct failure categories, and they require different responses.
Category A: Silent failures. Formality errors, register errors, medical terminology precision: these produce outputs that look correct. The Japanese is grammatical. The Vietnamese is fluent. There is no surface signal that anything went wrong. A monolingual user reading a single model's output has no way to know that the model made a register choice that a senior Japanese executive would notice, or that a clinical term was narrowed in a way that changes the population it applies to.
Category B: Visible failures. MiniMax translating "burning the midnight oil" as "burning torches." GPT-4.1-NANO resolving "That's sick" to the unambiguous "すごい." These are detectable, either by a speaker of the target language or by comparison with other model outputs.
The multi-model comparison SMART provides is most valuable in Category A. When five or ten models produce outputs and most agree on a formal register while one produces casual plain-form Japanese, the disagreement is visible in the comparison view. A user who speaks the target language can evaluate the options. A user who does not can see that a contested decision was made, and decide whether that sentence warrants human review.
For document-scale work, large files, layout-preserving translation of PDFs, contracts, or clinical materials, the platform's document processing capability handles file structure without breaking formatting. But the quality questions raised above apply regardless of file size. A 100-page clinical study where every instance of "hepatic impairment" is translated as "liver failure" is a larger version of the same problem identified in Test 2.
For the medical and legal categories specifically, human verification is the correct backstop. Tomedes' AI Post-Editing service, which pairs AI output with certified human review for exactly this kind of high-stakes content, is built for this. The suy gan / suy giảm chức năng gan split is exactly the kind of finding that should trigger that review, not because all models are wrong, but because the models that are most confident are not the most precise.
The value of seeing multiple outputs isn't that you'll always pick the majority. It's that you'll know a choice was made, which is different from assuming the output in front of you is correct.

Put the eight tests side by side and a pattern holds up: agreement and disagreement aren't randomly distributed. Idiomatic, everyday business language ("touch base," "burning the midnight oil") converged across nearly every model in the pool, in both rounds. Formality, legal precision, and medical terminology didn't. The CEO-formality test flipped from casual to formal only once the expanded pool was included. The indemnification clause surfaced a real legal distinction (免責 vs. 補償) that only one model, in one round, got right. The medical terminology split never resolved at all, staying at roughly 6-to-4 across both rounds regardless of which models were in the pool.
That's the actual finding, not a marketing claim: consensus doesn't make every sentence better, and it doesn't need to. It's most valuable exactly where a single model's fluency gives you no reason to doubt it, and where being wrong actually costs something.
There's a second, more practical layer to this test worth stating plainly. Running this comparison myself meant checking outputs from GPT-5.5, Claude Opus 4-7, Gemini 3.5-Flash, GLM-5-Turbo, and MiniMax M2.7 side by side with the existing baseline models, in one place, on one platform. Outside of MachineTranslation.com, that's not one subscription. It's several, one for each provider whose premium tier you want to test, at whatever each provider charges individually. Paying users here get that same comparison in one click, at a fraction of what maintaining five separate premium subscriptions would cost, without giving up the thing that actually matters: seeing where the models agree, and knowing exactly when they don't.
Neither is categorically better; it depends on the test category. Claude Sonnet and Claude Opus consistently chose the more precise Vietnamese medical term, and Claude Opus produced the most appropriate slang for "That's sick." But Claude Sonnet also produced casual Japanese in a formal CEO-context sentence, where GPT-5.5 got it right. Comparing outputs directly is more defensible than picking one model and trusting it.
There isn't one; accuracy is domain-dependent. Gemini 3.5-Flash and GLM-5-Turbo produced the most appropriate formal Japanese in this test. Both Claude models were most precise on Vietnamese medical terminology. DeepSeek V4-Pro was the only model to use the correct Japanese legal term, but only in Round 2, and it produced casual Japanese elsewhere. No single model dominated across all eight tests.
No, not automatically. Expanding the pool with newer premium-tier model variants shifted the consensus from casual to formal in the Japanese formality test. But in the Vietnamese medical terminology test, the split persisted at roughly 6-to-4 regardless of which models were included. More models surface more disagreement, which is useful signal, but the consensus still follows the majority, and the majority isn't always the most precise answer.
SMART is MachineTranslation.com's multi-model consensus system, spanning up to 22 AI models across providers including OpenAI, Anthropic, Google, and DeepSeek. It runs a translation through multiple models at once and surfaces the majority-consensus output alongside every individual model's answer, by default, with no configuration needed. The consensus shifts when the model pool shifts, as shown in this test's Japanese formality result: a casual consensus in Round 1 became formal in Round 2.
No single model; human review is the better answer. Claude models consistently chose the more precise Vietnamese medical term, and DeepSeek V4-Pro alone used the correct Japanese legal term, but only in Round 2. The medical terminology split never resolved across 10 models, which signals that domain expertise is required, not that a different model should be picked. A certified human post-editing review, like Tomedes offers, provides that specialist check.

By Rachelle Garcia
Connect on LinkedInRachelle leads product and AI at Tomedes, where she runs the experiments that turn internal data into better translation experiences. She writes about what actually happens when you build AI products such as MachineTranslation.com — the numbers, the surprises, and the parts that don't go to plan.