October 28, 2026

We expected AI translation into English to score higher. Our data said otherwise

If you translate into English because you assume AI is at its strongest there, our data says that assumption deserves a second look for some language pairs. Teams often treat English as the safe direction, and published benchmarks give them good reason to. When I started reading our quality scores by direction, I expected to see the same pattern. 

We looked at 28 days of MachineTranslation.com quality scores, from July 12 to August 8, 2026. For 32 of the 44 language pairs we could compare in both directions, translations out of English scored higher than translations into English. The typical gap is small, and a few pairs show a very large one. Published research reports the opposite direction, so this article covers what we measured, what might explain the disagreement, and what I cannot yet confirm.

Key takeaways

  • Translations out of English scored higher than translations into English for 32 of the 44 language pairs measured in both directions, in MachineTranslation.com quality scores from July 12 to August 8, 2026.
  • Across pairs with enough volume, 6.8% of out-of-English outputs scored below 8, against 10.0% of into-English outputs.
  • The gap is largest for Swedish and Amharic, where 47.0% and 46.3% of outputs into English scored below 8, compared with 11.7% and 18.1% out of English.
  • Published benchmarks, including FLORES-101 (2021) and a 2026 Kokborok study, report stronger results into English, so our data disagrees with the literature and I cannot yet confirm why.
  • The direction gap also appears outside English: in 11 of the 12 largest gaps, the pair with the less widely used language as the source scored lower.

Does translating into English really give better AI results?

It depends on the dataset, because published benchmarks say yes while our own scores mostly say no. The FLORES-101 evaluation benchmark (2021) found translation into English strong across the board and translation out of English much worse, and a 2026 study of Kokborok, a language of northeast India, reported the same direction gap across every system it tested.

Kokborok is a low-resource language, meaning one with little digital text available for training an AI model. The usual explanation is that English makes up a large share of the text on the web, so models see more of it and write it more fluently than they write languages with less text. A 2025 paper in Natural Language Processing, published by Cambridge University Press, turns that reasoning into a practical rule: when translation is necessary, go from the weaker language to the stronger one.

That reasoning is sound, and I treat the published result as a strong prior. A contradicting result from our data needs a careful explanation, and the sections below work toward one.

What do 28 days of MachineTranslation.com quality scores show?

Out of English scored higher for 32 of the 44 pairs we could compare in both directions. Out-of-English translations scored 9.100 and into-English translations scored 8.908. The share of outputs below 8 was 6.8% out of English and 10.0% into English.

Most scores on our scale sit in the 9s, so the average hides a lot. In my opinion the share of outputs below 8 is the more practical measure of bad output, and the table and chart below use it. The typical pair differs by only 0.11 points (the median), while the mean gap is 0.19, which tells me a handful of pairs pull the average. In the other 12 pairs, translation into English scored higher, including Czech and Romanian.

Language    Into English: average score     Into English: % below 8      Out of English: average score      Out of English: % below 8
Swedish7.67447.0%9.01111.7%
Amharic7.95846.3%8.88018.1%
Yoruba8.36727.7%9.0387.8%
Albanian8.37327.3%9.2574.1%
Nepali8.36327.2%9.2812.8%
Czech9.1634.6%8.83316.8%
Romanian9.0818.3%8.97414.2%
0%25%50%SwedishAmharicYorubaAlbanianNepaliInto EnglishOut of English

Share of translations scoring below 8, by direction, five language pairs. MachineTranslation.com quality scores, July 12 to August 8, 2026.

Why might our scores disagree with published benchmarks?

I see four possible reasons, and I cannot yet say which one explains the gap: how the score is produced, which text is translated, which languages are in the sample, and which AI models ran on each translation. Each one would change what our result means.

  • Scoring method. Benchmarks such as FLORES-101 compare each output against a human reference translation. Every score in our dataset comes from an automated process. If an automated scorer knows a language poorly, it may miss errors in that language and rate those outputs too high.
  • Text. Our scores come from live translations that vary in length and subject, while benchmark sets use fixed sentences chosen for the test.
  • Languages. Our comparison covers the 44 pairs with English that had enough volume in both directions, which is a different set from any single benchmark.
  • AI model mix. Each average blends the AI models that ran on those translations, so a different mix in one direction could shift its score.

My working view is that automated scores are least trustworthy in the languages where the scorer has seen the least text. I cannot prove that with this dataset, and I could be wrong on this.

Does the direction gap show up outside English?

Yes, because among the 85 pairs measured in both directions from July 12 to August 8, 2026, the pair with the less widely used language as the source scored lower in 11 of the 12 largest gaps. The exception was Arabic to Bengali, where Bengali as the target scored lower.

Amharic to Arabic averaged 7.489, while Arabic to Amharic averaged 9.183. Quechua to Spanish averaged 7.148, against 8.723 for Spanish to Quechua. Burmese to Chinese averaged 7.831, and Chinese to Burmese averaged 9.047. The lowest averages in our window were Quechua to Spanish (7.148), Amharic to Arabic (7.489) and Fijian to English (7.541), and in each of them the less widely used language was the source. Deciding which language is less widely used was my own judgment, made pair by pair, and a stricter definition could change the count by one or two.

As far as I understand, two readings fit this pattern. A model that has seen little Quechua may misread a Quechua source, and that misreading shows up as a low score. A scorer that has seen little Quechua may also fail to catch an error in Quechua output, which would inflate scores in the other direction. The two readings call for different advice, and the next section gives the steps that hold under either one.

What should you do when your language pair is the hard direction?

Do not rely on a score alone. Compare what several independent AI models produce, look at where they disagree, and have a speaker of the target language review anything that matters. Those steps hold under either reading of our data.

  1. Look for disagreement between AI models, not only a score. SMART consensus runs the same text through up to 22 AI models and returns the version most of them agree on. Low agreement marks the passages to read closely.
  2. Have a speaker review rarely used target languages. With Human Verification, a professional translator reviews the AI output and returns it with a 100% accuracy guarantee.
  3. Check the pair as well as the model. Our breakdown of which AI model handles each language pair best starts from the pair for the same reason.
  4. Treat minority-language work as higher risk by default. Our piece on AI translation for minority languages covers where the gaps tend to sit.

This is what reliability means in practice for us: agreement between independent AI models, plus a human check where it counts. If the language you are translating into is rare, I would have a speaker of that language check the output, and I would not treat a high score as that check.

Frequently asked questions

1. Is AI translation more accurate into English than out of English?

It depends on the language pair, because published benchmarks report stronger results into English, while our 28-day scores were higher out of English for 32 of 44 pairs. Treat direction as one more variable to check, especially when a less widely used language is involved.

2. Why is AI translation worse for some languages?

AI models see far less text in some languages during training, so they handle them less reliably. In our 28-day scores, 47.0% of Swedish to English outputs scored below 8, against 11.7% for English to Swedish. Automated scoring can add its own bias, so I read these numbers as a warning.

3. Which language pairs does AI translate worst?

Among pairs we measured in July and August 2026, the lowest averages were Quechua to Spanish at 7.15, Amharic to Arabic at 7.49, and Fijian to English at 7.54. In each case the less widely used language was the source. A pair needed at least 50 translations in the window to count.

4. Is it better to translate into your native language?

It depends on the language, because published research favors translating from a weaker language into a stronger one, while our scores for rarely used target languages ran high and may hide errors. For anything important, have a speaker of the target language review the output.

Photo of Rachelle Garcia

By Rachelle Garcia

Connect on LinkedIn

Rachelle leads product and AI at Tomedes, where she runs the experiments that turn internal data into better translation experiences. She writes about what actually happens when you build AI products such as MachineTranslation.com — the numbers, the surprises, and the parts that don't go to plan.

Share: