October 28, 2026
If you translate into English because you assume AI is at its strongest there, our data says that assumption deserves a second look for some language pairs. Teams often treat English as the safe direction, and published benchmarks give them good reason to. When I started reading our quality scores by direction, I expected to see the same pattern.
We looked at 28 days of MachineTranslation.com quality scores, from July 12 to August 8, 2026. For 32 of the 44 language pairs we could compare in both directions, translations out of English scored higher than translations into English. The typical gap is small, and a few pairs show a very large one. Published research reports the opposite direction, so this article covers what we measured, what might explain the disagreement, and what I cannot yet confirm.
It depends on the dataset, because published benchmarks say yes while our own scores mostly say no. The FLORES-101 evaluation benchmark (2021) found translation into English strong across the board and translation out of English much worse, and a 2026 study of Kokborok, a language of northeast India, reported the same direction gap across every system it tested.
Kokborok is a low-resource language, meaning one with little digital text available for training an AI model. The usual explanation is that English makes up a large share of the text on the web, so models see more of it and write it more fluently than they write languages with less text. A 2025 paper in Natural Language Processing, published by Cambridge University Press, turns that reasoning into a practical rule: when translation is necessary, go from the weaker language to the stronger one.
That reasoning is sound, and I treat the published result as a strong prior. A contradicting result from our data needs a careful explanation, and the sections below work toward one.
Out of English scored higher for 32 of the 44 pairs we could compare in both directions. Out-of-English translations scored 9.100 and into-English translations scored 8.908. The share of outputs below 8 was 6.8% out of English and 10.0% into English.
Most scores on our scale sit in the 9s, so the average hides a lot. In my opinion the share of outputs below 8 is the more practical measure of bad output, and the table and chart below use it. The typical pair differs by only 0.11 points (the median), while the mean gap is 0.19, which tells me a handful of pairs pull the average. In the other 12 pairs, translation into English scored higher, including Czech and Romanian.
| Language | Into English: average score | Into English: % below 8 | Out of English: average score | Out of English: % below 8 |
|---|---|---|---|---|
| Swedish | 7.674 | 47.0% | 9.011 | 11.7% |
| Amharic | 7.958 | 46.3% | 8.880 | 18.1% |
| Yoruba | 8.367 | 27.7% | 9.038 | 7.8% |
| Albanian | 8.373 | 27.3% | 9.257 | 4.1% |
| Nepali | 8.363 | 27.2% | 9.281 | 2.8% |
| Czech | 9.163 | 4.6% | 8.833 | 16.8% |
| Romanian | 9.081 | 8.3% | 8.974 | 14.2% |
Share of translations scoring below 8, by direction, five language pairs. MachineTranslation.com quality scores, July 12 to August 8, 2026.
I see four possible reasons, and I cannot yet say which one explains the gap: how the score is produced, which text is translated, which languages are in the sample, and which AI models ran on each translation. Each one would change what our result means.
My working view is that automated scores are least trustworthy in the languages where the scorer has seen the least text. I cannot prove that with this dataset, and I could be wrong on this.
Yes, because among the 85 pairs measured in both directions from July 12 to August 8, 2026, the pair with the less widely used language as the source scored lower in 11 of the 12 largest gaps. The exception was Arabic to Bengali, where Bengali as the target scored lower.
Amharic to Arabic averaged 7.489, while Arabic to Amharic averaged 9.183. Quechua to Spanish averaged 7.148, against 8.723 for Spanish to Quechua. Burmese to Chinese averaged 7.831, and Chinese to Burmese averaged 9.047. The lowest averages in our window were Quechua to Spanish (7.148), Amharic to Arabic (7.489) and Fijian to English (7.541), and in each of them the less widely used language was the source. Deciding which language is less widely used was my own judgment, made pair by pair, and a stricter definition could change the count by one or two.
As far as I understand, two readings fit this pattern. A model that has seen little Quechua may misread a Quechua source, and that misreading shows up as a low score. A scorer that has seen little Quechua may also fail to catch an error in Quechua output, which would inflate scores in the other direction. The two readings call for different advice, and the next section gives the steps that hold under either one.
Do not rely on a score alone. Compare what several independent AI models produce, look at where they disagree, and have a speaker of the target language review anything that matters. Those steps hold under either reading of our data.
This is what reliability means in practice for us: agreement between independent AI models, plus a human check where it counts. If the language you are translating into is rare, I would have a speaker of that language check the output, and I would not treat a high score as that check.
It depends on the language pair, because published benchmarks report stronger results into English, while our 28-day scores were higher out of English for 32 of 44 pairs. Treat direction as one more variable to check, especially when a less widely used language is involved.
AI models see far less text in some languages during training, so they handle them less reliably. In our 28-day scores, 47.0% of Swedish to English outputs scored below 8, against 11.7% for English to Swedish. Automated scoring can add its own bias, so I read these numbers as a warning.
Among pairs we measured in July and August 2026, the lowest averages were Quechua to Spanish at 7.15, Amharic to Arabic at 7.49, and Fijian to English at 7.54. In each case the less widely used language was the source. A pair needed at least 50 translations in the window to count.
It depends on the language, because published research favors translating from a weaker language into a stronger one, while our scores for rarely used target languages ran high and may hide errors. For anything important, have a speaker of the target language review the output.

By Rachelle Garcia
Connect on LinkedInRachelle leads product and AI at Tomedes, where she runs the experiments that turn internal data into better translation experiences. She writes about what actually happens when you build AI products such as MachineTranslation.com — the numbers, the surprises, and the parts that don't go to plan.