Is machine translation the greatest thing since sliced bread? : human and machine translation of food-related idioms

Abstract

This study examines the translation of food-related idiomatic expressions (IEs) from English into Brazilian Portuguese, by comparing the performance of trainee translators and machine translation (MT) systems operating under two distinct paradigms: neural machine translation (NMT) and large language models (LLMs). Contexts containing IEs are extracted from the Corpus of Contemporary American English and translated by undergraduates, and by four AI-based systems: Google Translate and DeepL (NMT-based), and ChatGPT-4 and Gemini 2.5 Pro (LLM-based). All translations are qualitatively evaluated for idiomaticity and conventionality using corpus evidence and classified into four categories: idiomatic and conventional, non-idiomatic but conventional, idiomatic but non-conventional, and inaccurate. When human translation (HT) and MT are compared overall, HT produces a higher proportion of idiomatic and conventional equivalents. While both approaches show similar proportions of non-idiomatic but conventional and idiomatic but non-conventional renderings, inaccurate translations are almost twice as frequent in MT. When the performance of HT is compared with that of the LLM-based systems, the results indicate that the latter, largely driven by Gemini’s performance, slightly outperform students in producing appropriate translations. A more fine-grained analysis, however, reveals important differences within MT: NMT systems perform markedly worse, generating few idiomatic and conventional equivalents and a high proportion of inaccurate translations. Statistical analyses confirmed a significant association between translator type (human, NMT, or LLM) and translation quality (idiomatic and conventional, non-idiomatic but conventional, idiomatic but non-conventional, or inaccurate) across the four evaluation categories. This association is driven primarily by the poorer performance of NMT systems, whereas no significant difference is found between human translators and LLMs. These results highlight the importance of integrating human expertise with advanced MT tools in idiom-rich translation tasks. They also highlight the need for translator training that emphasizes idiomaticity, conventional usage, and critical evaluation of machine-generated output, preparing translators to work effectively in increasingly hybrid translation environments.