This paper presents a comparative evaluation of Large Language Models (LLMs) and Neural Machine Translation (NMT) systems for Greek-to-English translation, a relatively underexplored setting for morphologically rich languages. We evaluate a representative LLM (GPT-5-class) against two widely used NMT systems (DeepL and Google Translate) across four text domains: News, Scientific, Technical, and Literary. Our methodology combines automatic evaluation metrics (BLEU, BERTScore, and TER) with expert human assessment, enabling a multi-dimensional analysis of translation quality. Results suggest that LLM-based translation consistently achieves higher fluency and stronger semantic alignment with reference texts, particularly in literary and syntactically complex scientific domains. In contrast, NMT systems remain more reliable in technical contexts requiring precise and concise terminology. Correlation analysis indicates moderate alignment between automatic metrics and human judgments, with BLEU and BERTScore showing the strongest positive associations in this pilot setting. Overall, our findings highlight a domain-dependent trade-off between fluency and terminological precision, and suggest that no single architecture uniformly outperforms the other. While limited in scale, this study provides preliminary evidence on the behavior of LLMs versus NMT systems for Greek translation and underscores the importance of domain-aware evaluation in machine translation research.
Dr. Despoina Mouratidis is a Researcher and Academic Tutor at the Ionian University, Greece. She holds a B.Sc. in Mathematics from the University of Ioannina, an M.Sc. in Computer Science (Informatics and Humanities), and a Ph.D. in Informatics (2021) from the Ionian University. Her doctoral thesis focused on the multimodal evaluation of stochastic machine translation. She has also successfully completed her postdoctoral research on "Machine Learning in Semantic Analysis of Cultural Data." Currently, Dr. Mouratidis has a broad teaching portfolio. She serves as a Tutor in the postgraduate Data Science and Machine Learning program at the Hellenic Open University and as a Contract Tutor at the Department of Informatics, Ionian University. Additionally, she delivers guest lectures at the Department of Foreign Languages, Translation and Interpreting, while also working as a mathematician in secondary education. As a member of the Humanistic & Social Informatics Laboratory, her research interests encompass natural language processing, deep learning architectures, fake news detection, and the evaluation of machine translation systems. She has actively contributed to major European and national research projects (Horizon 2020 TraMOOC, ENIRISST+). Dr. Mouratidis has authored numerous publications—receiving a Best Paper Award in 2021—and regularly serves on the program and organizing committees for AI and NLP conferences, including SETN and RANLP.
George Mastrogiannis is a recent graduate of the Department of Informatics at the Ionian University. His academic interests focus on Artificial Intelligence and their applications in Natural Language Processing and Machine Translation. During his studies, he gained practical experience in data analysis and evaluating large language models.
