A TRANSLATION PRODUCT ANALYSIS OF CHATGPT BASED ON THE ERROR TAXONOMY BY VILAR ET AL. (2006): PROBLEMS AND SOLUTIONS
DOI:
https://doi.org/10.35473/pho.v8i1.4358Keywords:
Translation error, Vilar et al. taxonomy, ChatGPTAbstract
This study aims to analyze the translation product of ChatGPT by identifying the errors based on the error taxonomy proposed by Vilar et al. (2006). As AI-powered machine translation (MT) becomes increasingly prevalent, evaluating the quality of the output and identifying specific weaknesses is crucial. This research used a descriptive qualitative approach to examine 30 English-to-Indonesian translation outputs generated by ChatGPT. The analysis was guided by Vilar et al.'s error categories: missing words, word order, incorrect words, unknown words, and punctuation. Results show that while ChatGPT performs well in terms of fluency, some errors still occur in the form of incorrect words (33,3%), word orders (23,3%), missing words (20%), and punctuation (13,3%) and unknown words (10%). It can be concluded that the most frequent errors are incorrect words, word orders, and missing words. Proposed solution involves post-editing strategies by adapting translation techniques by Molina and Albir in order to make the translation result more accurate, natural, and aligned with the norms of the target language
References
Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural machine translation by
jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
Bentivogli, L., Cettolo, M., Girardi, C., & Federico, M. (2016). Neural versus
phrase-based machine translation quality: A case study. EMNLP.
Brown, T. et al. (2020). Language Models are Few-Shot Learners. NeurIPS.
Dwyer, T. (2017). Translation and the machine: Technology, meaning, ethics.
Routledge.
Hutchins, W. J. (2005). The history of machine translation in a nutshell.
Retrieved from http://www.hutchinsweb.me.uk/Nutshell-2005.pdf
Koerner, E. F. K., & Asher, R. E. (Eds.). (1995). Concise History of the Language
Sciences: From the Sumerians to the Cognitivists. Pergamon
Lin, L.-C., & Chien, C.-H. (2009). An exploration of translation students’
learning styles and strategies. Asian EFL Journal, 11(1), 134–154
Molina, L., & Hurtado Albir, A. (2002). Translation techniques revisited: A
dynamic and functionalist approach. Meta: Translators’ Journal, 47(4),
498–512. https://doi.org/10.7202/008033ar
OpenAI. (2023). GPT-4 technical report. https://openai.com/research/gpt-4
Purwarianti, A., & Pribadi, F. (2020). Neural Machine Translation for
Indonesian-English using Transformer. IEEE ICCED.
Purnamasari, I., et al. (2022). Assessing Indonesian-English Machine
Translation Performance. Lingua, 18(1), 45–56
Suryawinata, A., et al. (2021). Exploring Indonesian-English Neural Machine
Translation. arXiv preprint arXiv:2106.00129.
Vilar, D., Xu, J., D’Haro, L. F., & Ney, H. (2006). Error analysis of statistical
machine translation output. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC).
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ...
& Polosukhin, I. (2017). Attention is all you need. In Advances in
neural information processing systems (pp. 5998-6008).
Wu, X., Zhang, Y., & Yang, H. (2023). Evaluating the fluency and fidelity of
ChatGPT translations. Journal of Computational Linguistics and
Translation Studies, 5(2), 66–81.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright notice:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License-ShareAlike 4.0 (CC-BY-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access)
