The impact of using text-to-speech (TTS) in post-editing machine translation (PEMT) workflows on translators’ cognitive effort, productivity, quality, and perceptions

• Dragoș Ciobanu ^[1] ; Miguel Rios ^[1] ; Alina Secară ^[1] ; Justus Brockmann ^[1] ; Raluca-Maria Chereji ^[1] ; Claudia Plieseis ^[1]
  1. [1] University of Vienna

     University of Vienna

     Innere Stadt, Austria

     
• Localización: Revista tradumàtica: traducció i tecnologies de la informació i la comunicació, ISSN-e 1578-7559, Nº. 22, 2024 (Ejemplar dedicado a: Interacció persona-ordinador en traducció i interpretació: programes i aplicacions), págs. 323-354
• Idioma: inglés
• DOI: 10.5565/rev/tradumatica.394
• Títulos paralelos:
  • El impacto del uso de texto a voz (TTS) en los flujos de trabajo de posedición de traducción automática (PEMT) sobre el esfuerzo cognitivo, la productividad, la calidad y las percepciones de los traductores.
  • L'impacte de l'ús de text a veu (TTS) en els fluxos de treball de postedició de traducció automàtica (PEMT) sobre l'esforç cognitiu, la productivitat, la qualitat i les percepcions dels traductors.
• Enlaces
  • Texto completo  1    2  
• Resumen
  • español

    Este experimento de seguimiento ocular investiga el impacto del uso de texto a voz (TTS) junto con la experiencia de traducción y posedición de traducción automática (PEMT) en el esfuerzo cognitivo, la productividad, la calidad y las percepciones de los traductores durante la posedición de traducción automática del inglés al alemán. El TTS redujo sustancialmente la velocidad de la PEMT y la duración media de las fijaciones en el texto meta, además de aumentar el número de fijaciones y el tiempo de permanencia en los textos origen y meta. El TTS no tuvo un efecto significativo en la distancia de edición ni en la calidad del texto meta. La experiencia tampoco mostró un impacto significativo. 14 de los 21 participantes prefirieron realizar la posedición con TTS.

  • català

    Aquest experiment de seguiment ocular investiga l’impacte de l’ús de text a veu (TTS) juntament amb l’experiència de traducció i postedició de traducció automàtica (PEMT) en l’esforç cognitiu, la productivitat, la qualitat i les percepcions dels traductors durant la postedició de traducció automàtica de l’anglès a l’alemany. El TTS ha reduït substancialment la velocitat de la PEMT i la durada mitja de les fixacions en el text meta, a banda d’augmentar el nombre de fixacions i el temps de permanència en els textos origen i meta. El TTS no ha tingut un efecte significatiu en la distància d’edició ni en la qualitat del text meta. L’experiència tampoc ha mostrat un impacte significatiu. 14 dels 21 participants han preferit realitzar la postedició amb TTS.

  • English

    This eye-tracking experiment investigates the impact of text-to-speech (TTS) alongside translation and post-editing machine translation (PEMT) experience on translators’ cognitive effort, productivity, quality, and perceptions during PEMT from English into German. TTS substantially lowered PEMT speed and target text mean fixation duration, and increased source and target text fixation counts and dwell time. TTS had no substantial effect on edit distance or target text quality. Experience had no substantial effect. 14 out of 21 participants preferred post-editing with TTS.

• Referencias bibliográficas
  • Allain, J.-F. (2010). Repenser la révision: Défense et illustration de la relecture croisée. Traduire, 223, 114–120. https://doi.org/10.4000/traduire.316....
  • Báez M. C. T. (2018). Machine Translation and Postediting: Impact of Training and Directionality on Quality and Productivity. Tradumàtica...
  • Balling, L. W. (2008). A brief introduction to regression designs and mixed-effects modelling by a recent convert. In S. Göpferich, A. L....
  • Briva-Iglesias, V., O’Brien, S., & Cowan, B. R. (2023). The impact of traditional and interactive post-editing on Machine Translation...
  • Brockmann, J., Secară, A., & Ciobanu, D. (forthcoming). The use of speech technologies and machine translation in institutional translation...
  • Bürkner, P.-C. (2017). brms: An R Package for Bayesian Multilevel Models Using Stan. Journal of Statistical Software, 80(1), 1–28. https://doi.org/10.18637/jss.v080.i01....
  • Cadwell, P., O’Brien, S., & Teixeira, C. S. C. (2018). Resistance and accommodation: Factors for the (non-) adoption of machine translation...
  • Canfora, C., & Ottmann, A. (2020). Risks in neural machine translation. Translation Spaces, 9(1), 57–77.
  • Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017)....
  • Castilho, S. (2016). Measuring acceptability of machine translated enterprise content [PhD Thesis]. Dublin City University.
  • Chen, F., & Jokinen, K. (2010). Speech technology: Theory and applications. Springer.
  • Chereji, R. (2024). What makes a medical translator? A survey on medical translators’ profiles, work-related challenges and use of Computer-Assisted...
  • Ciobanu, D. (2016). Automatic speech recognition in the professional translation process. Translation Spaces, 5(1), 124–144. https://doi.org/10.1075/ts.5.1.07cio....
  • Ciobanu, D., Ragni, V., & Secară, A. (2019). Speech Synthesis in the Translation Revision Process: Evidence from Error Analysis, Questionnaire,...
  • Conklin, K., Pellicer-Sánchez, A., & Carrol, G. (2018). Eye-Tracking: A Guide for Applied Linguistics Research. Cambridge University Press....
  • Daems, J., Carl, M., Vandepitte, S., Hartsuiker, R., & Macken, L. (2016). The Effectiveness of Consulting External Resources During Translation...
  • Daems, J., Vandepitte, S., Hartsuiker, R. J., & Macken, L. (2017a). Translation Methods and Experience: A Comparative Analysis of Human...
  • Daems, J., Vandepitte, S., Hartsuiker, R. J., & Macken, L. (2017b). Identifying the Machine Translation Error Types with the Greatest...
  • Dai, G. & Liu, S. (2024). Towards predicting post-editing effort with source text readability: An investigation for english-chinese machine...
  • Damerau, F. J. (1964). A technique for computer detection and correction of spelling errors. Commun. ACM, 7, 171–176.
  • De Almeida, G., & O’Brien, S. (2010). Analysing post-editing performance: Correlations with years of translation experience. Proceedings...
  • do Carmo, F. (2021). Editing Actions: A Missing Link Between Translation Process Research and Machine Translation Research. In M. Carl (Ed.),...
  • ELIA, EMT, EUATC, FIT EUROPE, GALA, LIND, & WOMEN IN LOCALIZATION. (2024). 2024 European Language Industry Survey. Trends, expectations...
  • EyeLink Data Viewer (4.2.1). (2021). [Computer software]. SR Research Ltd.
  • Farrell, M. (2023). Do translators use machine translation and if so, how? Results of a survey among professional translators. In J. Moorkens...
  • Ferreira, A., Schwieter, J. W., Gottardo, A., & Jones, J. (2016). Cognitive effort in direct and inverse translation performance: Insight...
  • GALA. (2024). AI & Automation Barometer Report. https://user.gala-global.org/files/ed13f63f-8c94-4a89-becf-7d982e5fe985.bin. [Accessed:...
  • Gaspari, F., Toral, A., Naskar, S. K., Groves, D., & Way, A. (2014). Perception vs. reality: Measuring machine translation post-editing...
  • Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2004). Bayesian Data Analysis (2nd ed.). Chapman and Hall/CRC.
  • Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. In Data Analysis Using Regression and...
  • Gius, E., Meister, J. C., Meister, M., Petris, M., Bruck, C., Jacke, J., Schumacher, M., Gersthofer, D., Flüh, M., & Horstmann, J. (2022)....
  • Heiss, A. (2021). Bayesian statistics resources. https://evalf21.classes.andrewheiss.com/resource/bayes/. [Accessed: 20241218].
  • Herbig, N., Düwel, T., Pal, S., Meladaki, K., Monshizadeh, M., Krüger, A., & van Genabith, J. (2020). MMPE: A Multi-Modal Interface for...
  • Holmqvist, K., & Andersson, R. (2017). Eye-tracking: A comprehensive guide to methods, paradigms and measures (2nd ed.). Lund Eye Tracking...
  • Hvelplund, K. T. (2014). Eye tracking and the translation process: Reflections on the analysis and interpretation of eye-tracking data. MonTI....
  • Hvelplund, K. T. (2017). Translators’ Use of Digital Resources during Translation. HERMES - Journal of Language and Communication in Business,...
  • Hvelplund, K. T. (2019). Digital resources in the translation process – attention, cognitive effort and processing flow. Perspectives, 27(4),...
  • ISO - International Organization for Standardization. (2017). ISO 18587:2017 Translation services – Post-editing of machine translation output...
  • ISO - International Organization for Standardization. (2024). ISO 5060:2024 Translation services – Evaluation of translation output – General...
  • Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329–354....
  • Koponen, M. (2012). Comparing human perceptions of post-editing effort with post-editing operations. Proceedings of the Seventh Workshop on...
  • Kruschke, J. K. (2018). Rejecting or Accepting Parameter Values in Bayesian Estimation. Advances in Methods and Practices in Psychological...
  • Kruschke, J. K. (2021). Bayesian Analysis Reporting Guidelines. Nature Human Behaviour, 5(10), 1282–1291. https://doi.org/10.1038/s41562-021-01177-7....
  • Levenshtein, V. I. (1965). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics. Doklady, 10, 707–710.
  • Liu, Y., & Zheng, B. (2022). Comparability of difficulty levels of translation tasks in CET-6 parallel test forms: Evidence from product...
  • Liyanapathirana, J., Bouillon, P., & Mesa-Lao, B. (2019). Surveying the potential of using speech technologies for post-editing purposes...
  • Lommel, A., Uszkoreit, H., & Burchardt, A. (2014). Multidimensional Quality Metrics (MQM): A Framework for Declaring and Describing Translation...
  • Macken, L., Prou, D., & Tezcan, A. (2020). Quantifying the Effect of Machine Translation in a High-Quality Human Translation Production...
  • McElreath, R. (2016). Statistical rethinking: A Bayesian course with examples in R and Stan. CRC Press/Taylor & Francis Group.
  • Meidert, U., Neumann, S., Ehrensberger-Dow, M., & Becker, H. (2016). Physical Ergonomics at Translators’ Workplaces: Findings from Ergonomic...
  • Moorkens, J., O’Brien, S., da Silva, I. A. L., de Lima Fonseca, N. B., & Alves, F. (2015). Correlations of perceived post-editing effort...
  • Moorkens, J., & O’Brien, S. (2017). Assessing user interface needs of post-editors of machine translation. In D. Kenny (Ed.), Human Issues...
  • Moorkens, J. (2018). Eye tracking as a measure of cognitive effort for post-editing of machine translation. In C. Walker & F. M. Federici...
  • O’Brien, S. (2009). Eye-tracking in translation process research: Methodological challenges and solutions. In I. M. Mees, S. Göpferich, &...
  • Orlando, M., Liao, S., & Kruger, J-L. (2024). Translation and Interpreting technologies and their impact on the industry. Macquarie University.
  • Parra Escartín, C., & Arcedillo, M. (2015). Machine translation evaluation made fuzzier: A study on post-editing productivity and evaluation...
  • R Core Team. (2023). R: A Language and Environment for Statistical Computing [Computer software]. R Foundation for Statistical Computing....
  • Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372–422. https://doi.org/10.1037/0033-2909.124.3.372....
  • Rios, M., Brockmann, J., Wiesinger, C., Chereji, R.-M., Secară, A., & Ciobanu, D. (2024). Bayesian Hierarchical Modelling for Analysing...
  • Rochard, M. (2021). Michel Rochard—Linguiste du mois de juin 2021 (A. Bulger, Interviewer) [Interview]. https://www.le-mot-juste-en-anglais.com/2021/06/le-linguiste-du-mois-juillet-2021-michel-rochard-est-dipl%C3%B4m%C3%A9-de-traduction-et-dinterpr%C3%A9tation-de-liaison-du-fas-germ.html....
  • Scocchera, G. (2017). Translation Revision as Rereading: Different Aspects of the Translator’s and Reviser’s Approach to the Revision Process....
  • Silva, B. B., Orrego-Carmona, D., & Szarkowska, A. (2022). Using linear mixed models to analyze data from eye-tracking research on subtitling....
  • SR Research WebLink (2.1.1). (2021). [Computer software]. SR Research Ltd.
  • Tan, L. (2023). Speech translation. In S.-W. Chan (Ed.), Routledge Encyclopedia of Translation Technology (2nd ed., pp. 726–737). Routledge....
  • Taylor, P. (2009). Text-to-Speech Synthesis. Cambridge University Press. https://doi.org/10.1017/CBO9780511816338. [Accessed: 20241218].
  • Temizöz, Ö. (2016). Postediting machine translation output: Subject matter experts versus professional translators. Perspectives, 24(4), 646–665....
  • Vieira, L. N. (2014). Indices of cognitive effort in machine translation post-editing. Machine Translation, 28(3), 187–216. https://doi.org/10.1007/s10590-014-9156-x....
  • Wang, Y., & Daghigh, A. J. (2023). Effect of text type on translation effort in human translation and neural machine translation post-editing...
  • Wiesinger, C., Brockmann, J., Secară, A., & Ciobanu, D. (2022). Speech-enabled machine translation post-editing in the context of translator...