Machine translation for subtitling: a large-scale evaluation

Bywood, L., Etchegoyhen, T., Georgakopoulou, P., Fishel, M., Jiang, J., Loenhout, G., Pozo, A., Turner, A., Volk, M. and Maucec, M. 2014. Machine translation for subtitling: a large-scale evaluation. LREC 2014, Ninth International Conference on Language Resources and Evaluation. Harpa Concert Hall and Conference Center, Reykjavik, Iceland 26 May 2014

TitleMachine translation for subtitling: a large-scale evaluation
AuthorsBywood, L., Etchegoyhen, T., Georgakopoulou, P., Fishel, M., Jiang, J., Loenhout, G., Pozo, A., Turner, A., Volk, M. and Maucec, M.
TypeConference paper
Abstract

This article describes a large-scale evaluation of the use of Statistical Machine Translation for professional subtitling. The work was carried out within the FP7 EU-funded project SUMAT and involved two rounds of evaluation: a quality evaluation and a measure of productivity gain/loss. We present the SMT systems built for the project and the corpora they were trained on, which combine professionally created and crowd-sourced data. Evaluation goals, methodology and results are presented for the eleven translation pairs that were evaluated by professional subtitlers. Overall, a majority of the machine translated subtitles received good quality ratings. The results were also positive in terms of productivity, with a global gain approaching 40%. We also evaluated the impact of applying quality estimation and filtering of poor MT output, which resulted in higher productivity gains for filtered files as opposed to fully machine-translated files. Finally, we present and discuss feedback from the subtitlers who participated in the evaluation, a key aspect for any eventual adoption of machine translation technology in professional subtitling

Keywordsstatistical machine translation, user evaluation, subtitling
Year2014
ConferenceLREC 2014, Ninth International Conference on Language Resources and Evaluation
Publisher's version
License
CC BY-SA 3.0
File Access Level
Open (open metadata and files)
Publication dates
Published26 May 2014
Journal citationpp. 46-53
Book titleProceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Web address (URL) of conference proceedingshttp://www.lrec-conf.org/proceedings/lrec2014/pdf/463_Paper.pdf

Related outputs

The W-ICAD model: Redefining museum access through the Workshop for Inclusive Co-created Audio Description
Eardley, A.F., Jones, V.E., Bywood, L., Thompson, H. and Husbands, D. 2024. The W-ICAD model: Redefining museum access through the Workshop for Inclusive Co-created Audio Description. Curator: The Museum Journal. Advanced online publication. https://doi.org/10.1111/cura.12649

Audiovisual Translators
Bywood, L. and Nikolić, K. 2024. Audiovisual Translators. in: Massey, G., Ehrensberger-Dow, M. and Angelone, E. (ed.) Handbook of the Language Industry de Gruyter Mouton.

Audiovisual Translation and Media Accessibility Training in the EMT Network
Valdez, S., Secara, A., Perez, E. and Bywood, L. 2023. Audiovisual Translation and Media Accessibility Training in the EMT Network. Journal of Audiovisual Translation. 6 (1), pp. 19-44. https://doi.org/10.47476/jat.v6i1.2023.238

______________________________________: Inclusive Football Commentary: Creating a Richer Experience for Audiences Who Cannot See
Bywood, L., Pedregosa Sanchez, I., Ginley, B and Eardley, A.F. 2023. ______________________________________: Inclusive Football Commentary: Creating a Richer Experience for Audiences Who Cannot See . in: Halder, S. and Squires, G. (ed.) Inclusion and Diversity: Communities, Practices, and Perspectives Abingdon Routledge.

Devisualizing the Museum: from Access to Inclusion
Eardley, A.F., Thompson, H., Fineman, A., Hutchinson, R., Bywood, L. and Cock, M. 2022. Devisualizing the Museum: from Access to Inclusion. Journal of Museum Education. 47 (2), pp. 150-165. https://doi.org/10.1080/10598650.2022.2077067

Audiovisual Translation
Nikolic, K. and Bywood, Lindsay 2021. Audiovisual Translation. Journal of Audiovisual Translation. 4 (1), pp. 50-70. https://doi.org/10.47476/jat.v4i1.2021.156

Lindsay Bywood interviews Carol Robertson on her experience of the early days of subtitling at the BBC
Bywood, L. 2020. Lindsay Bywood interviews Carol Robertson on her experience of the early days of subtitling at the BBC. JoSTrans - The Journal of Specialised Translation. 34.

Book review: Sanderson, John D. and Carla Botella-Tejera (eds) (2018) Focusing on Audiovisual Translation Research
Bywood, L. 2020. Book review: Sanderson, John D. and Carla Botella-Tejera (eds) (2018) Focusing on Audiovisual Translation Research. JoSTrans - The Journal of Specialised Translation. 34.

Technology and Audiovisual Translation
Bywood, L. 2020. Technology and Audiovisual Translation. in: Bogucki, Łukasz and Deckert, Mikołaj (ed.) The Palgrave Handbook of Audiovisual Translation and Media Accessibility Palgrave Macmillan. pp. 503-517

Post-Editing in Practice: Process, Product and Networks
Nunes Vieira, L., Alonso, E. and Bywood, L. 2019. Post-Editing in Practice: Process, Product and Networks. JoSTrans - The Journal of Specialised Translation. 31.

Testing the retranslation hypothesis for audiovisual translation: the films of Volker Schlöndorff subtitled into English
Bywood, L. 2019. Testing the retranslation hypothesis for audiovisual translation: the films of Volker Schlöndorff subtitled into English. Perspectives. 27 (6), pp. 815-832. https://doi.org/10.1080/0907676x.2019.1593467

Embracing the threat: machine translation as a solution for subtitling
Bywood, Lindsay, Georgakopoulou, Panayota and Etchegoyhen, Thierry 2017. Embracing the threat: machine translation as a solution for subtitling. Perspectives. 25 (3), pp. 492-508. https://doi.org/10.1080/0907676x.2017.1291695

Machine translation quality in an audiovisual context
Burchardt, A., Lommel, A., Bywood, L., Harris, K. and Popovic, M. 2016. Machine translation quality in an audiovisual context. Target. 28 (2), pp. 206-221. https://doi.org/10.1075/target.28.2.03bur

Book review: 'In translation: translators on their work and what it means' edited by Esther Allen and Susan Bernofsky (2013)
Bywood, L. 2014. Book review: 'In translation: translators on their work and what it means' edited by Esther Allen and Susan Bernofsky (2013). JoSTrans - The Journal of Specialised Translation. 21, pp. 206-208.

MT in subtitling and the rising profile of the post-editor
Georgakopoulou, P. and Bywood, L. 2014. MT in subtitling and the rising profile of the post-editor. Multilingual. 25 (1), pp. 24-28.

SUMAT: an online service for subtitling by machine translation
Bywood, L., Georgakopoulou, P., Etchegoyhen, T., Fishel, M., Jiang, J., Loenhout, G., Pozo, A., Spiliotopoulos, D., Turner, A. and Maucec, M. 2013. SUMAT: an online service for subtitling by machine translation. Machine Translation (MT) Summit XIV. ACROPOLIS Conference Centre, Nice, France 02 Sep 2013

Parallel subtitle corpora and their applications in machine translation and translatology
Bywood, L., Volk, M., Fishel, M. and Georgakopoulou, P. 2013. Parallel subtitle corpora and their applications in machine translation and translatology. Perspectives: Studies in Translatology. 21 (4), pp. 595-610. https://doi.org/10.1080/0907676X.2013.831920

Embracing the threat: machine translation as a solution
Bywood, L., Georgakopoulou, P. and Etchegoyhen, T. 2013. Embracing the threat: machine translation as a solution. Subtitling: a collective approach. University of Nottingham, Nottingham, UK 12 Jul 2013

Permalink - https://westminsterresearch.westminster.ac.uk/item/99821/machine-translation-for-subtitling-a-large-scale-evaluation


Share this

Usage statistics

750 total views
415 total downloads
These values cover views and downloads from WestminsterResearch and are for the period from September 2nd 2018, when this repository was created.