Parallel corpus alignment at the document, sentence and vocabulary levels

• Autores: Rogelio Nazar
• Localización: Procesamiento del lenguaje natural, ISSN 1135-5948, Nº. 47, 2011, págs. 129-136
• Idioma: inglés
• Enlaces
  • Texto completo
• Resumen
  • español

    Este artículo presenta un algoritmo independiente de lengua para la alineación de corpus paralelo a nivel de documento, de oración y de vocabulario, tomando como única fuente de información el mismo corpus a alinear. La entrada es un conjunto de documentos escritos en dos lenguas desconocidas A y B, donde cada documento en la lengua A tiene su correspondiente traducción a la lengua B. El problema consiste en: 1) dividir el conjunto de documentos en las dos lenguas; 2) alinear a nivel de documento: determinar qué documento en la lengua A es el original (o la traducción) de cada documento en la lengua B; 3) alinear a nivel de oración: determinar qué oración en el original corresponde a qué oración en la traducción y 4) alinear a nivel del vocabulario: determinar qué palabra en una lengua es equivalente a cada palabra en la traducción. El algoritmo es iterativo, ya que utiliza el vocabulario bilingüe resultante para realinear el corpus. La evaluación en inglés, castellano y francés muestra resultados competitivos en todos los niveles.

  • English

    This paper presents a language independent algorithm for the alignment of parallel corpora at the document, sentence and vocabulary levels using the to-be aligned corpus itself as the only source of information. The input is a set of documents written in two unknown languages A and B, where every document in language A has its corresponding translation into language B. The problem thus consists of: 1) dividing the set of documents in the two languages; 2) aligning at the document level to determine which document in language A is the original (or translation) of each document in language B; 3) aligning at the sentence level to determine which sentence in the original corresponds to each sentence in the translation and 4) aligning at the vocabulary level to determine which word in one language is equivalent to each word in the translation. The algorithm is iterative, using the resulting bilingual vocabulary to re-align the corpus. Evaluation figures in English, Spanish and French show competitive results at all levels of the alignment.

• Referencias bibliográficas
  • Braune, F. and A. Fraser. 2010. Improved unsupervised sentence alignment for symmetrical and asymmetrical parallel corpora. In Proc. of the...
  • Brown, P., V. Della Pietra, S. Della Pietra, and R. Mercer. 1993. The mathematics of statistical machine translation: parameter estimation....
  • Brown, P.F., J. C. Lai, and R. L. Mercer. 1991. Aligning sentences in parallel corpora. In Proc. of the 29th Annual Meeting of the ACL, pages...
  • Choueka, Y., E. Conley, and I. Dagan. 2000. A comprehensive bilingual word alignment system. application to disparate languages: Hebrew and...
  • Church, K.W. 1993. Charalign: a program for aligning parallel texts at the character level. In Proc. of the 31st Annual Meeting of the ACL,...
  • Fung, P. 1995. Compiling bilingual lexicon entries from a nonparallel english chinese corpus. In Proc. of the Third Workshop on Very Large...
  • Gale, W. A. and K.W. Church. 1991a. Identifying word correspondences in parallel texts. In Proc. of the DARPA Workshop on Speech and Natural...
  • Gale, W. A. and K.W. Church. 1991b. A program for aligning sentences in bilingual corpora. In Proc. of the 29th Annual Meeting of the ACL,...
  • Hiemstra, D. 1998. Multilingual domain modeling in twenty-one: automatic creation of a bi-directional translation lexicon from a parallel...
  • Kay, M. and M. R¨oschesein. 1988. Texttranslation alignment. Technical report, Xerox Palo Alto Research Center. McEnery, A. M. and M. P. Oakes....
  • Melamed, D. 2000. Pattern recognition for mapping bitext correspondence. In Parallel Text Processing: Alignment and Use of Translation Corpora....
  • Moore, R. 2002. Fast and accurate sentence alignment of bilingual corpora. In Proc. of the 5th Conference of the Association for Machine Translation...
  • Och, F. J. and H. Ney. 2000. Improved statistical alignment models. pages 440–447, Hongkong, China, October.
  • Rapp, R. 1999. Automatic identification of word translations from unrelated english and german corpora. In Proc. of 37th Annual Meeting of...
  • Resnik, P. 1999. Mining the web for bilingual text. In Proc. of 37th Annual Meeting of the ACL, pages 527–532.
  • Simoes, A. and J. Almeida. 2003. Natools - a statistical word aligner workbench. Procesamiento del Lenguaje Natural, 31:217–226, September.
  • Steinberger, R., B. Pouliquen, A. Widiger, C. Ignat, T. Erjavec, D. Tufis, and D. Varga. 2006. The jrc-acquis: A multilingual aligned parallel...
  • Tiedemann, J. 2006. Isa ica - two web interfaces for interactive alignment of bitexts. In Proc. of LREC 2006, Genova (Italy).
  • Varga, D., L. Németh, P. Halácsy, A. Kornai, V. Trón, and V. Nagy. 2005. Parallel corpora for medium density languages. In Proc. of the RANLP...
  • Véronis, J. 2000. From the rosetta stone to the information society: A survey of parallel text processing. In Parallel Text Processing: Alignment...