Overview of QuALES at IberLEF 2022:: Question Answering Learning from Examples in Spanish

• Autores: Aiala Rosá, Luis Chiruzzo, Lucía Bouza, Alina Dragonetti, Santiago Castro, Mathias Etcheverry, Santiago Góngora, Santiago Goycoechea, Juan Machado, Guillermo Moncecchi, Juan José Prada, Dina Wonsever
• Localización: Procesamiento del lenguaje natural, ISSN 1135-5948, Nº. 69, 2022, págs. 273-280
• Idioma: inglés
• Títulos paralelos:
  • Overview de QuALES en IberLEF 2022: Preguntas y Respuestas Automáticas sobre Ejemplos en Español
• Enlaces
  • Texto completo
• Resumen
  • español

    Presentamos los resultados de la tarea QuALES, que aborda el problema de Búsqueda de Respuestas extractiva a partir de textos. Tanto para entrenamiento como para evaluación utilizamos el corpus QuALES, un corpus de noticias de medios uruguayos sobre la pandemia por Covid-19 y temas relacionados. Describimos los sistemas desarrollados por siete participantes, todos ellos basados en diferentes modelos de lenguaje tipo BERT. Los mejores resultados se obtuvieron usando el modelo RoBERTa multilingüe preentrenado con SQUAD-Es-V2, con una fine tuning sobre el corpus QuALES.

  • English

    We present the results of the QuALES task, which addresses the problem of Extractive Question Answering from texts. For both training and evaluation we use the QuALES corpus, a corpus of Uruguayan media news about the Covid-19 pandemic and related topics. We describe the systems developed by seven participants, all of them based on different BERT-like language models. The best results were obtained using the multilingual RoBERTa model pre-trained with SQUAD-Es-V2, with a fine tuning on the QuALES corpus.

• Referencias bibliográficas
  • Baudis, P. and J. Sedivy. 2015. Modeling of the question answering task in the yodaqa system. In J. Mothe, J. Savoy, J. Kamps, K. Pinel-Sauvagnat,...
  • Carrino, C. P., M. R. Costa-jussa, and J. A. Fonollosa. 2019. Automatic spanish translation of the squad dataset for multilingual question...
  • Cañete, J., G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, and J. Perez. 2020. Spanish pre-trained bert model and evaluation data. Pml4dc at...
  • Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding....
  • Gutierrez Fandiño, A., J. Armengol Estape, M. Pamies, J. Llop Palao, J. Silveira Ocampo, C. Pio Carrino, C. Armentano Oller, C. Rodriguez...
  • Joshi, M., E. Choi, D. S. Weld, and L. Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension.
  • Jurafsky, D. and J. H. Martin. 2021. Speech and language processing. 3rd edition draft. US: Prentice Hall.
  • Kwiatkowski, T., J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, et al. 2019a....
  • Kwiatkowski, T., J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, M. Kelcey, J. Devlin, K. Lee, K....
  • Lan, Z., M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations....
  • Le, Q. and T. Mikolov. 2014. Distributed representations of sentences and documents. In International conference on machine learning, pages...
  • LeCun, Y., Y. Bengio, and G. Hinton. 2015. Deep learning. nature, 521(7553):436– 444.
  • Manning, C., P. Raghavan, and H. Schütze. 2010. Introduction to information retrieval. Natural Language Engineering, 16(1):100–103.
  • Min, S., J. Boyd-Graber, C. Alberti, D. Chen, E. Choi, M. Collins, K. Guu, H. Hajishirzi, K. Lee, J. Palomaki, et al. 2021. Neurips 2020 efficientqa...
  • Min, S., V. Zhong, R. Socher, and C. Xiong. 2018. Efficient and robust question answering from minimal context over documents. arXiv preprint...
  • Maximo, S. 2022. Supervised domain adaptation for extractive question answering in spanish.
  • Nakov, P., D. Hoogeveen, L. Marquez, A. Moschitti, H. Mubarak, T. Baldwin, and K. Verspoor. 2019a. Semeval-2017 task 3: Community question...
  • Nakov, P., L. Marquez, W. Magdy, A. Moschitti, J. Glass, and B. Randeree. 2019b. Semeval-2015 task 3: Answer selection in community question...
  • Peñas, A., P. Forner, A. Rodrigo, R. Sutclif- fe, C. Forascu, and C. Mota. 2010. Overview of respubliqa 2010: Question answering evaluation...
  • Peñas, A., P. Forner, R. Sutcliffe, A. Rodri- go, C. Forascu, I. Alegria, D. Giampiccolo, N. Moreau, and P. Osenova. 2009. Overview of respubliqa...
  • Rajpurkar, P., J. Zhang, K. Lopyrev, and P. Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
  • Reddy, S., D. Chen, and C. D. Manning. 2019. Coqa: A conversational question answering challenge. Transactions of the Association for Computational...
  • Rodrigo, A. and A. Peñas. 2022. Uned@quales 2022: Testing the performance of transformer-based language models for spanish question-answering.
  • Seo, M., A. Kembhavi, A. Farhadi, and H. Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. arXiv preprint arXiv:1611.01603.
  • Trischler, A., T. Wang, X. Yuan, J. Harris, A. Sordoni, P. Bachman, and K. Suleman. 2016. Newsqa: A machine comprehension dataset. arXiv preprint...
  • Vaca-Serrano, A. 2022. Adversarial question answering in spanish with transformer models.
  • Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L . Kaiser, and I. Polosukhin. 2017. Attention is all you need. Advances...
  • Xiong, C., V. Zhong, and R. Socher. 2017. Dcn+: Mixed objective and deep residual coattention for question answering. arXiv preprint arXiv:1711.00106.
  • Yu, L., K. M. Hermann, P. Blunsom, and S. Pulman. 2014. Deep learning for answer sentence selection. arXiv preprint arXiv:1412.1632.