Overview of ReCoRES at IberLEF 2022:: Reading Comprehension and Reasoning Explanation for Spanish

• Autores: Marco Antonio Sobrevilla Cabezudo, Diego Diestra, Rodrigo López, Erasmo Gómez, Arturo Oncevay, Fernando Alva Manchego
• Localización: Procesamiento del lenguaje natural, ISSN 1135-5948, Nº. 69, 2022, págs. 281-287
• Idioma: inglés
• Títulos paralelos:
  • Overview de ReCoRES en IberLEF 2022:: Comprensión de Lectura y Explicación de Razonamiento en Español
• Enlaces
  • Texto completo
• Dialnet Métricas: 1 Cita
• Resumen
  • español

    Este artículo presenta la tarea ReCoRES, organizada en IberLEF 2022, en el marco de la 38 edición de la Conferencia Internacional de la Sociedad Española para el Procesamiento del Lenguaje Natural. El objetivo de esta tarea es promover la tarea de Comprensión de Lectura y Razonamiento Verbal. Esta tarea es dividida en dos sub-tareas: (1) la identificación de la alternativa correcta en preguntas de comprensión de lectura y (2) la generación del razonamiento usado para seleccionar una alternativa. En general, 3 equipos participaron de este evento proponiendo mayormente modelos neuronales basados en transformers con algunas estrategias adicionales. Los resultados de este evento así como aprendizajes y algunos desafíos son presentados, abriendo un abanico de posibilidades como trabajos futuros.

  • English

    This paper presents the ReCoRES task, organized at IberLEF 2022, within the framework of the 38th edition of the International Conference of the Spanish Society for Natural Language Processing. The main goal of this shared-task is to promote the task of Reading Comprehension and Verbal Reasoning. This task is divided into two sub-tasks: (1) identifying the correct alternative in reading comprehension questions and (2) generating the reasoning used to select an alternative. In general, 3 teams participated in this event, mainly proposing transformer-based neural models in conjunction with additional strategies. The results of this event, insights and some challenges are presented, opening a range of possibilities for future work.

• Referencias bibliográficas
  • Baudis, P. and J. Sedivy. 2015. Modeling of the question answering task in the yodaqa system. In J. Mothe, J. Savoy, J. Kamps, K. Pinel-Sauvagnat,...
  • Carrino, C. P., M. R. Costa-jussa, and J. A. Fonollosa. 2019. Automatic spanish translation of the squad dataset for multilingual question...
  • Cañete, J., G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, and J. Perez. 2020. Spanish pre-trained bert model and evaluation data. Pml4dc at...
  • Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding....
  • Gutierrez Fandiño, A., J. Armengol Estape, M. Pamies, J. Llop Palao, J. Silveira Ocampo, C. Pio Carrino, C. Armentano Oller, C. Rodriguez...
  • Joshi, M., E. Choi, D. S. Weld, and L. Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension.
  • Jurafsky, D. and J. H. Martin. 2021. Speech and language processing. 3rd edition draft. US: Prentice Hall.
  • Kwiatkowski, T., J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, et al. 2019a....
  • Kwiatkowski, T., J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, M. Kelcey, J. Devlin, K. Lee, K....
  • Lan, Z., M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations....
  • Le, Q. and T. Mikolov. 2014. Distributed representations of sentences and documents. In International conference on machine learning, pages...
  • LeCun, Y., Y. Bengio, and G. Hinton. 2015. Deep learning. nature, 521(7553):436– 444.
  • Manning, C., P. Raghavan, and H. Schütze. 2010. Introduction to information retrieval. Natural Language Engineering, 16(1):100–103.
  • Min, S., J. Boyd-Graber, C. Alberti, D. Chen, E. Choi, M. Collins, K. Guu, H. Hajishirzi, K. Lee, J. Palomaki, et al. 2021. Neurips 2020 efficientqa...
  • Min, S., V. Zhong, R. Socher, and C. Xiong. 2018. Efficient and robust question answering from minimal context over documents. arXiv preprint...
  • Maximo, S. 2022. Supervised domain adaptation for extractive question answering in spanish.
  • Nakov, P., D. Hoogeveen, L. Marquez, A. Moschitti, H. Mubarak, T. Baldwin, and K. Verspoor. 2019a. Semeval-2017 task 3: Community question...
  • Nakov, P., L. Marquez, W. Magdy, A. Moschitti, J. Glass, and B. Randeree. 2019b. Semeval-2015 task 3: Answer selection in community question...
  • Peñas, A., P. Forner, A. Rodrigo, R. Sutclif- fe, C. Forascu, and C. Mota. 2010. Overview of respubliqa 2010: Question answering evaluation...
  • Peñas, A., P. Forner, R. Sutcliffe, A. Rodri- go, C. Forascu, I. Alegria, D. Giampiccolo, N. Moreau, and P. Osenova. 2009. Overview of respubliqa...
  • Rajpurkar, P., J. Zhang, K. Lopyrev, and P. Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
  • Reddy, S., D. Chen, and C. D. Manning. 2019. Coqa: A conversational question answering challenge. Transactions of the Association for Computational...
  • Rodrigo, A. and A. Peñas. 2022. Uned@quales 2022: Testing the performance of transformer-based language models for spanish question-answering.
  • Seo, M., A. Kembhavi, A. Farhadi, and H. Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. arXiv preprint arXiv:1611.01603.
  • Trischler, A., T. Wang, X. Yuan, J. Harris, A. Sordoni, P. Bachman, and K. Suleman. 2016. Newsqa: A machine comprehension dataset. arXiv preprint...
  • Vaca-Serrano, A. 2022. Adversarial question answering in spanish with transformer models.
  • Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L . Kaiser, and I. Polosukhin. 2017. Attention is all you need. Advances...
  • Xiong, C., V. Zhong, and R. Socher. 2017. Dcn+: Mixed objective and deep residual coattention for question answering. arXiv preprint arXiv:1711.00106.
  • Yu, L., K. M. Hermann, P. Blunsom, and S. Pulman. 2014. Deep learning for answer sentence selection. arXiv preprint arXiv:1412.1632.