Overview of GenoVarDis at IberLEF 2024: NER of Genomic Variants and Related Diseases in Spanish

• Autores: Marvin Matías Agüero Torales, Carlos Rodríguez, Marta Carcajona Mata, Juan I. Díaz Hernández, Mario Solís López, Antonio Miranda Escalada, Sergio López Alvárez, Jorge Mira Prats, Carlos Alberto Castaño Moraga, David Vilares Calvo , Luis Chiruzzo
• Localización: Procesamiento del lenguaje natural, ISSN 1135-5948, Nº. 73, 2024, págs. 381-392
• Idioma: inglés
• Títulos paralelos:
  • Resumen de GenoVarDis en IberLEF 2024: NER de Variantes Genómicas y Enfermedades Relacionadas en Español
• Enlaces
  • Texto completo
• Resumen
  • español

    Presentamos la primera competencia de Reconocimiento de Entidades Nombradas (NER) en literatura científica escrita en español sobre variantes genómicas, genes y sus enfermedades y síntomas asociados, GenoVarDis en la campaña IberLEF 2024. El reto consistía en identificar entidades relacionadas con variantes. Se anotó un corpus de 633 resúmenes extraídos de artículos de PubMed, con la información para las tareas. Siete equipos participaron en la fase de evaluación, obteniendo en general buenos resultados para la tarea.

  • English

    We present the first shared task for Named Entity Recognition (NER) in Spanish written scientific literature about genomic variants, genes, and its associated diseases and symptoms, GenoVarDis at IberLEF 2024 campaign. The challenge consisted on identifying entities related to genomic variants. We annotated a corpus of 633 abstracts extracted from PubMed articles, with the information for the tasks. Seven teams took part in the evaluation phase, obtaining in general good results for the task.

• Referencias bibliográficas
  • 1000 Genomes Project Consortium et al. 2010. A map of human genome variation from population scale sequencing. Nature, 467(7319):1061.
  • Agüero-Torales, M. M. and A. Miranda-Escalada. 2024. GenoVarDis-ST Baseline 1 - Lookup. https://github.com/mmaguero/genovardis-baseline.
  • AI@Meta. 2024. Llama 3 model card. GitHub repository.
  • Akiba, T., S. Sano, T. Yanase, T. Ohta, and M. Koyama. 2019. Optuna: A next-generation hyperparameter optimization framework. In Proceedings...
  • Carrino, C. P., J. Armengol-Estapé, A. Gutiérrez-Fandiño, J. Llop-Palao, M. Pàmies, A. Gonzalez-Agirre, and M. Villegas. 2021. Biomedical...
  • Carrino, C. P., J. Llop, M. Pàmies, A. Gutiérrez-Fandiño, J. Armengol-Estapé, J. Silveira-Ocampo, A. Valencia, A. Gonzalez-Agirre, and M....
  • Chen, E., F. M. Facio, K. W. Aradhya, S. Rojahn, K. E. Hatchell, S. Aguilar, K. Ouyang, S. Saitta, A. K. Hanson-Kwan, N. N. Capurro, E. Takamine,...
  • Cheng, C., F. Tan, and Z. Wei. 2020. Deepvar: an end-to-end deep learning approach for genomic variant recognition in biomedical literature....
  • Chiruzzo, L., S. M. Jiménez-Zafra, and F. Rangel. 2024. Overview of IberLEF 2024: Natural Language Processing Challenges for Spanish and other...
  • Cho, H. and H. Lee. 2019. Biomedical named entity recognition using deep neural networks with contextual information. BMC bioinformatics,...
  • Conneau, A., K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov. 2020. Unsupervised...
  • Demner-fushman, D., S. Ananiadou, and K. Cohen, editors. 2023. The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared...
  • Fandiño, A. G., J. A. Estapé, M. Pàmies, J. L. Palao, J. S. Ocampo, C. P. Carrino, C. A. Oller, C. R. Penagos, A. G. Agirre, and M. Villegas....
  • He, P., X. Liu, J. Gao, and W. Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention.
  • Hochreiter, S. and J. Schmidhuber. 1997. Long Short-Term Memory. Neural Computation, 9(8):1735–1780, 11.
  • Huang, Z., W. Xu, and K. Yu. 2015. Bidirectional lstm-crf models for sequence tagging.
  • Kodikara, M. and K. Verspoor. 2024a. Effectiveness of Cross-linguistic Extraction of Genetic Information using Generative Large Language Models....
  • Kodikara, M. and K. Verspoor. 2024b. The RMIT University system for NER of genetic entities in biomedical literature for the GenoVarDis shared...
  • Lafferty, J. D., A. McCallum, and F. C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence...
  • Lange, L., H. Adel, J. Strotgen, and D. Klakow. 2021. Clin-x: pre-trained language models and a study on cross-task transfer for concept extraction...
  • Lee, K., C.-H. Wei, and Z. Lu. 2020. Recent advances of automated methods for searching and extracting genomic variant information from biomedical...
  • Lezama-Sánchez, A. L. and M. Tovar Vidal. 2024. Named Entity Recognition in Scientific Texts Using Long Short-Term Memory Networks. In Proceedings...
  • Liu, Y., M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. 2019. Roberta: A robustly optimized...
  • Marco, P., E. Segarra, and L.-F. Hurtado. 2024. ELiRF at GenoVarDis Task: NER in Genomic Variants and related Diseases. In Proceedings of...
  • Martínez, A. 2024. FRE at GenoVarDis: A sane approach to Disease and Genomic Variant NER. In Proceedings of the Iberian Languages Evaluation...
  • Miranda-Escalada, A., E. Farré-Maduell, S. Lima-López, L. Gascó, V. Briva-Iglesias, M. Agüero-Torales, and M. Krallinger. 2021. The ProfNER...
  • Miranda-Escalada, A., F. Mehryary, J. Luoma, D. Estrada-Zavala, L. Gasco, S. Pyysalo, A. Valencia, and M. Krallinger. 2023. Overview of DrugProt...
  • Nuzzo, A., A. Riva, and R. Bellazzi. 2009. Phenotypic and genotypic data integration and exploration through a webservice architecture. BMC...
  • Oliveros, V. M. 2024a. Geno-VarDis@IberLEF2024: Automatic Genomic Variants and Related Diseases using Named Entity Recognition with Large...
  • Oliveros, V. M. 2024b. Geno-VarDis@IberLEF2024: Automatic Genomic Variants and Related Diseases using Named Entity Recognition with Large...
  • OpenAI. 2024. OpenAI GPT-3.5 Turbo API [gpt-3.5-turbo:0301]. Accessed: 2024.
  • Pavao, A., I. Guyon, A.-C. Letournel, X. Baró, H. Escalante, S. Escalera, T. Thomas, and Z. Xu. 2022. Codalab competitions: An open source...
  • Provilkov, I., D. Emelianenko, and E. Voita. 2020. BPE-dropout: Simple and effective subword regularization. In D. Jurafsky, J. Chai, N. Schluter,...
  • Ramos-Flores, O., H. Gómez-Adorno, and E. Galán-Vásquez. 2024. IMASnlp at GenoVarDis Task: Exploring Zero-shot and CRF Approaches to NER Task...
  • Rezarta, I., C. Arighi, I. Campbell, G. Gonzalez-Hernandez, L. Hirschman, M. Krallinger, S. Lima-L´opez, D. Weissenbacher, and Z. Lu, editors....
  • Stenetorp, P., S. Pyysalo, G. Topi´c, T. Ohta, S. Ananiadou, and J. Tsujii. 2012. brat: a web-based tool for NLP-assisted text annotation....
  • Sung, M., M. Jeong, Y. Choi, D. Kim, J. Lee, and J. Kang. 2022. BERN2: an advanced neural biomedical named entity recognition and normalization...
  • Vaca Serrano, A., G. García Subies, H. Montoro Zamorano, N. Aldama García, D. Samy, D. Betancur Sánchez, A. Moreno-Sandoval, M. Guerrero Nieto,...
  • Verspoor, K., A. Jimeno Yepes, L. Cavedon, T. McIntosh, A. Herten-Crabb, Z. Thomas, and J.-P. Plazzer. 2013. Annotating the biomedical literature...
  • Verspoor, K. M., G. E. Heo, K. Y. Kang, and M. Song. 2016. Establishing a baseline for literature mining human genetic variants and their...
  • Walsh, N., A. Cooper, A. Dockery, and J. J. O’Byrne. 2024. Variant reclassification and clinical implications. Journal of Medical Genetics,...
  • Wei, C.-H., A. Allot, K. Riehle, A. Milosavljevic, and Z. Lu. 2022. tmVar 3.0: an improved variant concept recognition and normalization tool....
  • Wei, C.-H., L. Phan, J. Feltz, R. Maiti, T. Hefferon, and Z. Lu. 2018. Tmvar 2.0: integrating genomic variant information from literature...
  • Yepes, A. J. and K. Verspoor. 2014. Mutation extraction tools can be combined for robust recognition of genetic variants in the literature....
  • Zaratiana, U., N. Tomeh, P. Holat, and T. Charnois. 2024. GLiNER: Generalist model for named entity recognition using bidirectional transformer....