Creation and validation of a bilingual test to estimate aural and written vocabulary size

• Autores: Martín Aoiz Pinillos
• Localización: Porta Linguarum: revista internacional de didáctica de las lenguas extranjeras, ISSN 1697-7467, Nº. 38, 2022, págs. 247-263
• Idioma: inglés
• DOI: 10.30827/portalin.vi38.23606
• Enlaces
  • Texto completo (pdf)
  • Texto completo  1    2  
• Resumen
  • español

    El tamaño del vocabulario de quienes aprenden una lengua es un predictor fiable de su éxito en la lengua meta porque se correlaciona claramente con mejores rendimientos. Las estimaciones precisas son esenciales para planificar la enseñanza de lenguas. Sin embargo, los instrumentos empleados pueden presentar problemas de validez y fiabilidad que afecten a su sensibilidad y precisión investigadoras.

    Este artículo describe la creación de un versión oral y escrita de un test bilingüe de vocabulario, administrado a 73 estudiantes adultos de inglés cuya lengua materna era el español. Sus respuestas fueron analizadas con el modelo Rasch para determinar los elementos del test que mejor se comportaban y mejorar la fiabilidad general del instrumento.

    La versión final del test presenta altos niveles de fiabilidad: .89 para la prueba oral de vocabulario y .82 para la prueba escrita. Además, las estadísticas descriptivas confirman que reconocer las palabras en su forma oral supone un reto mayor que hacerlo en su forma escrita porque hubo un 10,80% menos de respuestas correctas en el test oral. Este hallazgo confirma la afirmación de que el vocabulario oral y escrito son dos dimensiones distintas e influye en cómo se debería enseñar el vocabulario en las aulas.

  • English

    Language learners’ vocabulary size is a reliable predictor of their success in a second language as it clearly correlates with better performances in the target language. Being precise in those estimations is paramount to plan language teaching. However, the instruments employed by previous studies for those estimations might present validity and reliability issues that affect their research sensitivity and accuracy.

    This paper presents a step-by-step account of the creation of an aural and a written version of a bilingual vocabulary test. Test was delivered to 73 adult L1-Spanish students attending English classes. Their answers were analysed with Rasch model to determine the best performing items in the test so that the overall reliability of the instrument was enhanced.

    The final version of the test presents high levels of reliability: .89 for the listening vocabulary test and .82 for the written vocabulary test. Furthermore, descriptive statistics confirm that recognizing the words in their aural form is more challenging than in their written form: participants got 10.80% fewer correct answers in the listening vocabulary test. This finding confirms the claim that aural and written vocabulary are two separate dimensions, and impacts on how vocabulary should be taught in L2 classrooms.

• Referencias bibliográficas
  • Abbuhl, R., & Mackey, A. (2017). Second language acquisition research methods. In King, K. A., Lai, Y. J., & May, S. (Eds.). Research...
  • Andringa, S., Olsthoorn, N., van Beuningen, C., Schoonen, R., & Hulstijn, J. (2012). Determinants of success in native and non‐native...
  • Beglar, D., & Nation, P. (2007). A vocabulary size test. The Language Teacher, 31(7), 9-13.
  • Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: fundamental measurement in the human sciences (3rd ed.). Routledge.
  • Cheng, J., & Matthews, J. (2018). The relationship between three measures of L2 vocabulary knowledge and L2 listening and reading. Language...
  • Cobb, T. (2013). Frequency 2.0: Incorporating homoforms and multiword units in pedagogical frequency lists. In Bardel, C., Lindqvist, C.,...
  • Cobb, T. (2019). Compleat Web VP v.2 [computer program]. Accessed on 16 Jan 2019 at https://www.lextutor.ca/cgi-bin/range/texts/index.pl
  • Crossley, S. A., Cobb, T., & McNamara, D. S. (2013). Comparing count-based and band-based indices of word frequency: Implications for...
  • Hatch, E. M., & Lazaraton, A. (1991). The research manual: Design and statistics for applied linguistics. Heinle & Heinle Publishers.
  • Hazenberg, S., & Hulstijn, J. H. (1996). Defining a minimal receptive second language vocabulary for non-native university students: An...
  • Huang, H. T. (2010). How does second language vocabulary grow over time? A multi-methodological study of incremental vocabulary knowledge...
  • Karami, H. (2012). The development and validation of a bilingual version of the Vocabulary Size Test. RELC Journal, 43(1), 53-67. https://doi.org/10.1177/0033688212439359
  • Levitzky-Aviad, T., & Laufer, B. (2013). Lexical properties in the writing of foreign language learners over eight years of study: Single...
  • Linacre, J. M. (1997). KR-20 / Cronbach alpha or Rasch person reliability: Which tells the "truth"? Rasch Measurement Transactions,...
  • Linacre, J. M. (2012). A user’s guide to Winsteps Ministeps Rasch-model computer programs [version 3.74.0]. Retrieved from http://www.winsteps.com/index.htm
  • Linacre, J. M. (2012, 2019). Winsteps® Rasch Measurement, version 4.4.3. [Computer software] Downloaded from http://www.winsteps.com
  • Martinez, R., & Schmitt, N. (2012). A phrasal expressions list. Applied Linguistics, 33(3), 299-320. https://doi.org/10.1093/applin/ams010
  • Masrai, A. (2020). Exploring the impact of individual differences in aural vocabulary knowledge, written vocabulary knowledge and working...
  • Mathias, C.W. (2010). Sensitivity. In Salkind, N. J. (Ed.). (2010). Encyclopedia of research design (Vol. 3), (pp. 1337-1338). Sage.
  • Matthews, J. (2018). Vocabulary for listening: Emerging evidence for high and mid-frequency vocabulary knowledge. System, 72, 23-36. https://doi.org/10.1016/j.system.2017.10.005
  • McLean, S., Kramer, B., & Beglar, D. (2015). The creation and validation of a listening vocabulary levels test. Language Teaching Research,...
  • Meara, P. M., & Miralpeix, I. (2006). Y_Lex: The Swansea advanced vocabulary levels test. v2. 05. Lognostics.
  • Nation, I. S. P. (2001) Learning vocabulary in another language. Cambridge University Press.
  • Nation, I. S. P. (2012, 2019). The BNC/COCA word family lists (17 September 2012). Unpublished paper. [online] Retrieved from http://www.victoria.ac.nz/lals/about/staff/paul-nation
  • Nation, I. S. P. (2016). Making and using word lists for language learning and testing. John Benjamins. https://doi.org/10.1093/applin/amx052
  • Nation, I. S. P., & Webb, S. A. (2011). Researching and analyzing vocabulary. Heinle, Cengage Learning.
  • Nguyen, L.T.C., & Nation, P. (2011). A bilingual vocabulary size test of English for Vietnamese learners. RELC journal, 42(1), 86-99....
  • Ockey, G. J., & Green, B. A. (Eds.). (2020). Another generation of fundamental considerations in language assessment: A festschrift in...
  • Ovtcharov, V., Cobb, T., & Halter, R. (2006). La richesse lexicale des productions orales: mesure fiable du niveau de compétence langagière....
  • Schmitt, N. (2008). Instructed second language vocabulary learning. Language Teaching Research, 12(3), 329-363. https://doi.org/10.1177/1362168808089921
  • Schmitt, N., Cobb, T., Horst, M., & Schmitt, D. (2017). How much vocabulary is needed to use English? Replication of van Zeeland &...
  • Schmitt, N., Jiang, X., & Grabe, W. (2011). The percentage of words known in a text and reading comprehension. The Modern Language Journal,...
  • Schmitt, N., Schmitt, D. (2012). A reassessment of frequency and vocabulary size in L2 vocabulary teaching. Language Teaching, 47(4), 484-503....
  • Schmuckler, M. A. (2001). What is ecological validity? A dimensional analysis. Infancy, 2(4), 419-436. https://doi.org/10.1207/s15327078in0204_02
  • Silva, B. B., & Otwinowska, A. (2019). VST as a reliable academic placement tool despite cognate inflation effects. English for Specific...
  • UCLES (2012). Cambridge English: Preliminary Wordlist. Retrieved from https://www.cambridgeenglish.org/Images/84669-pet-vocabulary-list.pdf
  • UCLES (2021). B1 Preliminary and B1 Preliminary for Schools Vocabulary List. Retrieved from https://www.cambridgeenglish.org/images/506887-b1-Preliminary-2020-vocabulary-list.pdf
  • van Zeeland, H. (2014). Second language vocabulary knowledge in and from listening (Doctoral dissertation, University of Nottingham).
  • van Zeeland, H., & Schmitt, N. (2013). Lexical coverage in L1 and L2 listening comprehension: The same or different from reading comprehension?...
  • Zhao, P., & Ji, X. (2018). Validation of the Mandarin version of the Vocabulary Size Test. RELC Journal, 49(3), 308-321. https://doi.org/10.1177/0033688216639761