Investigating the use of readability metrics to detect differences in written productions of learners: a corpus-based study: a corpus-based study

• Autores: Paula Lissón
• Localización: Bellaterra: journal of teaching and learning language and literature, ISSN-e 2013-6196, Vol. 10, Nº. 4, 2017, págs. 68-86
• Idioma: inglés
• DOI: 10.5565/rev/jtl3.752
• Enlaces
  • Texto completo  1    2  
• Resumen
  • Multiple

    This paper deals with the use of readability metrics as indices of learners' linguistic features in a written corpus of Spanish learners of English L2. Seventeen measures of readability are presented and computed for 200 samples of written argumentative essays extracted from the corpus NOCE (Díaz-Negrillo, 2007). Support Vector Machines (SVM) are used in order to detect which are the metrics that perform better at detecting differences in learners’ productions belonging to students enrolled in the first or in the second year of an English major. Metrics based on sentence length, number of sentences, and number of polysyllabic words are reported to be the most accurate ones for the classification of learners' linguistic features.

  • English

    Este artículo trata sobre el uso de métricas de legibilidad como indicadores de las características lingüísticas propias a dos niveles de aprendices españoles de inglés L2. Presentamos y calculamos diecisiete medidas de legibilidad en 200 textos argumentativos extraídos del corpus NOCE (Díaz-Negrillo, 2007). Utilizamos SVM para averiguar qué métricas son capaces de detectar diferencias entre las 200 producciones, pertenecientes a alumnos de primer y segundo curso de Filología Inglesa, respectivamente. Las métricas basadas en la longitud de las frases, el número de frases y el número de palabras polisílabas son las que presentan mejores resultados.

  • français

    Cet article analyse les métriques de lisibilité dans une perspective de détection des niveaux des apprenants hispanophones en anglais L2 à partir d’un corpus de productions écrites. Dix-sept métriques de lisibilité ont été calculées pour 200 essais argumentatifs extraits du corpus NOCE (Díaz-Negrillo, 2007). On emploie la technique des Séparateurs à Vaste Marge (SVM) pour déterminer les métriques qui caractérisent le mieux les différences entre les productions des apprenants de première et deuxième année en anglais. Les métriques fondées sur la longueur des phrases, le nombre de phrases, ainsi que sur le nombre de mots polysyllabiques sont celles qui aident le mieux à détecter et à classer les étudiants selon leurs caractéristiques langagières.

• Referencias bibliográficas
  • Anderson, J. (1981). Analysing the readability of English and non-English texts in the classroom with Lix. Paper presented at the Seventh...
  • Anderson, J. (1983). LIX and RIX: Variations on a little-known readability index. Journal of Reading, 26(6), 490–496.
  • Béchara, H., Costa, H., Taslimipoor, S., Gupta, R., Orasan, C., Pastor, G. C., & Mitkov, R. (2015). MiniExperts: An SVM approach for measuring...
  • Björnsson, C. H. (1968). Läsbarhet. Liber.
  • Björnsson, C.-H. (1983). Readability of newspapers in 11 languages. Reading Research Quarterly, 480–497. DOI: https://doi.org/10.2307/747382
  • Bormuth, J. R. (1969). Development of readability analysis (Final report project no. 7-0052, contract no. OEC-3-7-070052-0326). US Department...
  • Campbell, W. M., Campbell, J. P., Reynolds, D. A., Singer, E., & Torres-Carrasquillo, P. A. (2006). Support vector machines for speaker...
  • Caylor, J. S., Stitch, T. G., & Fox. (1973). Methodologies for determining reading requirements of military occupational specialties....
  • Chall, J. S., & Dale, E. (1995). Readability revisited: The new Dale-Chall readability formula. Cambridge, MA: Brookline Books.
  • Coleman, M., & Liau, T. L. (1975). A computer readability formula designed for machine scoring. Journal of Applied Psychology, 60(2),...
  • Collins-Thompson, K., & Callan, J. (2005). Predicting reading difficulty with statistical language models. Journal of the Association...
  • Dale, E. (1931). A comparison of two word lists. Educational Research Bulletin, 10(18), 484– 489.
  • Dale, E., & Chall, J. S. (1948). A formula for predicting readability: Instructions. Educational Research Bulletin, 27(2), 37–54.
  • Díaz-Negrillo, A. (2007). A fine-grained error tagger for learner corpora (Doctoral dissertation). University of Jaen, Jaen.
  • Díaz-Negrillo, A. (2009). EARS: A user’s manual (vol. 1). Munich: LINCOM Academic Reference Books.
  • DuBay, W. H. (2004). The principles of readability. Costa Mesa: Impact Information. Retrieved from http://www.impact-information.com/impactinfo/readability02.pdf
  • Farr, J. N., Jenkins, J. J., & Paterson, D. G. (1951). Simplification of Flesch reading ease formula. Journal of Applied Psychology, 35(5),...
  • Feng, L., Jansche, M., Huenerfauth, M., & Elhadad, N. (2010). A comparison of features for automatic readability assessment. Proceedings...
  • Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32(3), 221– 233. DOI: https://doi.org/10.1037/h0057532
  • Flesch, R. F. (1949). Art of readable writing. USA: Hungry Minds Inc.
  • François, T. (2011). Les apports du traitement automatique des langues à la lisibilité du français langue étrangère. Université Catholique...
  • Giménez, J., & Marquez, L. (2004). Fast and accurate part-of-speech tagging: The SVM approach revisited. Recent Advances in Natural Language...
  • Granger, S., Gilquin, G., & Meunier, F. (Eds.). (2015). The Cambridge handbook of learner corpus research. Cambridge University Press....
  • Gunning, R. (1952). The technique of clear writing. New York: McGraw-Hill.
  • Harris, A. J., & Jacobson, M. D. (1974). Revised Harris-Jacobson readability formulas. Paper presented at the Annual Meeting of the College...
  • Hawkins, J. A., & Buttery, P. (2010). Criterial features in learner corpora: Theory and illustrations. English Profile Journal, 1(1),...
  • Hawkins, J. A., & Filipović, L. (2012). Criterial features in L2 English: Specifying the reference levels of the Common European Framework...
  • Heilman, M., Collins-Thompson, K., & Eskenazi, M. (2008). An analysis of statistical models and features for reading difficulty prediction....
  • Hsu, C.-W., Chang, C.-C., & Lin, C.-J. (2003). A practical guide to support vector classification. Retrieved from http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf
  • Jarvis, S. (2011). Data mining with learner corpora: choosing classifiers for L1 detection. In F. Meunier, S. De Cock, G. Gilquin, & M....
  • Kincaid, J. P., Fishburne Jr, R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability...
  • Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15(4),...
  • Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. Tesol...
  • McLaughlin, G. H. (1969). SMOG grading-a new readability formula. Journal of Reading, 12(8), 639–646.
  • McLaughlin, G. H. (1968). Proposals for British readability measures. In J. Downing & A. L. Brown (Eds.), Third international reading...
  • Michalke, M. (2017). Package koRpus: An R Package for Text Analysis (Version 0.10-2). Retrieved from http://reaktanz.de/?c=hacking&s=koRpus
  • Mullen, T., & Collier, N. (2004). Sentiment analysis using support vector machines with diverse information sources. Proceedings of EMNLP-04,...
  • Nivre, J., Hall, J., Nilsson, J., Eryiǧit, G., & Marinov, S. (2006). Labeled pseudo-projective dependency parsing with support vector...
  • O’hayre, J. (1966). Gobbledygook has gotta go. US Dept. of the Interior, Bureau of Land Management.
  • Pilán, I., Volodina, E., & Johansson, R. (2014). Rule-based and machine learning approaches for second language sentence-level readability....
  • Prabowo, R., & Thelwall, M. (2009). Sentiment analysis: A combined approach. Journal of Informetrics, 3(2), 143–157. DOI: https://doi.org/10.1016/j.joi.2009.01.003
  • Schmid, H. (1995). Treetagger: A language independent part-of-speech tagger (computer software). Institut Für Maschinelle Sprachverarbeitung:...
  • Senter, R., & Smith, E. A. (1967). Automated readability index. DTIC Document.
  • Shen, W., Williams, J., Marius, T., & Salesky, E. (2013). A language-independent approach to automatic text difficulty assessment for...
  • Song, J., He, Y., & Fu, G. (2015). Polarity classification of short product reviews via multiple cluster-based SVM classifiers. Paper...
  • Spache, G. (1953). A new readability formula for primary-grade reading materials. The Elementary School Journal, 53(7), 410–413. DOI: https://doi.org/10.1086/458513
  • Sung, Y., Lin, W., Dyson, S. B., Chang, K., & Chen, Y. (2015). Leveling L2 texts through readability: Combining multilevel linguistic...
  • Sung, Y.-T., Chen, J.-L., Cha, J.-H., Tseng, H.-C., Chang, T.-H., & Chang, K.-E. (2015). Constructing and validating readability models:...
  • Vajjala, S. (2018). Automated assessment of non-native learner essays: Investigating the role of linguistic features. International Journal...
  • Vajjala, S., & Meurers, D. (2012). On improving the accuracy of readability classification using insights from second language acquisition....
  • Wheeler, L. R., & Smith, E. H. (1954). A practical readability formula for the classroom teacher in the primary grades. Elementary English,...
  • Yamada, H., & Matsumoto, Y. (2003). Statistical dependency analysis with support vector machines. Proceedings of IWPT (Vol. 3, pp. 195–206).
  • Zalmout, N., Saddiki, H., & Habash, N. (2016). Analysis of foreign language teaching methods: An automatic readability approach. Proceedings...