Assessing the quality of ChatGPT’s generated output in light of human-written texts: a corpus study based on textual parameters

• Anna-Maria De Cesare ^[1]
  1. [1] Dresden University of Technology

     Dresden University of Technology

     Kreisfreie Stadt Dresden, Alemania

     
• Localización: CHIMERA: Romance Corpora and Linguistic Studies, ISSN-e 2386-2629, Nº. 10, 2023, págs. 179-210
• Idioma: inglés
• Enlaces
  • Texto completo
• Resumen

  • This contribution has an exploratory nature, marking the initial phase of a broader research project aimed at achieving both descriptive and theoretical objectives. The primary goal is to evaluate the quality of texts produced by Language Model Models (LLMs). Two key aspects are examined: the quality of generated texts in comparison to human-authored texts and the identification of distinctive features characterizing this emerging text typology. The analysis is centered on textual parameters, encompassing various phenomena related to text segmentation and three dimensions of text organization (the referential-thematic dimension, the logico-argumentative dimension, and the polyphonic-enunciative dimension). Results of different case studies based on a self-assemble corpus of biographies generated by ChatGPT-3.5 and published on Wikipedia are presented.

• Referencias bibliográficas
  • Andorno, C. 2003. Linguistica testuale. Un’introduzione. Roma: Carocci.
  • Bang, Y. et al. 2023. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity. https://doi.org/10.48550/arXiv.2302.04023...
  • Barbaresi, A. 2021. Trafilatura: A Web Scraping Library and Command-Line Tool for Text Discovery and Extraction. Proceedings of ACL/IJCNLP...
  • Cresti, E. & Panunzi, A. 2013. Introduzione ai corpora dell’italiano. Bologna: il Muli-no.
  • D’Achille, P. 2016. Architettura dell'italiano di oggi e linee di tendenza. In S. Lubello (ed.), Manuale di linguistica italiana. Berlin:...
  • De Cesare, A.-M. 2011. Espositivi, testi. In R. Simone (ed.), Enciclopedia dell’italiano. Roma: Treccani, 1474-1478.
  • De Cesare, A.-M. 2021a. Répétitions et variations des textes générés. Une analyse lin-guistique basée sur un corpus d’articles financiers...
  • De Cesare, A.-M. 2021b. Autour de la relation d’ajout. Définition et connecteurs adver-biaux du français. In A. Ferrari & F. Pecorari...
  • De Cesare, A.-M. (in press). Il Movimento Testuale seriale: forma prototipica e mani-festazione nei testi generati da ChatGPT. In L. Fesenmeier,...
  • De Cesare, A.-M., Eliasson, E. & Weidensdorfer, T. 2023. La coesione testuale (basata sulle strutture verbali) nella scrittura generata...
  • Ferrara, E. 2023. Should ChatGPT be Biased? Challenges and Risks of Bias in Large Lan-guage Models. https://doi.org/10.48550/arXiv.2304.03738...
  • Ferrari, A. 2010. Connettivi. In R. Simone (ed.), Enciclopedia dell’italiano. Roma: Trec-cani, 271-273.
  • Ferrari, A. 2014. The Basel Model for paragraph segmentation: the construction units, their relationships and linguistic indication. In S....
  • Ferrari, A., Cignetti, L., De Cesare, A.-M., Lala, L., Mandelli, M., Ricci, C. & Roggia, E. 2008. L’interfaccia lingua-testo. Natura e...
  • Ferrari, A. & De Cesare, A.-M. 2009. La progressione tematica rivisitata. Vox Romanica 68: 98-128.
  • Ferrari, A. & Pecorari, F. (2021). Introduzione. Denominazioni, definizioni, prospettive di analisi. In A. Ferrari & F. Pecorari (eds),...
  • Garrido-Muñoz, I., Martínez-Santiago, F. & Montejo-Ráez, A. 2023. MarIA and BETO are sexist: evaluating gender bias in large language...
  • Kotek, H., Dockum, R., Sun, D. Q. 2023. Gender bias and stereotypes in Large Language Models. https://aps.arxiv.org/pdf/2308.14921.pdf (accessed...
  • Lambrecht, K. 1994. Information Structure and Sentence Form. Topic, Focus and the Mental Representations of Discourse Referents. Cambridge:...
  • Nissim, M. & Pannitto, L. 2022. Che cos’è la linguistica computazionale? Roma: Ca-rocci.
  • Sabatini, A. 1993. Il sessismo nella lingua italiana. Commissione Nazionale per la parità e le pari opportunità tra uomo e donna. Presidenza...
  • Tavosanis, M. 2021. L’ideologia linguistica e le pratiche di Wikipedia in lingua italiana. In A. P. Alamán, F. Ruggiano & O. Walsh (eds),...
  • Viviani, A. 2011. Cognomi, articolo con. In R. Simone (ed.), Enciclopedia dell’italiano. Roma: Treccani, https://www.treccani.it/enciclopedia/articolo-con-prontuario-cognomi_%28Enciclopedia-dell%27Italiano%29/...
  • Werlich, E. 1975. Typologie der Texte. Entwurf eines Textlinguistischen Modells zur Grundlegung einer Textgrammatik. Heidelberg: Quelle &...
  • Permanent links to the Wikipedia entries (all accessed on September 18 and 19, 2023)
  • https://it.wikipedia.org/w/index.php?title=Dacia_Maraini&oldid=135501906.
  • https://it.wikipedia.org/w/index.php?title=Rosy_Bindi&oldid=135491102.
  • https://it.wikipedia.org/w/index.php?title=Grazia_Deledda&oldid=135390496.
  • https://it.wikipedia.org/w/index.php?title=Laura_Boldrini&oldid=135253541.
  • https://it.wikipedia.org/w/index.php?title=Elena_Cattaneo&oldid=134858998.
  • https://it.wikipedia.org/w/index.php?title=Paola_Cortellesi&oldid=135495025.
  • https://it.wikipedia.org/w/index.php?title=Eleonora_Duse&oldid=134989604.
  • https://it.wikipedia.org/w/index.php?title=Oriana_Fallaci&oldid=135283939.
  • https://it.wikipedia.org/w/index.php?title=Giorgia_Meloni&oldid=135501878.
  • https://it.wikipedia.org/w/index.php?title=Rita_Levi-Montalcini&oldid=135460237.
  • https://it.wikipedia.org/w/index.php?title=Liliana_Segre&oldid=135498117.
  • https://it.wikipedia.org/w/index.php?title=Antonietta_Brandeis&oldid=129060540.