Evaluation Benchmarks for Spanish Sentence Representations

Simple item page
dc.catalogadorgrr
dc.contributor.authorAraujo Vasquez, Vladimir Giovanny
dc.contributor.authorCarvallo, Andrés
dc.contributor.authorSoto A.
dc.contributor.authorMoens M.-F.
dc.contributor.authorKundu S.
dc.contributor.authorMercer R.E.
dc.contributor.authorCanete J.
dc.contributor.authorBravo-Marquez F.
dc.contributor.authorMendoza M.
dc.date.accessioned2024-05-28T20:02:34Z
dc.date.available2024-05-28T20:02:34Z
dc.date.issued2022
dc.description.abstract© European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0.Due to the success of pre-trained language models, versions of languages other than English have been released in recent years. This fact implies the need for resources to evaluate these models. In the case of Spanish, there are few ways to systematically assess the models' quality. In this paper, we narrow the gap by building two evaluation benchmarks. Inspired by previous work (Conneau and Kiela, 2018; Chen et al., 2019), we introduce Spanish SentEval and Spanish DiscoEval, aiming to assess the capabilities of stand-alone and discourse-aware sentence representations, respectively. Our benchmarks include considerable pre-existing and newly constructed datasets that address different tasks from various domains. In addition, we evaluate and analyze the most recent pre-trained Spanish language models to exhibit their capabilities and limitations. As an example, we discover that for the case of discourse evaluation tasks, mBERT, a language model trained on multiple languages, usually provides a richer latent representation than models trained only with documents in Spanish. We hope our contribution will motivate a fairer, more comparable, and less cumbersome way to evaluate future Spanish language models.
dc.description.funderANID FONDECYT
dc.description.funderNational Center for Artificial Intelligence CENIA
dc.description.funderU-Inicia VID
dc.description.funderANID
dc.fechaingreso.objetodigital2024-05-28
dc.format.extent11 páginas
dc.identifier.eisbn9791095546726
dc.identifier.scopusidSCOPUS_ID:85144404417
dc.identifier.scopusidSCOPUS_ID:2-s2.0-85144404417
dc.identifier.urihttps://aclanthology.org/2022.lrec-1.648.pdf
dc.identifier.urihttps://repositorio.uc.cl/handle/11534/85916
dc.information.autorucEscuela de Ingeniería; Araujo Vasquez, Vladimir Giovanny; S/I; 1081563
dc.language.isoen
dc.nota.accesocontenido completo
dc.pagina.final6034
dc.pagina.inicio6024
dc.publisherEuropean Language Resources Association (ELRA)
dc.relation.ispartof2022 Language Resources and Evaluation Conference, LREC 2022
dc.revista2022 Language Resources and Evaluation Conference, LREC 2022
dc.rightsacceso abierto
dc.subjectdiscourse evaluation
dc.subjectlanguage models
dc.subjectrepresentation learning
dc.subjectsentence evaluation
dc.titleEvaluation Benchmarks for Spanish Sentence Representations
dc.typecomunicación de congreso
sipa.codpersvinculados1081563
sipa.trazabilidadSCOPUS;02-03-2023
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
2022.lrec-1.648.pdf
Size:
410.16 KB
Format:
Adobe Portable Document Format
Description:
Download
Collections
Artículos de conferencia