Learning High-Quality and General-Purpose Phrase Representations - Département Informatique et Réseaux Access content directly
Conference Papers Year : 2024

Learning High-Quality and General-Purpose Phrase Representations

Abstract

Phrase representations play an important role in data science and natural language processing, benefiting various tasks like Entity Alignment, Record Linkage, Fuzzy Joins, and Paraphrase Classification. The current state-of-theart method involves fine-tuning pre-trained language models for phrasal embeddings using contrastive learning. However, we have identified areas for improvement. First, these pretrained models tend to be unnecessarily complex and require to be pre-trained on a corpus with context sentences. Second, leveraging the phrase type and morphology gives phrase representations that are both more precise and more flexible. We propose an improved framework to learn phrase representations in a context-free fashion. The framework employs phrase type classification as an auxiliary task and incorporates character-level information more effectively into the phrase representation. Furthermore, we design three granularities of data augmentation to increase the diversity of training samples. Our experiments across a wide range of tasks show that our approach generates superior phrase embeddings compared to previous methods while requiring a smaller model size. The code is available at https: //github.com/tigerchen52/PEARL
Fichier principal
Vignette du fichier
eacl-2024.pdf (701.87 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04465022 , version 1 (19-02-2024)

Identifiers

  • HAL Id : hal-04465022 , version 1

Cite

Lihu Chen, Gaël Varoquaux, Fabian M. Suchanek. Learning High-Quality and General-Purpose Phrase Representations. EACL 2024 - The 18th Conference of the European Chapter of the Association for Computational Linguistics, Mar 2024, La Valette, Malta. ⟨hal-04465022⟩
54 View
13 Download

Share

Gmail Facebook X LinkedIn More