Améliorer les prédictions des modèles de language avec les prompts enrichies par les Graphes de Connaissances ⋆
| Autor: | Brate, Ryan, Dang, Minh-Hoang, Hoppe, Fabian, He, Yuan, Meroño-Peñuela, Albert, Sadashivaiah, Vijay |
|---|---|
| Přispěvatelé: | KNAW Humanities Cluster, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), FIZ Karlsruhe - Leibniz Institute for Information Infrastructure, University of Oxford, King‘s College London, Rensselaer Polytechnic Institute (RPI), ANR-19-CE23-0014,DeKaloG,Graphes de connaissances décentralisés(2019) |
| Jazyk: | angličtina |
| Rok vydání: | 2022 |
| Předmět: | |
| Zdroj: | DL4KG@ ISWC2022 DL4KG@ ISWC2022, Oct 2022, Hangzhou, China King's College London |
| Popis: | International audience; Despite advances in deep learning and knowledge graphs (KGs), using language models for natural language understanding and question answering remains a challenging task. Pre-trained language models (PLMs) have shown to be able to leverage contextual information, to complete cloze prompts, next sentence completion and question answering tasks in various domains. Unlike structured data querying in e.g. KGs, mapping an input question to data that may or may not be stored by the language model is not a simple task. Recent studies have highlighted the improvements that can be made to the quality of information retrieved from PLMs by adding auxiliary data to otherwise naive prompts. In this paper, we explore the effects of enriching prompts with additional contextual information leveraged from the Wikidata KG on language model performance. Specifically, we compare the performance of naive vs. KG-engineered cloze prompts for entity genre classification in the movie domain. Selecting a broad range of commonly available Wikidata properties, we show that enrichment of cloze-style prompts with Wikidata information can result in a significantly higher recall for the investigated BERT and RoBERTa large PLMs. However, it is also apparent that the optimum level of data enrichment differs between models. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|