Local and Global Context-Based Pairwise Models for Sentence Ordering
Autor: | Aditya Jyoti Paul, Ruskin Raj Manku |
---|---|
Jazyk: | angličtina |
Rok vydání: | 2021 |
Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Computer Science - Logic in Computer Science Information Systems and Management Computer Science - Computation and Language Computer Science - Artificial Intelligence I.2.7 H.3.3 H.3.1 Management Information Systems Machine Learning (cs.LG) Logic in Computer Science (cs.LO) Computer Science - Information Retrieval Artificial Intelligence (cs.AI) Artificial Intelligence Computation and Language (cs.CL) Software Information Retrieval (cs.IR) |
ISSN: | 0950-7051 |
Popis: | Sentence Ordering refers to the task of rearranging a set of sentences into the appropriate coherent order. For this task, most previous approaches have explored global context-based end-to-end methods using Sequence Generation techniques. In this paper, we put forward a set of robust local and global context-based pairwise ordering strategies, leveraging which our prediction strategies outperform all previous works in this domain. Our proposed encoding method utilizes the paragraph's rich global contextual information to predict the pairwise order using novel transformer architectures. Analysis of the two proposed decoding strategies helps better explain error propagation in pairwise models. This approach is the most accurate pure pairwise model and our encoding strategy also significantly improves the performance of other recent approaches that use pairwise models, including the previous state-of-the-art, demonstrating the research novelty and generalizability of this work. Additionally, we show how the pre-training task for ALBERT helps it to significantly outperform BERT, despite having considerably lesser parameters. The extensive experimental results, architectural analysis and ablation studies demonstrate the effectiveness and superiority of the proposed models compared to the previous state-of-the-art, besides providing a much better understanding of the functioning of pairwise models. This is a post-print of an article published in Knowledge-Based Systems. For the journal-typeset version, please see https://www.sciencedirect.com/science/article/abs/pii/S0950705122001873?via%3Dihub |
Databáze: | OpenAIRE |
Externí odkaz: |