Perspectives Generation via Multi-Head Attention Mechanism and Common-Sense Knowledge

Autor: Fatima Alkhawaldeh, Tommy Yuan, Dimitar Kazakov
Rok vydání: 2022
Zdroj: International Journal on Cybernetics & Informatics. 11:1-13
ISSN: 2277-548X
DOI: 10.5121/ijci.2022.110201
Popis: Consideration of multiple viewpoints on a contentious issue is critical for avoiding bias and assisting in the formulation of rational decisions. We observe that the current model imposes a constraint on diversity. This is because the conventional attention mechanism is biased toward a single semantic aspect of the claim, whereas the claim may contain multiple semantic aspects. Additionally, disregarding common-sense knowledge may result in generating perspectives that violate known facts about the world. The proposed approach is divided into two stages: the first stage considers multiple semantic aspects, which results in more diverse generated perspectives; the second stage improves the quality of generated perspectives by incorporating common-sense knowledge. We train the model on each stage using reinforcement learning and automated metric scores. The experimental results demonstrate the effectiveness of our proposed model in generating a broader range of perspectives on a contentious subject.
Databáze: OpenAIRE