Fractal Analysis of GPT-2 Token Embedding Spaces: Stability and Evolution of Correlation Dimension

Autor: Minhyeok Lee

Publikováno v: Fractal and Fractional, Vol 8, Iss 10, p 603 (2024)

This paper explores the fractal properties of token embedding spaces in GPT-2 language models by analyzing the stability of the correlation dimension, a measure of geometric complexity. Token embeddings represent words or subwords as vectors in a hig

Externí odkaz: https://doaj.org/article/1c3f0e391b9d4777bf590b094549562e

Tento výsledek nelze pro nepřihlášené uživatele zobrazit.
K zobrazení výsledku je třeba se přihlásit.

Vers des représentations contextualisées de mots

Autor: Popa, Diana-Nicoleta

Publikováno v: Computers and Society [cs.CY]. Université Grenoble Alpes, 2019. English. ⟨NNT : 2019GREAM037⟩

Continuous word representations (word type embeddings) are at the basis of most modern natural language processing systems, providing competitive results particularly when input to deep learning models. However, important questions are raised concern

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=od______2592::afab8a84a1e2a71a9c3e206514dbd2bf
https://tel.archives-ouvertes.fr/tel-02478383/file/POPA_2019_archivage.pdf

Tento výsledek nelze pro nepřihlášené uživatele zobrazit.
K zobrazení výsledku je třeba se přihlásit.

MidiTok: A Python Package for MIDI File Tokenization

Autor: Nathan Fradet, Jean-Pierre Briot, Fabien Chhel, Amal El Fallah-Seghrouchni, Nicolas Gutowski

Publikováno v: HAL
The 22nd International Society for Music Information Retrieval Conference (ISMIR 2021)
The 22nd International Society for Music Information Retrieval Conference (ISMIR 2021), Nov 2021, Online, United States
Extended Abstracts for the Late-Breaking Demo Session of the 22nd International Society for Music Information Retrieval Conference
22nd International Society for Music Information Retrieval Conference
22nd International Society for Music Information Retrieval Conference, Nov 2021, Online, United States

International audience; This article presents MidiTok, a Python package to encode MIDI files into sequences of tokens to be used with sequential Deep Learning models like Transformers or Recurrent Neural Networks. It allows researchers and developers

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=dedup_wf_001::aefb93203b2f468f193ad5908ce00ada
https://hal.sorbonne-universite.fr/hal-03418930