Zobrazeno 1 - 10
of 39 123
pro vyhledávání: '"memorisation"'
Autor:
Dankers, Verna, Titov, Ivan
Memorisation is a natural part of learning from real-world data: neural models pick up on atypical input-output combinations and store those training examples in their parameter space. That this happens is well-known, but how and where are questions
Externí odkaz:
http://arxiv.org/abs/2408.04965
Autor:
Speicher, Till, Khan, Mohammad Aflah, Wu, Qinyuan, Nanda, Vedant, Das, Soumi, Ghosh, Bishwamittra, Gummadi, Krishna P., Terzi, Evimaria
Understanding whether and to what extent large language models (LLMs) have memorised training data has important implications for the reliability of their output and the privacy of their training data. In order to cleanly measure and disentangle memo
Externí odkaz:
http://arxiv.org/abs/2407.19262
Publikováno v:
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2024)
Understanding memorisation in language models has practical and societal implications, e.g., studying models' training dynamics or preventing copyright infringements. Prior work defines memorisation as the causal effect of training with an instance o
Externí odkaz:
http://arxiv.org/abs/2406.04327
When training a neural network, it will quickly memorise some source-target mappings from your dataset but never learn some others. Yet, memorisation is not easily expressed as a binary feature that is good or bad: individual datapoints lie on a memo
Externí odkaz:
http://arxiv.org/abs/2311.05379
Large language models have gained significant popularity because of their ability to generate human-like text and potential applications in various fields, such as Software Engineering. Large language models for code are commonly trained on large uns
Externí odkaz:
http://arxiv.org/abs/2312.11658
Quantifying the impact of individual data samples on machine learning models is an open research problem. This is particularly relevant when complex and high-dimensional relationships have to be learned from a limited sample of the data generating di
Externí odkaz:
http://arxiv.org/abs/2311.03075
A distinction is often drawn between a model's ability to predict a label for an evaluation sample that is directly memorised from highly similar training samples versus an ability to predict the label via some method of generalisation. In the contex
Externí odkaz:
http://arxiv.org/abs/2311.12337
Autor:
CANGLONG WANG1 canglongwang6@gmail.com, SHUO WANG2 shuo.wang@beds.ac.uk
Publikováno v:
China Perspectives. 2023, Issue 135, p61-70. 10p.
Autor:
Li, Yucheng
Data contamination in model evaluation is getting increasingly prevalent as the massive training corpora of large language models often unintentionally include benchmark samples. Therefore, contamination analysis has became an inevitable part of reli
Externí odkaz:
http://arxiv.org/abs/2309.10677