Zobrazeno 1 - 10
of 135
pro vyhledávání: '"Seddik, Mohamed"'
Autor:
Firdoussi, Aymane El, Seddik, Mohamed El Amine, Hayou, Soufiane, Alami, Reda, Alzubaidi, Ahmed, Hacid, Hakim
Synthetic data has gained attention for training large language models, but poor-quality data can harm performance (see, e.g., Shumailov et al. (2023); Seddik et al. (2024)). A potential solution is data pruning, which retains only high-quality data
Externí odkaz:
http://arxiv.org/abs/2410.08942
Autor:
Alami, Reda, Almansoori, Ali Khalifa, Alzubaidi, Ahmed, Seddik, Mohamed El Amine, Farooq, Mugariya, Hacid, Hakim
We demonstrate that preference optimization methods can effectively enhance LLM safety. Applying various alignment techniques to the Falcon 11B model using safety datasets, we achieve a significant boost in global safety score (from $57.64\%$ to $99.
Externí odkaz:
http://arxiv.org/abs/2409.07772
Autor:
Malartic, Quentin, Chowdhury, Nilabhra Roy, Cojocaru, Ruxandra, Farooq, Mugariya, Campesan, Giulia, Djilali, Yasser Abdelaziz Dahou, Narayan, Sanath, Singh, Ankit, Velikanov, Maksim, Boussaha, Basma El Amel, Al-Yafeai, Mohammed, Alobeidli, Hamza, Qadi, Leen Al, Seddik, Mohamed El Amine, Fedyanin, Kirill, Alami, Reda, Hacid, Hakim
We introduce Falcon2-11B, a foundation model trained on over five trillion tokens, and its multimodal counterpart, Falcon2-11B-vlm, which is a vision-to-text model. We report our findings during the training of the Falcon2-11B which follows a multi-s
Externí odkaz:
http://arxiv.org/abs/2407.14885
This paper provides theoretical insights into high-dimensional binary classification with class-conditional noisy labels. Specifically, we study the behavior of a linear classifier with a label noisiness aware loss function, when both the dimension o
Externí odkaz:
http://arxiv.org/abs/2405.14088
The phenomenon of model collapse, introduced in (Shumailov et al., 2023), refers to the deterioration in performance that occurs when new models are trained on synthetic data generated from previously trained models. This recursive training loop make
Externí odkaz:
http://arxiv.org/abs/2404.05090
This paper explores the effects of various forms of regularization in the context of language model alignment via self-play. While both reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO) require to collect cost
Externí odkaz:
http://arxiv.org/abs/2404.04291
We study the estimation of a planted signal hidden in a recently introduced nested matrix-tensor model, which is an extension of the classical spiked rank-one tensor model, motivated by multi-view clustering. Prior work has theoretically examined the
Externí odkaz:
http://arxiv.org/abs/2402.10677
Autor:
Maniparambil, Mayug, Akshulakov, Raiymbek, Djilali, Yasser Abdelaziz Dahou, Narayan, Sanath, Seddik, Mohamed El Amine, Mangalam, Karttikeya, O'Connor, Noel E.
Aligned text-image encoders such as CLIP have become the de facto model for vision-language tasks. Furthermore, modality-specific encoders achieve impressive performances in their respective domains. This raises a central question: does an alignment
Externí odkaz:
http://arxiv.org/abs/2401.05224
Autor:
Seddik, Mohamed El Amine, Guillaud, Maxime, Decurninge, Alexis, Goulart, José Henrique de Morais
This work introduces an asymptotic study of Hotelling-type tensor deflation in the presence of noise, in the regime of large tensor dimensions. Specifically, we consider a low-rank asymmetric tensor model of the form $\sum_{i=1}^r \beta_i{\mathcal{A}
Externí odkaz:
http://arxiv.org/abs/2310.18717
In this paper, we propose a nested matrix-tensor model which extends the spiked rank-one tensor model of order three. This model is particularly motivated by a multi-view clustering problem in which multiple noisy observations of each data point are
Externí odkaz:
http://arxiv.org/abs/2305.19992