To Code, or Not To Code? Exploring Impact of Code in Pre-training

Autor: Aryabumi, Viraat, Su, Yixuan, Ma, Raymond, Morisot, Adrien, Zhang, Ivan, Locatelli, Acyr, Fadaee, Marzieh, Üstün, Ahmet, Hooker, Sara

Including code in the pre-training data mixture, even for models not specifically designed for code, has become a common practice in LLMs pre-training. While there has been anecdotal consensus among practitioners that code data plays a vital role in

Externí odkaz: http://arxiv.org/abs/2408.10914

Changes in bodyweight after initiating antiretroviral therapy close to HIV-1 seroconversion: an international cohort collaboration

Autor: Moreno, Santiago, Burns, Fiona, Campo, Rafael Eduardo, Garges, Harmony, Mussini, Cristina, Pantazis, Nikos, Kamel, Moustafa, Porter, Kholoud, Sabin, Caroline, Tariq, Shema, Touloumi, Giota, Vannappagari, Vani, Anne, Alain Volny, Young, Lital, Gill, John, Carlander, Christina, Grabar, Sophie, Jarrín, Inma, Meyer, Laurence, van der Valk, Marc, Wittkop, Linda, Aisam, Agnes, Barger, Diana, Davidovich, Udi, Dos Santos, Marie, Eriksson, Lars, Fitzgerald, Eli, Karakosta, Argyro, Krentz, Hartmut, Nicholls, Emily Jay, Policek, Nicoletta, Ruiz-Burga, Elisa, Sandford, Chris, Spire, Bruno, Suárez-García, Inés, Abgrall, Sophie, Andriantsoanirina, Valerie, Avettand-Fenoel, Veronique, Bourgeois, Christine, Chaix, Marie-Laure, Cheret, Antoine, Fischer, Hugues, Goujard, Cecile, Lascoux-Combe, Caroline, Le Palec, Annie, Petrov-Sanchez, Ventzlislava, Saez-Cirion, Asier, Seng, Remonie, Stefic, Karl, Tine, Josephine, Piet, E, Gagneux-Brunon, A, Jacomet, C, Piroth, L, Benezit, F, Goussef, M, Tattevin, P, Bani Sadr, B, Lamaury, I, Bazus, H, Robineau, O, Calin, R, Katlama, J, Denis, B, Ghosn, J, Joly, V, Khuong, M A, Caby, F C, Rouveix Nordon, E, de Truchis, P, Abgrall, S, Chéret, A, Duvivier, C, Becker, A, Miailhes, P, Abel, S, Unal, G, Makinson, A, Martin-Blondel, G, Morisot, A, Bregigeon, S, Enel, P, Allavena, C, Rabier, V, Vallet, L, Marchand, L, Saïdi, T, Costagliola, D, Grabar, S, Andriantsoanirina, V, Fischer, H, Marchand T Saïdi, L, Tattevin, Pierre, de Truchis, Pierre, Fischer, Hughes, Dalmau, David, Navarro, M Luisa, González, M Isabel, Garcia, Federico, Poveda, Eva, Iribarren, Jose Antonio, Gutiérrez, Félix, Rubio, Rafael, Vidal, Francesc, Berenguer, Juan, Muñoz-Fernández, M Ángeles, Adamis, G, Chini, M, Chrysos, G, Marangos, M, Katsarou, O, Kofteridis, D, Metallidis, S, Panagopoulos, P, Papadopoulos, A, Paparizos, V, Psychogiou, M, Sambatakou, H, Sipsas, N V, Touloumi, G, Fox, Julie, Terry, Louise, Waters, Anele, Uriel, Alison, Ustianowski, Andrew, Hackney, Pamela, Fahd, Niaz, Fidler, Sarah, Ayap, Wilbert, Molina, Marcelino, Waters, Laura, Nur, Fowsiya, Fernandez, Thomas, Nugent, Diarmuid, Pinedo, Javier, Reeves, Iain, Fong, Tracy, Nicholls, Jane, Cunningham, Laura, Pangan, Jaydee, Mackintosh, Claire, Sharp, Louise, Sabin, Caroline A, Van der Valk, Marc, Jarrin, Inma, van Sighem, Ard, Volny Anne, Alain, Costagliola, Dominique

Publikováno v: In The Lancet HIV October 2024 11(10):e660-e669

Mitigating harm in language models with conditional-likelihood filtration

Autor: Ngo, Helen, Raterink, Cooper, Araújo, João G. M., Zhang, Ivan, Chen, Carol, Morisot, Adrien, Frosst, Nicholas

Language models trained on large-scale unfiltered datasets curated from the open web acquire systemic biases, prejudices, and harmful views from their training data. We present a methodology for programmatically identifying and removing harmful text

Externí odkaz: http://arxiv.org/abs/2108.07790

Add a SideNet to your MainNet

Autor: Morisot, Adrien

As the performance and popularity of deep neural networks has increased, so too has their computational cost. There are many effective techniques for reducing a network's computational footprint (quantisation, pruning, knowledge distillation), but th

Externí odkaz: http://arxiv.org/abs/2007.13512

Real‐life challenges using personalized prognostic scoring systems in acute myeloid leukemia

Autor: Anne Calleja, Michael Loschi, Laurent Bailly, Adeline Morisot, Alice Marceau, Lionel Mannone, Guillaume Robert, Patrick Auberger, Claude Preudhomme, Sophie Raynaud, Fabien Subtil, Pierre Sujobert, Thomas Cluzeau

Publikováno v: Cancer Medicine, Vol 12, Iss 5, Pp 5656-5660 (2023)

Abstract Personalized medicine is a challenge for patients with acute myeloid leukemia (AML). The identification of several genetic mutations in several AML trials led to the creation of a personalized prognostic scoring algorithm known as the Knowle

Externí odkaz: https://doaj.org/article/d992efe825594a8d9a57c31cda98a4ed