Zobrazeno 1 - 10
of 11
pro vyhledávání: '"Ábrego, Gustavo Hernández"'
Autor:
Gomez, Frank Palma, Sanabria, Ramon, Sung, Yun-hsuan, Cer, Daniel, Dalmia, Siddharth, Abrego, Gustavo Hernandez
Large language models (LLMs) are trained on text-only data that go far beyond the languages with paired speech and text data. At the same time, Dual Encoder (DE) based retrieval systems project queries and documents into the same embedding space and
Externí odkaz:
http://arxiv.org/abs/2404.01616
Autor:
Lee, Jinhyuk, Dai, Zhuyun, Ren, Xiaoqi, Chen, Blair, Cer, Daniel, Cole, Jeremy R., Hui, Kai, Boratko, Michael, Kapadia, Rajvi, Ding, Wen, Luan, Yi, Duddu, Sai Meher Karthik, Abrego, Gustavo Hernandez, Shi, Weiqiang, Gupta, Nithi, Kusupati, Aditya, Jain, Prateek, Jonnalagadda, Siddhartha Reddy, Chang, Ming-Wei, Naim, Iftekhar
We present Gecko, a compact and versatile text embedding model. Gecko achieves strong retrieval performance by leveraging a key idea: distilling knowledge from large language models (LLMs) into a retriever. Our two-step distillation process begins wi
Externí odkaz:
http://arxiv.org/abs/2403.20327
Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval
Autor:
Thakur, Nandan, Ni, Jianmo, Ábrego, Gustavo Hernández, Wieting, John, Lin, Jimmy, Cer, Daniel
There has been limited success for dense retrieval models in multilingual retrieval, due to uneven and scarce training data available across multiple languages. Synthetic training data generation is promising (e.g., InPars or Promptagator), but has b
Externí odkaz:
http://arxiv.org/abs/2311.05800
Autor:
Moiseev, Fedor, Abrego, Gustavo Hernandez, Dornbach, Peter, Zitouni, Imed, Alfonseca, Enrique, Dong, Zhe
Dual encoders have been used for retrieval tasks and representation learning with good results. A standard way to train dual encoders is using a contrastive loss with in-batch negatives. In this work, we propose an improved contrastive learning objec
Externí odkaz:
http://arxiv.org/abs/2306.02516
Autor:
Anil, Rohan, Dai, Andrew M., Firat, Orhan, Johnson, Melvin, Lepikhin, Dmitry, Passos, Alexandre, Shakeri, Siamak, Taropa, Emanuel, Bailey, Paige, Chen, Zhifeng, Chu, Eric, Clark, Jonathan H., Shafey, Laurent El, Huang, Yanping, Meier-Hellstern, Kathy, Mishra, Gaurav, Moreira, Erica, Omernick, Mark, Robinson, Kevin, Ruder, Sebastian, Tay, Yi, Xiao, Kefan, Xu, Yuanzhong, Zhang, Yujing, Abrego, Gustavo Hernandez, Ahn, Junwhan, Austin, Jacob, Barham, Paul, Botha, Jan, Bradbury, James, Brahma, Siddhartha, Brooks, Kevin, Catasta, Michele, Cheng, Yong, Cherry, Colin, Choquette-Choo, Christopher A., Chowdhery, Aakanksha, Crepy, Clément, Dave, Shachi, Dehghani, Mostafa, Dev, Sunipa, Devlin, Jacob, Díaz, Mark, Du, Nan, Dyer, Ethan, Feinberg, Vlad, Feng, Fangxiaoyu, Fienber, Vlad, Freitag, Markus, Garcia, Xavier, Gehrmann, Sebastian, Gonzalez, Lucas, Gur-Ari, Guy, Hand, Steven, Hashemi, Hadi, Hou, Le, Howland, Joshua, Hu, Andrea, Hui, Jeffrey, Hurwitz, Jeremy, Isard, Michael, Ittycheriah, Abe, Jagielski, Matthew, Jia, Wenhao, Kenealy, Kathleen, Krikun, Maxim, Kudugunta, Sneha, Lan, Chang, Lee, Katherine, Lee, Benjamin, Li, Eric, Li, Music, Li, Wei, Li, YaGuang, Li, Jian, Lim, Hyeontaek, Lin, Hanzhao, Liu, Zhongtao, Liu, Frederick, Maggioni, Marcello, Mahendru, Aroma, Maynez, Joshua, Misra, Vedant, Moussalem, Maysam, Nado, Zachary, Nham, John, Ni, Eric, Nystrom, Andrew, Parrish, Alicia, Pellat, Marie, Polacek, Martin, Polozov, Alex, Pope, Reiner, Qiao, Siyuan, Reif, Emily, Richter, Bryan, Riley, Parker, Ros, Alex Castro, Roy, Aurko, Saeta, Brennan, Samuel, Rajkumar, Shelby, Renee, Slone, Ambrose, Smilkov, Daniel, So, David R., Sohn, Daniel, Tokumine, Simon, Valter, Dasha, Vasudevan, Vijay, Vodrahalli, Kiran, Wang, Xuezhi, Wang, Pidong, Wang, Zirui, Wang, Tao, Wieting, John, Wu, Yuhuai, Xu, Kelvin, Xu, Yunhan, Xue, Linting, Yin, Pengcheng, Yu, Jiahui, Zhang, Qiao, Zheng, Steven, Zheng, Ce, Zhou, Weikang, Zhou, Denny, Petrov, Slav, Wu, Yonghui
We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Thr
Externí odkaz:
http://arxiv.org/abs/2305.10403
Autor:
Ni, Jianmo, Qu, Chen, Lu, Jing, Dai, Zhuyun, Ábrego, Gustavo Hernández, Ma, Ji, Zhao, Vincent Y., Luan, Yi, Hall, Keith B., Chang, Ming-Wei, Yang, Yinfei
It has been shown that dual encoders trained on one domain often fail to generalize to other domains for retrieval tasks. One widespread belief is that the bottleneck layer of a dual encoder, where the final score is simply a dot-product between a qu
Externí odkaz:
http://arxiv.org/abs/2112.07899
Autor:
Ni, Jianmo, Ábrego, Gustavo Hernández, Constant, Noah, Ma, Ji, Hall, Keith B., Cer, Daniel, Yang, Yinfei
We provide the first exploration of sentence embeddings from text-to-text transformers (T5). Sentence embeddings are broadly useful for language processing tasks. While T5 achieves impressive performance on language tasks cast as sequence-to-sequence
Externí odkaz:
http://arxiv.org/abs/2108.08877
In this paper we explore the effects of negative sampling in dual encoder models used to retrieve passages for automatic question answering. We explore four negative sampling strategies that complement the straightforward random sampling of negatives
Externí odkaz:
http://arxiv.org/abs/2010.12523
Autor:
Yang, Yinfei, Cer, Daniel, Ahmad, Amin, Guo, Mandy, Law, Jax, Constant, Noah, Abrego, Gustavo Hernandez, Yuan, Steve, Tar, Chris, Sung, Yun-Hsuan, Strope, Brian, Kurzweil, Ray
We introduce two pre-trained retrieval focused multilingual sentence encoding models, respectively based on the Transformer and CNN model architectures. The models embed text from 16 languages into a single semantic space using a multi-task trained d
Externí odkaz:
http://arxiv.org/abs/1907.04307
Autor:
Yang, Yinfei, Abrego, Gustavo Hernandez, Yuan, Steve, Guo, Mandy, Shen, Qinlan, Cer, Daniel, Sung, Yun-hsuan, Strope, Brian, Kurzweil, Ray
In this paper, we present an approach to learn multilingual sentence embeddings using a bi-directional dual-encoder with additive margin softmax. The embeddings are able to achieve state-of-the-art results on the United Nations (UN) parallel corpus r
Externí odkaz:
http://arxiv.org/abs/1902.08564