Zobrazeno 1 - 10
of 352
pro vyhledávání: '"SCHWARTZ, ROY"'
We consider two classic problems: maximum coverage and monotone submodular maximization subject to a cardinality constraint. [Nemhauser--Wolsey--Fisher '78] proved that the greedy algorithm provides an approximation of $1-1/e$ for both problems, and
Externí odkaz:
http://arxiv.org/abs/2411.05553
Natural language is composed of words, but modern LLMs process sub-words as input. A natural question raised by this discrepancy is whether LLMs encode words internally, and if so how. We present evidence that LLMs engage in an intrinsic detokenizati
Externí odkaz:
http://arxiv.org/abs/2410.05864
Autor:
Ben-Artzy, Amit, Schwartz, Roy
In decoder-based LLMs, the representation of a given layer serves two purposes: as input to the next layer during the computation of the current token; and as input to the attention mechanism of future tokens. In this work, we show that the importanc
Externí odkaz:
http://arxiv.org/abs/2409.03621
Autor:
Kuznetsov, Ilia, Afzal, Osama Mohammed, Dercksen, Koen, Dycke, Nils, Goldberg, Alexander, Hope, Tom, Hovy, Dirk, Kummerfeld, Jonathan K., Lauscher, Anne, Leyton-Brown, Kevin, Lu, Sheng, Mausam, Mieskes, Margot, Névéol, Aurélie, Pruthi, Danish, Qu, Lizhen, Schwartz, Roy, Smith, Noah A., Solorio, Thamar, Wang, Jingyan, Zhu, Xiaodan, Rogers, Anna, Shah, Nihar B., Gurevych, Iryna
The number of scientific articles produced every year is growing rapidly. Providing quality control over them is crucial for scientists and, ultimately, for the public good. In modern science, this process is largely delegated to peer review -- a dis
Externí odkaz:
http://arxiv.org/abs/2405.06563
Autor:
Mamou, Jonathan, Pereg, Oren, Korat, Daniel, Berchansky, Moshe, Timor, Nadav, Wasserblat, Moshe, Schwartz, Roy
Speculative decoding is commonly used for reducing the inference latency of large language models. Its effectiveness depends highly on the speculation lookahead (SL)-the number of tokens generated by the draft model at each iteration. In this work we
Externí odkaz:
http://arxiv.org/abs/2405.04304
Autor:
Reif, Yuval, Schwartz, Roy
Large language models (LLMs) have shown remarkable adaptability to diverse tasks, by leveraging context prompts containing instructions, or minimal input-output examples. However, recent work revealed they also exhibit label bias -- an undesirable pr
Externí odkaz:
http://arxiv.org/abs/2405.02743
It is a common belief that large language models (LLMs) are better than smaller-sized ones. However, larger models also require significantly more time and compute during inference. This begs the question: what happens when both models operate under
Externí odkaz:
http://arxiv.org/abs/2404.00725
Transformers are considered conceptually different from the previous generation of state-of-the-art NLP models - recurrent neural networks (RNNs). In this work, we demonstrate that decoder-only transformers can in fact be conceptualized as unbounded
Externí odkaz:
http://arxiv.org/abs/2401.06104
We study graph ordering problems with a min-max objective. A classical problem of this type is cutwidth, where given a graph we want to order its vertices such that the number of edges crossing any point is minimized. We give a $ \log^{1+o(1)}(n)$ ap
Externí odkaz:
http://arxiv.org/abs/2311.15639
In this paper we consider the online Submodular Welfare (SW) problem. In this problem we are given $n$ bidders each equipped with a general (not necessarily monotone) submodular utility and $m$ items that arrive online. The goal is to assign each ite
Externí odkaz:
http://arxiv.org/abs/2308.07746