Zobrazeno 1 - 10
of 20
pro vyhledávání: '"Holzenberger, Nils"'
Autor:
Hamdani, Rajaa El, Bonald, Thomas, Malliaros, Fragkiskos, Holzenberger, Nils, Suchanek, Fabian
This paper investigates the factuality of large language models (LLMs) as knowledge bases in the legal domain, in a realistic usage scenario: we allow for acceptable variations in the answer, and let the model abstain from answering when uncertain. F
Externí odkaz:
http://arxiv.org/abs/2409.11798
Autor:
Hou, Abe Bohan, Jurayj, William, Holzenberger, Nils, Blair-Stanek, Andrew, Van Durme, Benjamin
Large Language Models (LLMs) show promise as a writing aid for professionals performing legal analyses. However, LLMs can often hallucinate in this setting, in ways difficult to recognize by non-professionals and existing text evaluation metrics. In
Externí odkaz:
http://arxiv.org/abs/2409.09947
Autor:
Hou, Abe Bohan, Weller, Orion, Qin, Guanghui, Yang, Eugene, Lawrie, Dawn, Holzenberger, Nils, Blair-Stanek, Andrew, Van Durme, Benjamin
Legal professionals need to write analyses that rely on citations to relevant precedents, i.e., previous case decisions. Intelligent systems assisting legal professionals in writing such documents provide great benefits but are challenging to design.
Externí odkaz:
http://arxiv.org/abs/2406.17186
Statutory reasoning refers to the application of legislative provisions to a series of case facts described in natural language. We re-frame statutory reasoning as an analogy task, where each instance of the analogy task involves a combination of two
Externí odkaz:
http://arxiv.org/abs/2401.06715
We find that the best publicly available LLMs like GPT-4 and Claude currently perform poorly on basic legal text handling. This motivates the creation of a benchmark consisting of examples that lawyers and paralegals would expect LLMs to handle zero-
Externí odkaz:
http://arxiv.org/abs/2311.09693
Publikováno v:
180 TAX NOTES FEDERAL 1101 (AUG. 14, 2023)
The authors explain where OpenAI got the tax law example in its livestream demonstration of GPT-4, why GPT-4 got the wrong answer, and how it fails to reliably calculate taxes.
Comment: 5 pages
Comment: 5 pages
Externí odkaz:
http://arxiv.org/abs/2309.09992
LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
Autor:
Guha, Neel, Nyarko, Julian, Ho, Daniel E., Ré, Christopher, Chilton, Adam, Narayana, Aditya, Chohlas-Wood, Alex, Peters, Austin, Waldon, Brandon, Rockmore, Daniel N., Zambrano, Diego, Talisman, Dmitry, Hoque, Enam, Surani, Faiz, Fagan, Frank, Sarfaty, Galit, Dickinson, Gregory M., Porat, Haggai, Hegland, Jason, Wu, Jessica, Nudell, Joe, Niklaus, Joel, Nay, John, Choi, Jonathan H., Tobia, Kevin, Hagan, Margaret, Ma, Megan, Livermore, Michael, Rasumov-Rahe, Nikon, Holzenberger, Nils, Kolt, Noam, Henderson, Peter, Rehaag, Sean, Goel, Sharad, Gao, Shang, Williams, Spencer, Gandhi, Sunny, Zur, Tom, Iyer, Varun, Li, Zehua
The advent of large language models (LLMs) and their adoption by the legal community has given rise to the question: what types of legal reasoning can LLMs perform? To enable greater study of this question, we present LegalBench: a collaboratively co
Externí odkaz:
http://arxiv.org/abs/2308.11462
Statutory reasoning is the task of reasoning with facts and statutes, which are rules written in natural language by a legislature. It is a basic legal skill. In this paper we explore the capabilities of the most capable GPT-3 model, text-davinci-003
Externí odkaz:
http://arxiv.org/abs/2302.06100
Information Extraction (IE) researchers are mapping tasks to Question Answering (QA) in order to leverage existing large QA resources, and thereby improve data efficiency. Especially in template extraction (TE), mapping an ontology to a set of questi
Externí odkaz:
http://arxiv.org/abs/2205.12643
Statutory reasoning is the task of determining whether a legal statute, stated in natural language, applies to the text description of a case. Prior work introduced a resource that approached statutory reasoning as a monolithic textual entailment pro
Externí odkaz:
http://arxiv.org/abs/2105.07903