Výsledky vyhledávání

Report

Reasoning Beyond Bias: A Study on Counterfactual Prompting and Chain of Thought Reasoning

Autor: Moore, Kyle, Roberts, Jesse, Pham, Thao, Fisher, Douglas

Language models are known to absorb biases from their training data, leading to predictions driven by statistical regularities rather than semantic relevance. We investigate the impact of these biases on answer choice preferences in the Massive Multi

Externí odkaz: http://arxiv.org/abs/2408.08651

Zobrazit plný text záznamu

Report

Large Language Model Recall Uncertainty is Modulated by the Fan Effect

Autor: Roberts, Jesse, Moore, Kyle, Pham, Thao, Ewaleifoh, Oseremhen, Fisher, Doug

This paper evaluates whether large language models (LLMs) exhibit cognitive fan effects, similar to those discovered by Anderson in humans, after being pre-trained on human textual data. We conduct two sets of in-context recall experiments designed t

Externí odkaz: http://arxiv.org/abs/2407.06349

Zobrazit plný text záznamu

Report

The Base-Rate Effect on LLM Benchmark Performance: Disambiguating Test-Taking Strategies from Benchmark Performance

Autor: Moore, Kyle, Roberts, Jesse, Pham, Thao, Ewaleifoh, Oseremhen, Fisher, Doug

Cloze testing is a common method for measuring the behavior of large language models on a number of benchmark tasks. Using the MMLU dataset, we show that the base-rate probability (BRP) differences across answer tokens are significant and affect task

Externí odkaz: http://arxiv.org/abs/2406.11634

Zobrazit plný text záznamu

Report

Do Large Language Models Learn Human-Like Strategic Preferences?

Autor: Roberts, Jesse, Moore, Kyle, Fisher, Doug

In this paper, we evaluate whether LLMs learn to make human-like preference judgements in strategic scenarios as compared with known empirical results. Solar and Mistral are shown to exhibit stable value-based preference consistent with humans and ex

Externí odkaz: http://arxiv.org/abs/2404.08710

Zobrazit plný text záznamu

Report

Using Artificial Populations to Study Psychological Phenomena in Neural Models

Autor: Roberts, Jesse, Moore, Kyle, Wilenzick, Drew, Fisher, Doug

The recent proliferation of research into transformer based natural language processing has led to a number of studies which attempt to detect the presence of human-like cognitive behavior in the models. We contend that, as is true of human psycholog

Externí odkaz: http://arxiv.org/abs/2308.08032

Zobrazit plný text záznamu