Zobrazeno 1 - 10
of 26
pro vyhledávání: '"Sonkar, Shashank"'
This paper introduces MalAlgoQA, a novel dataset designed to evaluate the counterfactual reasoning capabilities of Large Language Models (LLMs) through a pedagogical approach. The dataset comprises mathematics and reading comprehension questions, eac
Externí odkaz:
http://arxiv.org/abs/2407.00938
Autor:
Sonkar, Shashank, Baraniuk, Richard G.
We introduce Many-Shot Regurgitation (MSR) prompting, a new black-box membership inference attack framework for examining verbatim content reproduction in large language models (LLMs). MSR prompting involves dividing the input text into multiple segm
Externí odkaz:
http://arxiv.org/abs/2405.08134
This paper presents a novel exploration into the regressive side effects of training Large Language Models (LLMs) to mimic student misconceptions for personalized education. We highlight the problem that as LLMs are trained to more accurately mimic s
Externí odkaz:
http://arxiv.org/abs/2404.15156
Autor:
Sonkar, Shashank, Ni, Kangqi, Lu, Lesa Tran, Kincaid, Kristi, Hutchinson, John S., Baraniuk, Richard G.
We introduce a new area of study in the field of educational Natural Language Processing: Automated Long Answer Grading (ALAG). Distinguishing itself from Automated Short Answer Grading (ASAG) and Automated Essay Grading (AEG), ALAG presents unique c
Externí odkaz:
http://arxiv.org/abs/2404.14316
In this paper, we introduce "Marking", a novel grading task that enhances automated grading systems by performing an in-depth analysis of student responses and providing students with visual highlights. Unlike traditional systems that provide binary
Externí odkaz:
http://arxiv.org/abs/2404.14301
In this paper, we introduce the novel concept of pedagogically aligned Large Language Models (LLMs) that signifies a transformative shift in the application of LLMs within educational contexts. Rather than providing direct responses to user queries,
Externí odkaz:
http://arxiv.org/abs/2402.05000
We propose novel evaluations for mathematical reasoning capabilities of Large Language Models (LLMs) based on mathematical misconceptions. Our primary approach is to simulate LLMs as a novice learner and an expert tutor, aiming to identify the incorr
Externí odkaz:
http://arxiv.org/abs/2310.02439
Autor:
Sonkar, Shashank, Le, MyCo, Chen, Xinghe, Liu, Naiming, Mallick, Debshila Basu, Baraniuk, Richard G.
High-quality conversational datasets are crucial for the successful development of Intelligent Tutoring Systems (ITS) that utilize a Large Language Model (LLM) backend. Synthetic student-teacher dialogues, generated using advanced GPT-4 models, are a
Externí odkaz:
http://arxiv.org/abs/2309.12161
Deduction under Perturbed Evidence: Probing Student Simulation Capabilities of Large Language Models
Autor:
Sonkar, Shashank, Baraniuk, Richard G.
We explore whether Large Language Models (LLMs) are capable of logical reasoning with distorted facts, which we call Deduction under Perturbed Evidence (DUPE). DUPE presents a unique challenge to LLMs since they typically rely on their parameters, wh
Externí odkaz:
http://arxiv.org/abs/2305.14507
Autor:
Sonkar, Shashank, Baraniuk, Richard G.
This paper investigates the key role of Feed-Forward Networks (FFNs) in transformer models by utilizing the Parallel Attention and Feed-Forward Net Design (PAF) architecture, and comparing it to their Series Attention and Feed-Forward Net Design (SAF
Externí odkaz:
http://arxiv.org/abs/2305.13297