Zobrazeno 1 - 10
of 67
pro vyhledávání: '"Kumar, Ramana"'
Autor:
Phuong, Mary, Aitchison, Matthew, Catt, Elliot, Cogan, Sarah, Kaskasoli, Alexandre, Krakovna, Victoria, Lindner, David, Rahtz, Matthew, Assael, Yannis, Hodkinson, Sarah, Howard, Heidi, Lieberum, Tom, Kumar, Ramana, Raad, Maria Abi, Webson, Albert, Ho, Lewis, Lin, Sharon, Farquhar, Sebastian, Hutter, Marcus, Deletang, Gregoire, Ruoss, Anian, El-Sayed, Seliem, Brown, Sasha, Dragan, Anca, Shah, Rohin, Dafoe, Allan, Shevlane, Toby
To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four
Externí odkaz:
http://arxiv.org/abs/2403.13793
One of the most surprising puzzles in neural network generalisation is grokking: a network with perfect training accuracy but poor generalisation will, upon further training, transition to perfect generalisation. We propose that grokking occurs when
Externí odkaz:
http://arxiv.org/abs/2309.02390
One of the gnarliest challenges in reinforcement learning (RL) is exploration that scales to vast domains, where novelty-, or coverage-seeking behaviour falls short. Goal-directed, purposeful behaviours are able to overcome this, but rely on a good g
Externí odkaz:
http://arxiv.org/abs/2302.04693
Autor:
Uesato, Jonathan, Kushman, Nate, Kumar, Ramana, Song, Francis, Siegel, Noah, Wang, Lisa, Creswell, Antonia, Irving, Geoffrey, Higgins, Irina
Recent work has shown that asking language models to generate reasoning steps improves performance on many reasoning tasks. When moving beyond prompting, this raises the question of how we should supervise such models: outcome-based approaches which
Externí odkaz:
http://arxiv.org/abs/2211.14275
Autor:
Shah, Rohin, Varma, Vikrant, Kumar, Ramana, Phuong, Mary, Krakovna, Victoria, Uesato, Jonathan, Kenton, Zac
The field of AI alignment is concerned with AI systems that pursue unintended goals. One commonly studied mechanism by which an unintended goal might arise is specification gaming, in which the designer-provided specification is flawed in a way that
Externí odkaz:
http://arxiv.org/abs/2210.01790
Autor:
Kenton, Zachary, Kumar, Ramana, Farquhar, Sebastian, Richens, Jonathan, MacDermott, Matt, Everitt, Tom
Causal models of agents have been used to analyse the safety aspects of machine learning systems. But identifying agents is non-trivial -- often the causal model is just assumed by the modeler without much justification -- and modelling failures can
Externí odkaz:
http://arxiv.org/abs/2208.08345
Agents should avoid unsafe behaviour during both training and deployment. This typically requires a simulator and a procedural specification of unsafe behaviour. Unfortunately, a simulator is not always available, and procedurally specifying constrai
Externí odkaz:
http://arxiv.org/abs/2201.08102
Autor:
Sarma, Gopal, Koppel, James, Malecha, Gregory, Schultz, Patrick, Drexler, Eric, Kumar, Ramana, Roux, Cody, Zucker, Philip
Formal Methods for the Informal Engineer (FMIE) was a workshop held at the Broad Institute of MIT and Harvard in 2021 to explore the potential role of verified software in the biomedical software ecosystem. The motivation for organizing FMIE was the
Externí odkaz:
http://arxiv.org/abs/2104.00739
How can we design agents that pursue a given objective when all feedback mechanisms are influenceable by the agent? Standard RL algorithms assume a secure reward function, and can thus perform poorly in settings where agents can tamper with the rewar
Externí odkaz:
http://arxiv.org/abs/2011.08827
This paper describes REALab, a platform for embedded agency research in reinforcement learning (RL). REALab is designed to model the structure of tampering problems that may arise in real-world deployments of RL. Standard Markov Decision Process (MDP
Externí odkaz:
http://arxiv.org/abs/2011.08820