Content area
Full Text
Keywords: Oracle AI, AI safety, CAS, theorem proving, math oracles
Received: June 24, 2013
In the context of superintelligent AI systems, the term "oracle" has two meanings. One refers to modular systems queried for domain-speciñc tasks. Another usage, referring to a class of systems which may be useful for addressing the value alignment and AI control problems, is a superintelligent AI system that only answers questions. The aim of this manuscript is to survey contemporary research problems related to oracles which align with long-term research goals of AI safety. We examine existing question answering systems and argue that their high degree of architectural heterogeneity makes them poor candidates for rigorous analysis as oracles. On the other hand, we identify computer algebra systems (CASs) as being primitive examples of domain-speciñc oracles for mathematics and argue that efforts to integrate computer algebra systems with theorem provers, systems which have largely been developed independent of one another, provide a concrete set of problems related to the notion of provable safety that has emerged in the AI safety community. We review approaches to interfacing CASs with theorem provers, describe well-deñned architectural deñciencies that have been identiñed with CASs, and suggest possible lines of research and practical software projects for scientists interested in AI safety.
Povzetek: Obravnavani so raziskovalni problemi, povezani z racunskimi sistemi s prerokom in varnostjo umetne inteligence.
1Introduction
Recently, significant public attention has been drawn to the consequences of achieving human-level artificial intelligence. While there have been small communities analyzing the long-term impact of AI and related technologies for decades, these forecasts were made before the many recent breakthroughs that have dramatically accelerated the pace of research in areas as diverse as robotics, computer vision, and autonomous vehicles, to name just a few [1-3].
Most researchers and industrialists view advances in artificial intelligence as having the potential to be overwhelmingly beneficial to humanity. Medicine, transportation, and fundamental scientific research are just some of the areas that are actively being transformed by advances in artificial intelligence. On the other hand, issues of privacy and surveillance, access and inequality, or economics and policy are also of utmost importance and are distinct from the specific technical challenges posed by most cuttingedge research problems [4, 5].
In the context of...