{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T23:15:58Z","timestamp":1771629358965,"version":"3.50.1"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2012,6,6]],"date-time":"2012-06-06T00:00:00Z","timestamp":1338940800000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Minds & Machines"],"published-print":{"date-parts":[[2012,11]]},"DOI":"10.1007\/s11023-012-9282-2","type":"journal-article","created":{"date-parts":[[2012,6,5]],"date-time":"2012-06-05T17:45:41Z","timestamp":1338918341000},"page":"299-324","source":"Crossref","is-referenced-by-count":64,"title":["Thinking Inside the Box: Controlling and Using an Oracle AI"],"prefix":"10.1007","volume":"22","author":[{"given":"Stuart","family":"Armstrong","sequence":"first","affiliation":[]},{"given":"Anders","family":"Sandberg","sequence":"additional","affiliation":[]},{"given":"Nick","family":"Bostrom","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2012,6,6]]},"reference":[{"key":"9282_CR1","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511978036","volume-title":"Machine ethics","author":"M Anderson","year":"2011","unstructured":"Anderson, M., & Anderson, S. L. (2011). Machine ethics. Cambridge: Cambridge University Press."},{"key":"9282_CR2","unstructured":"Armstrong, S. (2010). Utility indifference. Technical Report. Future of Humanity Institute, Oxford University, no. 2010-1."},{"key":"9282_CR3","unstructured":"Asimov, I. (1942). Runaround. In Street & Smith (Eds.). Astounding Science Fiction."},{"issue":"211","key":"9282_CR4","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1111\/1467-9213.00309","volume":"53","author":"N Bostrom","year":"2003","unstructured":"Bostrom, N. (2003a). Are you living in a computer simulation? Philosophical Quarterly, 53(211), 243\u2013255.","journal-title":"Philosophical Quarterly"},{"key":"9282_CR5","unstructured":"Bostrom, N. (2003b). Ethical issues in advanced artificial intelligence. Cognitive, Emotive and Ethical Aspects of Decision Making in Humans 2, 12\u201317."},{"key":"9282_CR6","unstructured":"Bostrom, N. (2001). Existential risks: Analyzing human extinction scenarios and related hazards. Journal of Evolution and Technology 9."},{"key":"9282_CR7","first-page":"44","volume":"10","author":"N Bostrom","year":"2011","unstructured":"Bostrom, N. (2011). Information hazards: A typology of potential harms from knowledge. Review of Contemporary Philosophy, 10, 44\u201379.","journal-title":"Review of Contemporary Philosophy"},{"key":"9282_CR8","unstructured":"Bostrom, N. (2000). Predictions from philosophy? Coloquia Manilana (PDCIS) 7."},{"key":"9282_CR10","first-page":"339","volume-title":"Death and anti-death: Two hundred years after Kant, fifty years after Turing","author":"N Bostrom","year":"2004","unstructured":"Bostrom, N. (2004). The future of human evolution. In C. Tandy (Ed.), Death and anti-death: Two hundred years after Kant, fifty years after Turing (pp. 339\u2013371). Palo Alto, California: Ria University Press."},{"key":"9282_CR9","unstructured":"Bostrom, N. (2012a). Superintelligence: An analysis of the coming machine intelligence revolution (in preparation)."},{"key":"9282_CR11","doi-asserted-by":"crossref","unstructured":"Bostrom, N. (2012b). The superintelligent will: Motivation and instrumental rationality in advanced artificial agents. Minds and Machines (Forthcoming).","DOI":"10.1007\/s11023-012-9281-3"},{"key":"9282_CR12","first-page":"504","volume-title":"Global catastrophic risks","author":"B Caplan","year":"2008","unstructured":"Caplan, B. (2008). The totalitarian threat. In N. Bostrom & M. Cirkovic (Eds.), Global catastrophic risks (pp. 504\u2013519). Oxford: Oxford University Press."},{"key":"9282_CR13","first-page":"7","volume":"17","author":"DJ Chalmers","year":"2010","unstructured":"Chalmers, D. J. (2010). The singularity: A philosophical analysis. Journal of Consciousness Studies, 17, 7\u201365.","journal-title":"Journal of Consciousness Studies"},{"key":"9282_CR14","unstructured":"de Blanc, P. (2011). Ontological Crises in Artificial Agents\u2019 Value Systems. arXiv:1105.3821v1 [cs.AI]."},{"key":"9282_CR15","doi-asserted-by":"crossref","unstructured":"Good, I. J. (1965). Speculations concerning the first ultraintelligent machine. Advances in Computers, 6, 31\u201383.","DOI":"10.1016\/S0065-2458(08)60418-0"},{"key":"9282_CR30","volume-title":"Beyond AI: Creating the conscience of the machine","author":"JS Hall","year":"2007","unstructured":"Hall, J. S. (2007). Beyond AI: Creating the conscience of the machine. Amherst, NY: Prometheus Books."},{"key":"9282_CR16","unstructured":"Hanson, R. (2001). Economic growth given machine intelligence. Journal of Artificial Intelligence Research."},{"key":"9282_CR17","doi-asserted-by":"crossref","DOI":"10.1353\/book13178","volume-title":"Golem: Jewish magical and mystical traditions on the artificial anthropoid","author":"M Idel","year":"1990","unstructured":"Idel, M. (1990). Golem: Jewish magical and mystical traditions on the artificial anthropoid. Albany, New York: State University of New York Press."},{"key":"9282_CR18","unstructured":"Kaas, S., Rayhawk, S., Salamon, A., & Salamon, P. (2010). Economic implications of software minds. http:\/\/www.sci.sdsu.edu\/~salamon\/SoftwareMinds.pdf ."},{"key":"9282_CR19","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511809477","volume-title":"Judgement under uncertainty: Heuristics and biases","author":"D Kahneman","year":"1982","unstructured":"Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgement under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press."},{"key":"9282_CR20","volume-title":"The singularity is near","author":"R Kurzweil","year":"2005","unstructured":"Kurzweil, R. (2005). The singularity is near. New York, NY: Penguin Group."},{"key":"9282_CR21","volume-title":"Thinking about foreign policy: Finding an appropriate role for artificial intelligence computers","author":"JC Mallery","year":"1988","unstructured":"Mallery, J. C. (1988). Thinking about foreign policy: Finding an appropriate role for artificial intelligence computers. Cambridge, MA: MIT Political Science Department."},{"key":"9282_CR22","unstructured":"McCarthy, J., Minsky M., Rochester N., & Shannon, C. (1956). Dartmouth Summer Research Conference on Artificial Intelligence."},{"key":"9282_CR23","unstructured":"Omohundro, S. (2008). The basic AI drives. In B. G. P. Wang (Ed.). Proceedings of the First AGI Conference (p. 171). Frontiers in Artificial Intelligence and Applications, IOS Press."},{"key":"9282_CR24","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1080\/13669870903126267","volume":"13","author":"T Ord","year":"2010","unstructured":"Ord, T., Hillerbrand, R., & Sandberg, A. (2010). Probing the improbable: Methodological challenges for risks with low probabilities and high stakes. Journal of Risk Research, 13, 191\u2013205.","journal-title":"Journal of Risk Research"},{"key":"9282_CR25","unstructured":"Russell, S., & Norvig, P. (2009). Artificial intelligence: A modern approach. 3. Englewood Cliffs, NJ: Prentice-Hall."},{"key":"9282_CR26","unstructured":"Sandberg, A. (2001). Friendly superintelligence. Presentation at Extro 5 conference, 15\u201317 June 2001. http:\/\/www.nada.kth.se\/~asa\/Extro5\/Friendly%20Superintelligence.htm ."},{"key":"9282_CR27","volume-title":"The strategy of conflict","author":"T Schelling","year":"1960","unstructured":"Schelling, T. (1960). The strategy of conflict. Massachusetts: Harvard University Press."},{"key":"9282_CR28","unstructured":"Shulman, C. (2010). Omohundro\u2019s \u201cbasic AI drives\u201d and catastrophic risks. http:\/\/singinst.org\/upload\/ai-resource-drives.pdf ."},{"key":"9282_CR29","volume-title":"The shape of automation for men and management","author":"HA Simon","year":"1965","unstructured":"Simon, H. A. (1965). The shape of automation for men and management. New York: Harper & Row."},{"key":"9282_CR31","volume-title":"Reinforcement learning: An introduction","author":"R Sutton","year":"1998","unstructured":"Sutton, R., & Barto, A. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press."},{"key":"9282_CR32","volume-title":"Theory of games and economic behavior","author":"J Neumann von","year":"1944","unstructured":"von Neumann, J., & Morgenstern, O. (1944). Theory of games and economic behavior. Princeton, NJ: Princeton University Press."},{"key":"9282_CR33","first-page":"308","volume-title":"Global catastrophic risks","author":"E Yudkowsky","year":"2008","unstructured":"Yudkowsky, E. (2008). Artificial intelligence as a positive and negative factor in global risk. In N. Bostrom & M. Cirkovic (Eds.), Global catastrophic risks (pp. 308\u2013345). Oxford: Oxford University Press."},{"key":"9282_CR34","unstructured":"Yudkowsky, E. (2001a). Creating friendly AI. Singularity Institute. http:\/\/singinst.org\/CFAI\/ ."},{"key":"9282_CR35","unstructured":"Yudkowsky, E. (2001b). Friendly AI 0.9. Singularity Institute. http:\/\/singinst.org\/CaTAI\/friendly\/contents.html ."},{"key":"9282_CR36","unstructured":"Yudkowsky, E. (2001c). General intelligence and seed AI 2.3. http:\/\/singinst.org\/ourresearch\/publications\/GISAI\/ ."},{"key":"9282_CR37","unstructured":"Yudkowsky, E. (2009). Paperclip maximiser. Less Wrong. http:\/\/wiki.lesswrong.com\/wiki\/Paperclip_maximizer ."},{"key":"9282_CR38","unstructured":"Yudkowsky, E. (2002). The AI-box experiment. Singularity Institute. http:\/\/yudkowsky.net\/singularity\/aibox ."}],"container-title":["Minds and Machines"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11023-012-9282-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s11023-012-9282-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11023-012-9282-2","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,23]],"date-time":"2023-06-23T03:01:37Z","timestamp":1687489297000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s11023-012-9282-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,6,6]]},"references-count":38,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2012,11]]}},"alternative-id":["9282"],"URL":"https:\/\/doi.org\/10.1007\/s11023-012-9282-2","relation":{},"ISSN":["0924-6495","1572-8641"],"issn-type":[{"value":"0924-6495","type":"print"},{"value":"1572-8641","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,6,6]]}}}