{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T16:28:31Z","timestamp":1770740911157,"version":"3.49.0"},"reference-count":54,"publisher":"MIT Press - Journals","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2018,4]]},"abstract":"<jats:p> We extend the neural Turing machine (NTM) model into a dynamic neural Turing machine (D-NTM) by introducing trainable address vectors. This addressing scheme maintains for each memory cell two separate vectors, content and address vectors. This allows the D-NTM to learn a wide variety of location-based addressing strategies, including both linear and nonlinear ones. We implement the D-NTM with both continuous and discrete read and write mechanisms. We investigate the mechanisms and effects of learning to read and write into a memory through experiments on Facebook bAbI tasks using both a feedforward and GRU controller. We provide extensive analysis of our model and compare different variations of neural Turing machines on this task. We show that our model outperforms long short-term memory and NTM variants. We provide further experimental results on the sequential [Formula: see text]MNIST, Stanford Natural Language Inference, associative recall, and copy tasks. <\/jats:p>","DOI":"10.1162\/neco_a_01060","type":"journal-article","created":{"date-parts":[[2018,1,30]],"date-time":"2018-01-30T21:36:27Z","timestamp":1517348187000},"page":"857-884","source":"Crossref","is-referenced-by-count":33,"title":["Dynamic Neural Turing Machine with Continuous and Discrete Addressing Schemes"],"prefix":"10.1162","volume":"30","author":[{"given":"Caglar","family":"Gulcehre","sequence":"first","affiliation":[{"name":"University of Montreal, Montreal QC H3T 1J4, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sarath","family":"Chandar","sequence":"additional","affiliation":[{"name":"University of Montreal, Montreal QC H3T 1J4, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kyunghyun","family":"Cho","sequence":"additional","affiliation":[{"name":"CIFAR Azrieli Global Scholar, New York University, New York 10003, NY, U.S.A."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yoshua","family":"Bengio","sequence":"additional","affiliation":[{"name":"CIFAR Senior Fellow, University of Montreal, Montreal QC H3T 1J4, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.279"},{"key":"B2","author":"Arjovsky M.","year":"2016","journal-title":"Proceedings of the International Conference on Machine Learning"},{"key":"B3","author":"Ba J. L.","year":"2016","journal-title":"Layer normalization"},{"key":"B4","author":"Bahdanau D.","year":"2015","journal-title":"Proceedings of the International Conference on Representation Learning"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1109\/72.279181"},{"key":"B6","author":"Bordes A.","year":"2015","journal-title":"Large-scale simple question answering with memory networks"},{"key":"B7","author":"Bowman S. R.","year":"2015","journal-title":"A large annotated corpus for learning natural language inference"},{"key":"B8","author":"Chandar S.","year":"2016","journal-title":"Hierarchical memory networks"},{"key":"B9","author":"Cho K.","year":"2014","journal-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing"},{"key":"B10","author":"Chorowski J.","year":"2015","journal-title":"Attention-based models for speech recognition"},{"key":"B11","author":"Cooijmans T.","year":"2017","journal-title":"Proceedings of the International Conference on Representation Learning"},{"key":"B12","author":"Dodge J.","year":"2015","journal-title":"Evaluating prerequisite qualities for learning end-to-end dialog systems"},{"key":"B13","author":"Goodfellow I.","year":"2016","journal-title":"Deep learning"},{"key":"B14","author":"Graves A.","year":"2014","journal-title":"Neural Turing machines"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1038\/nature20101"},{"key":"B16","first-page":"1819","volume-title":"Advances in neural information processing systems","volume":"29","author":"Grefenstette E.","year":"2015"},{"key":"B17","author":"Gulcehre C.","year":"2016","journal-title":"Proceedings of the International Conference on Machine Learning"},{"key":"B18","author":"Henaff M.","year":"2016","journal-title":"Tracking the world state with recurrent entity networks"},{"key":"B19","author":"Hermann K. M.","year":"2015","journal-title":"Teaching machines to read and comprehend"},{"key":"B20","author":"Hill F.","year":"2015","journal-title":"The Goldilocks principle: Reading children's books with explicit memory representations"},{"key":"B21","author":"Hochreiter S.","year":"1991","journal-title":"Untersuchungen zu dynamischen neuronalen netzen"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177703732"},{"key":"B24","first-page":"190","volume-title":"Advances in neural information processing systems","volume":"28","author":"Joulin A.","year":"2015"},{"key":"B25","author":"Kingma D. P.","year":"2014","journal-title":"Adam: A method for stochastic optimization"},{"key":"B26","author":"Krueger D.","year":"2016","journal-title":"Zoneout: Regularizing RNNs by randomly preserving hidden activations"},{"key":"B27","author":"Kumar A.","year":"2015","journal-title":"Ask me anything: Dynamic memory networks for natural language processing"},{"key":"B28","author":"Le Q. V.","year":"2015","journal-title":"A simple way to initialize recurrent networks of rectified linear units"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1166"},{"key":"B30","author":"Miller A.","year":"2016","journal-title":"Key-value memory networks for directly reading documents"},{"key":"B31","author":"Mnih A.","year":"2014","journal-title":"Proceedings of the International Conference on Machine Learning"},{"key":"B32","first-page":"807","author":"Nair V.","year":"2010","journal-title":"Proceedings of the 27th International Conference on Machine Learning"},{"key":"B33","first-page":"1532","author":"Pennington J.","year":"2014","journal-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing"},{"key":"B34","author":"Rae J. W.","year":"2016","journal-title":"30th Conference on Neural Information Processing Systems"},{"key":"B35","author":"Reed S.","year":"2016","journal-title":"Proceedings of the International Conference on Representation Learning"},{"key":"B36","author":"Rockt\u00e4schel T.","year":"2015","journal-title":"Reasoning about entailment with neural attention"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1044"},{"key":"B38","author":"Santoro A.","year":"2016","journal-title":"Proceedings of the International Conference on Machine Learning"},{"key":"B39","first-page":"1047","volume-title":"Advances in neural information processing systems","volume":"7","author":"Schmidhuber J.","year":"1995"},{"key":"B40","author":"Semeniuta S.","year":"2016","journal-title":"Recurrent dropout without memory loss"},{"key":"B41","author":"Seo M.","year":"2016","journal-title":"Query-reduction networks for question answering"},{"key":"B42","author":"Serban I. V.","year":"2016","journal-title":"Proceedings of the 30th AAAI Conference on Artificial Intelligence"},{"key":"B43","author":"Sukhbaatar S.","year":"2015","journal-title":"End-to-end memory networks"},{"key":"B44","volume-title":"Adaptive processing of sequences and data structures","author":"Sun G.","year":"1997"},{"key":"B45","author":"Vinyals O.","year":"2015","journal-title":"A neural conversational model"},{"key":"B47","author":"Weston J.","year":"2015","journal-title":"Proceedings of the International Conference on Representation Learning"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992696"},{"key":"B49","author":"Xiong C.","year":"2016","journal-title":"Dynamic memory networks for visual and textual question answering"},{"key":"B50","author":"Xu K.","year":"2015","journal-title":"Proceedings of the International Conference on Representation Learning"},{"key":"B51","author":"Yang G.","year":"2016","journal-title":"Lie access neural Turing machine"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.512"},{"key":"B53","author":"Zaremba W.","year":"2015","journal-title":"Learning simple algorithms from examples"},{"key":"B54","author":"Zaremba W.","year":"2015","journal-title":"Reinforcement learning neural Turing machines"},{"key":"B55","author":"Zhang W.","year":"2015","journal-title":"Structured memory for neural Turing machines"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/neco_a_01060","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:42:22Z","timestamp":1615585342000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/30\/4\/857-884\/8367"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,4]]},"references-count":54,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,4]]}},"alternative-id":["10.1162\/neco_a_01060"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_01060","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,4]]}}}