{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T20:23:43Z","timestamp":1776889423884,"version":"3.51.2"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2024,2,8]],"date-time":"2024-02-08T00:00:00Z","timestamp":1707350400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,8]],"date-time":"2024-02-08T00:00:00Z","timestamp":1707350400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Scientific Research Grant of Shantou University, China","award":["NTF17016"],"award-info":[{"award-number":["NTF17016"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["82071992"],"award-info":[{"award-number":["82071992"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100021171","name":"Basic and Applied Basic Research Foundation of Guangdong Province","doi-asserted-by":"crossref","award":["2020B1515120061"],"award-info":[{"award-number":["2020B1515120061"]}],"id":[{"id":"10.13039\/501100021171","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The retinal impulse signal is the basic carrier of visual information. It records the distribution of light on the retina. However, its direct conversion to a scene image is difficult due to the nonlinear characteristics of its distribution. Therefore, the use of artificial neural network to reconstruct the scene from retinal spikes has become an important research area. This paper proposes the architecture of a neural network based on vector quantization, where the feature vectors of spike trains are extracted, compressed, and stored using a feature extraction and compression network. During the decoding process, the nearest neighbour search method is used to find the nearest feature vector corresponding to each feature vector in the feature map. Finally, a reconstruction network is used to decode a new feature map composed of matching feature vectors to obtain a visual scene. This paper also verifies the impact of vector quantization on the characteristics of pulse signals by comparing experiments and visualizing the characteristics before and after vector quantization. The network delivers promising performance when evaluated on different datasets, demonstrating that this research is of great significance for improving relevant applications in the fields of retinal image processing and artificial intelligence.<\/jats:p>","DOI":"10.1007\/s40747-023-01333-8","type":"journal-article","created":{"date-parts":[[2024,2,8]],"date-time":"2024-02-08T09:02:49Z","timestamp":1707382969000},"page":"3445-3458","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Retinal spike train decoder using vector quantization for visual scene reconstruction"],"prefix":"10.1007","volume":"10","author":[{"given":"Kunwu","family":"Ma","sequence":"first","affiliation":[]},{"given":"Alex Noel Joseph","family":"Raj","sequence":"additional","affiliation":[]},{"given":"Vijayarajan","family":"Rajangam","sequence":"additional","affiliation":[]},{"given":"Tardi","family":"Tjahjadi","sequence":"additional","affiliation":[]},{"given":"Minying","family":"Liu","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7236-6859","authenticated-orcid":false,"given":"Zhemin","family":"Zhuang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,2,8]]},"reference":[{"key":"1333_CR1","doi-asserted-by":"publisher","DOI":"10.3389\/fneur.2021.661938","author":"US Kim","year":"2021","unstructured":"Kim US, Mahroo OA, Mollon JD, Yu-Wai-Man P (2021) Retinal ganglion cells-diversity of cell types and clinical relevance. Front Neurol. https:\/\/doi.org\/10.3389\/fneur.2021.661938","journal-title":"Front Neurol"},{"key":"1333_CR2","doi-asserted-by":"publisher","first-page":"877","DOI":"10.1038\/nn0901-877","volume":"4","author":"RH Masland","year":"2001","unstructured":"Masland RH (2001) The fundamental plan of the retina. Nat Neurosci 4:877\u2013886. https:\/\/doi.org\/10.1038\/nn0901-877","journal-title":"Nat Neurosci"},{"key":"1333_CR3","doi-asserted-by":"publisher","first-page":"995","DOI":"10.1038\/nature07140","volume":"454","author":"JW Pillow","year":"2008","unstructured":"Pillow JW, Shlens J, Paninski L, Sher A, Litke AM, Chichilnisky EJ, Simoncelli EP (2008) Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature 454:995\u2013999. https:\/\/doi.org\/10.1038\/nature07140","journal-title":"Nature"},{"issue":"5","key":"1333_CR4","doi-asserted-by":"publisher","first-page":"1412","DOI":"10.1109\/TBME.2014.2314733","volume":"61","author":"JD Weiland","year":"2014","unstructured":"Weiland JD, Humayun MS (2014) Retinal prosthesis. IEEE Trans Biomed Eng 61(5):1412\u20131424. https:\/\/doi.org\/10.1109\/TBME.2014.2314733","journal-title":"IEEE Trans Biomed Eng"},{"key":"1333_CR5","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1146\/annurev-vision-091517-034055","volume":"4","author":"WN Grimes","year":"2018","unstructured":"Grimes WN, Songco-Aguas A, Rieke F (2018) Parallel processing of rod and cone signals: retinal function and human perception. Ann Rev Vis Sci 4:123\u2013141. https:\/\/doi.org\/10.1146\/annurev-vision-091517-034055","journal-title":"Ann Rev Vis Sci"},{"issue":"1","key":"1333_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0053063","volume":"8","author":"R G\u00fctig","year":"2013","unstructured":"G\u00fctig R, Gollisch T, Sompolinsky H, Meister M (2013) Computing complex visual features with retinal spike times. PLoS ONE 8(1):1\u201315. https:\/\/doi.org\/10.1371\/journal.pone.0053063","journal-title":"PLoS ONE"},{"issue":"2","key":"1333_CR7","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1016\/j.neuron.2009.12.009","volume":"65","author":"T Gollisch","year":"2010","unstructured":"Gollisch T, Meister M (2010) Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron 65(2):150\u2013164","journal-title":"Neuron"},{"key":"1333_CR8","unstructured":"Gershenson, C.: Artificial neural networks for beginners (2003). CoRR arXiv:cs.NE\/0308031"},{"key":"1333_CR9","unstructured":"Rivest F, Bengio Y, Kalaska J (2004) Brain inspired reinforcement learning. In: Saul L, Weiss Y, Bottou L (eds) Advances in neural information processing systems 17. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2004\/file\/d37b3ca37106b2bfdeaa12647e3bb1c9-Paper.pdf"},{"issue":"4","key":"1333_CR10","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1631\/FITEE.1700833","volume":"20","author":"Z-B Wu","year":"2019","unstructured":"Wu Z-B, Yu J-Q (2019) Vector quantization: a review. Front Inf Technol Electron Eng 20(4):507\u2013524. https:\/\/doi.org\/10.1631\/FITEE.1700833","journal-title":"Front Inf Technol Electron Eng"},{"issue":"7","key":"1333_CR11","doi-asserted-by":"publisher","first-page":"3236","DOI":"10.1109\/TIP.2018.2817044","volume":"27","author":"J Li","year":"2018","unstructured":"Li J, Li B, Xu J, Xiong R, Gao W (2018) Fully connected network-based intra prediction for image coding. IEEE Trans Image Process 27(7):3236\u20133247. https:\/\/doi.org\/10.1109\/TIP.2018.2817044","journal-title":"IEEE Trans Image Process"},{"key":"1333_CR12","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1016\/j.patcog.2017.10.013","volume":"77","author":"J Gu","year":"2018","unstructured":"Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J et al (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354\u2013377. https:\/\/doi.org\/10.1016\/j.patcog.2017.10.013","journal-title":"Pattern Recogn"},{"issue":"5","key":"1333_CR13","doi-asserted-by":"publisher","first-page":"1218","DOI":"10.1109\/TPAMI.2019.2893965","volume":"42","author":"H Gao","year":"2020","unstructured":"Gao H, Yuan H, Wang Z, Ji S (2020) Pixel transposed convolutional networks. IEEE Trans Pattern Anal Mach Intell 42(5):1218\u20131227. https:\/\/doi.org\/10.1109\/TPAMI.2019.2893965","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"2","key":"1333_CR14","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1038\/nn.2733","volume":"14","author":"AB Graf","year":"2011","unstructured":"Graf AB, Kohn A, Jazayeri M, Movshon JA (2011) Decoding the activity of neuronal populations in macaque primary visual cortex. Nat Neurosci 14(2):239\u2013245. https:\/\/doi.org\/10.1038\/nn.2733","journal-title":"Nat Neurosci"},{"key":"1333_CR15","doi-asserted-by":"publisher","unstructured":"Hinrikus H, Karai D, Lass J, Rodina A (2010) Effect of noise in processing of visual information. In: Nonlinear biomedical physics, vol 4. Springer, pp 1\u20137. https:\/\/doi.org\/10.1186\/1753-4631-4-S1-S5","DOI":"10.1186\/1753-4631-4-S1-S5"},{"issue":"7801","key":"1333_CR16","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1038\/s41586-020-2130-2","volume":"580","author":"OI Rumyantsev","year":"2020","unstructured":"Rumyantsev OI, Lecoq JA, Hernandez O, Zhang Y, Savall J, Chrapkiewicz R, Li J, Zeng H, Ganguli S, Schnitzer MJ (2020) Fundamental bounds on the fidelity of sensory cortical coding. Nature 580(7801):100\u2013105. https:\/\/doi.org\/10.1038\/s41586-020-2130-2","journal-title":"Nature"},{"issue":"47","key":"1333_CR17","doi-asserted-by":"publisher","first-page":"11003","DOI":"10.1523\/JNEUROSCI.3305-05.2005","volume":"25","author":"JW Pillow","year":"2005","unstructured":"Pillow JW, Paninski L, Uzzell VJ, Simoncelli EP, Chichilnisky E (2005) Prediction and decoding of retinal ganglion cell responses with a probabilistic spiking model. J Neurosci 25(47):11003\u201311013. https:\/\/doi.org\/10.1523\/JNEUROSCI.3305-05.2005","journal-title":"J Neurosci"},{"key":"1333_CR18","doi-asserted-by":"publisher","unstructured":"Nee R, Zelst A, Awater G (2000) Maximum likelihood decoding in a space division multiplexing system. In: VTC2000-Spring. 2000 IEEE 51st vehicular technology conference proceedings (Cat. No.00CH37026), vol 1, pp 6\u2013101. https:\/\/doi.org\/10.1109\/VETECS.2000.851407","DOI":"10.1109\/VETECS.2000.851407"},{"key":"1333_CR19","doi-asserted-by":"crossref","unstructured":"D\u00edaz-Tahoces A, Martinez-Alvarez A, Garcia-Moll A, Humphreys L, Bolea J\u00c1, Fern\u00e1ndez E (2015) Towards the reconstruction of moving images by populations of retinal ganglion cells. In: Artificial computation in biology and medicine: international work-conference on the interplay between natural and artificial computation, IWINAC 2015, Elche, Spain, June 1\u20135, 2015, Proceedings, Part I 6. Springer, pp 220\u2013227","DOI":"10.1007\/978-3-319-18914-7_23"},{"key":"1333_CR20","doi-asserted-by":"publisher","unstructured":"Kurita T (2019) Principal component analysis (pca). In: Computer vision: a reference guide, pp 1\u20134. https:\/\/doi.org\/10.1007\/978-3-030-03243-2_649-1","DOI":"10.1007\/978-3-030-03243-2_649-1"},{"issue":"7","key":"1333_CR21","doi-asserted-by":"publisher","first-page":"1719","DOI":"10.1162\/neco_a_01395","volume":"33","author":"YJ Kim","year":"2021","unstructured":"Kim YJ, Brackbill N, Batty E, Lee J, Mitelut C, Tong W, Chichilnisky E, Paninski L (2021) Nonlinear decoding of natural images from large-scale primate retinal ganglion recordings. Neural Comput 33(7):1719\u20131750. https:\/\/doi.org\/10.1162\/neco_a_01395","journal-title":"Neural Comput"},{"key":"1333_CR22","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1016\/j.neunet.2020.01.033","volume":"125","author":"Y Zhang","year":"2020","unstructured":"Zhang Y, Jia S, Zheng Y, Yu Z, Tian Y, Ma S, Huang T, Liu JK (2020) Reconstruction of natural visual scenes from neural spikes with deep neural networks. Neural Netw 125:19\u201330. https:\/\/doi.org\/10.1016\/j.neunet.2020.01.033","journal-title":"Neural Netw"},{"key":"1333_CR23","doi-asserted-by":"publisher","unstructured":"Xi E, Bing S, Jin Y (2017) Capsule network performance on complex data https:\/\/doi.org\/10.48550\/arXiv.1712.03480","DOI":"10.48550\/arXiv.1712.03480"},{"issue":"13","key":"1333_CR24","doi-asserted-by":"publisher","first-page":"15164","DOI":"10.1007\/s10489-022-03402-w","volume":"52","author":"W Li","year":"2022","unstructured":"Li W, Joseph Raj AN, Tjahjadi T, Zhuang Z (2022) Fusion of ANNs as decoder of retinal spike trains for scene reconstruction. Appl Intell 52(13):15164\u201315176. https:\/\/doi.org\/10.1007\/s10489-022-03402-w","journal-title":"Appl Intell"},{"key":"1333_CR25","unstructured":"Van Den Oord A, Vinyals O et al (2017) Neural discrete representation learning. In: Guyon I, Von Luxburg U, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/7a98af17e63a0ac09ce2e96d03992fbc-Paper.pdf"},{"key":"1333_CR26","unstructured":"Kingma DP, Welling M (2022) Auto-encoding variational bayes. arXiv:1312.6114 [stat.ML]"},{"key":"1333_CR27","unstructured":"Razavi A, Oord A, Vinyals O (2019) Generating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems, vol 32. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2019\/file\/5f8e2fa1718d1bbcadf1cd9c7a54fb8c-Paper.pdf"},{"key":"1333_CR28","doi-asserted-by":"crossref","unstructured":"Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)","DOI":"10.1109\/CVPR.2017.298"},{"key":"1333_CR29","doi-asserted-by":"crossref","unstructured":"Chen Y, Dai X, Liu M, Chen D, Yuan L, Liu Z (2020) Dynamic relu. In: Computer vision\u2013ECCV 2020: 16th European conference, Glasgow, UK, August 23\u201328, 2020, proceedings, Part XIX 16, pp 351\u2013367","DOI":"10.1007\/978-3-030-58529-7_21"},{"issue":"7","key":"1333_CR30","doi-asserted-by":"publisher","first-page":"1344","DOI":"10.1109\/TUFFC.2020.2972573","volume":"67","author":"H Lee","year":"2020","unstructured":"Lee H, Park J, Hwang JY (2020) Channel attention module with multiscale grid average pooling for breast cancer segmentation in an ultrasound image. IEEE Trans Ultrason Ferroelectr Freq Control 67(7):1344\u20131353. https:\/\/doi.org\/10.1109\/TUFFC.2020.2972573","journal-title":"IEEE Trans Ultrason Ferroelectr Freq Control"},{"key":"1333_CR31","doi-asserted-by":"crossref","unstructured":"Liu Z, Cheng K-T, Huang D, Xing EP, Shen Z (2022) Nonuniform-to-uniform quantization: towards accurate quantization via generalized straight-through estimation. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 4942\u20134952","DOI":"10.1109\/CVPR52688.2022.00489"},{"issue":"2","key":"1333_CR32","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1109\/MSP.2010.939537","volume":"28","author":"I To\u0161i\u0107","year":"2011","unstructured":"To\u0161i\u0107 I, Frossard P (2011) Dictionary learning. IEEE Signal Process Mag 28(2):27\u201338. https:\/\/doi.org\/10.1109\/MSP.2010.939537","journal-title":"IEEE Signal Process Mag"},{"issue":"5","key":"1333_CR33","doi-asserted-by":"publisher","first-page":"1302","DOI":"10.1162\/NECO_a_00434","volume":"25","author":"C-P Lee","year":"2013","unstructured":"Lee C-P, Lin C-J (2013) A study on l2-loss (squared hinge-loss) multiclass svm. Neural Comput 25(5):1302\u20131323. https:\/\/doi.org\/10.1162\/NECO_a_00434","journal-title":"Neural Comput"},{"key":"1333_CR34","doi-asserted-by":"publisher","unstructured":"Haynes D, Corns S, Venayagamoorthy GK (2012) An exponential moving average algorithm. In: 2012 IEEE Congress on evolutionary computation. IEEE, pp 1\u20138. https:\/\/doi.org\/10.1109\/CEC.2012.6252962","DOI":"10.1109\/CEC.2012.6252962"},{"key":"1333_CR35","doi-asserted-by":"crossref","unstructured":"Tjandra A, Sisman B, Zhang M, Sakti S, Li H, Nakamura S (2019) VQVAE unsupervised unit discovery and multi-scale code2spec inverter for zerospeech challenge 2019. CoRR arXiv:1905.11449","DOI":"10.21437\/Interspeech.2019-3232"},{"issue":"3","key":"1333_CR36","doi-asserted-by":"publisher","first-page":"1009925","DOI":"10.1371\/journal.pcbi.1009925","volume":"18","author":"JK Liu","year":"2022","unstructured":"Liu JK, Karamanlis D, Gollisch T (2022) Simple model for encoding natural images by retinal ganglion cells with nonlinear spatial integration. PLoS Comput Biol 18(3):1009925. https:\/\/doi.org\/10.1371\/journal.pcbi.1009925","journal-title":"PLoS Comput Biol"},{"key":"1333_CR37","doi-asserted-by":"publisher","first-page":"49","DOI":"10.3389\/fninf.2017.00049","volume":"11","author":"B Cessac","year":"2017","unstructured":"Cessac B, Kornprobst P, Kraria S, Nasser H, Pamplona D, Portelli G, Vi\u00e9ville T (2017) Pranas: a new platform for retinal analysis and simulation. Front Neuroinform 11:49. https:\/\/doi.org\/10.3389\/fninf.2017.00049","journal-title":"Front Neuroinform"},{"issue":"4","key":"1333_CR38","doi-asserted-by":"publisher","first-page":"1488","DOI":"10.1109\/TIP.2011.2173206","volume":"21","author":"D Brunet","year":"2012","unstructured":"Brunet D, Vrscay ER, Wang Z (2012) On the mathematical properties of the structural similarity index. IEEE Trans Image Process 21(4):1488\u20131499. https:\/\/doi.org\/10.1109\/TIP.2011.2173206","journal-title":"IEEE Trans Image Process"},{"key":"1333_CR39","doi-asserted-by":"publisher","unstructured":"Schluchter MD (2005). Mean square error. https:\/\/doi.org\/10.1002\/0470011815.b2a15087","DOI":"10.1002\/0470011815.b2a15087"},{"key":"1333_CR40","doi-asserted-by":"publisher","unstructured":"Johnson DH (2006) Signal-to-noise ratio. Scholarpedia 1(12):2088. https:\/\/doi.org\/10.4249\/scholarpedia.2088","DOI":"10.4249\/scholarpedia.2088"},{"key":"1333_CR41","unstructured":"Arora S, Hu W, Kothari PK (2018) An analysis of the t-sne algorithm for data visualization. In: Conference on learning theory. PMLR, pp 1455\u20131462. https:\/\/proceedings.mlr.press\/v75\/arora18a.html"},{"key":"1333_CR42","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1016\/j.neunet.2018.12.002","volume":"111","author":"A Tavanaei","year":"2019","unstructured":"Tavanaei A, Ghodrati M, Kheradpisheh SR, Masquelier T, Maida A (2019) Deep learning in spiking neural networks. Neural Netw 111:47\u201363. https:\/\/doi.org\/10.1016\/j.neunet.2018.12.002","journal-title":"Neural Netw"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01333-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-023-01333-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01333-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T18:13:09Z","timestamp":1715883189000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-023-01333-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,8]]},"references-count":42,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["1333"],"URL":"https:\/\/doi.org\/10.1007\/s40747-023-01333-8","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,8]]},"assertion":[{"value":"20 July 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 December 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 February 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}