{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T07:16:27Z","timestamp":1761808587427,"version":"3.37.3"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"8","license":[{"start":{"date-parts":[[2022,9,10]],"date-time":"2022-09-10T00:00:00Z","timestamp":1662768000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,9,10]],"date-time":"2022-09-10T00:00:00Z","timestamp":1662768000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"engineering and physical sciences research council","doi-asserted-by":"publisher","award":["EP\/P010040\/1"],"award-info":[{"award-number":["EP\/P010040\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Auton Robot"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Focal-plane Sensor-processors (FPSPs) are a camera technology that enables low power, high frame rate computation in the image sensor itself, making them suitable for edge computation. To fit into the sensor array, FPSPs are highly resource-constrained, with limited instruction set and few registers - which makes developing complex algorithms difficult. In this work, we present Cain, a compiler for convolutional filters that targets SCAMP-5, a general-purpose FPSP. Cain generates code to evaluate multiple convolutional kernels at the same time. It generates code that avoids the need for hardware multipliers, while orchestrating the exploitation of common sub-terms\u2014leading to a large reduction in instruction count compared to both straightforward and prior optimized approaches. We demonstrate the capability enabled by Cain on SCAMP-5 with robotic navigation for near-sensor high-speed and low-power computation, by using Cain to implement a neural network on the focal plane.\n<\/jats:p>","DOI":"10.1007\/s10514-022-10053-w","type":"journal-article","created":{"date-parts":[[2022,9,10]],"date-time":"2022-09-10T13:02:36Z","timestamp":1662814956000},"page":"893-910","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Compiling CNNs with Cain: focal-plane processing for robot navigation"],"prefix":"10.1007","volume":"46","author":[{"given":"Edward","family":"Stow","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abrar","family":"Ahsan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yingying","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ali","family":"Babaei","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Riku","family":"Murai","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sajad","family":"Saeedi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul H. J.","family":"Kelly","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,9,10]]},"reference":[{"key":"10053_CR1","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., & Kudlur, M. (2016). Tensorflow: A system for large-scale machine learning. In 12th USENIX symposium on operating systems design and implementation OSDI 16), (pp. 265\u2013283)."},{"key":"10053_CR2","doi-asserted-by":"crossref","unstructured":"Barthels, H., Psarras, C., & Bientinesi, P. (2019). Linnea: Automatic generation of efficient linear algebra programs. arXiv:1912.12924.","DOI":"10.1145\/3394277.3401836"},{"key":"10053_CR3","doi-asserted-by":"crossref","unstructured":"Bose, L., Chen, J., Carey, S. J., Dudek, P., & Mayol-Cuevas, W. (2017). Visual odometry for pixel processor arrays. In 2017 IEEE international conference on computer vision (ICCV), (pp. 4614\u20134622).","DOI":"10.1109\/ICCV.2017.493"},{"key":"10053_CR4","doi-asserted-by":"crossref","unstructured":"Bose, L., Chen, J., Carey, S. J., Dudek, P., & Mayol-Cuevas, W. (2019). A camera that CNNs: Towards embedded neural networks on pixel processor arrays. In Proceedings of the IEEE international conference on computer vision (ICCV), (pp. 1335\u20131344).","DOI":"10.1109\/ICCV.2019.00142"},{"key":"10053_CR5","doi-asserted-by":"crossref","unstructured":"Carey, S. J., Barr, D. R. W., Wang, B., Lopich, A., & Dudek, P. (2012). Locating high speed multiple objects using a scamp-5 vision-chip. In 2012 13th international workshop on cellular nanoscale networks and their applications, (pp. 1\u20132).","DOI":"10.1109\/CNNA.2012.6331468"},{"key":"10053_CR6","unstructured":"Carey, S. J., Lopich, A., Barr, D. R. W., Wang, B., & Dudek, P. (2013). A 100,000 fps vision sensor with embedded 535GOPS\/W $$256 \\times 256$$ SIMD processor array. In 2013 symposium on VLSI circuits, (pp. C182\u2013C1830)."},{"issue":"1","key":"10053_CR7","doi-asserted-by":"publisher","first-page":"212","DOI":"10.1016\/j.jestch.2015.06.006","volume":"19","author":"A Chandra","year":"2016","unstructured":"Chandra, A., & Chattopadhyay, S. (2016). Design of hardware efficient fir filter: A review of the state-of-the-art approaches. Engineering Science and Technology, an International Journal, 19(1), 212\u2013226.","journal-title":"Engineering Science and Technology, an International Journal"},{"key":"10053_CR8","unstructured":"Chen, J. (2020). scamp5 kernel api macro analog.hpp file reference. https:\/\/scamp.gitlab.io\/scamp5d_doc\/."},{"key":"10053_CR9","doi-asserted-by":"crossref","unstructured":"Chen, J., Liu, Y., Carey, S. J., & Dudek, P. (2020). Proximity estimation using vision features computed on sensor. In 2020 IEEE international conference on robotics and automation (ICRA), (pp. 2689\u20132695).","DOI":"10.1109\/ICRA40945.2020.9197370"},{"issue":"1","key":"10053_CR10","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1109\/JSSC.2016.2616357","volume":"52","author":"YH Chen","year":"2016","unstructured":"Chen, Y. H., Krishna, T., Emer, J. S., & Sze, V. (2016). Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks. IEEE Journal of Solid-state Circuits, 52(1), 127\u2013138.","journal-title":"IEEE Journal of Solid-state Circuits"},{"key":"10053_CR11","unstructured":"Debrunner, T., Saeedi, S., Bose, L., Davison, A. J., & Kelly, P. H. J. (2019a). Camera Tracking on Focal-Plane Sensor-Processor Arrays."},{"key":"10053_CR12","doi-asserted-by":"crossref","unstructured":"Debrunner, T., Saeedi, S., & Kelly, P. H. J. (2019b). AUKE: Automatic kernel code generation for an analogue SIMD focal-plane sensor-processor array. ACM Transactions on Architecture and Code Optimization 15(4).","DOI":"10.1145\/3291055"},{"issue":"2","key":"10053_CR13","doi-asserted-by":"publisher","first-page":"661","DOI":"10.1109\/LRA.2015.2509024","volume":"1","author":"A Giusti","year":"2016","unstructured":"Giusti, A., Guzzi, J., Ciresan, D. C., He, F. L., Rodriguez, J. P., Fontana, F., Faessler, M., Forster, C., Schmidhuber, J., Caro, G. D., Scaramuzza, D., & Gambardella, L. M. (2016). A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots. IEEE Robotics and Automation Letters, 1(2), 661\u2013667.","journal-title":"IEEE Robotics and Automation Letters"},{"key":"10053_CR14","unstructured":"Greatwood, C., Bose, L., Richardson, T., Mayol-Cuevas, W., Clarke, R., Chen, J., & Carey, S. (2019). Towards drone racing with a pixel processor array. In 11th international micro air vehicle competition and conference (IMAV), (pp. 76\u201382)."},{"key":"10053_CR15","unstructured":"Guillard, B. (2019). Cnns-on-fpsps. https:\/\/github.com\/brouwa\/CNNs-on-FPSPs\/tree\/c6b5c51839e9e3c453681e5b0a3e3ef541ba3cce."},{"key":"10053_CR16","unstructured":"Kim, D. K., & Chen, T. (2015). Deep neural network for real-time autonomous indoor navigation. arXiv:1511.04668."},{"issue":"11","key":"10053_CR17","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y LeCun","year":"1998","unstructured":"LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278\u20132324.","journal-title":"Proceedings of the IEEE"},{"key":"10053_CR18","unstructured":"Liu, Y., Bose, L., Chen, J., Carey, S., Dudek, P., & Mayol-Cuevas, W. (2020). High-speed light-weight cnn inference via strided convolutions on a pixel processor array. In British machine vision virtual conference 2020."},{"issue":"9","key":"10053_CR19","doi-asserted-by":"publisher","first-page":"1883","DOI":"10.1049\/ipr2.12158","volume":"15","author":"Y Liu","year":"2021","unstructured":"Liu, Y., Bose, L., Greatwood, C., Chen, J., Fan, R., Richardson, T., Carey, S. J., Dudek, P., & Mayol-Cuevas, W. (2021). Agile reactive navigation for a non-holonomic mobile robot using a pixel processor array. IET Image Processing, 15(9), 1883\u20131892.","journal-title":"IET Image Processing"},{"key":"10053_CR20","unstructured":"Liu, Y., Chen, J., Bose, L., Dudek, P., & Mayol-Cuevas, W. (2021b). Bringing a robot simulator to the scamp vision system. arXiv:2105.10479."},{"issue":"2","key":"10053_CR21","doi-asserted-by":"publisher","first-page":"1088","DOI":"10.1109\/LRA.2018.2795643","volume":"3","author":"A Loquercio","year":"2018","unstructured":"Loquercio, A., Maqueda, A. I., Del-Blanco, C. R., & Scaramuzza, D. (2018). DroNet: Learning to Fly by Driving. IEEE Robotics and Automation Letters, 3(2), 1088\u20131095.","journal-title":"IEEE Robotics and Automation Letters"},{"key":"10053_CR22","unstructured":"Martel, J. (2019). Unconventional processing with unconventional visual sensing. PhD thesis, Institut National des Sciences Appliqu\u00e9es de Lyon."},{"issue":"7","key":"10053_CR23","doi-asserted-by":"publisher","first-page":"1642","DOI":"10.1109\/TPAMI.2020.2986944","volume":"42","author":"JNP Martel","year":"2020","unstructured":"Martel, J. N. P., M\u00fcller, L. K., Carey, S. J., Dudek, P., & Wetzstein, G. (2020). Neural sensors: Learning pixel exposures for HDR imaging and video compressive sensing with programmable sensors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(7), 1642\u20131653.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"10053_CR24","doi-asserted-by":"crossref","unstructured":"Murai, R., Saeedi, S., & Kelly, P. H. J. (2020). BIT-VO: Visual Odometry at 300 FPS using Binary Features from the Focal Plane. In IEEE\/RSJ international conference on intelligent robots and systems (IROS).","DOI":"10.1109\/IROS45743.2020.9341151"},{"issue":"11","key":"10053_CR25","doi-asserted-by":"publisher","first-page":"2020","DOI":"10.1109\/JPROC.2018.2856739","volume":"106","author":"S Saeedi","year":"2018","unstructured":"Saeedi, S., Bodin, B., Wagstaff, H., et al. (2018). Navigating the landscape for real-time localization and mapping for robotics and virtual and augmented reality. Proceedings of the IEEE, 106(11), 2020\u20132039.","journal-title":"Proceedings of the IEEE"},{"key":"10053_CR26","unstructured":"Stow, E. (2020). Automatic code generation for simultaneous convolutional kernels on cellular processor arrays. Master\u2019s thesis, Imperial College London."},{"key":"10053_CR27","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1007\/978-3-030-95953-1_13","volume-title":"Languages and compilers for parallel computing","author":"E Stow","year":"2022","unstructured":"Stow, E., Murai, R., Saeedi, S., & Kelly, P. H. J. (2022). Cain: Automatic code generation for simultaneous convolutional kernels on focal-plane sensor-processors. In B. Chapman & J. Moreira (Eds.), Languages and compilers for parallel computing (pp. 181\u2013197). Cham: Springer International Publishing."},{"key":"10053_CR28","unstructured":"Wong, M. Z., Guillard, B., Murai, R., Saeedi, S., & Kelly, P. H. J. (2020). AnalogNet: convolutional neural network inference on analog focal plane sensor processors. arXiv preprint arXiv:2006.01765."},{"key":"10053_CR29","unstructured":"XIMEA. (2021). xiB - PCI Express Cameras with high speed and resolution. https:\/\/www.ximea.com\/pci-express-camera\/pci-express-camera."},{"key":"10053_CR30","doi-asserted-by":"crossref","unstructured":"Zar\u00e1ndy, \u00c1. (2011). Focal-Plane Sensor-Processor Chips. SpringerLink: B\u00fccher, Springer, New York.","DOI":"10.1007\/978-1-4419-6475-5"}],"container-title":["Autonomous Robots"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-022-10053-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10514-022-10053-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-022-10053-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,5]],"date-time":"2022-11-05T10:14:24Z","timestamp":1667643264000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10514-022-10053-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,10]]},"references-count":30,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10053"],"URL":"https:\/\/doi.org\/10.1007\/s10514-022-10053-w","relation":{},"ISSN":["0929-5593","1573-7527"],"issn-type":[{"type":"print","value":"0929-5593"},{"type":"electronic","value":"1573-7527"}],"subject":[],"published":{"date-parts":[[2022,9,10]]},"assertion":[{"value":"1 July 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 June 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 September 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}