{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T07:53:32Z","timestamp":1782892412154,"version":"3.54.5"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1010512","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,10,7]],"date-time":"2022-10-07T00:00:00Z","timestamp":1665100800000}}],"reference-count":45,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2022,9,27]],"date-time":"2022-09-27T00:00:00Z","timestamp":1664236800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"publisher","award":["FA9550-14-1-0398"],"award-info":[{"award-number":["FA9550-14-1-0398"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100001906","name":"Washington Research Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100001906","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000936","name":"Gordon and Betty Moore Foundation","doi-asserted-by":"publisher","award":["2013-10-29"],"award-info":[{"award-number":["2013-10-29"]}],"id":[{"id":"10.13039\/100000936","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000879","name":"Alfred P. Sloan Foundation","doi-asserted-by":"publisher","award":["3835"],"award-info":[{"award-number":["3835"]}],"id":[{"id":"10.13039\/100000879","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Science Foundation Graduate Research Fellowship"},{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"publisher","award":["FA9550-19-1-0386"],"award-info":[{"award-number":["FA9550-19-1-0386"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"publisher","award":["FA9550-19-1-0386"],"award-info":[{"award-number":["FA9550-19-1-0386"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Insect flight is a strongly nonlinear and actuated dynamical system. As such, strategies for understanding its control have typically relied on either model-based methods or linearizations thereof. Here we develop a framework that combines model predictive control on an established flight dynamics model and <jats:italic>deep neural networks<\/jats:italic> (DNN) to create an efficient method for solving the inverse problem of flight control. We turn to natural systems for inspiration since they inherently demonstrate network pruning with the consequence of yielding more efficient networks for a specific set of tasks. This bio-inspired approach allows us to leverage network pruning to optimally sparsify a DNN architecture in order to perform flight tasks with as few neural connections as possible, however, there are limits to sparsification. Specifically, as the number of connections falls below a critical threshold, flight performance drops considerably. We develop sparsification paradigms and explore their limits for control tasks. Monte Carlo simulations also quantify the statistical distribution of network weights during pruning given initial random weights of the DNNs. We demonstrate that on average, the network can be pruned to retain a small amount of original network weights and still perform comparably to its fully-connected counterpart. The relative number of remaining weights, however, is highly dependent on the initial architecture and size of the network. Overall, this work shows that sparsely connected DNNs are capable of predicting the forces required to follow flight trajectories. Additionally, sparsification has sharp performance limits.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1010512","type":"journal-article","created":{"date-parts":[[2022,9,27]],"date-time":"2022-09-27T17:38:59Z","timestamp":1664300339000},"page":"e1010512","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["Pruning deep neural networks generates a sparse, bio-inspired nonlinear controller for insect flight"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3292-4446","authenticated-orcid":true,"given":"Olivia","family":"Zahn","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0185-2216","authenticated-orcid":true,"given":"Jorge","family":"Bustamante","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0901-169X","authenticated-orcid":true,"given":"Callin","family":"Switzer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5706-1096","authenticated-orcid":true,"given":"Thomas L.","family":"Daniel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6004-2275","authenticated-orcid":true,"given":"J. Nathan","family":"Kutz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2022,9,27]]},"reference":[{"issue":"7","key":"pcbi.1010512.ref001","doi-asserted-by":"crossref","first-page":"1759","DOI":"10.1162\/089976698300017124","article-title":"Synaptic Pruning in Development: A Computational Account","volume":"10","author":"G Chechik","year":"1998","journal-title":"Neural Computation"},{"issue":"3","key":"pcbi.1010512.ref002","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1016\/j.tics.2006.01.007","article-title":"Cognition through the lifespan: mechanisms of change","volume":"10","author":"FIM Craik","year":"2006","journal-title":"Trends in Cognitive Sciences"},{"key":"pcbi.1010512.ref003","doi-asserted-by":"crossref","first-page":"23","DOI":"10.3389\/fnsys.2017.00023","article-title":"Activity-Dependent Synaptic Refinement: New Insights from Drosophila","volume":"11","author":"F Vonhoff","year":"2017","journal-title":"Frontiers in Systems Neuroscience"},{"issue":"1","key":"pcbi.1010512.ref004","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.conb.2006.12.003","article-title":"Evolution and development of neural circuits in invertebrates","volume":"17","author":"PS Katz","year":"2007","journal-title":"Current Opinion in Neurobiology"},{"issue":"5880","key":"pcbi.1010512.ref005","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1038\/299250a0","article-title":"Metamorphosis of the insect nervous system: changes in morphology and synaptic interactions of identified neurones","volume":"299","author":"RB Levine","year":"1982","journal-title":"Nature"},{"issue":"11","key":"pcbi.1010512.ref006","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1038\/s41583-021-00507-y","article-title":"Mechanisms governing activity-dependent synaptic pruning in the developing mammalian CNS","volume":"22","author":"TE Faust","year":"2021","journal-title":"Nature Reviews Neuroscience"},{"issue":"1","key":"pcbi.1010512.ref007","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1113\/jphysiol.1962.sp006837","article-title":"Receptive fields, binocular interaction and functional architecture in the cat\u2019s visual cortex","volume":"160","author":"DH Hubel","year":"1962","journal-title":"The Journal of physiology"},{"issue":"1","key":"pcbi.1010512.ref008","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1113\/jphysiol.1978.sp012421","article-title":"Ocular dominance in layer IV of the cat\u2019s visual cortex and the effects of monocular deprivation","volume":"281","author":"CJ Shatz","year":"1978","journal-title":"The Journal of physiology"},{"issue":"5","key":"pcbi.1010512.ref009","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1016\/0893-6080(89)90020-8","article-title":"Multilayer feedforward networks are universal approximators","volume":"2","author":"K Hornik","year":"1989","journal-title":"Neural networks"},{"issue":"7553","key":"pcbi.1010512.ref010","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep Learning","volume":"521","author":"Y LeCun","year":"2015","journal-title":"Nature"},{"key":"pcbi.1010512.ref011","volume-title":"Deep learning","author":"I Goodfellow","year":"2016"},{"issue":"1","key":"pcbi.1010512.ref012","doi-asserted-by":"crossref","DOI":"10.1038\/s41467-018-07210-0","article-title":"Deep learning for universal linear embeddings of nonlinear dynamics","volume":"9","author":"B Lusch","year":"2018","journal-title":"Nature Communications"},{"issue":"4","key":"pcbi.1010512.ref013","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1007\/s00162-020-00520-4","article-title":"Deep model predictive flow control with limited sensor data and online learning","volume":"34","author":"K Bieker","year":"2020","journal-title":"Theoretical and Computational Fluid Dynamics"},{"key":"pcbi.1010512.ref014","unstructured":"Gomez AN, Zhang I, Kamalakara SR, Madaan D, Swersky K, Gal Y, et al. Learning sparse networks using targeted dropout. arXiv preprint arXiv:190513678. 2019;."},{"issue":"56","key":"pcbi.1010512.ref015","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"N Srivastava","year":"2014","journal-title":"Journal of Machine Learning Research"},{"key":"pcbi.1010512.ref016","unstructured":"Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, et al. Language Models are Few-Shot Learners; 2020."},{"key":"pcbi.1010512.ref017","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.neucom.2017.02.029","article-title":"Group sparse regularization for deep neural networks","volume":"241","author":"S Scardapane","year":"2017","journal-title":"Neurocomputing"},{"issue":"1","key":"pcbi.1010512.ref018","doi-asserted-by":"crossref","DOI":"10.1038\/s41467-018-04316-3","article-title":"Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science","volume":"9","author":"DC Mocanu","year":"2018","journal-title":"Nature Communications"},{"key":"pcbi.1010512.ref019","article-title":"Optimal brain damage","volume":"2","author":"Y LeCun","year":"1989","journal-title":"Advances in neural information processing systems"},{"key":"pcbi.1010512.ref020","unstructured":"Hassibi B, Stork DG, Wolff GJ. Optimal brain surgeon and general network pruning. In: IEEE international conference on neural networks. IEEE; 1993. p. 293\u2013299."},{"key":"pcbi.1010512.ref021","unstructured":"Louizos C, Welling M, Kingma DP. Learning Sparse Neural Networks through L0 Regularization; 2018."},{"key":"pcbi.1010512.ref022","unstructured":"Louizos C, Ullrich K, Welling M. Bayesian Compression for Deep Learning; 2017."},{"key":"pcbi.1010512.ref023","unstructured":"Kuzmin A, Nagel M, Pitre S, Pendyam S, Blankevoort T, Welling M. Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks; 2019."},{"issue":"31","key":"pcbi.1010512.ref024","doi-asserted-by":"crossref","first-page":"9728","DOI":"10.1073\/pnas.1421997112","article-title":"Robust network oscillations during mammalian respiratory rhythm generation driven by synaptic dynamics","volume":"112","author":"C Guerrier","year":"2015","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"33","key":"pcbi.1010512.ref025","doi-asserted-by":"crossref","first-page":"11772","DOI":"10.1523\/JNEUROSCI.1099-11.2011","article-title":"Cellular-resolution population imaging reveals robust sparse coding in the Drosophila mushroom body","volume":"31","author":"KS Honegger","year":"2011","journal-title":"Journal of neuroscience"},{"key":"pcbi.1010512.ref026","doi-asserted-by":"crossref","first-page":"102","DOI":"10.3389\/fncom.2018.00102","article-title":"Biological mechanisms for learning: a computational model of olfactory learning in the Manduca sexta moth, with applications to neural nets","volume":"12","author":"CB Delahunt","year":"2018","journal-title":"Frontiers in computational neuroscience"},{"issue":"6","key":"pcbi.1010512.ref027","doi-asserted-by":"crossref","first-page":"537","DOI":"10.1093\/icb\/36.6.537","article-title":"Unsteady Mechanisms of Force Generation in Aquatic and Aerial Locomotion","volume":"36","author":"MH Dickinson","year":"1996","journal-title":"American Zoologist"},{"issue":"23","key":"pcbi.1010512.ref028","doi-asserted-by":"crossref","first-page":"4191","DOI":"10.1242\/jeb.00663","article-title":"The aerodynamics of insect flight","volume":"206","author":"SP Sane","year":"2003","journal-title":"Journal of Experimental Biology"},{"key":"pcbi.1010512.ref029","doi-asserted-by":"crossref","first-page":"3114","DOI":"10.1242\/jeb.02363","article-title":"Flight control in the hawkmoth Manduca sexta: the inverse problem of hovering","volume":"209","author":"TL Hedrick","year":"2006","journal-title":"The Journal of Experimental Biology"},{"key":"pcbi.1010512.ref030","doi-asserted-by":"crossref","first-page":"1523","DOI":"10.1242\/jeb.077644","article-title":"Flexible strategies for flight control: an active role for the abdomen","volume":"216","author":"JP Dyhr","year":"2013","journal-title":"The Journal of Experimental Biology"},{"key":"pcbi.1010512.ref031","article-title":"Abdominal movements in insect flight reshape the role of non-aerodynamic structures for flight maneuverability I: Model predictive control for flower tracking","author":"J Bustamante","year":"2022","journal-title":"Integrative Organismal Biology"},{"key":"pcbi.1010512.ref032","unstructured":"Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al.. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems; 2015. Available from: https:\/\/www.tensorflow.org\/."},{"key":"pcbi.1010512.ref033","unstructured":"Zahn O. MothPruning; 2021. https:\/\/github.com\/oliviatessa\/MothPruning."},{"issue":"022712","key":"pcbi.1010512.ref034","first-page":"1","article-title":"Wing-pitch modulation in maneuvering fruit flies is explained by an interplay between aerodynamics and a torsional spring","volume":"92","author":"T Beatus","year":"2015","journal-title":"Physical Review E"},{"key":"pcbi.1010512.ref035","article-title":"Neural evidence supports a dual sensory-motor role for insect wings","volume":"284","author":"B Pratt","year":"2017","journal-title":"Proceedings of the Royal Society B: Biological Sciences"},{"key":"pcbi.1010512.ref036","doi-asserted-by":"crossref","unstructured":"Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, et al. SciPy 1.0\u2013Fundamental Algorithms for Scientific Computing in Python. arXiv e-prints. 2019; p. arXiv:1907.10121.","DOI":"10.1038\/s41592-020-0772-5"},{"key":"pcbi.1010512.ref037","unstructured":"Chollet F, et al. Keras; 2015. https:\/\/keras.io."},{"key":"pcbi.1010512.ref038","unstructured":"Bradbury J, Frostig R, Hawkins P, Johnson MJ, Leary C, Maclaurin D, et al. JAX: composable transformations of Python+NumPy programs; 2018. Available from: http:\/\/github.com\/google\/jax."},{"key":"pcbi.1010512.ref039","unstructured":"Kingma DP, Ba J. Adam: A Method for Stochastic Optimization; 2017."},{"key":"pcbi.1010512.ref040","unstructured":"Zhu M, Gupta S. To prune, or not to prune: exploring the efficacy of pruning for model compression; 2017."},{"issue":"7825","key":"pcbi.1010512.ref041","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1038\/s41586-020-2649-2","article-title":"Array programming with NumPy","volume":"585","author":"CR Harris","year":"2020","journal-title":"Nature"},{"key":"pcbi.1010512.ref042","unstructured":"NVIDIA, Vingelmann P, Fitzek FHP. CUDA, release: 10.2.89; 2020. Available from: https:\/\/developer.nvidia.com\/cuda-toolkit."},{"issue":"3","key":"pcbi.1010512.ref043","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1016\/j.neuron.2011.10.006","article-title":"Computational Mechanisms of Sensorimotor Control","volume":"72","author":"D Franklin","year":"2011","journal-title":"Neuron"},{"key":"pcbi.1010512.ref044","unstructured":"Li Z, Wallace E, Shen S, Lin K, Keutzer K, Klein D, et al. Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers; 2020. Available from: https:\/\/arxiv.org\/abs\/2002.11794."},{"issue":"6","key":"pcbi.1010512.ref045","article-title":"Feedback through graph motifs relates structure and function in complex networks","volume":"98","author":"Y Hu","year":"2018","journal-title":"Physical Review E"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1010512","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,10,7]],"date-time":"2022-10-07T00:00:00Z","timestamp":1665100800000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010512","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,7]],"date-time":"2022-10-07T17:47:34Z","timestamp":1665164854000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010512"}},"subtitle":[],"editor":[{"given":"Joseph","family":"Ayers","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2022,9,27]]},"references-count":45,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,9,27]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1010512","relation":{"new_version":[{"id-type":"doi","id":"10.1371\/journal.pcbi.1010512","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,27]]}}}