{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T18:38:54Z","timestamp":1782239934982,"version":"3.54.5"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,16]],"date-time":"2025-01-16T00:00:00Z","timestamp":1736985600000},"content-version":"vor","delay-in-days":15,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Centrale Nantes"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>The promising outcomes of dynamical system identification techniques, such as SINDy (Brunton et al. in Proc Natl Acad Sci 113(15):3932\u20133937, 2016), highlight their advantages in providing qualitative interpretability and extrapolation compared to non-interpretable deep neural networks (Rudin in Nat Mach Intell 1(5):206\u2013215, 2019). These techniques suffer from parameter updating in real-time use cases, especially when the system parameters are likely to change during or between processes. Recently, the OASIS (Bhadriraju et al. in AIChE J 66(11):16980, 2020) framework introduced a data-driven technique to address the limitations of real-time dynamical system parameters updating, yielding interesting results. Nevertheless, we show in this work that superior performance can be achieved using more advanced model architectures. We present an innovative encoding approach, based mainly on the use of Set Encoding methods of sequence data, which give accurate adaptive model identification for complex dynamic systems, with variable input time series length. Two Set Encoding methods are used: the first is Deep Set (Zaheer et al. in Adv Neural Inf Process Syst 30, 2017), and the second is Set Transformer (Lee et al. in: International conference on machine learning, PMLR, pp 3744\u20133753 2019). Comparing Set Transformer to OASIS framework on Lotka\u2013Volterra for real-time local dynamical system identification and time series forecasting, we find that the Set Transformer architecture is well adapted to learning relationships within data sets. We then compare the two Set Encoding methods based on the Lorenz system for online global dynamical system identification. Finally, we trained a Deep Set model to perform identification and characterization of abnormalities for 1D heat-transfer problem.<\/jats:p>","DOI":"10.1007\/s10994-024-06732-7","type":"journal-article","created":{"date-parts":[[2025,1,16]],"date-time":"2025-01-16T16:27:09Z","timestamp":1737044829000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Adaptive parameters identification for nonlinear dynamics using deep permutation invariant networks"],"prefix":"10.1007","volume":"114","author":[{"given":"Mouad","family":"Elaarabi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Domenico","family":"Borzacchiello","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philippe Le","family":"Bot","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yves L. E.","family":"Guennec","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastien","family":"Comas-Cardona","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,1,16]]},"reference":[{"key":"6732_CR1","unstructured":"Ba, J. L., Kiros, J. R., & Hinton, G. E. (2016). Layer normalization. arXiv:1607.06450"},{"key":"6732_CR2","unstructured":"Becker, S., Klein, M., Neitz, A., Parascandolo, G., & Kilbertus, N. (2023). Predicting ordinary differential equations with transformers. In: International conference on machine learning (pp. 1978\u20132002). PMLR."},{"issue":"11","key":"6732_CR3","doi-asserted-by":"publisher","first-page":"16980","DOI":"10.1002\/aic.16980","volume":"66","author":"B Bhadriraju","year":"2020","unstructured":"Bhadriraju, B., Bangi, M. S. F., Narasingam, A., & Kwon, J.S.-I. (2020). Operable adaptive sparse identification of systems: Application to chemical processes. AIChE Journal, 66(11), 16980.","journal-title":"AIChE Journal"},{"key":"6732_CR4","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1016\/j.jprocont.2021.10.006","volume":"107","author":"B Bhadriraju","year":"2021","unstructured":"Bhadriraju, B., Kwon, J.S.-I., & Khan, F. (2021). OASIS-P: Operable adaptive sparse identification of systems for fault prognosis of chemical processes. Journal of Process Control, 107, 114\u2013126.","journal-title":"Journal of Process Control"},{"issue":"15","key":"6732_CR5","doi-asserted-by":"publisher","first-page":"3932","DOI":"10.1073\/pnas.1517384113","volume":"113","author":"SL Brunton","year":"2016","unstructured":"Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2016). Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15), 3932\u20133937.","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"18","key":"6732_CR6","doi-asserted-by":"publisher","first-page":"710","DOI":"10.1016\/j.ifacol.2016.10.249","volume":"49","author":"SL Brunton","year":"2016","unstructured":"Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2016). Sparse identification of nonlinear dynamics with control (SINDYc). IFAC-PapersOnLine, 49(18), 710\u2013715.","journal-title":"IFAC-PapersOnLine"},{"key":"6732_CR7","doi-asserted-by":"crossref","unstructured":"Chartrand, R. (2011). Numerical differentiation of noisy, nonsmooth data. International Scholarly Research Notices, 2011.","DOI":"10.5402\/2011\/164564"},{"issue":"1","key":"6732_CR8","doi-asserted-by":"publisher","first-page":"6136","DOI":"10.1038\/s41467-021-26434-1","volume":"12","author":"Z Chen","year":"2021","unstructured":"Chen, Z., Liu, Y., & Sun, H. (2021). Physics-informed learning of governing equations from scarce data. Nature Communications, 12(1), 6136.","journal-title":"Nature Communications"},{"issue":"2","key":"6732_CR9","doi-asserted-by":"publisher","first-page":"023174","DOI":"10.1103\/PhysRevResearch.4.023174","volume":"4","author":"Y Chen","year":"2022","unstructured":"Chen, Y., Luo, Y., Liu, Q., Xu, H., & Zhang, D. (2022). Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE). Physical Review Research, 4(2), 023174.","journal-title":"Physical Review Research"},{"key":"6732_CR10","doi-asserted-by":"publisher","first-page":"623","DOI":"10.7717\/peerj-cs.623","volume":"7","author":"D Chicco","year":"2021","unstructured":"Chicco, D., Warrens, M. J., & Jurman, G. (2021). The coefficient of determination r-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. Peerj Computer Science, 7, 623.","journal-title":"Peerj Computer Science"},{"key":"6732_CR11","unstructured":"Cranmer, M. (2023). Interpretable machine learning for science with PySR and Symbolic Regression. jl arXiv:2305.01582"},{"key":"6732_CR12","unstructured":"d\u2019Ascoli, S., Becker, S., Mathis, A., Schwaller, P., & Kilbertus, N. (2023). Odeformer: Symbolic regression of dynamical systems with transformers. arXiv:2310.05573"},{"key":"6732_CR13","doi-asserted-by":"crossref","unstructured":"Fasel, U., Kaiser, E., Kutz, J. N., Brunton, B. W., & Brunton, S. L. (2021). Sindy with control: A tutorial. In: 2021 60th IEEE conference on decision and control (CDC) (pp. 16\u201321). IEEE.","DOI":"10.1109\/CDC45484.2021.9683120"},{"issue":"6","key":"6732_CR14","doi-asserted-by":"publisher","first-page":"061006","DOI":"10.1115\/1.4043148","volume":"14","author":"P Gel\u00df","year":"2019","unstructured":"Gel\u00df, P., Klus, S., Eisert, J., & Sch\u00fctte, C. (2019). Multidimensional approximation of nonlinear dynamical systems. Journal of Computational and Nonlinear Dynamics, 14(6), 061006.","journal-title":"Journal of Computational and Nonlinear Dynamics"},{"issue":"2","key":"6732_CR15","doi-asserted-by":"publisher","first-page":"788","DOI":"10.1007\/s10618-022-00894-5","volume":"37","author":"H Hewamalage","year":"2023","unstructured":"Hewamalage, H., Ackermann, K., & Bergmeir, C. (2023). Forecast evaluation for data scientists: Common pitfalls and best practices. Data Mining and Knowledge Discovery, 37(2), 788\u2013832.","journal-title":"Data Mining and Knowledge Discovery"},{"issue":"2219","key":"6732_CR16","doi-asserted-by":"publisher","first-page":"20180335","DOI":"10.1098\/rspa.2018.0335","volume":"474","author":"E Kaiser","year":"2018","unstructured":"Kaiser, E., Kutz, J. N., & Brunton, S. L. (2018). Sparse identification of nonlinear dynamics for model predictive control in the low-data limit. Proceedings of the Royal Society A, 474(2219), 20180335.","journal-title":"Proceedings of the Royal Society A"},{"key":"6732_CR17","first-page":"10269","volume":"35","author":"P-A Kamienny","year":"2022","unstructured":"Kamienny, P.-A., d\u2019Ascoli, S., Lample, G., & Charton, F. (2022). End-to-end symbolic regression with transformers. Advances in Neural Information Processing Systems, 35, 10269\u201310281.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6732_CR18","unstructured":"Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv:1412.6980"},{"key":"6732_CR19","unstructured":"Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., & Teh, Y. W. (2019). Set transformer: A framework for attention-based permutation-invariant neural networks. In: International conference on machine learning (pp. 3744\u20133753). PMLR."},{"issue":"5","key":"6732_CR20","doi-asserted-by":"publisher","first-page":"1523","DOI":"10.1007\/s10994-023-06315-y","volume":"112","author":"C Lejeune","year":"2023","unstructured":"Lejeune, C., Mothe, J., Soubki, A., & Teste, O. (2023). Data driven discovery of systems of ordinary differential equations using nonconvex multitask learning. Machine Learning, 112(5), 1523\u20131549.","journal-title":"Machine Learning"},{"key":"6732_CR21","doi-asserted-by":"publisher","first-page":"108925","DOI":"10.1016\/j.jcp.2019.108925","volume":"399","author":"Z Long","year":"2019","unstructured":"Long, Z., Lu, Y., & Dong, B. (2019). PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network. Journal of Computational Physics, 399, 108925.","journal-title":"Journal of Computational Physics"},{"issue":"1","key":"6732_CR22","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1007\/s10462-023-10622-0","volume":"57","author":"N Makke","year":"2024","unstructured":"Makke, N., & Chawla, S. (2024). Interpretable scientific discovery with symbolic regression: A review. Artificial Intelligence Review, 57(1), 2.","journal-title":"Artificial Intelligence Review"},{"issue":"2","key":"6732_CR23","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1515\/cmam-2021-0131","volume":"22","author":"M Manucci","year":"2022","unstructured":"Manucci, M., Aguado, J. V., & Borzacchiello, D. (2022). Sparse data-driven quadrature rules via l p-quasi-norm minimization. Computational Methods in Applied Mathematics, 22(2), 389\u2013411.","journal-title":"Computational Methods in Applied Mathematics"},{"key":"6732_CR24","unstructured":"Martius, G., & Lampert, C. H. (2016). Extrapolation and learning equations. arXiv:1610.02995"},{"key":"6732_CR25","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1016\/j.compchemeng.2017.07.002","volume":"106","author":"A Narasingam","year":"2017","unstructured":"Narasingam, A., & Kwon, J.S.-I. (2017). Development of local dynamic mode decomposition with control: Application to model predictive control of hydraulic fracturing. Computers & Chemical Engineering, 106, 501\u2013511.","journal-title":"Computers & Chemical Engineering"},{"issue":"3","key":"6732_CR26","doi-asserted-by":"publisher","first-page":"464","DOI":"10.1111\/insr.12554","volume":"91","author":"JS North","year":"2023","unstructured":"North, J. S., Wikle, C. K., & Schliep, E. M. (2023). A review of data-driven discovery for dynamic systems. International Statistical Review, 91(3), 464\u2013492.","journal-title":"International Statistical Review"},{"issue":"1","key":"6732_CR28","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1137\/15M1013857","volume":"15","author":"JL Proctor","year":"2016","unstructured":"Proctor, J. L., Brunton, S. L., & Kutz, J. N. (2016). Dynamic mode decomposition with control. SIAM Journal on Applied Dynamical Systems, 15(1), 142\u2013161.","journal-title":"SIAM Journal on Applied Dynamical Systems"},{"key":"6732_CR29","doi-asserted-by":"crossref","unstructured":"Quade, M., Abel, M., Nathan\u00a0Kutz, J., & Brunton, S. L. (2018). Sparse identification of nonlinear dynamics for rapid model recovery. Chaos: An Interdisciplinary Journal of Nonlinear Science, 28(6).","DOI":"10.1063\/1.5027470"},{"key":"6732_CR30","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1016\/j.jcp.2018.10.045","volume":"378","author":"M Raissi","year":"2019","unstructured":"Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686\u2013707.","journal-title":"Journal of Computational Physics"},{"issue":"5","key":"6732_CR31","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","volume":"1","author":"C Rudin","year":"2019","unstructured":"Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206\u2013215.","journal-title":"Nature Machine Intelligence"},{"issue":"4","key":"6732_CR32","doi-asserted-by":"publisher","first-page":"1602614","DOI":"10.1126\/sciadv.1602614","volume":"3","author":"SH Rudy","year":"2017","unstructured":"Rudy, S. H., Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2017). Data-driven discovery of partial differential equations. Science Advances, 3(4), 1602614.","journal-title":"Science Advances"},{"key":"6732_CR33","doi-asserted-by":"crossref","unstructured":"Silva, B. M., Champion, K., Quade, M., Loiseau, J.-C., Kutz, J. N., & Brunton, S. L. (2020). Pysindy: A python package for the sparse identification of nonlinear dynamics from data. arXiv:2004.08424","DOI":"10.21105\/joss.02104"},{"key":"6732_CR34","first-page":"784","volume":"7","author":"I Sobol","year":"1967","unstructured":"Sobol, I. (1967). The distribution of points in a cube and the accurate evaluation of integrals (in Russian). Zhurnal Vychislitel\u2019noi Matematiki i Matematicheskoi Fiziki, 7, 784\u2013802.","journal-title":"Zhurnal Vychislitel\u2019noi Matematiki i Matematicheskoi Fiziki"},{"issue":"1","key":"6732_CR35","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","volume":"58","author":"R Tibshirani","year":"1996","unstructured":"Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1), 267\u2013288.","journal-title":"Journal of the Royal Statistical Society Series B: Statistical Methodology"},{"key":"6732_CR36","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, \u0141., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30."},{"issue":"3","key":"6732_CR37","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","volume":"17","author":"P Virtanen","year":"2020","unstructured":"Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020). SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 17(3), 261\u2013272.","journal-title":"Nature Methods"},{"issue":"1","key":"6732_CR38","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1146\/annurev.es.09.110178.001201","volume":"9","author":"PJ Wangersky","year":"1978","unstructured":"Wangersky, P. J. (1978). Lotka\u2013Volterra population models. Annual Review of Ecology and Systematics, 9(1), 189\u2013218.","journal-title":"Annual Review of Ecology and Systematics"},{"key":"6732_CR39","unstructured":"Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., & Smola, A. J. (2017). Deep sets. Advances in Neural Information Processing Systems, 30."},{"key":"6732_CR40","doi-asserted-by":"publisher","first-page":"1404","DOI":"10.1109\/ACCESS.2018.2886528","volume":"7","author":"P Zheng","year":"2018","unstructured":"Zheng, P., Askham, T., Brunton, S. L., Kutz, J. N., & Aravkin, A. Y. (2018). A unified framework for sparse relaxed regularized regression: SR3. IEEE Access, 7, 1404\u20131423.","journal-title":"IEEE Access"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06732-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-024-06732-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06732-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,7]],"date-time":"2025-02-07T16:21:22Z","timestamp":1738945282000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-024-06732-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1]]},"references-count":39,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["6732"],"URL":"https:\/\/doi.org\/10.1007\/s10994-024-06732-7","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1]]},"assertion":[{"value":"5 May 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 August 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 September 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 January 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"We affirm that there are no known Conflict of interest that have influenced this work.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"We confirm that this research complies with authors ethical standards. All authors consent to participate in the submitted work.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}}],"article-number":"22"}}