{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T06:19:26Z","timestamp":1787552366841,"version":"build-2736575974"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T00:00:00Z","timestamp":1787529600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004052","name":"King Abdullah University of Science and Technology","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004052","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Existing multi-layer perceptron (MLP)-mixer architectures either rely heavily on extensive training data or employ rigid, hand-engineered mixing operations. This study introduces a Basis-Driven Learnable Operator (BDLO) for MLP-mixers, targeting a balance between high-capacity flexible learning and computationally efficient, yet constrained, handcrafted solutions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>BDLO is a drop-in replacement for the shifting block of shifting-based MLP-mixers. It approximates shifting-based mixing operations by learning the coefficients of a real, complete, discrete basis, yielding an input-dependent transformation matrix that is applied to both rows and columns of the token table, while the patch embedding, channel-mixing MLPs, skip connections and classification head are left unchanged. The operator was integrated into CycleMLP, HireMLP and AS-MLP at three model sizes each, and all models were trained from scratch under an identical configuration on CIFAR10, CIFAR100, a reduced ImageNet (32 \u00d7 32, 500 classes) and full ImageNet1K. Standard and discrete cosine transform bases were compared, hyperparameter sensitivity was assessed with Optuna, and differences were tested using the Wilcoxon signed-rank test.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      BDLO reduced the cost of the mixing layer to 1.68 GFLOPs, against 3.33\u20135.08 GFLOPs for the original layers, and reduced whole-model parameter counts by 21.8%\u201356.7% (mean reduction 12.84M), with a mean accuracy difference of 0.38% in favor of BDLO. The Wilcoxon test confirmed a significantly lower parameter distribution (\n                      <jats:italic>p<\/jats:italic>\n                      = 0.0039) and no statistically significant accuracy difference (\n                      <jats:italic>p<\/jats:italic>\n                      = 0.496 on CIFAR100,\n                      <jats:italic>p<\/jats:italic>\n                      = 0.0625 on CIFAR10,\n                      <jats:italic>p<\/jats:italic>\n                      = 0.5 on the reduced ImageNet). Results were invariant to the choice of basis (mean cosine distance 0.0049 between the learned coefficient vectors) and insensitive to the BDLO-specific hyperparameters.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>Comparability holds in aggregate and in the parameter-constrained regime, with model-specific exceptions for baselines that include channel mixing (HireMLP) and for high-capacity baselines on larger datasets (AS-MLP). BDLO behaves as an input-adaptive spectral modulator whose bounded coefficients provide implicit regularization. This confirms the effectiveness of BDLO as an efficient operator replacement for shifting-based MLP-mixers, most suitable for parameter-constrained, small-to-medium-scale models in image recognition tasks.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frai.2026.1769436","type":"journal-article","created":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T05:30:35Z","timestamp":1787549435000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Basis-driven learnable operator for MLP-mixers"],"prefix":"10.3389","volume":"9","author":[{"given":"Ahmed","family":"Elsheikh","sequence":"first","affiliation":[{"name":"Mathematics and Engineering Physics Department, Faculty of Engineering, Cairo University","place":["Giza, Egypt"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohammed E.","family":"Fouda","sequence":"additional","affiliation":[{"name":"Compumacy for Artificial Intelligence Solutions","place":["Giza, Egypt"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmed M.","family":"Eltawil","sequence":"additional","affiliation":[{"name":"CEMSE Division, King Abdullah University of Science and Technology","place":["Thuwal, Saudi Arabia"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,8,24]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1109\/T-C.1974.223784","article-title":"Discrete cosine transform","volume":"100","author":"Ahmed","year":"1974","journal-title":"IEEE Trans. Comput"},{"key":"B2","doi-asserted-by":"publisher","first-page":"2623","DOI":"10.1145\/3292500.3330701","article-title":"\u201cOptuna: a next-generation hyperparameter optimization framework,\u201d","author":"Akiba","year":"2019","journal-title":"Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery &data mining, pages"},{"key":"B3","volume-title":"Pattern Recognition and Machine Learning, volume 4","author":"Bishop","year":"2006"},{"key":"B4","article-title":"CycleMLP: an MLP-like architecture for dense prediction","author":"Chen","year":"2021","journal-title":"arXiv preprint arXiv:2107.10224"},{"key":"B5","article-title":"A downsampled variant of imagenet as an alternative to the cifar datasets","author":"Chrabaszcz","year":"2017","journal-title":"arXiv preprint arXiv:1707.08819"},{"key":"B6","article-title":"AutoAugment: learning augmentation policies from data","author":"Cubuk","year":"2018","journal-title":"arXiv preprint arXiv:1805.09501"},{"key":"B7","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/BF02551274","article-title":"Approximation by superpositions of a sigmoidal function","volume":"2","author":"Cybenko","year":"1989","journal-title":"Mathem. Control Signals Syst"},{"key":"B8","article-title":"Pixelated butterfly: simple and efficient sparse training for neural network models","author":"Dao","year":"2021","journal-title":"arXiv preprint arXiv:2112.00029"},{"key":"B9","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1109\/CVPR.2009.5206848","article-title":"\u201cImageNet: a large-scale hierarchical image database,\u201d","author":"Deng","year":"2009","journal-title":"2009 IEEE conference on computer vision and pattern recognition"},{"key":"B10","volume-title":"Abstract Algebra, volume 3","author":"Dummit","year":"2004"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00090","article-title":"\u201cHire-MLP: vision mlp via hierarchical rearrangement,\u201d","author":"Guo","year":"2022","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B12","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1109\/TPAMI.2022.3152247","article-title":"A survey on vision transformer","volume":"45","author":"Han","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B13","article-title":"Gaussian error linear units (GELUs)","author":"Hendrycks","year":"2016","journal-title":"arXiv preprint arXiv:1606.08415"},{"key":"B14","doi-asserted-by":"publisher","first-page":"1328","DOI":"10.1109\/TPAMI.2022.3145427","article-title":"Vision permutator: a permutable MLP-like architecture for visual recognition","volume":"45","author":"Hou","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B15","article-title":"\u201cWhy do better loss functions lead to less transferable features,\u201d","author":"Kornblith","year":"2021","journal-title":"Advances in neural information processing systems (NeurIPS)"},{"key":"B16","article-title":"Learning Multiple Layers of Features from Tiny Images","volume-title":"Technical Report","author":"Krizhevsky","year":"2009"},{"key":"B17","doi-asserted-by":"publisher","first-page":"6999","DOI":"10.1109\/TNNLS.2021.3084827","article-title":"A survey of convolutional neural networks: analysis, applications, and prospects","volume":"33","author":"Li","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B18","article-title":"AS-MLP: an axial shifted mlp architecture for vision","author":"Lian","year":"2021","journal-title":"arXiv preprint arXiv:2107.08391"},{"key":"B19","first-page":"9204","article-title":"\u201cPay attention to mlps,\u201d","author":"Liu","year":"2021","journal-title":"Advances in neural information processing systems"},{"key":"B20","doi-asserted-by":"publisher","first-page":"100520","DOI":"10.1016\/j.patter.2022.100520","article-title":"Are we ready for a new paradigm shift? A survey on visual deep mlp","volume":"3","author":"Liu","year":"2022","journal-title":"Patterns"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00986","article-title":"\u201cSwin transformer: hierarchical vision transformer using shifted windows,\u201d","author":"Liu","year":"2021","journal-title":"Proceedings of the IEEE\/CVF international conference on computer vision"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01167","article-title":"\u201cA convnet for the 2020s,\u201d","author":"Liu","year":"2022","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B23","article-title":"SGDR: stochastic gradient descent with warm restarts","author":"Loshchilov","year":"2016","journal-title":"arXiv preprint arXiv:1608.03983"},{"key":"B24","article-title":"\u201cWhen does label smoothing help?,\u201d","author":"M\u00fcller","year":"2019","journal-title":"Advances in neural information processing systems"},{"key":"B25","doi-asserted-by":"publisher","first-page":"3362","DOI":"10.1109\/TAI.2024.3351116","article-title":"Deep model intellectual property protection with compression-resistant model watermarking","volume":"5","author":"Nie","year":"2024","journal-title":"IEEE Trans. Artif. Intell"},{"key":"B26","volume-title":"Random Butterfly Transformations With Applications in Computational Linear Algebra","author":"Parker","year":"1995"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1072","DOI":"10.1109\/TSP.2012.2226449","article-title":"Learning sparsifying transforms","volume":"61","author":"Ravishankar","year":"2012","journal-title":"IEEE Trans. Signal Proc"},{"key":"B28","volume-title":"Computational Science and Engineering","author":"Strang","year":"2007"},{"key":"B29","article-title":"Pre-training of lightweight vision transformers on small datasets with minimally scaled images","author":"Tan","year":"2024","journal-title":"arXiv preprint arXiv:2402.03752"},{"key":"B30","first-page":"24261","article-title":"\u201cMLP-Mixer: an all-mlp architecture for vision,\u201d","author":"Tolstikhin","year":"2021","journal-title":"Advances in neural information processing systems"},{"key":"B31","doi-asserted-by":"publisher","first-page":"5314","DOI":"10.1109\/TPAMI.2022.3206148","article-title":"ResMLP: feedforward networks for image classification with data-efficient training","volume":"45","author":"Touvron","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B32","first-page":"10347","article-title":"\u201cTraining data-efficient image transformers &distillation through attention,\u201d","volume-title":"International conference on machine learning","author":"Touvron","year":"2021"},{"key":"B33","doi-asserted-by":"publisher","first-page":"531","DOI":"10.3390\/s25020531","article-title":"A comprehensive survey of deep learning approaches in image processing","volume":"25","author":"Trigka","year":"2025","journal-title":"Sensors"},{"key":"B34","article-title":"\u201cGmNet: Revisiting gating mechanisms from a frequency view,\u201d","author":"Wang","year":"2026","journal-title":"International conference on learning representations (ICLR)"},{"key":"B35","first-page":"22691","article-title":"\u201cDynaMixer: a vision mlp architecture with dynamic mixing,\u201d","volume-title":"International conference on machine learning","author":"Wang","year":"2022"},{"key":"B36","article-title":"Tree-structured parzen estimator: understanding its algorithm components and their roles for better empirical performance","author":"Watanabe","year":"2023","journal-title":"arXiv preprint arXiv:2304.11127"},{"key":"B37","doi-asserted-by":"publisher","first-page":"2759","DOI":"10.1609\/aaai.v37i3.25376","article-title":"\u201cActive token mixer,\u201d","author":"Wei","year":"2023","journal-title":"Proceedings of the AAAI conference on artificial intelligence"},{"key":"B38","doi-asserted-by":"publisher","first-page":"8502","DOI":"10.1109\/ACCESS.2017.2695239","article-title":"A survey of dictionary learning algorithms for face recognition","volume":"5","author":"Xu","year":"2017","journal-title":"IEEE Access"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1109\/WACV51458.2022.00367","article-title":"\u201cS2-MLP: Spatial-shift mlp architecture for vision,\u201d","author":"Yu","year":"2022","journal-title":"Proceedings of the IEEE\/CVF winter conference on applications of computer vision"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1769436\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T05:30:38Z","timestamp":1787549438000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1769436\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,24]]},"references-count":39,"alternative-id":["10.3389\/frai.2026.1769436"],"URL":"https:\/\/doi.org\/10.3389\/frai.2026.1769436","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,24]]},"article-number":"1769436"}}