{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T19:51:22Z","timestamp":1769716282002,"version":"3.49.0"},"reference-count":40,"publisher":"SAGE Publications","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IFS"],"published-print":{"date-parts":[[2023,8,24]]},"abstract":"<jats:p>The method based on entropy was used for the Bayesian optimization. Based on compelling information theory, Entropy Search (ES) and Predictive Entropy Search (PES) maximized information about the unknown function when the loss function reaches the maximum value. However, both methods were plagued by complicated calculations for estimating entropy. The most important motivation of this article is to improve and modularize the entropy search itself, making this method more flexible and effective for model adaptation. After the initial optimization and pruning module processing, a reasonable initial configuration for the complex model was successfully established, further reducing the space required for secondary optimization hyper-parameter search. The advantage of this method is that, on the one hand, the basic method of Bayesian optimization is used to get the best result of the iteration, while ensuring that the algorithm has theoretical boundedness. On the other hand, through the maximum entropy, the information features of the original space and data set are retained as much as possible to reduce the loss of information due to the initialization process, so as to improve the precision of the secondary optimization of the model. Further, a new algorithm framework is proposed, integrating MES and Sequential Model-Based Optimization (SMBO). With MES as the final module of the whole optimization process, a more accurate and reasonable algorithmic model was built, which lays a solid mathematical basis for the final empirical analysis.<\/jats:p>","DOI":"10.3233\/jifs-230470","type":"journal-article","created":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T10:18:12Z","timestamp":1689070692000},"page":"4991-5006","source":"Crossref","is-referenced-by-count":0,"title":["Quadratic optimization for the hyper-parameter based on maximum entropy search"],"prefix":"10.1177","volume":"45","author":[{"given":"Yuqi","family":"Li","sequence":"first","affiliation":[{"name":"School of Economics and Management, Beijing University of Posts and Telecommunications, Beijing, China"}]}],"member":"179","reference":[{"key":"10.3233\/JIFS-230470_ref1","first-page":"397","article-title":"Using confidence bounds for exploitation-exploration tradeoffs","volume":"3","author":"Auer","year":"2002","journal-title":"Journal of Machine Learning Research"},{"issue":"8","key":"10.3233\/JIFS-230470_ref2","doi-asserted-by":"crossref","first-page":"1889","DOI":"10.1162\/089976600300015187","article-title":"Gradient-based optimization of hyperparameters","volume":"12","author":"Bengio","year":"2000","journal-title":"Neural Computation"},{"key":"10.3233\/JIFS-230470_ref3","volume-title":"A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning","author":"Brochu","year":"2009"},{"key":"10.3233\/JIFS-230470_ref4","doi-asserted-by":"crossref","unstructured":"Calandra R. , Seyfarth A. , Peters J. and Deisenroth M.P. , An experimental comparison of Bayesian optimization for bipedal locomotion. In International Conference on Robotics and Automation (ICRA), May 31-June 7, Hong Kong, China, 2014.","DOI":"10.1109\/ICRA.2014.6907117"},{"key":"10.3233\/JIFS-230470_ref5","volume-title":"Model selection and model averaging","author":"Claeskens","year":"2008"},{"key":"10.3233\/JIFS-230470_ref6","unstructured":"Djolonga J. , Krause A. and Cevher V. , Highdimensional Gaussian process bandits. In Neural Information Processing Systems, December 5-8, Lake Tahoe, Nevada, 2013, 2013."},{"key":"10.3233\/JIFS-230470_ref7","first-page":"137","author":"Dean","year":"2004","journal-title":"MapReduce: Simplified data processing on large clusters"},{"key":"10.3233\/JIFS-230470_ref8","doi-asserted-by":"crossref","DOI":"10.5962\/bhl.title.27468","volume-title":"The Genetical Theory of Natural Selection: A Complete Variorum Edition","author":"Fisher","year":"1930"},{"key":"10.3233\/JIFS-230470_ref9","first-page":"1809","article-title":"Entropy search for information-efficient global optimization","volume":"13","author":"Hennig","year":"2012","journal-title":"Journal of Machine Learning Research"},{"key":"10.3233\/JIFS-230470_ref10","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/9780262028356.001.0001","volume-title":"Empirical model discovery and theory evaluation: automatic selection methods in econometrics","author":"Hendry","year":"2014"},{"issue":"1","key":"10.3233\/JIFS-230470_ref11","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s00477-018-1600-7","article-title":"Multiple hydrological models comparison and an improved Bayesian model averaging approach for ensemble prediction over semi-humid regions","volume":"33","author":"Huo","year":"2019","journal-title":"Stochastic Environmental Research and Risk Assessment"},{"key":"10.3233\/JIFS-230470_ref12","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1613\/jair.2861","article-title":"ParamILS: An automatic algorithm configuration framework","volume":"36","author":"Hutter","year":"2009","journal-title":"Journal of Artificial Intelligence Research"},{"key":"10.3233\/JIFS-230470_ref13","doi-asserted-by":"crossref","first-page":"108338","DOI":"10.1016\/j.knosys.2022.108338","article-title":"A nondestruc-tive automatic defect detection method with pixelwise segmentation","volume":"242","author":"Yang","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"10.3233\/JIFS-230470_ref14","doi-asserted-by":"crossref","unstructured":"Kaelbling L.P. and Lozano-Perez T. , Learning composable models of primitive actions. In International Conference on Robotics and Automation (ICRA), May 29-June 3, Singapore, 2017.","DOI":"10.1109\/ICRA.2017.7989109"},{"issue":"1","key":"10.3233\/JIFS-230470_ref15","first-page":"6:1","article-title":"Cross-disciplinary perspectives on meta-learning for algorithm selection","volume":"41","author":"Smith-Miles","year":"2008","journal-title":"ACM Computing Surveys"},{"key":"10.3233\/JIFS-230470_ref16","unstructured":"Muandet K. , Balduzzi D. and Sch\u00f6lkopf B. , Domain generalization via invariant feature representation. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16-21 June 2013, pages 10\u201318, 2013."},{"issue":"1","key":"10.3233\/JIFS-230470_ref17","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1613\/jair.4742","article-title":"Global continuous optimization with error bound and fast convergence","volume":"56","author":"Kawaguchi","year":"2016","journal-title":"Journal of Artificial Intelligence Research"},{"issue":"241","key":"10.3233\/JIFS-230470_ref18","doi-asserted-by":"crossref","first-page":"108274","DOI":"10.1016\/j.knosys.2022.108274","article-title":"Bi-CLKT: bi-graph contrastive learning based knowledge tracing","author":"Song","year":"2022","journal-title":"Knowl.-Based Syst."},{"issue":"240","key":"10.3233\/JIFS-230470_ref19","doi-asserted-by":"crossref","first-page":"108149","DOI":"10.1016\/j.knosys.2022.108149","article-title":"Weighted quantile discrepancy-based deep domain adaptation network for intelligent fault diagnosis","author":"Fan","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"10.3233\/JIFS-230470_ref20","first-page":"2011","article-title":"Contextual Gaussian process bandit optimization. In","author":"Krause","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"issue":"1","key":"10.3233\/JIFS-230470_ref21","first-page":"97","article-title":"A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise","volume":"86","author":"Kushner","year":"1964","journal-title":"Journal of Fluids Engineering"},{"issue":"18","key":"10.3233\/JIFS-230470_ref22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/math10183247","article-title":"Runge\u2013Kutta Embedded Methods of Orders 8 (7) for Use in Quadruple Precision Computations","volume":"10","author":"Kovalnogov","year":"2022","journal-title":"Mathematics"},{"key":"10.3233\/JIFS-230470_ref23","doi-asserted-by":"crossref","first-page":"6628889","DOI":"10.1155\/2021\/6628889","article-title":"A new initialization approach in particle swarm optimization for global optimization problems","volume":"2021","author":"Waqas","year":"2021","journal-title":"Computational Intelligence and Neuroscience"},{"key":"10.3233\/JIFS-230470_ref24","doi-asserted-by":"crossref","first-page":"5990999","DOI":"10.1155\/2021\/5990999","article-title":"A systematic literature review on particle swarm optimization techniques for medical diseases detection","volume":"2021","author":"Pervaiz","year":"2021","journal-title":"Computational and Mathematical Methods in Medicine"},{"issue":"16","key":"10.3233\/JIFS-230470_ref25","doi-asserted-by":"crossref","first-page":"7591","DOI":"10.3390\/app11167591","article-title":"Comparative analysis of low discrepancy sequence-based initialization approaches using population-based algorithms for solving the global optimization problems","volume":"11","author":"Bangyal","year":"2021","journal-title":"Applied Sciences"},{"key":"10.3233\/JIFS-230470_ref26","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.jsc.2022.01.003","article-title":"Heuristics to sift extraneous factors in Dixon resultants","volume":"112","author":"Qin","year":"2022","journal-title":"Journal of Symbolic Computation"},{"issue":"20","key":"10.3233\/JIFS-230470_ref27","doi-asserted-by":"crossref","first-page":"6844","DOI":"10.1016\/j.eswa.2015.05.006","article-title":"A fuzzy-rough nearest neighbor classifier combined with consistency-based subset evaluation and instance selection for automated diagnosis of breast cancer","volume":"42","author":"Onan","year":"2015","journal-title":"Expert Systems with Applications"},{"issue":"23","key":"10.3233\/JIFS-230470_ref28","doi-asserted-by":"crossref","first-page":"e5909","DOI":"10.1002\/cpe.5909","article-title":"Sentiment analysis on product reviews based on weighted word embeddings and deep neural networks","volume":"33","author":"Onan","year":"2021","journal-title":"Concurrency and Computation: Practice and Experience"},{"issue":"1","key":"10.3233\/JIFS-230470_ref29","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1177\/0165551515613226","article-title":"A feature selection model based on genetic rank aggregation for text sentiment classification","volume":"43","author":"Onan","year":"2017","journal-title":"Journal of Information Science"},{"issue":"1","key":"10.3233\/JIFS-230470_ref30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.52460\/src.2021.004","article-title":"Ensemble of Classifiers and Term Weighting Schemes for Sentiment Analysis in Turkish","volume":"1","author":"Onan","year":"2021","journal-title":"Scientific Research Communications"},{"key":"10.3233\/JIFS-230470_ref31","doi-asserted-by":"crossref","first-page":"7701","DOI":"10.1109\/ACCESS.2021.3049734","article-title":", A term weighted neural language model and stacked bidirectional LSTM based framework for sarcasm identification","volume":"9","author":"Onan","year":"2021","journal-title":"IEEE Access"},{"issue":"3","key":"10.3233\/JIFS-230470_ref32","doi-asserted-by":"crossref","first-page":"572","DOI":"10.1002\/cae.22253","article-title":"Sentiment analysis on massive open online course evaluations: A text mining and deep learning approach","volume":"29","author":"Onan","year":"2021","journal-title":"Computer Applications in Engineering Education"},{"key":"10.3233\/JIFS-230470_ref33","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1016\/j.isprsjprs.2022.02.011","article-title":"Continuous space ant colony algorithm for automatic selection of orthophoto mosaic seamline network","volume":"186","author":"Wang","year":"2022","journal-title":"ISPRS Journal of Photogrammetry and Remote Sensing"},{"key":"10.3233\/JIFS-230470_ref34","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1007\/BF00116827","article-title":"Learning quickly when irrelevant attributes are abound: A new linear threshold algorithm","volume":"2","author":"Littlestone","year":"1988","journal-title":"Machine Learning"},{"key":"10.3233\/JIFS-230470_ref35","first-page":"1996","volume-title":"Part of The Lecture Notes in Statistics Book Series","author":"Neal"},{"key":"10.3233\/JIFS-230470_ref36","unstructured":"Bardenet R. , Brendel M.B. , K\u00e9gl B. and Sebag M. , Collaborative hyperparameter tuning. Proceedings of the30th International Conference on Machine Learning (ICML), Atlanta, Georgia, USA, pp. 199\u2013207, 2013."},{"issue":"C","key":"10.3233\/JIFS-230470_ref37","article-title":"An adaptive boosting charging strategy optimization based on thermoelectric-aging model, surrogates and multi-objective optimization","volume":"312","author":"Su","year":"2022","journal-title":"Applied Energy"},{"issue":"1","key":"10.3233\/JIFS-230470_ref38","doi-asserted-by":"crossref","first-page":"1113","DOI":"10.1109\/TTE.2022.3204843","article-title":"A hybrid battery equivalent circuit model, deep learning, and transfer learning for battery state monitoring","volume":"9","author":"Su","year":"2023","journal-title":"IEEE Transactions on Transportation Electrification"},{"key":"10.3233\/JIFS-230470_ref39","doi-asserted-by":"crossref","unstructured":"Rasmussen C.E. and Williams C.K.I. , Gaussian Processes for Machine Learning. The MIT Press, 2006.","DOI":"10.7551\/mitpress\/3206.001.0001"},{"key":"10.3233\/JIFS-230470_ref40","unstructured":"Rudin W. , Fourier Analysis on Groups. John Wiley & Sons, 2011."}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/JIFS-230470","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T09:16:39Z","timestamp":1769678199000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/JIFS-230470"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,24]]},"references-count":40,"journal-issue":{"issue":"3"},"URL":"https:\/\/doi.org\/10.3233\/jifs-230470","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,24]]}}}