{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T04:03:36Z","timestamp":1768449816809,"version":"3.49.0"},"reference-count":43,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2023,11,2]],"date-time":"2023-11-02T00:00:00Z","timestamp":1698883200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"KAKENHI","doi-asserted-by":"publisher","award":["20H04160"],"award-info":[{"award-number":["20H04160"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001691","name":"KAKENHI","doi-asserted-by":"publisher","award":["20H00590"],"award-info":[{"award-number":["20H00590"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Digital microfluidic biochips (DMFBs), which are used in various fields like DNA analysis, clinical diagnosis, and PCR testing, have made biochemical experiments more compact, efficient, and user-friendly than the previous methods. However, their reliability is often compromised by their inability to adapt to all kinds of errors. Errors in biochips can be categorized into two types: known errors, and unknown errors. Known errors are detectable before the start of the routing process using sensors or cameras. Unknown errors, in contrast, only become apparent during the routing process and remain undetected by sensors or cameras, which can unexpectedly stop the routing process and diminish the reliability of biochips. This paper introduces a deep reinforcement learning-based routing algorithm, designed to manage not only known errors but also unknown errors. Our experiments demonstrated that our algorithm outperformed the previous ones in terms of the success rate of the routing, in the scenarios including both known errors and unknown errors. Additionally, our algorithm contributed to detecting unknown errors during the routing process, identifying the most efficient routing path with a high probability.<\/jats:p>","DOI":"10.3390\/s23218924","type":"journal-article","created":{"date-parts":[[2023,11,2]],"date-time":"2023-11-02T09:28:13Z","timestamp":1698917293000},"page":"8924","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A Deep Reinforcement Learning Approach to Droplet Routing for Erroneous Digital Microfluidic Biochips"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-7449-3934","authenticated-orcid":false,"given":"Tomohisa","family":"Kawakami","sequence":"first","affiliation":[{"name":"Graduate School of Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chiharu","family":"Shiro","sequence":"additional","affiliation":[{"name":"Graduate School of Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9626-0944","authenticated-orcid":false,"given":"Hiroki","family":"Nishikawa","sequence":"additional","affiliation":[{"name":"Graduate School of Information Science and Technology, Osaka University, Osaka 565-0871, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiangbo","family":"Kong","sequence":"additional","affiliation":[{"name":"Department of Intelligent Robotics, Faculty of Engineering, Toyama Prefectural University, Imizu 939-0398, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hiroyuki","family":"Tomiyama","sequence":"additional","affiliation":[{"name":"Graduate School of Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2279-4644","authenticated-orcid":false,"given":"Shigeru","family":"Yamashita","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Azizipour, N., Avazpour, R., Rosenzweig, D.H., Sawan, M., and Ajji, A. (2020). Evolution of biochip technology: A review from lab-on-a-chip to organ-on-a-chip. Micromachines, 11.","DOI":"10.3390\/mi11060599"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1324177.1324178","article-title":"High-level synthesis of digital microfluidic biochips","volume":"3","author":"Su","year":"2008","journal-title":"ACM J. Emerg. Technol. Comput. Syst. (JETC)"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sista, R.S., Ng, R., Nuffer, M., Basmajian, M., Coyne, J., Elderbroom, J., Hull, D., Kay, K., Krishnamurthy, M., and Roberts, C. (2020). Digital microfluidic platform to maximize diagnostic tests with low sample volumes from newborns and pediatric patients. Diagnostics, 10.","DOI":"10.3390\/diagnostics10010021"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Huang, S., Connolly, J., Khlystov, A., and Fair, R.B. (2020). Digital microfluidics for the detection of selected inorganic ions in aerosols. Sensors, 20.","DOI":"10.3390\/s20051281"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"22727","DOI":"10.1073\/pnas.2014739117","article-title":"Rapid isothermal amplification and portable detection system for SARS-CoV-2","volume":"117","author":"Ganguli","year":"2020","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yang, C., Gan, X., Zeng, Y., Xu, Z., Xu, L., Hu, C., Ma, H., Chai, B., Hu, S., and Chai, Y. (2023). Advanced design and applications of digital microfluidics in biomedical fields: An update of recent progress. Biosens. Bioelectron., 242.","DOI":"10.1016\/j.bios.2023.115723"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1080\/14737159.2021.1898950","article-title":"Future potential of Rapid Acceleration of Diagnostics (RADx Tech) in molecular diagnostics","volume":"21","author":"Schachter","year":"2021","journal-title":"Expert Rev. Mol. Diagn."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Dkhar, D.S., Kumari, R., Malode, S.J., Shetti, N.P., and Chandra, P. (2023). Integrated lab-on-a-chip devices: Fabrication methodologies, transduction system for sensing purposes. Pharm. Biomed. Anal., 223.","DOI":"10.1016\/j.jpba.2022.115120"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Chaudhary, V., Khanna, V., Awan, H.T.A., Singh, K., Khalid, M., Mishra, Y.K., Bhansali, S., Li, C.Z., and Kaushik, A. (2023). Towards hospital-on-chip supported by 2D MXenes-based 5th generation intelligent biosensors. Biosens. Bioelectron., 220.","DOI":"10.1016\/j.bios.2022.114847"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1126\/science.1076996","article-title":"Microfluidic large-scale integration","volume":"298","author":"Thorsen","year":"2002","journal-title":"Science"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"930","DOI":"10.1109\/JPROC.2003.813570","article-title":"Microfluidics meets MEMS","volume":"91","author":"Verpoorte","year":"2003","journal-title":"Proc. IEEE"},{"key":"ref_12","unstructured":"Pollack, M.G. (2001). Electrowetting-Based Microactuation of Droplets for Digital Microfluidics, The Duke University."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/JMEMS.2002.807467","article-title":"Creating, transporting, cutting, and merging liquid droplets by electrowetting-based actuation for digital microfluidic circuits","volume":"12","author":"Cho","year":"2003","journal-title":"J. Microelectromech. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1725","DOI":"10.1063\/1.1308534","article-title":"Electrowetting-based actuation of liquid droplets for microfluidic applications","volume":"77","author":"Pollack","year":"2000","journal-title":"Appl. Phys. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"6616","DOI":"10.1021\/la990548n","article-title":"Reversible electrowetting and trapping of charge: Model and experiments","volume":"15","author":"Verheijen","year":"1999","journal-title":"Langmuir"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1002\/biot.201000324","article-title":"Picoliter DNA sequencing chemistry on an electrowetting-based digital microfluidic platform","volume":"6","author":"Welch","year":"2011","journal-title":"Biotechnol. J."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1109\/TCAD.2005.855956","article-title":"Microfluidics-based biochips: Technology issues, implementation platforms, and design-automation challenges","volume":"25","author":"Su","year":"2006","journal-title":"IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"817","DOI":"10.1109\/TCAD.2012.2183369","article-title":"Cross-contamination avoidance for droplet routing in digital microfluidic biochips","volume":"31","author":"Zhao","year":"2012","journal-title":"IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst."},{"key":"ref_19","unstructured":"Liang, T.C., and Zhong, Z. (2020, January 13\u201318). Adaptive droplet routing in digital microfluidic biochips using deep reinforcement learning. Proceedings of the 37th International Conference on Machine Learning, ICML 2020, Online."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1705","DOI":"10.1039\/D0LC00144A","article-title":"Current commercialization status of electrowetting-on-dielectric (EWOD) digital microfluidics","volume":"20","author":"Li","year":"2020","journal-title":"Lab Chip"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1145\/122344.122377","article-title":"Dyna, an integrated architecture for learning, planning, and reacting","volume":"2","author":"Sutton","year":"1991","journal-title":"ACM Sigart Bull."},{"key":"ref_22","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"350","DOI":"10.1038\/s41586-019-1724-z","article-title":"Grandmaster level in StarCraft II using multi-agent reinforcement learning","volume":"575","author":"Vinyals","year":"2019","journal-title":"Nature"},{"key":"ref_26","first-page":"27730","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","article-title":"Highly accurate protein structure prediction with AlphaFold","volume":"596","author":"Jumper","year":"2021","journal-title":"Nature"},{"key":"ref_28","unstructured":"Su, F., and Chakrabarty, K. (2004, January 7\u201311). Architectural-level synthesis of digital microfluidics-based biochips. Proceedings of the IEEE\/ACM International Conference on Computer Aided Design, 2004. ICCAD-2004, San Jose, CA, USA."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1001","DOI":"10.1109\/TCAD.2010.2049153","article-title":"Design tools for digital microfluidic biochips: Toward functional diversification and more than moore","volume":"29","author":"Chakrabarty","year":"2010","journal-title":"IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Huang, T.W., and Ho, T.Y. (2009, January 4\u20137). A fast routability-and performance-driven droplet routing algorithm for digital microfluidic biochips. Proceedings of the 2009 IEEE International Conference on Computer Design, Lake Tahoe, CA, USA.","DOI":"10.1109\/ICCD.2009.5413119"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Keszocze, O., Wille, R., and Drechsler, R. (2014, January 2\u20136). Exact routing for digital microfluidic biochips with temporary blockages. Proceedings of the 2014 IEEE\/ACM International Conference on Computer-Aided Design (ICCAD), San Jose, CA, USA.","DOI":"10.1109\/ICCAD.2014.7001383"},{"key":"ref_32","unstructured":"Pan, I., and Samanta, T. (2013). Recent Advances in Intelligent Informatics, Proceedings of the Second International Symposium on Intelligent Informatics (ISI\u201913), Mysore, India, 23\u201324 August 2013, Springer."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1145\/1148015.1148017","article-title":"Yield enhancement of reconfigurable microfluidics-based biochips using interstitial redundancy","volume":"2","author":"Su","year":"2006","journal-title":"ACM J. Emerg. Technol. Comput. Syst. (JETC)"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Xu, T., and Chakrabarty, K. (2007, January 4\u20138). Integrated droplet routing in the synthesis of microfluidic biochips. Proceedings of the DAC07: The 44th Annual Design Automation Conference 2007, San Diego, CA, USA.","DOI":"10.1109\/DAC.2007.375301"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1109\/TCAD.2011.2177836","article-title":"Simultaneous optimization of droplet routing and control-pin mapping to electrodes in digital microfluidic biochips","volume":"31","author":"Zhao","year":"2012","journal-title":"IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1109\/TCAD.2012.2211104","article-title":"Error recovery in cyberphysical digital microfluidic biochips","volume":"32","author":"Luo","year":"2012","journal-title":"IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Willsey, M., Stephenson, A.P., Takahashi, C., Vaid, P., Nguyen, B.H., Piszczek, M., Betts, C., Newman, S., Joshi, S., and Strauss, K. (2019, January 13\u201317). Puddle: A dynamic, error-correcting, full-stack microfluidics platform. Proceedings of the ASPLOS\u201919: Architectural Support for Programming Languages and Operating Systems, Providence, RI, USA.","DOI":"10.1145\/3297858.3304027"},{"key":"ref_38","unstructured":"Su, F., and Chakrabarty, K. (2005, January 7\u201311). Design of fault-tolerant and dynamically-reconfigurable microfluidic biochips. Proceedings of the Design, Automation and Test in Europe, Munich, Germany."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kawakami, T., Shiro, C., Nishikawa, H., Kong, X., Tomiyama, H., and Yamashita, S. (2023, January 26\u201328). A Deep Reinforcement Learning-based Routing Algorithm for Unknown Erroneous Cells in DMFBs. Proceedings of the 2023 21st IEEE Interregional NEWCAS Conference (NEWCAS), Edinburgh, UK.","DOI":"10.1109\/NEWCAS57931.2023.10198067"},{"key":"ref_40","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst., 25."},{"key":"ref_41","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_42","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016, January 20\u201322). Asynchronous methods for deep reinforcement learning. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_43","unstructured":"Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015, January 7\u20139). Trust region policy optimization. Proceedings of the 32nd International Conference on Machine Learning, Lille, France."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/21\/8924\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:16:00Z","timestamp":1760130960000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/21\/8924"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,2]]},"references-count":43,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2023,11]]}},"alternative-id":["s23218924"],"URL":"https:\/\/doi.org\/10.3390\/s23218924","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,2]]}}}