{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T16:14:18Z","timestamp":1773072858350,"version":"3.50.1"},"reference-count":30,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T00:00:00Z","timestamp":1695859200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62027812"],"award-info":[{"award-number":["62027812"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62273185"],"award-info":[{"award-number":["62273185"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2022A1515010107"],"award-info":[{"award-number":["2022A1515010107"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"GuangDong Basic and Applied Basic Research Foundation","award":["62027812"],"award-info":[{"award-number":["62027812"]}]},{"name":"GuangDong Basic and Applied Basic Research Foundation","award":["62273185"],"award-info":[{"award-number":["62273185"]}]},{"name":"GuangDong Basic and Applied Basic Research Foundation","award":["2022A1515010107"],"award-info":[{"award-number":["2022A1515010107"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>A patch clamp is the \u201cgold standard\u201d method for studying ion-channel biophysics and pharmacology. Due to the complexity of the operation and the heavy reliance on experimenter experience, more and more researchers are focusing on patch-clamp automation. The existing automated patch-clamp system focuses on the process of completing the experiment; the detection method in each step is relatively simple, and the robustness of the complex brain film environment is lacking, which will increase the detection error in the microscopic environment, affecting the success rate of the automated patch clamp. To address these problems, we propose a method that is suitable for the contact between pipette tips and neuronal cells in automated patch-clamp systems. It mainly includes two key steps: precise positioning of pipettes and contact judgment. First, to obtain the precise coordinates of the tip of the pipette, we use the Mixture of Gaussian (MOG) algorithm for motion detection to focus on the tip area under the microscope. We use the object detection model to eliminate the encirclement frame of the pipette tip to reduce the influence of different shaped tips, and then use the sweeping line algorithm to accurately locate the pipette tip. We also use the object detection model to obtain a three-dimensional bounding frame of neuronal cells. When the microscope focuses on the maximum plane of the cell, which is the height in the middle of the enclosing frame, we detect the focus of the tip of the pipette to determine whether the contact between the tip and the cell is successful, because the cell and the pipette will be at the same height at this time. We propose a multitasking network CU-net that can judge the focus of pipette tips in complex contexts. Finally, we design an automated contact sensing process in combination with resistance constraints and apply it to our automated patch-clamp system. The experimental results show that our method can increase the success rate of pipette contact with cells in patch-clamp experiments.<\/jats:p>","DOI":"10.3390\/s23198144","type":"journal-article","created":{"date-parts":[[2023,9,29]],"date-time":"2023-09-29T07:42:08Z","timestamp":1695973328000},"page":"8144","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Neuron Contact Detection Based on Pipette Precise Positioning for Robotic Brain-Slice Patch Clamps"],"prefix":"10.3390","volume":"23","author":[{"given":"Ke","family":"Li","sequence":"first","affiliation":[{"name":"Institute of Robotics and Automatic Information System, Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin 300350, China"},{"name":"Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen 518083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huiying","family":"Gong","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Automatic Information System, Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin 300350, China"},{"name":"Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen 518083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8480-744X","authenticated-orcid":false,"given":"Jinyu","family":"Qiu","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Automatic Information System, Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin 300350, China"},{"name":"Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen 518083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ruimin","family":"Li","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Automatic Information System, Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin 300350, China"},{"name":"Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen 518083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qili","family":"Zhao","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Automatic Information System, Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin 300350, China"},{"name":"Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen 518083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin","family":"Zhao","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Automatic Information System, Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin 300350, China"},{"name":"Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen 518083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mingzhu","family":"Sun","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Automatic Information System, Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin 300350, China"},{"name":"Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen 518083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,28]]},"reference":[{"key":"ref_1","unstructured":"Verkhratsky, A., and Parpura, V. (2014). Patch-Clamp Methods and Protocols, Springer."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1038\/nrd2552","article-title":"High-throughput electrophysiology: An emerging paradigm for ion-channel screening and physiology","volume":"7","author":"Dunlop","year":"2008","journal-title":"Nat. Rev. Drug Discov."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"046003","DOI":"10.1088\/1741-2552\/ab1834","article-title":"PatcherBot: A single-cell electrophysiology robot for adherent cells and brain slices","volume":"16","author":"Kolb","year":"2019","journal-title":"J. Neural Eng."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2341","DOI":"10.1152\/jn.00738.2018","article-title":"Autonomous patch-clamp robot for functional characterization of neurons in vivo: Development and application to mouse visual cortex","volume":"121","author":"Holst","year":"2019","journal-title":"J. Neurophysiol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-021-21291-4","article-title":"Automatic deep learning-driven label-free image-guided patch clamp system","volume":"12","author":"Koos","year":"2021","journal-title":"Nat. Commun."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.compmedimag.2018.09.008","article-title":"Automatic localization of normal active organs in 3D PET scans","volume":"70","author":"Afshari","year":"2018","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1177\/0278364902021010833","article-title":"Biological cell injection using an autonomous microrobotic system","volume":"21","author":"Sun","year":"2002","journal-title":"Int. J. Robot. Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1012","DOI":"10.1039\/b600238b","article-title":"Automated MEMS-based Drosophila embryo injection system for high-throughput RNAi screens","volume":"6","author":"Zappe","year":"2006","journal-title":"Lab Chip"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1002\/jemt.20118","article-title":"Autofocusing in computer microscopy: Selecting the optimal focus algorithm","volume":"65","author":"Sun","year":"2004","journal-title":"Microsc. Res. Tech."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1007\/s00542-020-04891-w","article-title":"Macro-to-micro positioning and auto focusing for fully automated single cell microinjection","volume":"27","author":"Su","year":"2021","journal-title":"Microsyst. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1099","DOI":"10.1109\/TBME.2016.2590995","article-title":"Autofocusing and polar body detection in automated cell manipulation","volume":"64","author":"Wang","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1037","DOI":"10.1016\/j.neuron.2017.08.011","article-title":"Closed-loop real-time imaging enables fully automated cell-targeted patch-clamp neural recording in vivo","volume":"95","author":"Suk","year":"2017","journal-title":"Neuron"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1152\/jn.00025.2015","article-title":"MATLAB-based automated patch-clamp system for awake behaving mice","volume":"114","author":"Desai","year":"2015","journal-title":"J. Neurophysiol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1564","DOI":"10.1152\/jn.00386.2016","article-title":"Integration of autopatching with automated pipette and cell detection in vitro","volume":"116","author":"Wu","year":"2016","journal-title":"J. Neurophysiol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wang, Z., Gong, H., Li, K., Yang, B., Du, Y., Liu, Y., Zhao, X., and Sun, M. (2022, January 23\u201327). Simultaneous Depth Estimation and Localization for Cell Manipulation Based on Deep Learning. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.","DOI":"10.1109\/IROS47612.2022.9982228"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Li, R., and Peng, B. (2022). Implementing monocular visual-tactile sensors for robust manipulation. Cyborg Bionic Syst., 2022.","DOI":"10.34133\/2022\/9797562"},{"key":"ref_17","first-page":"1","article-title":"Robotic Patch Clamp Based on Noninvasive 3-D Cell Morphology Measurement for Higher Success Rate","volume":"71","author":"Zhao","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"634","DOI":"10.1038\/nprot.2016.007","article-title":"Assembly and operation of the autopatcher for automated intracellular neural recording in vivo","volume":"11","author":"Kodandaramaiah","year":"2016","journal-title":"Nat. Protoc."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"e24656","DOI":"10.7554\/eLife.24656","article-title":"Multi-neuron intracellular recording in vivo via interacting autopatching robots","volume":"7","author":"Kodandaramaiah","year":"2018","journal-title":"Elife"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"108357","DOI":"10.1016\/j.jneumeth.2019.108357","article-title":"Advances in the automation of whole-cell patch clamp technology","volume":"326","author":"Suk","year":"2019","journal-title":"J. Neurosci. Methods"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhao, Q., Han, Y., Jia, Y., Yu, N., Sun, M., and Zhao, X. (2020, January 13\u201317). Robotic whole-cell patch clamping based on three dimensional location for adherent cells. Proceedings of the 2020 International Conference on Manipulation, Automation and Robotics at Small Scales (MARSS), Toronto, ON, Canada.","DOI":"10.1109\/MARSS49294.2020.9307890"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zivkovic, Z. (2004, January 23\u201326). Improved adaptive Gaussian mixture model for background subtraction. Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, Cambridge, UK.","DOI":"10.1109\/ICPR.2004.1333992"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1016\/j.patrec.2005.11.005","article-title":"Efficient adaptive density estimation per image pixel for the task of background subtraction","volume":"27","author":"Zivkovic","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1016\/0026-2862(73)90046-0","article-title":"Characteristics of the servo-controlled micropipet pressure system","volume":"5","author":"Fox","year":"1973","journal-title":"Microvasc. Res."},{"key":"ref_25","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015). Advances in Neural Information Processing Systems, Available online: https:\/\/papers.nips.cc\/paper_files\/paper\/2015\/hash\/14bfa6bb14875e45bba028a21ed38046-Abstract.html."},{"key":"ref_26","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Liu, R., Ren, C., Fu, M., Chu, Z., and Guo, J. (2022). Platelet detection based on improved yolo_v3. Cyborg Bionic Syst., 2022.","DOI":"10.34133\/2022\/9780569"},{"key":"ref_28","unstructured":"Reis, D., Kupec, J., Hong, J., and Daoudi, A. (2023). Real-Time Flying Object Detection with YOLOv8. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Schimmack, M., and Mercorelli, P. (2019). An adaptive derivative estimator for fault-detection using a dynamic system with a suboptimal parameter. Algorithms, 12.","DOI":"10.3390\/a12050101"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3077","DOI":"10.1016\/j.jfranklin.2021.01.041","article-title":"A survey of interval observers design methods and implementation for uncertain systems","volume":"358","author":"Khan","year":"2021","journal-title":"J. Frankl. Inst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/19\/8144\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:01:08Z","timestamp":1760130068000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/19\/8144"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,28]]},"references-count":30,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["s23198144"],"URL":"https:\/\/doi.org\/10.3390\/s23198144","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,28]]}}}