Design of a Wheelchair Control System Based on Hand Gesture Recognition Using ResNet18
Published online: 10/11/2025
Corressponding author's email:
dotv@hcmute.edu.vnDOI:
https://doi.org/10.54644/jte.2025.1813Keywords:
Deep learning, Powered wheelchair, Hand gesture classification, ResNet18, Biomedical engineeringAbstract
The development of wheelchairs utilizing advanced technology at low costs is gaining attention to improve the quality of life for approximately 800 million people with disabilities worldwide. These individuals often face challenges in mobility, access to education, and social integration. Among various wheelchair control methods, hand gesture control is considered optimal due to its efficiency and health safety. However, in Vietnam, research in this field remains limited. This project focuses on designing a smart wheelchair system using computer vision to recognize hand gestures and convert them into control commands, combining hardware and software solutions. The study employs deep learning models such as ResNet-18 for image processing, integrated on a Jetson Nano device, and hardware optimization to achieve the highest efficiency. Although challenges remain, such as ensuring accuracy in diverse environments and maintaining stable control under real-world conditions, this research promises not only to enhance user independence but also to open new opportunities in biomedical engineering. It contributes to improving the quality of life and fostering social inclusion for people with disabilities.
Downloads: 0
References
World Health Organization, “Disability.” Accessed: Jan. 18, 2025. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/disability-and-health
J. C. Luna, J. M. Rocha, E. Monacelli, G. Foggea, Y. Hirata, and S. Delaplace, “WISP, wearable inertial sensor for online wheelchair propulsion detection,” Sensors, vol. 22, no. 11, 2022, doi: 10.3390/s22114221. DOI: https://doi.org/10.3390/s22114221
Y. Li and O. Ming, “Hand gesture recognition using Kinect,” doi: 10.1109/ICSESS.2012.6269439. DOI: https://doi.org/10.1109/ICSESS.2012.6269439
O. O. Adebayo, E. Adetiba, and O. T. Ajayi, “Hand gesture recognition-based control of motorized wheelchair using electromyography sensors and recurrent neural network,” IOP Conf. Ser. Mater. Sci. Eng., vol. 1107, no. 1, p. 012063, Apr. 2021, doi: 10.1088/1757-899X/1107/1/012063. DOI: https://doi.org/10.1088/1757-899X/1107/1/012063
R. K. Nasare and G. K. Yenurkar, “Hand gesture based navigation control for automated wheelchair,” 2017, doi: 10.21172/1.81.031. DOI: https://doi.org/10.21172/1.81.031
A. Stroh and J. Desai, “Hand gesture-based artificial neural network trained hybrid human–machine interface system to navigate a powered wheelchair,” J. Bionic Eng., vol. 18, no. 5, pp. 1045–1058, 2021, doi: 10.1007/s42235-021-00074-z. DOI: https://doi.org/10.1007/s42235-021-00074-z
A. I. Iskanderani et al., “Voice controlled artificial intelligent smart wheelchair,” in Proc. 8th Int. Conf. Intelligent and Advanced Systems (ICIAS), 2021, pp. 1–5, doi: 10.1109/ICIAS49414.2021.9642607. DOI: https://doi.org/10.1109/ICIAS49414.2021.9642607
A. Jayakody, A. Nawarathna, I. Wijesinghe, S. Liyanage, and J. Dissanayake, “Smart wheelchair to facilitate disabled individuals,” in Proc. Int. Conf. Advancements in Computing (ICAC), 2019, pp. 249–254, doi: 10.1109/ICAC49085.2019.9103409. DOI: https://doi.org/10.1109/ICAC49085.2019.9103409
A. B. Karim, A. Ul Haq, A. Noor, B. Khan, and Z. Hussain, “Raspberry Pi based voice controlled smart wheelchair,” in Proc. Int. Conf. Emerging Trends in Smart Technologies (ICETST), 2022, pp. 1–5, doi: 10.1109/ICETST55735.2022.9922929. DOI: https://doi.org/10.1109/ICETST55735.2022.9922929
W. Luo, J. Cao, K. Ishikawa, and D. Ju, “A human-computer control system based on intelligent recognition of eye movements and its application in wheelchair driving,” Multimodal Technol. Interact., vol. 5, no. 9, 2021, doi: 10.3390/mti5090050. DOI: https://doi.org/10.3390/mti5090050
D. Cojocaru, L. F. Manta, I. C. Vladu, A. Dragomir, and A. M. Mariniuc, “Using an eye gaze new combined approach to control a wheelchair movement,” in Proc. 23rd Int. Conf. System Theory, Control and Computing (ICSTCC), 2019, pp. 626–631, doi: 10.1109/ICSTCC.2019.8886158. DOI: https://doi.org/10.1109/ICSTCC.2019.8886158
N. Wanluk, S. Visitsattapongse, A. Juhong, and C. Pintavirooj, “Smart wheelchair based on eye tracking,” in Proc. 9th Biomed. Eng. Int. Conf. (BMEiCON), 2016, pp. 1–4, doi: 10.1109/BMEiCON.2016.7859594. DOI: https://doi.org/10.1109/BMEiCON.2016.7859594
D. Bai et al., “Design of an eye movement-controlled wheelchair using Kalman filter algorithm,” in Proc. IEEE Int. Conf. Information and Automation (ICIA), 2016, pp. 1664–1668, doi: 10.1109/ICInfA.2016.7832085. DOI: https://doi.org/10.1109/ICInfA.2016.7832085
F. P. Chee, M. D. S., H. W. J. Chang, A. Awang, A. S. Francis, and J. Dayou, “Eye tracking electronic wheelchair for physically challenged person,” ARPN J. Eng. Appl. Sci., vol. 12, pp. 4078–4083, Jan. 2017.
E. Antoniou et al., “EEG-based eye movement recognition using brain–computer interface and random forests,” Sensors, vol. 21, no. 7, 2021, doi: 10.3390/s21072339. DOI: https://doi.org/10.3390/s21072339
K. Joshi, P. Soni, S. Joshi, A. Vyas, and R. Joshi, “Cognitive-chair: AI-based advanced brain sensing wheelchair for paraplegic/quadriplegic people,” in Proc. 4th Int. Conf. Artificial Intelligence and Speech Technology (AIST), 2022, pp. 1–6, doi: 10.1109/AIST55798.2022.10065338. DOI: https://doi.org/10.1109/AIST55798.2022.10065338
B. A. Fadheel, A. J. Mahdi, H. F. Jaafar, M. S. Nazir, M. S. Obaid, and S. H. Musa, “Speed control of a wheelchair prototype driven by a DC motor through real EEG brain signals,” IOP Conf. Ser. Mater. Sci. Eng., vol. 671, no. 1, p. 012036, Jan. 2020, doi: 10.1088/1757-899X/671/1/012036. DOI: https://doi.org/10.1088/1757-899X/671/1/012036
F. Ferracuti, A. Freddi, S. Iarlori, S. Longhi, A. Monteriù, and C. Porcaro, “Augmenting robot intelligence via EEG signals to avoid trajectory planning mistakes of a smart wheelchair,” J. Ambient Intell. Humaniz. Comput., vol. 14, no. 1, pp. 223–235, 2023, doi: 10.1007/s12652-021-03286-7. DOI: https://doi.org/10.1007/s12652-021-03286-7
V. Do Tran and C. T. Nguyen, “A computer vision-based eye-tracking system toward an eye-controlled powered wheelchair,” in Lecture Notes in Networks and Systems, Springer, 2024, pp. 290–300, doi: 10.1007/978-3-031-76197-3_25. DOI: https://doi.org/10.1007/978-3-031-76197-3_25
H. P. Hai, N. Van Hoa, and D. T. Nghi, “Comparison of deep learning model with other machine learning methods in microarray gene expression data classification,” in Proc. 10th Nat. Conf. Fundamental and Applied Information Technology (FAIR), Da Nang, Vietnam, Aug. 2017.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778, doi: 10.1109/CVPR.2016.90. DOI: https://doi.org/10.1109/CVPR.2016.90
K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” ResNet paper notes, doi: 10.48550/arXiv.1512.03385.
V. D. Quang, N. H. Yen, M. H. Man, and N. T. Nha, “Ứng dụng kỹ thuật học sâu trong hỗ trợ chẩn đoán bệnh viêm phổi thông qua ảnh chụp X-quang,” Tạp chí Khoa học, Trường Đại học Vinh, vol. 50, no. 4A, pp. 29–39, 2021.
Downloads
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2025 Journal of Technical Education Science
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright © JTE.
