A Computer Vision-Based Respiratory Rate Monitoring and Alarm System
Corressponding author's email:
thanh.nguyenthimy@eiu.edu.vnDOI:
https://doi.org/10.54644/jte.78B.2023.1384Keywords:
Computer vision-based, breathing rate detection, sleep apnea, optical flow, Principal component analysisAbstract
Breathing rate is one of the most important vital signals for monitoring health status and reflecting conditions of dangerous diseases. Previous contactless breath monitoring methods were more convenient than contact methods, but they were not suitable for the actual sleeping environment because of the narrow field of vision (FoV). This study proposed a breathing rate monitoring strategy using a mono camera to track and detect sleep apnea phenomena. Breathing rates were first tracked among consecutive image frames. The human body area was then isolated and magnified using a deep neural network (DNN) model before applying the optical flow algorithm to extract and monitor the up and down changes caused by respiration. The most variated directions of the body feature’s motions were detected based on the Principal Component Analysis (PCA) method. Breathing rate was the number of times the signal amplitude peaks per minute. The comparison between predicted values and manually estimated was used for evaluating the accuracy of the method. The accuracy of our method in various light, position, and distance conditions is 2 breaths/minute (<10%) for children and less than 1 breath/minute (<5%) for adults. The study has two main contributions: (1) monitoring breathing rate at home gives comfortable feelings to patients and caregivers, expanding the potential of applying modern technology to clinics, (2) the study has solved the problem of tracking small movements in videos with relatively large FoV in real-time. Perspectively, we will be employed the method in a home-based respiratory rate monitoring system.
Downloads: 0
References
M. A. Cretikos, R. Bellomo, K. Hillman, J. Chen, S. Finfer, and A. Flabouris, “Respiratory rate: the neglected vital sign,” Medical Journal of Australia, vol. 188, no. 11, pp. 657–659, 2008.
R. M. H. Schein, N. Hazday, M. Pena, B. H. Ruben, and C. L. Sprung, “Clinical Antecedents to In-Hospital Cardiopulmonary Arrest,” Chest, vol. 98, no. 6, pp. 1388–1392, Dec. 1990, doi: 10.1378/chest.98.6.1388.
D. Farmakis, J. Parissis, J. Lekakis, and G. Filippatos, “Acute Heart Failure: Epidemiology, Risk Factors, and Prevention,” Revista Española de Cardiología (English Edition), vol. 68, no. 3, pp. 245–248, Mar. 2015, doi: 10.1016/j.rec.2014.11.004.
O. Ruuskanen, E. Lahti, L. C. Jennings, and D. R. Murdoch, “Viral pneumonia,” The Lancet, vol. 377, no. 9773, pp. 1264–1275, Apr. 2011, doi: 10.1016/S0140-6736(10)61459-6.
N. M. Punjabi, “The epidemiology of adult obstructive sleep apnea,” Proc. Am. Thorac. Soc., vol. 5, no. 2, pp. 136–143, 2008.
E. S. Katz, R. B. Mitchell, and C. M. D’Ambrosio, “Obstructive sleep apnea in infants,” Am. J. Respir. Crit. Care Med., vol. 185, no. 8, pp. 805–816, 2012.
H. K. Yaggi, J. Concato, W. N. Kernan, J. H. Lichtman, L. M. Brass, and V. Mohsenin, “Obstructive sleep apnea as a risk factor for stroke and death,” New England Journal of Medicine, vol. 353, no. 19, pp. 2034–2041, 2005.
M. Heron, “Comparability of Race-specific Mortality Data Based on 1977 Versus 1997 Reporting Standards,” National vital statistics reports, vol. 70, no. 3, pp. 1-31, Apr. 2021.
H. Liu, J. Allen, D. Zheng, and F. Chen, “Recent development of respiratory rate measurement technologies,” Physiol. Meas., vol. 40, no. 7, p. 07TR01, 2019, doi: 10.1088/1361-6579/ab299e.
X. Chen, Y. Xiao, Y. Tang, J. F. Mendoza, and G. Cao, “ApneaDetector: Detecting Sleep Apnea with Smartwatches,” Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 5, no. 2, pp. 1–22, Jun. 2021, doi: 10.1145/3463514.
P. H. Charlton et al., “Breathing rate estimation from the electrocardiogram and photoplethysmogram: A review,” IEEE Rev. Biomed. Eng., vol. 11, pp. 2–20, 2017.
C. Massaroni, A. Nicolò, D. L. Presti, M. Sacchetti, S. Silvestri, and E. Schena, “Contact-based methods for measuring respiratory rate,” Sensors, vol. 19, no. 4, p. 908, 2019.
R. Ruangsuwana, G. Velikic, and M. Bocko, “Methods to extract respiration information from ECG signals,” in 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, 2010, pp. 570–573. doi: 10.1109/ICASSP.2010.5495584.
A. John, B. Cardiff, and D. John, “A 1D-CNN based deep learning technique for sleep apnea detection in iot sensors,” in 2021 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, 2021, pp. 1–5.
I. M. Costanzo, D. Sen, L. Rhein, and U. Guler, “Respiratory monitoring: Current state of the art and future roads,” IEEE Rev. Biomed. Eng., vol. 15, pp. 103–121, 2022.
E. Dafna, T. Rosenwein, A. Tarasiuk, and Y. Zigel, “Breathing rate estimation during sleep using audio signal analysis,” in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, 2015, pp. 5981–5984.
A. D. Droitcour, “Non-contact measurement of heart and respiration rates with a single-chip microwave Doppler radar,” Ph.D thesis, Stanford University, 2006.
W. Li, B. Tan, and R. J. Piechocki, “Non-contact breathing detection using passive radar,” in 2016 IEEE International Conference on Communications (ICC), IEEE, 2016, pp. 1–6.
C. Mak and Y. Lui, “The effect of sound on office productivity,” Building Services Engineering Research and Technology, vol. 33, no. 3, pp. 339–345, Aug. 2012, doi: 10.1177/0143624411412253.
Y. Shi and F. D. Real, “Smart cameras: Fundamentals and classification,” in Smart cameras, Springer, 2009, pp. 19–34.
M. H. Li, A. Yadollahi, and B. Taati, “A non-contact vision-based system for respiratory rate estimation,” in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2014, pp. 2119–2122.
G. Jayatilaka, H. Weligampola, S. Sritharan, P. Pathmanathan, R. Ragel, and I. Nawinne, “Non-contact infant sleep apnea detection,” in 2019 14th Conference on Industrial and Information Systems (ICIIS), IEEE, 2019, pp. 260–265.
K. S. Tan, R. Saatchi, H. Elphick, and D. Burke, “Real-time vision based respiration monitoring system,” in 2010 7th International Symposium on Communication Systems, Networks & Digital Signal Processing (CSNDSP 2010), IEEE, 2010, pp. 770–774.
A. Sobral and A. Vacavant, “A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos,” Computer Vision and Image Understanding, vol. 122, pp. 4–21, May 2014, doi: 10.1016/j.cviu.2013.12.005.
C. Y. Fang, H. H. Hsieh, and S. W. Chen, “A vision-based infant respiratory frequency detection system,” in 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA), IEEE, 2015, pp. 1–8.
K. O’Shea and R. Nash, “An introduction to convolutional neural networks,” arXiv preprint arXiv:1511.08458, 2015.
S. Baker and I. Matthews, “Lucas-kanade 20 years on: A unifying framework,” Int. J. Comput. Vis., vol. 56, no. 3, pp. 221–255, 2004.
S. P. Mishra et al., “Multivariate statistical data analysis-principal component analysis (PCA),” International Journal of Livestock Research, vol. 7, no. 5, pp. 60–78, 2017.
T. M. T. Nguyen, V. C. Pham, V. L. Tran, and X. Q. Dao, “Respiratory rate monitoring during sleep using camera,” Eastern International University Scientific Research Conference (EIUSRC2022), Dec. 2022.
A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, “Yolov4: Optimal speed and accuracy of object detection,” arXiv preprint arXiv:2004.10934, 2020.
J. Shi, “Good features to track,” in 1994 Proceedings of IEEE conference on computer vision and pattern recognition, IEEE, 1994, pp. 593–600.
N. Sharmin and R. Brad, “Optimal filter estimation for Lucas-Kanade optical flow,” Sensors, vol. 12, no. 9, pp. 12694–12709, 2012.
A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, “Yolov4: Optimal speed and accuracy of object detection,” arXiv preprint arXiv:2004.10934, 2020.
C. Harris and M. Stephens, “A combined corner and edge detector,” in Alvey vision conference, Manchester, UK, 1988, pp. 10–5244.
H. A. Kadhim and W. A. Araheemah, “A comparative between corner-detectors (Harris, Shi-Tomasi & FAST) in images noisy using non-local means filter,” Journal of Al-Qadisiyah for computer science and mathematics, vol. 11, no. 3, p. 86, 2019.
Downloads
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2023 Journal of Technical Education Science
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright © JTE.
