Submission title: “Research Vitals“
Authors: Aamna Amer and Mohamad Abdel Rida, University of Calgary
Layperson Summary:
COVID-19 struck us like a scene straight out of a movie. In a matter of months, it overwhelmed our healthcare systems and our health care workers. During these radical times, we needed a new weapon in our fight. Enter Remote Vitals. A tool that helps health care workers fight COVID-19. It works by a patient simply placing a mobile phone on their chest for 180 seconds. During this time, their phone is scanning and collecting thousands of data points. Finally, patterns in their breathing, such as coughs, whizzing, and abnormalities, emerge. Artificial intelligence recognizes these patterns that are hard to see with the naked eye and predicts what kind of breathing ailment the patient has. The greatest feature about Remote Vitals is that as more people use it, the more its accuracy improves. Just like a large project that needs a big group of people to work together, Remote Vitals enables users’ devices to work together to help detect new patterns. Remote vitals aims to reduce unnecessary checkups from doctors and nurses ultimately speeding up the discovery of a cure and the rolling out vaccines.
But we didn’t stop there. The treatment of COVID19 needs time and patience. So, we added a symptom tracking feature that helps patients who may be experiencing COVID-like symptoms to monitor and self-isolate with confidence. The patient can then share this information with health care workers in order to help them understand the patient’s condition and recent medical history with more accuracy.
Abstract:
The onset of the Coronavirus disease (COVID-19), or severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has resulted in a perceived need to accurately diagnose and continuously monitor non-severe COVID-19 patients remotely. The COVID-19 pandemic has highlighted the existence of inequalities corresponding to existing political, economical, and psychosocial factors that pose problems in the delivery and access of healthcare [1,2,3]. As such, the Remote Vitals application aims to bridge this gap by providing remote diagnostics and monitoring to individuals from the comfort of their home, while also keeping low-acuity patients and those that are immunocompromised safely isolated. Remote Vitals accomplishes this by providing three distinct services: a symptom tracker, a symptom query in collaboration with MediTriage Assistant, and a vitals monitoring tool. Patient vitals, particularly their breathing rate and heartbeat, are determined by employing smartphone sensors to collect accelerometer and gyroscope data that are first rigorously processed using signal processing techniques[8], such as Fast Fourier Transforms, autocorrelation, bandpass filtration, and Cloud-Based[7] machine learning[7] . Alternative preliminary diagnostics to COVID-19 is offered by MediTriage Assistant, a program that compares symptoms experienced against key words retrieved from medical ontologies. Remote Vitals, in conjunction with MediTriage Assistant, has the potential to significantly increase the capacity of tracking COVID-19 related symptoms, as well as detect and monitor infected patients and mitigate the risk of coronavirus spread.
Introduction:
Throughout history there have been several occurrences of communicable diseases spreading rapidly within a select region. With the onset of globalization and increase in population density, the transfer of infectious diseases has become exponentially easier. The term “epidemic” refers to the often unexpected increase in the number of infectious diseases in a locality or region that goes above the normal or predicted value in that specific area. A pandemic distinctly has a global or nation-wide social and economic impact while an epidemic is more contained in terms of not only the region, but also in the casualty experienced. For instance, major and renowned pandemics that have negatively impacted societies in the past include the Bubonic Plague, the Sixth Cholera Pandemic, with the most relevant being the Coronavirus pandemic.
One significant symptom that occurs in COVID-19 patients is Tachypnea, which is defined as abnormally rapid breathing. People infected with COVID-19 have more rapid respiration and thus have difficulty breathing or shortness of breath. Normally, doppler radar, thermal imaging technology and cameras based on motion detection are used for recording respiratory signals, but they each have their drawbacks. Contact measurement devices are heavy, expensive, and inconvenient for patients to use in comparison to cell phones that many individuals have on hand.
A research paper in the field of machine learning[7] demonstrated how machine vision and classifiers powered by deep learning could be used in the detection of abnormal respiratory breathing patterns[5]. A depth camera was used to calculate measurements based on the variation in displacement instead of the changes in pixel intensity. The proposed model with the depth camera could classify 6 different respiratory patterns with the accuracy and precision of 94.5% and 94.4% respectively [5]. However, one significant drawback with the research conducted is that depth cameras are cumbersome, require technical expertise to use, and are not readily available, whereas many individuals do have smartphones. The Remote Vitals research can be used to diagnose or monitor the health of patients from the comfort of their home, thus decreasing the reliance and pressure on hospitals and medical personnels, which is a characteristic unique to the Remotes Vitals App in comparison to other technologies that can detect respiratory signals.
By performing a Fast Fourier Transform and signal processing techniques[8] on the accelerometer and gyroscope data collected from a smartphone, breathing rate and heart rate can be extracted. The ultimate goal is to have a fully functioning app that, with the assistance of machine learning[7], is able to help analyze breathing and heart rates. More information about the nature of a patient’s breathing can be inferred through the properties of the signal, such as the breathing signal regularity, and then be used to monitor and offer preliminary diagnosis to the user.
Methods:
Signal Processing
Signal Processing [8] is a branch of electrical engineering that involves analyzing a plethora of signals, such as sound waves, images, and time-varying measurement values, and then mathematically manipulating them. The techniques used to process signals retrieved from accelerometer and gyroscope data from smartphone sensors in the Remote Vitals Application included: Fast Fourier Transforms, autocorrelation, bandpass frequency filtration and template matching [6].
Fourier Transform
A Fourier transform (FT) is a reversible mathematical method that transforms a non-periodic function of time into a function of frequency or vice versa. The term “Forward Fourier Transform” is used to describe the transformation between a function of time into a function of frequency, while the term “Inverse Fourier Transform” describes the opposite. The Fourier transform of a function f(𝓍) and the inverse Fourier Transform is represented by the following equations:
A Fourier transform has seven basic properties: linearity, translation (time shifting), modulation (frequency shifting), conjugation, time scaling, real and imaginary part in time, and finally integration. These unique mathematical properties enable a Fourier transform to be indicative of the frequencies and their proportions in a signal. Using the three parameters, amplitude, phase, and frequency, a frequency domain representation can be mapped. A Fourier transform expressed as a series of sine waves is superior to square waves, sawtooths, and other aperiodic signals due to the fact that it does not change shape when subjected to a LTI or linear variant system. As such, this form has widespread applications and is commonly used in control systems and signal processing[8].
A fast Fourier transform or a FFT is a highly efficient algorithm that lessens the computation time for a discrete Fourier transform (DFT) by eliminating the redundant calculations. FFT is based upon the divide-and-conquer paradigm, which requires a problem to be divided into smaller subproblems, solved, and then eventually combined into one larger solution. Generally, the subproblems are solved recursively or with a direct method, such as dynamic programming technique. In the Remote Vitals Application, the algorithm utilizes Fast Fourier Transforms to undergo a transformation between a function of time into a function of frequency with the data collected, in a timely and efficient manner that is superior to the regular Fourier Transforms, without compromising on the accuracy.
Autocorrelation
Autocorrelation is a mathematical representation of the degree of similarity between a signal and a delayed version of itself over successive time intervals. It is calculated as a function of mean and variance. The repetitive nature of a positive autocorrelation can help predict the behaviour of data, as such it is useful is forecasting trends. It can also be used, however, to determine the randomness of a dataset. Additionally, autocorrelation is also useful in determining linear dependence between the original signal and the lagged values of a time series.
Bandpass Frequency Filtration
A filter in the signal processing[8] field is a system that performs mathematical operations to enhance or reduce certain aspects of a signal. This form of mathematical manipulation encompasses two main systems: digital filters and spectrum analyzers. Digital filters perform signal filtering[8] in the time domain, whereas spectrum analyzers provide signal representation in the frequency domain. A low-pass filter falls under the branch of digital filters, and is responsible for weakening signals above a cutoff frequency, thus enabling lower frequencies to pass through the filter. Likewise, a high-pass filter weakens signals below a cutoff frequency, thus allowing higher frequencies to pass through the filter. Medical instrumentation like the EEG/ECG for example operate at low frequencies, as such they must pass through an LPF to eliminate ambient noise, whereas small speakers will use a high pass filter to remove the bass. A bandpass frequency filter combines aspects of both a low pass filter and a high pass filter to focus on a certain range of frequency.
Template Matching & Artificial Intelligence
Artificial intelligence has opened possibilities for efficient and complex algorithms to impact the lives of users in ways we previously thought impossible. In this project, we aim to use an ensemble of machine learning[7] models to hone in the predictive power of the algorithms that we have developed and provide an experience that is assuring, helpful, and truly empowers medical professionals with a wider set of tools at their disposal.
Once users take a recording, approximately 32 thousand data points per recording are sent to our servers which then filter, dissect, and absorb the necessary information that is embedded in this data. Patterns, anomalies, and insights are then used to update the existing models to improve their predictive power. The more people use Remote Vitals, the faster and more enhanced the experience is and in turn, the more clean, labeled data there is available at our disposal to enhance our machine learning[7] algorithms in the medical sphere. While each user has a standalone algorithm preshipped in their device, once they connect to the internet, they have access to an algorithm that has been trained on millions of data points from thousands of users.
Another interesting approach to make diagnosis and monitoring precise is the use of template matching[6] which is a method that attempts to find the template that matches the incoming data the closets. This method is only accessible to respiratory diseases, especially COVID-19. For example, in the case of COVID-19, patients experience tachypnea which displays a unique pattern in their incoming breathing data. The template matching[6] algorithm then sifts through libraries of preexisting knowledge to make connections with the users breathing patterns and the textbook breathing patterns of tachypnea. Furthermore, the more breathing data we obtain, the more efficient the template matching[6] algorithm becomes as we are able to identify cases where the data actually might correspond to users that have conducted COVID-19.
The incorporation of Artificial Intelligence in any application always has a drawback which is the need for data to fuel the learning of the models. Regardless of how advanced a model may be, without data, models can not truly provide value. Which is why we have incorporated a strong data pipeline between the users and our servers in order to speed up the training of our models and make sure that our users are always one step away from the magic. However, as this application gains traction, it will depend on the users to take recordings and interact with the application daily in order to improve the predictive quality of our models. These issues can easily be solved through incorporating pre existing data of breathing patterns and by creating mathematical models that use are adopted from literature regarding breathing pattern deficiencies such as bachichardia, trachichardia, and tachypnea to generate idealized data in order to give the models enough training so that deployment may still result in enough user traction.
MediTriage Assistant
The Meditrage assistant comprises two components: the client and the server. The client allows a user to input how they are feeling in terms of phrases and keywords. An example input from the user may be symptoms such as headache and persistent coughing. The assistant would then suggest the top three conditions that could entail their symptoms. These results are then paired with the signals provided from the device’s sensors to help label data corresponding to a user. For instance, if a user provides symptoms pertaining to COVID-19, the MediTriage system will carefully label the user’s data so long as they report experiencing the specified symptoms. This will then be incorporated into the machine learning algorithms to allow for clustering and classification of the data with labels uniquely associated with each recording.
Results:
Raw Sensor Data
Processed Data
Fourier Transform:
Low Pass Filter:
Conclusion
Remote Vitals tackles the issue of inadequate rapid testing and reduces unnecessary exposure to healthcare staff, among many other things. The app can also be tailored to provide an integrated service for other respiratory conditions in the future. Our data driven approaches that use built-in sensors to collect data can be employed to combat a wider set of diseases so long as the relevant sensors are available. Although we can sense heartbeats through the built-in mobile sensors, we can still use additional sensors that are found on wearable devices to tailor algorithms to the early identification and detection of heart disease. Additionally, we aim to further integrate with Alberta Health Services (AHS) and other government agencies by creating data pipelines that allow for the transfer of data between Remote Vitals and health agencies more efficiently.
By utilizing sensors that are available on every smartphone and creating a user interface that is accessible to both Android and IOS users, we are able to make Remote Vitals accessible to as many people as possible. By applying the Fast Fourier Transform, Band Pass filters, and Autocorrelation techniques, we are able to deliver more accurate results while curating the data that is inputted into our machine learning models in a cost effective and scalable manner. Furthermore, our results show that there is a lot of information that can be obtained from data that is collected by simple mobile sensors. In Figures 1.2 and 1.3, we were able to predict the heartbeat and breathing rate of the subjects within 5 bpms and 7 bpm of accuracy respectively. Collecting more data to accurately diagnose and infer the state of a patient’s breathing and respiratory health will be the next step in the progress of this project.
This report incorporated information from various reliable sources, including the organizations devoted to diminishing the impacts of epidemics, as well as trustworthy research papers in the field of academia. Research into epidemic origins and spread was mostly collected from disease prevention centers, including the World Health Organization and the Centers for Disease Control and Prevention to name a few. Information regarding epidemic modeling was taken from distinguished scientific magazines that present the work of scientists, such as the Nature Methods, which in this report was used to discuss the various models that exist today to help describe epidemics. To supplement the content encompassed in the research papers, notes taken from numerous university institutes were utilized as well.
References:
- Tai, D. B. G., Shah, A., Doubeni, C. A., Sia, I. G. & Wieland, M. L. The disproportionate
impact of COVID-19 on racial and ethnic minorities in the United States. Clin. Infect. Dis.
https://doi.org/10.1093/cid/ciaa815 (2020). - Williamson, E. J. et al. Factors associated with COVID-19-related death using
OpenSAFELY. Nature 584, 430–436 (2020). - Sirotich, E., Hausmann, J.S. Removing barriers and disparities in health: lessons from
the COVID-19 pandemic. Nat Rev Rheumatol (2020).
https://doi.org/10.1038/s41584-020-00524-8 - L. (2016, June 10). Template matching. Retrieved November, 2020, from
https://en.wikipedia.org/wiki/Template_matching#:~:text=Template%20matching%20is%
20a%20technique,to%20detect%20edges%20in%20images. http://www.learndatasci.com
Licence Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) - Wang, Y., Hu, M., Li, Q., Zhang, X., Zhai, G., & Yao, N. (2020, February 12). Abnormal
respiratory patterns classifiers may contribute to large-scale screening of people infected
with COVID-19 in an accurate and unobtrusive manner. Retrieved November 24, 2020,
from https://arxiv.org/abs/2002.05534 - (PDF) Fast and high-performance template matching method. (n.d.). Retrieved
November 24, 2020, from
https://www.researchgate.net/publication/221363967_Fast_and_high-performance_temp
late_matching_method - G, Y. (2017, October 27). Distributed training in the cloud: Cloud Machine Learning
Engine. Retrieved November 24, 2020, from
https://towardsdatascience.com/distributed-training-in-the-cloud-cloud-machine-learningengine-9e264ddde27f - Pal, K., Mitra, K., Bit, A., Bhattacharyya, S., & Dey, A. (2018, June 12). Medical Signal
Processing in Biomedical and Clinical Applications. Retrie
Appendix A: User Interface
