|Home | About | Journals | Submit | Contact Us | Français|
Heart rate is an important physiological parameter for healthcare. Among measurement methods, photoplethysmography (PPG) is an easy and convenient method for pulse rate detection. However, as the PPG signal faces the challenge of motion artifacts and is constrained by the position chosen, the purpose of this paper is to implement a comfortable and easy-to-use multi-PPG sensor module combined with a stable and accurate real-time pulse rate detection method on a computer mouse. A weighted average method for multi-PPG sensors is used to adjust the weight of each signal channel in order to raise the accuracy and stability of the detected signal, therefore reducing the disturbance of noise under the environment of moving effectively and efficiently. According to the experiment results, the proposed method can increase the usability and probability of PPG signal detection on palms.
The statistic of World Health Organization (WHO) in 2016 has shown that the cardiovascular diseases (CVDs) are the number 1 cause of death globally . A review of the European Society of Cardiology has mentioned that heart rate is an independent cardiovascular risk factor , while the American Heart Association (AHA) has noticed that to control the heart rate within a relatively normal range is one of the treatment goals . Therefore, a comfortable and easy-to-use design of heart rate monitoring device will be the trend in the following years.
Photoplethysmography (PPG) is one of the optical techniques that has been developed for experimental use in vascular disease  and has applications on measuring the oxygen saturation, blood pressure, cardiac output , and determining heat stress level by using frequency analysis of PPG and its derivatives . To compare PPG with other heart rate detection methods like electrocardiography (ECG), PPG is more convenient to use and easier to wear . Users can easily wear the PPG devices on the earlobe, on the finger, on the wrist, or on the toe [7,8,9,10,11]. Different positions are chosen to detect the signal in order to improve the signal quality. Though PPG devices have advantages like being light-weighted, portable, and easy-to-use, motion artifacts are still the problems that have to overcome [5,12,13,14,15].
In the previous studies, problems such as weight of device, or whether the device is comfortable to wear or not are highly related to the acceptability and popularity of the users. Several later studies have proposed PPG detection on the side surface of the computer mouse [16,17,18]. However, users have to force their thumbs to contact the surface of the PPG sensor, which may not be comfortable. In this study, we suggest an alternate sensing position, developed by placing the multi-PPG sensor on the top surface of the mouse, which makes contact with the user’s palm. In this solution, multiple sensors can increase the contact area and the probability of signal detection, and users can use the mouse comfortably instead of forcing their finger skin to touch the sensor.
The PPG signal suffers from motion artifacts easily, and the signal quality is constrained by the position chosen for signal detection [10,19]. Lee et al. have indicated that the greatest and worst cause of artifact noise that contaminates the PPG signals is motion artifacts produced from physical activity and body movement . In order to achieve clear PPG and improve the accuracy of pulse rate detection, filter and amplification are required .
Tran and Chung have provided a robust algorithm for peak detection that can eliminate the fake peaks and correct error peaks in the original PPG signal under challenging movement conditions in real time on a personal computer mouse . Nevertheless, the error rate of the algorithm still increases due to mouse movements that cause more motion artifacts [14,15,20]. Alzahrani et al. and Warren et al. have constructed a design of multi-PPG sensors combining with an accelerometer on wearable devices to reduce motion artifacts [22,23]. These structures, however, have the disadvantages of increasing costs and algorithm complexities.
The purpose of our study is to implement a comfortable and easy-to-use module with a stable and accurate pulse rate detection on a computer mouse that most people are using everyday either in the office or at home, as multi-channel sensors can handle displacement and misalignment of the palm . We proposed a multi-sensor module design (which is smaller than the previous mentioned studies [16,17,18]) on the top surface of the mouse, and other circuits including the microcontroller are inside the mouse. Therefore, users don’t have to force their hands holding the mouse while they are using the mouse. The multi-channel sensor structure can not only raise the area contacted by the users, but also increase the sensitivity and predictivity. Furthermore, a weighted average method for the mixed signal can emphasize the better signal with a higher weight and the poorer signal with a lower weight to improve the quality of the signal and to increase the accuracy of detection.
The paper is organized as follows: Section 2 describes the hardware structure and the proposed method to perform pulse rate detection based on a multi-sensor; the experimental device and the experimental procedures of each stage are stated in Section 3; Section 4 summarizes the experimental results of the proposed method and the comparisons to each channel and both the average and the weighted average method; and finally, some conclusions are discussed in Section 5.
The proposed structure consists of a microcontroller (NUC120, Nuvoton Technology Corporation, Hsinchu City, Taiwan, which is an ARM®Cortex™-M0-based microcontroller), a Bluetooth module, the four-PPG sensor with four sets of band-pass filter with an amplifier, and a programmable-gain amplifier (PGA117, Texas Instruments, Dallas, TX, United States). The signal is extracted from each PPG sensor (uPI Semiconductor, Hsinchu County, Taiwan, with a wavelength of 850 nm) as one signal channel, which includes an infrared LED and a photodiode (PD).
The designed system structure is shown in Figure 1, which is based on the design of our previous study . The LED emits infrared light to the skin, and then the photodiode receives the reflected signal that is dependent on the changes in the blood pulse and generates the PPG signal at the output of the sensor. The PPG signal will then input through a band-pass filter and an amplifier. The filtered and amplified signal will next input into a microcontroller to evaluate the pulse rate by using the algorithm and finally output the PPG waveform of each channel and a mixed PPG to a PC, which is a user interface, for waveforms and pulse rate display, and for health data storage.
The Cortex™-M0 based microcontroller NUC120 (Nuvoton Technology Corporation, Hsinchu City, Taiwan) is used as the main controller in our design, and it is embedded in the computer mouse combined with other related circuits. The Inter Integrated Circuit (I2C) is used to control the PPG sensors, and the Serial Peripheral Interface (SPI) is used to control the PGA switching on modifying the magnitude factor of different PPG channels. The filtered and amplified PPG signal will input through a built-in analog-to-digital converter (ADC) with a resolution of 12 bits and the sampling rate is 200 Hz, and then the output signal will be communicated to the personal computer by the Bluetooth module. The waveform and the evaluated pulse rate will be displayed on the computer and the data is stored and can be analyzed by the application program.
In this work, we propose a pulse rate detection device using a computer mouse (TCN183, T.C. STAR, New Taipei City, Taiwan, with dpi of 800). Our proposed system is embedded in the internal of the mouse and only the multi-PPG sensor of the PPG sensor module is placed on the top surface of the mouse as shown in Figure 2 with 2 cm wide and 2 cm long. The PPG sensors 1, 2, 3, and 4 show the position of each PPG channel on the sensor, respectively.
In the previous hardware structure section, we have acquired the PPG signals from the hardware. All algorithms including the peak detection, pulse rate calculation, and digital signal processing are done in the microcontroller. In order to improve the quality of the PPG acquisition, we first input the PPG signal into a digital filtering process to eliminate the noise. Then, a waveforms mixing process is to emphasize the needed PPG signal to increase the accuracy of peaks detection and pulse rate calculation. The system flow of the designed process in this paper is depicted in Figure 3.
After analog signal processing in the previous hardware structure section, the extracted PPG signal has become more obvious and much cleaner than before, for which the signal is small and noise contaminated. However, there is still some high frequency noise disturbing the PPG signal due to hardware structure. Therefore, a 40th-order low-pass FIR (Finite Impulse Response) filter with a cutoff frequency of 5 Hz is designed to reduce the high frequency noise.
Since blood vessels spreading in the palm are diverse, the difficulties of PPG detection may be increased according to the position of palm that contacts the PPG sensors. Therefore, this study proposes to expand the area of detection using the structure of multi-sensors, and then mix the signals of each channel to form a new PPG signal that improves the quality of the signal and makes the detection easier and more convenient.
Two methods are used as comparisons in our proposed design, the average, and the weighted average method, with the purpose of observing whether the weighted average method is a better way to improve signal quality than the average method.
Therefore, a mixed waveform of four channels can be decided by two different methods above. According to the result of each sample either in the average method Equation (1), or the weighted average method Equation (2), we can then combine all sample points from each method into a whole waveform .
We can observe from the PPG signal that each pulse raises rapidly. Thus, the slope value of PPG signals and dynamic thresholding are used to detect the peaks. Figure 4 depicts the peak detection flow used in this study . In the beginning of the test, the Flag is set to false. When the slope value is larger than the threshold (Th), the Flag is set to true and this slope is recorded in the buffer. While the Flag is true and the slope value is smaller than or equals to 0, a peak index (peak position) is recorded. The maximum slope value from the buffer will be used to update the new threshold as adaptive thresholding and then clean the buffer to restart the peak detection.
A PPI check is then applied after peak detection to prevent noise contaminations. When the PPI is in the requested range, a peak is detected and finally outputs the pulse rate (). The requested PPI range is set to based on our previous work . The pulse rate is calculated as Equation (3), where the is determined by the peak-to-peak interval between the nth and the th peak and is the sampling rate. The peak detection and pulse rate calculation procedure are referenced and adapted based on our previous research [24,25]:
A reference ECG lead I signal is measured by using electrodes on the right and the left wrists and are connected to the microcontroller to synchronized with the PPG signal [26,27]. In order to verify the performance of the proposed algorithm that forms the mixed signal, experiments are designed in four stages (six movements): rest, slow movement (horizontal and vertical), rapid movement (horizontal and vertical), and browsing stage; details will be described in the section of Movement Stages. Furthermore, the results of each channel and the mixed waveform of two methods: the average method and the weighted average method are shown as comparisons.
There are 21 healthy volunteered subjects who participated in the experiments. The demographic data of the volunteers in this study is listed in Table 1: 17 males and four females with the mean ± standard deviation age of 25.14 ± 5.66 years old (from 20 to 41), the weight is 70.19 ± 15.52 kg (from 45 to 101), the height is 170.24 ± 6.99 cm (from 155 to 183), and the average heart rate (HR) is 88.84 ± 10.63 bpm (from 66.74 to 102.60). Every subject has to perform four stages of experiments, in which six movements are included, with a recording time of one minute per movement. For the reason of the initialization of our proposed device and the reference device, the first and the last five seconds of recording time will be discarded before data analyzing.
The main goal in this section is to verify whether the mixed signal of multiple channels from the average method and the weighted average method can have more stable signal than a single channel signal. In addition, the purpose of each stage is to test the accuracy of the proposed device across different experimental conditions—for instance, different speed and directions of movements.
Subjects should hold the mouse and make contact with their palms, without doing any actions like moving, or clicking, therefore controlling the variables.
Figure 5 shows the experimental user interface of the slow and rapid movement stages. The numbers 1 and 3 in the figure depict the slow horizontal (left and right) and vertical (up and down) movement, respectively. Furthermore, the numbers 2 and 4 in the figure depict the rapid horizontal and vertical movement, respectively. The scrollbar in each described movement (numbers 1 to 4) will move automatically with a constant speed (slow and rapid defined in Section 3.2.2 and Section 3.2.3, respectively) according to the direction shown by each bidirectional arrow.
In this experiment, subjects have to move the mouse cursor following the moving scrollbar shown on the numbers 1 and 3 in Figure 5, which is the experiment of the slow horizontal and vertical movement. The speed of slow movement experiments, either the horizontal or the vertical, is 10 s per round with a distance of 720 pixels (2 cm in the real movement) set by the authors. Furthermore, the recording time for each slow horizontal or vertical movement is one minute. The speed of both movements are recorded as well.
In this experiment, subjects have to move the mouse to follow the scrollbar shown on the numbers 2 and 4 in Figure 5, which is the experiment of the rapid horizontal (left and right) and vertical (up and down) movement. The speed of this experiment, either the horizontal or the vertical, is 2 s per round with a distance of 720 pixels, which is set by the authors. Furthermore, the recording time for each movement is one minute. The speed of both movements is recorded as previously.
The subject has to use the computer mouse either with moving, or left clicking the mouse quickly to browse the electronic document with the speed of 10 s per page for one minute. With respect to data observation and analysis, the PPG raw data of the proposed device and the reference device will be recorded, and the clicking time and times of clicking will be documented simultaneously.
To evaluate the performance of the designed algorithm, the signal acquired from the proposed and the reference device have been compared and a sensitivity as Equation (4), positive predictivity as Equation (5), and failed detection rate as Equation (6) test has been carried out for different configuration parameters :
where (True Positive) is the number of peaks detected; (False Negative) is the number of peaks non-detected; and (False Positive) is the number of artifacts or noise that classified as peaks. The definition of each parameter is shown as Figure 6.
Different parameter presents different meaning: the higher the , the lower probability of the peaks non-detected; represents the ratio of true peaks detected, which means the correctness of the detection; and the is the ratio of failure to correction. Otherwise, if high and low are evaluated, the detection result is contributed to a high accuracy.
In order to provide a comparison of the proposed algorithm, we compare the acquired PPG signal from our proposed device and the reference ECG signal from the reference device to evaluate the configuration parameters , , and .
Table 2 summarizes the results of the , , and of each channel and the mixed signal under the rest environment for all subjects. The performance of the mixed signal from the average method has an of and an of , and the performance of the weighted average method has an of and an of . While the worst performance under the rest environment is the signal from , it still has the of and a low of . Therefore, a high and a low of the proposed system is proved under the rest environment.
In the stages of slow movement and rapid movement, we have performed two directions of each stage: the horizontal and the vertical movement. Table 3 and Table 4 show a comparison among the results of the (%) of each channel and the mixed waveform from the average method and the proposed method of every subject for the slow horizontal movement and the rapid horizontal movement, respectively. The average of each channel and both mixed waveform methods of all subjects are also listed.
The waveforms of the rapid movement stage of each channel and the mixed signal by the average and the proposed weighted average method for subject 6 is shown in Figure 7. As stated in the previous research , a PPG peak comes after the ECG R-wave in the same cardiac cycle. We can obviously observe from each channel that there may be more than one peak appearing between the ECG R-peak and the following R-peak, which may affect the accuracy of detection. Moreover, the signal channels are motion contaminated due to a rapid movement of the mouse, and the peak detection result of the mixed signal by the average method is still affected by the fluctuations of motions, while the proposed weighted average method is almost close to the reference ECG signal. In the circle where some non-detected peaks of the average method occurred is due to the fact that the slope between the time period of 3 to 3.5 s does not exceed the maximum slope and leads to undetected peaks in the following few samples, while the weighted average method does amplify those points and detected the peaks.
Table 5 is the (%) result of each channel and the mixed waveform from the average and the weighted average method of every subject for the browsing experiment. The combination results of each channel and both mixed waveform methods of all subjects are also listed. As can be observed, though the low channels differ from subjects, the evaluated performance of the proposed weighted average method still results in lower failure of detection than the mixed signal of the average method. The waveform of the browsing stage for subject 9 is shown as Figure 8. It is shown that the signal is disturbed by moving and clicking. Especially in , the consecutive fluctuations in 0.5 to 2 s appear when clicking occurs. In the circle is the period where current PPI exceeds the requested range and therefore comes out as a non-detected peak in the weighted average method.
Table 6 and Table 7 summarized the results of all experiment stages by the average method and the weighted average method of all subjects, respectively. Comparing the of each method, we can observe that the proposed weighted average method has lower than the average method. The proposed method also has higher and than the average method of all experiment stages, as it was known that a higher is desirable to provide better prediction accuracy . The is used to assess the repeatability, the precision, and the reproducibility of the methods, and a higher is desirable to provide better performance . Therefore, it can be concluded from the results that a mixed waveform by the weighted average method has better performance including high accuracy and detection rate on peak detection than the average method.
In the four stages of six movements, the better performance signal channel differs from each subject. However, after the process of waveform mixing by the proposed algorithm, the and the of the mixed signal maintain the best or the second best. These have proved that even different habits of mouse use will affect the position detected, and the structure of multi-sensor can increase the probability of signal extraction.
There are two important movements of mouse usage: moving and clicking. From the stage of slow movement and rapid movement in the previous section, more motion artifacts are found in rapid movement stage than in slow movement stage. Though the sensitivity and the detection rate are reduced for rapid movement stage either for the results of the proposed algorithm or the average method, the evaluated result still remains a high and a low .
In the browsing stage, the tester has to use the mouse to browse the electronic document that combines mouse clicking and moving, therefore causing the decreasing of the sensitivity as compared to the rest environment. The reason for the loss of sensitivity is that the palm may leave a distance from the sensor when clicking occurs; thus, unexpected peaks appear and the difficulties of detection also increase. From the result of browsing by the proposed method in Table 5, failure of detection appears in most channels for almost all the subjects, but this issue is overcome with the proposed method, which will amplify the cleaner signal and reduce the effects of noise. The overall results of the proposed method in Table 7 also show the and the of the detected PPG signal.
As optical measurement like PPG has been widely used nowadays, a growing concern about the possible use of LED arrays that might pose a potential threat has been derived in some studies . Concerning the eye safety related to the power of infrared LEDs when the palm is not covering the mouse, a power saving mode is designed in this study to detect whether the user’s palm is contacting the mouse or not. When no palm (pulse rate) is detected, the turn-on time of the LED lights will be reduced by lowering its modulation frequency. Therefore, the controls of the PPG sensors are added not only for power saving, but also for protecting our eyes.
There are still some limitations in this study. The limitation of the contact sensor is that the sensors must be contacted with the palm while doing the measurement. In the normal computer use, the signals recorded with the system can be relatively piecewise periods of PPG signals since the user often loosens his/her grip on the mouse when he/she is typing the text with a keyboard. Therefore, the PPG signals were recorded only when the user is gripping the mouse.
The results in the study are carried out by limits of young and healthy subjects; thus, in the future, more subjects from different ages (young to old) and more conditions, like small and large palms, people with poorer circulation, or people with arrhythmias will be tested.
Moreover, motion artifacts still negatively impact measurement accuracy. Therefore, how to determine the signal quality is important [12,31,32]. In the results of the experiments, we found some sensitivity loss in the movement stages, which is due to the peaks being non-detected. Therefore, a more robust peak detection by event-related moving averages with dynamic threshold is considered  to improve the performance of peak detection in the future.
A real-time pulse rate monitoring mouse with a multi-sensor structure has been proposed in this study, and a weighted average method to adjust the weight of the signal according to the quality of the channel, which increase the accuracy and the stability of PPG peak detection. The mixed waveform process by the proposed method emphasizes the cleaner signal, therefore reducing the disturbance of noise effectively.
Despite the performance results by the weighted average method of each subject possibly not being the best, the overall sensitivity and the failed detection rate of the processed signal outperform most single channels. The four experiment stages have proved that the proposed multi-sensor structure and the weighted average method for the mixed waveform process can raise the usability of the detected PPG signal efficiently and effectively, therefore increasing the probability of PPG signal detection on palms.
Since the proposed design is simple, easy-to-use, and low-cost, the device could be served as a measurement tool to collect physiological signals and play a major role in early detection of diseases. Therefore, the proposed design is suitable as a solution to overcome barriers to improve health care in low- and middle-income countries .
This work was supported in part by uPI Semiconductor. We would like to acknowledge the support for their sponsorship to the project. This work was also supported in part by a research grant from the Ministry of Science and Technology under Grant MOST 105-3011-E-011-001.
The following abbreviations are used in this manuscript:
|WHO||World Health Organization|
|AHA||American Heart Association|
|I2C||Inter Intergrated Circuit|
|SPI||Serial Peripheral Interface|
|FIR||Finite Impulse Response|
|BMI||Body Mass Index|
|FDR||Failed Detection Rate|
S.-T.L., W.-H.C. and Y.-H.L. contributed in development of the multi-PPG sensor and the weighted average method; W.-H.C. implemented the hardware circuit of the multi-PPG sensor module and the experimental tools; S.-T.L., W.-H.C. and Y.-H.L. contributed in the analysis of the performance; S.-T.L. and Y.-H.L. wrote the paper; and Y.-H.L. contributed as a research advisor.
The authors declare no conflict of interest.
All subjects agreed with the informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology.