3.1 HR and RR detection
) signals for the G and B channels (R not shown), extracted from a 290 s movie of a volunteer who was asked to perform 50 knee bends. Five seconds after the exercise ended, the volunteer sat on a chair and was asked to hold his/her unsupported head still. The ROI for which signals are shown is a rectangular area on the forehead (ROI I in ). Dips in PVraw
= 35 s and t
= 90 s) are likely due to involuntary movements, changing the angle of the forehead slightly, resulting in a lower signal. DC levels or low frequencies (< 0.1 Hz) are typically not considered in PPG due to the fact that calibration of such signals is difficult [13
]. In our approach, using ambient light and a camera which applies several automatic brightness and color functions on each channel, calibration is an even more difficult task. Herein, we will not address the DC and low frequency variations and instead focus on the AC signals.
Fig. 1 (a): PVraw(t) signals for G and B channels as indicated and a ROI on the forehead (ROI I in ). Movie recording started 5 s after the subject finished physical exercise. Boxed areas in (a) are shown as insert graphs. In the left insert (t = 30–50s), (more ...)
Fig. 2 (a) PVAC(t) signals (G channel) for four ROI (I–IV) indicated in (b). (c) Corresponding power spectra. Signals for ROI’s II and III are reduced (× 0.1) for clarity. The bar in (a) represents 10 pixel values for II and III and 1 (more ...)
The plethysmographic information is visible in the inserts. Oscillations for HR and RR are indicated in the G and B channels, respectively (left insert). The HR is better visible in , displaying the joint time-frequency diagrams of the signals (G and B, respectively) in for a time window of 10 s (300 frames). The HR gradually decreases from t = 0 to 100 s, as the volunteer recuperates from the physical exercise. The HR and RR signals are most pronounced in the G and B channels, respectively, and both decrease gradually during recuperation. The fundamental HR frequency and, even the 2nd through 4th harmonics, can be identified in . Detection of the harmonics indicates that both the HR but and the shape of the plethysmogram were determined.
= 180 s the volunteer appeared to be fully recuperated and was asked to in- and exhale deeply and, at a more rapid pace, until t
= 230 s whereupon the volunteer was asked to breath normally. During this voluntary hyperventilation RR increases in frequency and amplitude (). Simultaneously, HR gradually increases from 1.1 to 1.5 Hz at t
=215 s after which it gradually decreases. HR peaks before
hyperventilation stopped, indicating that HR is autonomically controlled and measured independent of RR. The observed RR and HR evolutions are consistent with the literature [14
Quantification of HR, both at rest and after physical exercise, was found to be in excellent agreement with HR quantified by a commercial pressure cuff with a digital HR display. We are aware, however, that HR related signals could also be introduced through movements of the head. Any slight movement could move a relatively dark area in and out of the ROI with a periodicity equal to the HR and would also appear as plethysmographic signals. Such signals, however, would be fundamentally different from what is normally referred to as PPG: a signal induced by temporal variation in blood volume and corresponding variations in light absorption. Comparison of PVAC(t) signals are shown in for three additional ROI’s (II – IV), indicated in , provides evidence that the signals in are true PPG signals. If HR related movements combined with contrast variations were due to the strong HR signal shown in at 1.1 Hz, a smaller ROI than I should provide an even stronger HR signal since the many pixels in the bulk of ROI I, do not contribute to the HR signal (only contrast variations at the edges of ROI I would contribute). However, even a single pixel (ROI II) has the same power at the HR as ROI I (). The main difference is that the signal for ROI II contains more noise as compared to than that for ROI I. Apparently, each pixel in ROI contributes to the HR signal while camera noise (per pixel) is reduced by averaging leading to a much higher SNR than for a single pixel. The PVAC(t) signal for ROI III is strong due to the high contrast combined with involuntary movements. However, the power spectrum () does not indicate that movements are predominant at HR: the spectrum basically indicates noise, associated with movements at random frequencies. Both comparisons are consistent with the hypothesis that the signals in are true PPG signals. Finally, the fact that the G channel almost always features a much stronger HR signal as compared to the R and B channels, is also strong evidence that the signals are filtered by variations in blood volume (due to the absorption bands for oxy-and de-oxy hemoglobin for yellow and green light).
ROI IV, encompassing the entire face, illustrates that selection of the ROI is not critical for the HR determination. The amplitude of the HR signal is reduced (due to the inclusion of many background pixels) but the noise is also reduced, leaving the SNR intact: even the 3rd harmonic is well above camera noise level (). Movement artifacts combined with stark contrasts such as the eyes, nose, mouth or face compared to image background PV’s cancel out in the signal for ROI IV. The RR can be clearly identified at 0.27 Hz. The noise level as a function of frequency in (IV) is consistent with 1/frequency noise (not shown), which is characteristic for CCD detectors.
3.2 Modulation of RR and HR
Whereas – show the RR and HR as baseline modulated signals (DC modulation), an example of pulse amplitude modulation (AC modulation) is shown in . Data was extracted from a movie (shown in the left panel in media 1) recorded at 30 fps and 320×240 pixel resolution. are PVAC(t) and PVBP(t) maps, respectively, for t = 6 s. The display (false color) amplitude for is 5× that of to compensate for the much lower HR amplitude. The full PVAC(t) and PVBP(t) signals (30 s) are shown in where dashed vertical lines indicate the time for . The power spectrum for the G channel is shown in . The strongest signal at 0.12 Hz represents RR (harmonics are indicated) while 3 peaks in the HR frequency range (1, 1.12 and 1.24 Hz) are consistent with an amplitude modulation of HR at 1.12 Hz with RR at 0.12 Hz. shows PVBP(t) (0.8 – 6 Hz) also illustrating the ampltidue modulation of RR and HR.
Fig. 3 (a–c) A movie excerpt (frame 179, t = 6 s, (Media1)), selected to demonstrate a low signal for PVAC(t), (b) and high signal for PVBP(t) (c). (d) PVAC (t) for the ROI indicated in (a), for the R, G and B channels, displayed up to t = 30 s (media (more ...)
The amplitude relationship between R, G and B channels shown in is typical for most data we have obtained so far: the strongest plethysmographic signal is for the G channel, although the RR signal is sometimes more pronounced in the R or B channel (as in ). Note that the RR signals in for the three channels are different in shape as well as amplitude, suggesting that they may contain complementary information regarding oxygenation or depth origin of the pulsation (red light penetrates deeper in human skin as compared to blue). The RR induced R and B channel signals are well above the noise level and are lower than the G channel signal possibly due to a low absorption coefficient by blood for the R channel [15
] and a small probing volume for the B channel. The HR signals for the R and B channels in are close to the noise level.
3.3 Pulse amplitude mapping
As discussed in section 3.1 and illustrated by the signal for ROI III in , movement artifacts are likely not HR related, however, they are still important when generating pulsatility (power) maps. Areas with high contrast moving in and out of the ROI will feature strong fluctuations and, thus, correspond to increased powers for a large range of frequencies, including the HR (e.g., see the power spectrum for ROI III in ,).
A simple method to reduce such artifacts in power maps is illustrated in . Comparing the G channel of a movie frame and the corresponding power map () for the movie at the HR of this volunteer (1.06 Hz, power map is for a 23 s signal at 30 fps) shows relatively high powers in areas with high contrast. Under the assumption that movement artifacts mostly induce ‘white noise’ (no dominant frequency) around the HR frequency (as is plausible from the power spectrum for ROI III, ), we determine a ‘movement artifact map’ by averaging the powers at bandwidths around the HR. Such a map, () indicates that areas with high contrast () are indeed associated with high powers. After subtracting the movement artifact map from that in 4b, we obtain the map corrected for movement artifacts (. A range of better artifact reductions could be applied such as software to laterally synchronize the movie frame by frame. Such software is currently unavailable, however, and we used the described correction method as a preliminary attempt. All power maps shown herein are corrected in this way.
Fig. 4 (a) Still (G channel only) from a movie. (b) Corresponding power map (at HR = 1.06 Hz) including artifactual high powers in areas with high contrast. (c) The movement artifact map consisting of average powers for bandwidths (0.80 – 0.95 and 1.17– (more ...)
A critical look at the power map in in comparison with the image in 4(a) suggests that the pulse power map correlates with the intensity of the G channel. Lighter areas in 4(a) often feature a higher pulse signal in 4(b), and vice versa. Until we have established better ways to deal with shading (due to facial curvature etc.) the presented power maps should not be interpreted as robust maps of relative pulsatilities. Nevertheless, in pulse amplitude maps of several PWS patients, we have often observed clear pulsatility contrasts between normal and adjacent PWS skin indicating the feasibility of plethysmographic imaging. In normal skin HR pulse amplitudes (G channel) were typically 2 – 4 times higher than in adjacent PWS skin: e.g. 0.75 and 0.25 PV, respectively.
From calibration experiments (unpublished) we have determined that the pixel values of the used digital cameras can be related to light intensities as Y ≈ 73 ln(I) + k, where Y is a channel (R, G or B) pixel value, I is the actual light intensity to which a CCD pixel is exposed and k is a constant which depends on camera parameters such as exposure time and aperture. Using this relationship we estimate the variation in reflectance during a heart beat cycle from the amplitude of the HR plethysmograms (in PV). For example, a HR pulse amplitude of 0.75 PV corresponds to a 1% variation in reflectance (e.g.
from 20 to 20.2% reflectance [16
] for green/yellow light).
3.4 Phase mapping
An example of mapping phase differences using plethysmographic imaging is shown in and a movie (media 2). shows a still from a PWS patient who underwent laser therapy 5 minutes prior to the movie recording. Media 2 shows the PVBP(t) movie (BP filter: 0.8 – 6 Hz., G channel) of which excerpts (frames 9 and 22) are shown in . In the PWS area has a higher intensity as compared to surrounding normal skin (note, this is true for the AC signal, the DC intensity is still lower) and shows that 0.43 s later, the opposite is true. A phase map for the HR frequency of 1.43 Hz is shown in . The approximate phases of the frames in are indicated by the circle and triangle in . The Lissajous presentation () shows that the phase difference persists during the full 33 s. (43 heart beat cycles) of the movie recording. The Lissajous curve for the PWS signal shifted by 2 frames () shows that the phase difference can be quantified as approximately 0.067 s, or 34 degrees at a HR of 1.43 Hz.
Fig. 5 (a) Treated PWS area (dashed line) and 2 ROI (PWS and normal skin). (b–c) Frames 9 and 22, respectively, (Media 2). Intensities in (b–c) and Media 2 are linearly proportional to PVBP(t) (BP filter: 0.8 – 6 Hz). (d) Phase map (computed (more ...)
Some areas in can be identified as distinctly lower pulsatility (arrows) and correspond to areas that show some purpura (discoloration) due to laser therapy. Interestingly, they seem to be surrounded by areas with relatively higher pulsatility. The strong contrast between PWS and normal skin in the phase map () is remarkable, in particular when compared to the relative lack of contrast in the power map. To our knowledge such a phase difference in PWS has never been reported nor discussed in literature. We have not yet been able to explain these intriguing hemodynamic phenomena.
3.5 Cardio-vascular wave propagation
To illustrate interesting spatio-temporal features of the PPG signals and contrast between pulsatility in PWS and adjacent normal skin plethysmographic images are shown in and media 3. The panels in Media 3 are (from left to right) PVBP(t) maps (BP filter: 0.8 – 2 Hz) for the R, G and B channels and the actual movie (reduced resolution), respectively. A movie excerpt from media 3 (t = 3.4 s, frame 103) is shown in . In the carotid artery area (neck to ear), the PVBP movies for each of the channels often appear to show an upward direction of relative intensity. We tentatively interpret this as the cardio-vascular wave. Although for most heart beats the periodic intensity variation is spread out laterally and lacks a clear definition of vascular structure, the arrow in indicates a structure which may indeed be the carotid artery. The power maps for the full 30 s movie () average out the seemingly random lateral dispersion of the wave and show a similar longitudinal structure as indicated in . This structure is best defined in the power map for the R channel (, arrow).
Fig. 6 (a–c) PVBP(t) for R, G and B channels, respectively and (d) the original movie, a screenshot of (Media 3) (t = 3.3 s.). (e–g) Corresponding power maps for R, G and B for the HR frequency (1.06 Hz) and the full 30 s. movie. Arrows in (b,e) (more ...)
The phase map for the G channel () shows a small but distinct gradient (arrow) which corresponds to the general upward direction of the wave propagation displayed in Media 3. Although the carotid artery, presumably imaged in , is not visible on the phase map, a phase contrast between the general carotid artery area and the rest of the imaged face can be seen.
In all three power maps () the relatively high powers in the lower right corner (arrows in ) are movement artifacts. In the original, non compressed movie, this area can be clearly seen to pulsate as the cardio-vascular waves propagate through the carotid artery. Even the adjacent hair moves synchronously. Since these displacements are clearly dominated by the HR, the described correction method (section 3.3) fails to remove these artifacts. Given the strong physical displacement of skin by the left carotid artery, we are aware that the relatively strong signals in the patient’s right carotid artery area may also be (partly) due to displacements. A slight displacement may easily modify the illumination (angle) of this area, synchronously with the HR. For a further discussion regarding the interpretation of these results see below.