A RI sensor’s sensitivity is governed by the fraction of the optical mode that interacts with the sample. For many resonant-based RI sensors, a change in sample RI, Δn_s, results in a spectral shift Δλ is given by

where η is the fraction of optical intensity that exists in the liquid sample [5,14] and n_eff is the effective RI experienced by the resonant mode. Two resonant RI sensor configurations of note do not fit Eq. (1). For the first configuration, LPFG, the sensitivity is also related to the fraction of light interacting with the sample. The sensitivity increases as a function of η_c − η_cl, where η_c and η_cl are the fraction of optical intensity in solution in the core mode and the cladding mode, respectively [4]. The expressions for the second one, SPR, sensitivity in a prism-coupled and in a grating-coupled configuration are given in Homola [16].

Demonstrated RI sensitivities for various RI sensors are listed in Table 1. In general, designs that enable high interaction between the resonant mode and the sample have high sensitivity. In contrast, in a conventional ring resonator, only around 1–3% of the optical mode interacts with the sample [8,12], which leads to a relatively low sensitivity.

However, the magnitude of the spectral shift (i.e., the sensitivity) does not wholly articulate the capability of the device to detect and quantify the properties of the sample. Equally important is the ability to precisely and accurately measure the spectral shift that results from the sample. Here we introduce the concept of the sensor resolution, which characterizes the smallest possible spectral shift that can be accurately measured. As described later, this term takes into account the spectral resolution of the system and a number of noise parameters. The sensitivity (S) and the sensor resolution (R) combine to form the detection limit (DL) of the device:

For refractometric sensing, the DL reports the smallest sample RI change that can accurately be measured. For biomolecule sensing, the DL describes the minimum amount of analyte that the RI sensor can accurately quantify.

Spectral resolution and system noise factor into the DL because of the methodology for measuring the spectral shift in response to a sample. In most resonant-based RI sensors, the resonant mode has a Lorentzian spectral profile. One simple method to monitor the spectral shift is to track the position of the extremum (i.e., minimum or maximum, depending on the measurement configuration). The total sensor response is the spectral difference between the final and initial extrema, as illustrated in Fig. 1(A). If infinitely high spectral resolution were available and absolutely zero system noise were present, the sensor performance could be characterized by the sensitivity alone. However, spectral resolution and system noise detract from the precision and accuracy with which the true center of the resonant mode can be located. The minimum error in determining the actual mode spectral position leads to finite sensor resolution, and subsequently to the need for the sensor DL.

There are two classes of noise that contribute to errors in determining the positions of the resonant mode: amplitude variations and spectral variations. Amplitude noise refers to the cumulative noise added to the spectral mode profile. Noise sources include thermal and shot noise in the photodetector, laser relative intensity noise, and quantization error. Figure 2(A) shows a spectral mode trace with noise signals added to it. In a typical system, the extremum is located quickly and automatically by the system’s processor, such as a computer. With added amplitude noise, the actual extremum is not likely to be at the exact center frequency of the Lorentzian shape as it should be. Thus, the amplitude noise processes result in a random spectral deviation of the measured spectral location of the resonant mode.

It is important to analyze the random process of this deviation and its dependence on the magnitude of the noise and the linewidth of the mode. Although the noise processes are well characterized by standardized statistical distributions, the extremum operation prevents a direct analytical solution for the statistical distribution of the resulting spectral variation of the actual extremum. However, the system can be studied numerically using a Monte Carlo simulation. Here we use this technique to determine the statistical variance of the extremum of a Lorentzian-shaped mode when a random noise signal is added to the amplitude of the mode. For simplicity we draw each amplitude noise data point from a white Gaussian distribution (in reality, shot noise is produced by a Poisson process).

The Lorentzian modes pictured in Fig. 2(A) have a noise signal added to produce a signal to noise ratio (SNR) of 40 dB and 60 dB (60 dB is a good SNR for a typical photonic link), respectively, where the signal is defined as the height of the signal (power normalized to 1) and the noise is the variance of the noise distribution. In the Monte Carlo simulations, these signals are created repeatedly while the maximum is found for each resulting mode profile in each iteration. The deviation of the spectral location of the maximum amplitude from the actual center wavelength of the mode is determined for each iteration and then the resulting data set is statistically analyzed to determine the standard deviation. Figure 2(B) plots the three standard deviations (3σ) value of the determined maxima for SNRs ranging from 20 dB to 80 dB, and for Q-factors between 10³ and 10⁸. The inset shows a numerically generated probability density function (PDF) for the relative spectral position of the maximum value for a particular simulation run. The PDF is similar to a Gaussian distribution. The simulation results reveal that having a high Q-factor is advantageous in reducing the spectral noise of the sensor. This phenomenon can be explained by the fact that the noise signal results in a finite probability for samples within the optical bandwidth of the resonant mode to be identified as the extremum [see Fig. 2(A)]. Resonant modes with narrower linewidth filter the spectral noise more effectively, which leads to lower spectral deviation from the actual center of the mode. As a result, Q-factor plays an important role in the DL of the sensor.

Although an analytical solution for the statistical variance of the location of the mode amplitude maximum value is not available, the Monte Carlo simulation results provide an opportunity to approximate the relationship between Q-factor, SNR, and the resulting spectral variation of the mode’s maximum value. The simulation results reveal a linear dependence of the standard deviation on the linewidth and an exponential relationship with the SNR. It is found that within the ranges of parameters considered here, the numerical results can be approximated by: where σ is the standard deviation of the resulting spectral variation and Δλ is the full-width-half-max of the mode amplitude and is related to the Q-factor by Q = λ/Δλ. SNR is in linear units in this expression (e.g., 60 dB = 10⁶). The results of this estimation are plotted in Fig. 2(B), and show good agreement with the Monte Carlo simulation results. Equation (3) is generally applicable to any RI sensor technology in which the measured extremum value is used to identify the spectral location of the resonant mode.

In addition to amplitude noise, there are spectral noise factors that also contribute to variation in the spectral location of the resonant mode and thus to the sensor resolution. First is the thermal variation of the system. All RI sensors are susceptible to thermal-induced fluctuations because the sensor material and the sample have non-zero thermo-optic coefficients and thermal expansion coefficients. Thus, slight changes in temperature cause the RI of the material and of the sample to change, which, as shown by Eq. (1), causes the spectral position of the resonant mode to shift. This results in another noise source contributing to error in determining the spectral shift produced by the analyte.

The spectral resolution of the system setup also may limit the precision with which the resonant mode position may be identified. Depending on the measurement setup, either the laser or the optical detection mechanism may limit the resolution. In one common setup that is used for very narrow resonant mode linewidth RI sensors, the laser is tuned over a very small spectral range – on the order of pm – at the output of the system. It is expected that the processing unit in the setup can sample the photodetector voltage at a very high rate compared to the laser tuning speed. The limitation in this case is due to the linewidth of the laser, which is typically less than 1 MHz. At 1550 nm, 1 MHz is equivalent to about 8 fm.

In a setup that is common for relatively large resonant mode linewidth RI sensors, a broadband optical source and an optical spectrum analyzer or spectrometer are used to record the resonant mode. In this case, it is clear that the spectral resolution of the detection device provides the limitation, which may be on the order of 1 pm. The resulting error due to spectral limitations of the detection device can be modeled as quantization error. As an example, for a device with a spectral resolution of 1 pm, the error in determining the position of the resonant mode is uniformly distributed between −0.5 pm and 0.5 pm, and has a resulting standard deviation of σ_spec-res = 0.29 pm.

Once the statistical properties of all of the noise sources are understood, the sensor resolution can be determined. Here we use the typical convention of establishing the resolution as 3σ of the noise in the system. The total system noise variance can be approximated by summing all of the individual noise variances, i.e. .

For illustration, we can compare the DL of two very different cases. Consider one RI sensor with a sensitivity of 1000 nm/RIU, a Q-factor of 10⁴, a SNR of 60 dB, a spectral resolution of 1 pm, and temperature stabilization with standard deviation of 10 fm. Consider another RI sensor with a sensitivity of 25 nm/RIU, a Q-factor of 10⁷, a SNR of 60 dB, a spectral resolution of 1 fm, and a temperature stabilization with standard deviation of 10 fm. Assuming an operating wavelength of 1550 nm and using Eq. (3), we can compare the refractometric detection limit of these two RI sensors. The parameters and the resulting DLs are compared in Table 2. Despite the lower sensitivity, the high-Q sensor has slightly better performance because of the dramatically lower spectral variation due to amplitude noise. This result accents the need for a rigorous method of performance quantification for RI sensors instead of relying on the RI sensitivity value to indicate the sensor’s performance.

This illustration is also designed to emphasize the fact that high-Q-factor RI sensors are typically limited in performance by temperature stabilization, while low-Q-factor RI sensors are typically limited by amplitude noise and spectral resolution. Sensor design should focus on the minimization of these issues in order to minimize the detection limit.

Our analysis can also be used to investigate the role of the optical mode fraction that interacts with the sample, η. Equation (1) shows that sensitivity increases linearly with η, and Eq. (2) states that DL decreases linearly with S. Therefore, it seems logical to conclude that a higher η results in a lower detection limit. However, closer examination reveals that this is not always the case. Although sensitivity increases with increasing η, the absorption of photons by the sample medium also increases with increasing η. This decreases the Q-factor of the resonant mode, which is equivalent to an increase in the mode linewidth. This increasing linewidth serves to counteract the increasing sensitivity, and, depending on the absorption coefficient of the medium, may have an equalizing effect on the actual DL.

Consider an optical ring resonator sensor as an example. The ratio of the evanescent intensity in the sample to the total mode intensity is η. The sample has an absorption given by α, where α is related to the Q-factor of the mode by: where Q_α is the Q factor due to the absorption of photons and the observed Q-factor is defined as 1/Q = 1/Q_α + 1/Q₀. Figure 3(A) compares two Lorentzian modes from a ring resonator with η = 0.05 and η = 0.01 when the evanescent field is interacting with water, which has α ≈ 1000 m⁻¹ at 1550 nm [21]. As shown in the previous analysis, the broader linewidth leads to poorer sensor resolution and degrades the DL. Using Eqs. (3) and (4) to calculate the noise term due to the mode linewidth, we can calculate the DL for various sample absorption α and mode fraction in the sample η. Figure 3(B) plots the calculated detection limit for η of 0.01, 0.05, and 0.1, assuming a wavelength of 1550 nm, a mode amplitude SNR of 60 dB, and a Q₀ of 10⁸. While higher η results in a larger sensitivity, the calculations show that for α > 10 m⁻¹, there is no advantage in terms of detection limit because the increasing linewidth counteracts the increasing sensitivity. Alternatively, for α < 1 m⁻¹, an increasing η results in a corresponding improvement to the detection limit, as intuitively expected.

The dashed lines in Fig. 3(B) show the detection limit for η = 0.05 and η = 0.1 if a temperature-induced spectral fluctuation of σ = 10 fm is also factored into the calculation. The calculation indicates that for α < 100 m⁻¹, the temperature-induced fluctuation dominates, while for α > 100 m⁻¹, the effect on linewidth of the optical absorption due to the sample dominates the DL calculation. When the temperature-induced spectral fluctuation is dominating, an increase in η results in a linear improvement in detection limit. A direct consequence of the above result is that under the same experimental condition, the detection limit is better for the solution of higher RI, as more light is pulled into the solution [19].

This analysis offers clues for sensor design parameters that can improve performance. For samples that have low absorption, increasing the mode fraction of light interacting with the sample improves the sensor detection limit. For samples with higher absorption, there is no advantage to increasing the interaction between the resonant mode and the sample. Furthermore, consider the case of water, which has two orders of magnitude lower absorption for visible light as compared to 1550 nm. Figure 3(B) shows the significant advantage in sensor performance by operating at the lower wavelength and with higher interaction between the resonant mode and the sample. Armani et al. recently reported significant reduction in solvent absorption at near infrared by replacing H₂O with D₂O [10]. This is another strategy to improve the RI sensor performance.

Altogether, the analyses presented here provide insight into how to optimize sensor design to improve the detection limit. In the case of high-Q sensors, controlling the temperature-induced spectral fluctuations is the key to an excellent detection limit. Design options include temperature control using a thermal electric cooler (TEC) [22], the use of materials with advantageous thermal properties [23], or utilizing a reference channel to subtract the common mode noise, which includes thermally-induced spectral fluctuations [24]. Also, optimizing the wavelength of operation and the absorption of the sample contribute to improved detection limit, and lead to the opportunity to further improve the performance by increasing the interaction between the resonant mode an the sample. For low Q-factor sensing devices, these parameters are not expected to be particularly important because the detection limit is dominated by amplitude noise added to the recorded mode spectrum and by spectral resolution. Reducing the amplitude noise and applying signal processing techniques to reduce the recorded noise are expected to improve the detection performance of these devices. With these improvements, it is possible to break the RIU detection limit, which is currently on the order of 10⁻⁷ RIU [25, 26].

It is relatively straight-forward to convert a sensor’s RI sensitivity into a sensitivity for biomolecules because the capture of biomolecules at the sensor surface changes the RI in the region probed by the sensor’s evanescent field. An analysis to convert RI sensitivity to biomolecule sensitivity has been presented and experimentally verified for generalized ring resonator sensors by Zhu, et al., in [14]. In that work, the authors began with Eq. (1) above. The analysis is generally applicable to all resonant-based RI sensors that also conform to Eq. (1). The analysis leads to an expression for the spectral shift of the resonant mode, δλ, for biomolecule capture given an bulk RI sensitivity of S:

In this expression, σ_p is the surface density of the captured biomolecules, α_ex is the excess polarizability of the molecule, n_m is the RI of the sensor material (e.g., glass in a typical ring resonator), and n_s is the RI of the sample buffer. From this, we can define a sensitivity for biomolecule detection: