Search tips
Search criteria 


Logo of lwwopenLippincott Williams & WilkinsThis article
Epidemiology (Cambridge, Mass.)
Epidemiology. 2017 January; 28(1): 72–76.
Published online 2016 November 30. doi:  10.1097/EDE.0000000000000567
PMCID: PMC5380105

Brief Report: Investigating Uncertainty in the Minimum Mortality Temperature

Methods and Application to 52 Spanish Cities



The minimum mortality temperature from J- or U-shaped curves varies across cities with different climates. This variation conveys information on adaptation, but ability to characterize is limited by the absence of a method to describe uncertainty in estimated minimum mortality temperatures.


We propose an approximate parametric bootstrap estimator of confidence interval (CI) and standard error (SE) for the minimum mortality temperature from a temperature–mortality shape estimated by splines.


The coverage of the estimated CIs was close to nominal value (95%) in the datasets simulated, although SEs were slightly high. Applying the method to 52 Spanish provincial capital cities showed larger minimum mortality temperatures in hotter cities, rising almost exactly at the same rate as annual mean temperature.


The method proposed for computing CIs and SEs for minimums from spline curves allows comparing minimum mortality temperatures in different cities and investigating their associations with climate properly, allowing for estimation uncertainty.

The temperature–mortality relationship has been described as a J- or U-shaped curve, with a temperature at which risk of mortality is at a minimum.1 Minimum mortality temperature is found at different temperature levels, varying greatly across countries and regions possibly due to acclimatization.2,3 The value of the minimum mortality temperature and in particular how it varies across places with different climates4 and over time5 has been investigated, as this is relevant to adaptation to climate. However, such comparison of minimum mortality temperatures, formal or informal, has been limited by the absence of a method to describe uncertainty in estimation of minimum mortality temperature. Temperature–mortality curves frequently do not identify clearly the point within the long bottom of the J or U shape where the minimum mortality temperature is, suggesting its estimation is imprecise.

This article presents a method to obtain an estimate, confidence interval (CI), and standard error (SE) for the minimum mortality temperature based on the temperature–mortality shape estimated by splines, and in particular distributed lag nonlinear models. We apply this method to investigate the patterns of minimum mortality temperature in the 52 provincial capital cities in Spain and explore the climatic determinants of variation in the city-specific minimum mortality temperature.


Temperature–Mortality Relationship

We collected daily counts of all-natural cause mortality (International Classification of Diseases-9th revision: 1–799) and daily mean temperature for the 52 provincial capital cities in Spain for the study period 1990–2010 (data provided by the Spain National Institute of Statistics and the Spain National Meteorology Agency, respectively). For each city, we applied a standard time-series quasi-Poisson regression, including a natural cubic spline of time with 10 degrees of freedom per year to control for seasonal and long-term trends and an indicator of day of the week. We modeled the association with temperature using distributed lag nonlinear models.6,7 This class of models can describe the complex nonlinear and lagged dependencies typically found for temperature and mortality through the combination of two functions that define the conventional exposure–response relationship and the additional lag–response relationship, respectively. The latter is applied to model the delay between the exposure occurrence and the associated increase in risk. We specifically modeled the exposure–response curves with a natural cubic spline with three internal knots placed at the 10th, 75th, and 90th centiles of location-specific temperature distributions. We did so because the temperature–mortality association is typically more curved toward the tails, in particular the right tail for heat, than across the milder, more central range, and the lag–response curve with a natural cubic with three internal knots placed at equally spaced values in the log scale.8 The lag period is extended to 21 days to capture the long delay in the effects of cold and account for short-term harvesting.8 We tested these modeling choices further in sensitivity analyses.

Point of Minimum Temperature Mortality

Identifying a minimum mortality temperature from any estimated function associating temperature to mortality is straightforward. One merely needs to scan through the function to find the value of t that minimizes it. Finding a SE and CI is not so simple. We propose the following algorithm, which can be described as an approximate parametric bootstrap estimate:

  1. Simulate a large number (we used N = 10,000) of parametric bootstrap splines from the estimated spline coefficients and their covariance matrix.
  2. For each simulated spline, identify the minimum mortality temperature.
  3. From the N estimated minimum mortality temperature of the simulated bootstrap splines, use the 2.5th and 97.5th centiles and the SD to estimate the 95% CI and SE, respectively.

A detailed algorithm to find SE and CI for the minimum mortality temperature of the temperature-mortality spline and R code for the estimation process, jointly with examples of use and simulations, are given in eAppendix 1 (

We used the estimates and SEs obtained by the above procedure to describe the distribution of the 52 minimum mortality temperatures and explore their association with the cities’ climates, summarized by the mean annual temperature, allowing for the estimation precision using random effects meta-analysis and meta-regression, estimating the between-city component of variance by the method of moments.9


We first tested the coverage of the CI and bias in the standard error estimated from this method by applying splines simulated from an underlying association with known minimum mortality temperature. Coverage of the 95% CI (from 1,000 simulations) was 96.4%. Mean of estimated SEs were higher (+41%) but median just slightly lower (−4%) than the SD of minimum mortality temperatures in the datasets simulated. Estimates did not change when using a negative binomial rather than a Poisson distribution (eAppendix 1 Table;

The city-specific temperature–mortality curves are shown in eAppendix 2 ( Most cities exhibited J or U shapes, showing evidence of increased mortality at temperatures higher and lower than the minimum mortality temperatures. However, we noticed that in some smaller cities, the minimum mortality temperature was at or close to one of the imprecisely estimated tails of the curve. To avoid what were likely spuriously high or low estimated minimum mortality temperatures, the point estimate was therefore constrained to the 1st–99th centile range, although we allowed the bootstrap minimum mortality temperatures, and hence CIs fall outside that range.

Figure Figure11 shows the curves for six cities selected to illustrate issues in estimating minimum mortality temperature and its CI. The top three cities in Figure Figure11 are typical in having convex (broadly U shaped) temperature–mortality curves, although for all except Barcelona the CIs widths suggest considerable uncertainty in the minimum mortality temperature. Also the uncertainty is notably asymmetric. The long, flat bottom of the curve for Madrid suggests the possibility of minimum mortality temperature range, a possible extension of the minimum mortality temperature concept. Each of the bottom three cities illustrates the main problems that can occur in smaller cities. In Tenerife and Cáceres, the unconstrained estimated minimum mortality temperatures are at the minimum and maximum of their temperature ranges, respectively, although each city also has a local, more central minimum, which is the estimate of minimum mortality temperature if the minimum mortality temperature is constrained to be in the 1st–99th percentile range. But again the CI widths suggest considerable uncertainty in the minimum mortality temperature, especially in Caceres where includes the entire range of temperature. In Teruel, the temperature–mortality curve is monotonic increasing, so the 1st–99th percentile constraint merely increases the minimum mortality temperature estimate from the minimum to the 1st percentile. The eAppendix 3 ( shows unconstrained and constrained estimates of minimum mortality temperatures for the cities for which the constraint was applies, and also for three others that would be constrained if 5th–95th or 10th–90th percentile range constraints had been adopted.

Temperature–mortality associations for six provincial capital cities in Spain (with 95% CI shaded grey) selected to illustrate issues in estimating minimum mortality temperatures. The top three cities show unconstrained minimum mortality temperatures ...

The city-specific minimum mortality temperatures ranged from −2°C, in a small Northern city (Teruel) up to 26°C in the Southern Mediterranean region (Tarragona and Castellόn), with random effects meta-analysis mean of 19.7°C (95% CI = 18.1°C, 21.4°C). In terms of the percentile scale minimum mortality temperatures ranged from the 1st to 93rd percentiles of the city-specific distribution of daily mean temperature (mean 15.5°C, ranged from 10.6°C to 21.6°C). Many of the minimum mortality temperatures are estimated imprecisely (mean standard error of 7.2°C), and the overlap of CIs suggests caution in comparing them between individual cities. Estimated underlying heterogeneity as a proportion of total variation is moderate (I2 = 24%), but there is a discernible pattern of higher thresholds in cities with higher annual mean temperatures (Figure (Figure2).2). A random effects meta-regression model confirmed this, estimating that the minimum mortality temperature increased on average 1°C for a mean rise of 1°C of the mean annual temperature (b = 1.09°C, 95% CI = 0.56°C, 1.63°C), with almost no residual heterogeneity (I2 = 2.8%). The patterns were broadly robust to alternative model assumptions. The use knots placed at the quartiles provided somewhat larger heterogeneity between-city estimates, while lag periods at 14 and 28 days provided slightly lower and upper pooled estimates, respectively (eAppendix 4;

Forest plot for the estimated minimum mortality temperatures, and its corresponding percentile, for the 52 provincial capital cities in Spain. Cities are sorted by annual mean temperature, which is indicated in parentheses after the city name. *Cities ...


By deriving and applying a method for computing CIs and SEs for minimums from spline curves, we were able to properly compare minimum mortality temperatures in different cities and investigate associations of them with climate allowing for estimation uncertainty, which we found often to be large. The method is applicable to describing uncertainty in any minimum or maximum of a fitted spline curve.

The principle difficulty was handling minimum mortality temperatures that were apparently at minimum or maximum temperatures, where spline curves are imprecise, mainly in smaller cities. We believe that our somewhat ad hoc procedure of constraining estimates to exclude temperatures in the extreme tails worked well in this data. However, it would seem sensible to review these procedures in future applications. It is possible for true minimum mortality temperatures to fall at or close to the minimum or maximum observed temperatures, where spline curves are imprecise. Large cities showing empirical minimum mortality temperatures at extreme temperature might give stronger evidence for this than the small cities with extreme minimum mortality temperatures seen in Spain. The use of minimum mortality temperatures from, for example, an empirical Bayes best linear unbiased temperature–mortality curves approach8 could be a more measured way to combine uncertain evidence from a city with prior expectations, but at the cost of complexity. Using the mean of the bootstrap simulated minimum mortality temperatures (as given in eAppendix 1; or natural cubic splines, which limit curvature of splines at tails,1,10 helps simplify problem cases more simply.

The availability of standard errors should make meta-analysis of minimum mortality temperatures more powerful and robust, but given often asymmetric confidence intervals, some caution is needed. Meta-analytic methods making minimum distributional assumptions to estimate the between-study component of variance, such as the DerSimonian and Laird approach,9 may have some advantages in this context.

Applying the method to 52 Spanish cities confirmed the previously observed pattern of higher minimum mortality temperatures in hotter cities2,3 and was able to quantify this, with minimum mortality temperatures rising almost exactly at the same rate as annual mean temperature (1°C/°C). This suggests that Spanish communities had broadly adapted to their local climate to the same extent. We hope that the method proposed will be useful in clarifying to what extent this is also true elsewhere and where climate varies over time, as found to some extent in France using similar analyses that, however, did not account for minimum mortality temperature uncertainty.5

Supplementary Material


A.T. was supported by the Ministry of Education of the Spanish Government (Salvador Madariaga’s Grant PRX12/00515), B.A. was supported by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Environmental Change and Health, and A.G. by the UK Medical Research Council (Grant ID: MR/M022625/1).

The authors report no conflicts of interest.

Supplemental digital content is available through direct URL citations in the HTML and PDF versions of this article (


1. Armstrong B. Models for the relationship between ambient temperature and daily mortality. Epidemiology. 2006;17:624–631. [PubMed]
2. Curriero FC, Heiner KS, Samet JM, Zeger SL, Strug L, Patz JA. Temperature and mortality in 11 cities of the eastern United States. Am J Epidemiol. 2002;155:80–87. [PubMed]
3. Armstrong BG, Chalabi Z, Fenn B, et al. Association of mortality with high temperatures in a temperate climate: England and Wales. J Epidemiol Community Health. 2011;65:340–345. [PubMed]
4. Guo Y, Gasparrini A, Armstrong B, et al. Global variation in the effects of ambient temperature on mortality: a systematic evaluation. Epidemiology. 2014;25:781–789. [PMC free article] [PubMed]
5. Todd N, Valleron AJ. Space-time covariation of mortality with temperature: a systematic study of deaths in France, 1968-2009. Environ Health Perspect. 2015;123:659–664. [PMC free article] [PubMed]
6. Gasparrini A, Armstrong B, Kenward MG. Distributed lag non-linear models. Stat Med. 2010;29:2224–2234. [PMC free article] [PubMed]
7. Gasparrini A. Distributed lag linear and non-linear models in R: the package dlnm. J Stat Softw. 2011;43:1–20. [PMC free article] [PubMed]
8. Gasparrini A, Guo Y, Hashizume M, et al. Mortality risk attributable to high and low ambient temperature: a multicountry observational study. Lancet. 2015;386:369–375. [PMC free article] [PubMed]
9. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–188. [PubMed]
10. Hastie T, Tibshirani R. Generalized Additive Models. 2nd ed. London: Chapman & Hall/CRC; 1990.

Articles from Lippincott Williams & Wilkins Open Access are provided here courtesy of Wolters Kluwer Health