During the pre-pH1N1 and the pH1N1 period overall, both models' estimates were highly correlated with ILINet data (). However, during pH1N1 Wave 1, the original model did not correlate highly with ILINet data (0.290), whereas the updated model showed high correlation with ILINet data (r
0.945). Both the original and updated model estimates were highly correlated with ILINet data during pH1N1 Wave 2 (r
0.916 and r
Correlation and RMSE between United States Google Flu Trends estimates and ILINet data.
The magnitude of the ILI activity estimated by the original model was lower than both ILINet and the updated model estimates during the pH1N1 period overall, as evidenced by the threefold increase in RMSE compared to the pre-pH1N1 period (). The overall data trend for the two models was comparable, however, as evidenced by both models' high (r>0.9) correlations with ILINet data. The peak estimates of ILI activity occur during the same week in both models, and coincide with peak ILI activity as measured by ILINet ().
We can also see from that both models provided accurate estimates of ILINet data during early 2009, when seasonal influenza was circulating. Over the entire pre-pH1N1 period, the updated model slightly outperformed the original model, both in terms of correlation with ILINet data (original model: r
0.906; updated model: r
0.942) and RMSE (original model: RMSE
0.006; updated model: RMSE
0.005; see ). The updated model's peaks coincided with ILINet in four of the six pre-pH1N1 seasons (2003–04, 2004–05, 2005–06, and 2006–07); the original model's peaks coincided with ILINet in three previous seasons (2003–04, 2005–06, and 2007–08).
The updated model included approximately 160 search query terms related to influenza activity, compared with approximately 40 in the original model. Although the updated model uses four times as many queries as the original model, it has only one-fourth the query volume of the original model due to the inclusion of less common queries than in the original model. The two models share 11 queries, which comprise 50% of the updated model's query volume but only 11% of the original model's query volume.
The updated model queries are more directly related to influenza, rather than complications associated with influenza infection, such as “pnumonia” (misspelling is intentional and reflects the actual query spelling), which were a large composition of the original model (). Queries in the categories “influenza complication” and “symptoms of an influenza complication” made up 48% of the volume of the original model; in the updated model, these categories comprise only 17% of the volume. Queries in the categories “general influenza symptoms” and “specific influenza symptoms” comprise 69% of the updated model volume, compared with only 8% of original model volume. In addition, 72% of the updated model queries contain the word ‘flu’ (38% of volume), compared to only 14% of original model queries (2% of volume).
Search Behavior During pH1N1
Throughout the pH1N1 period, the total query volume for queries in the original model was lower than expected, given the previous relationship with ILINet data, and the original model therefore underestimated ILI activity. During the pH1N1 period, the original model underestimated ILINet data by an average of 0.014, a near three-fold increase in average error compared to the next-least-accurate season (2003), and a more than five-fold increase relative to the six prior seasons overall. Search query volume was low for nearly all query categories. In single-category models created to examine the volume decrease, all but one query category produced underestimates during the pH1N1 period. For example, queries in the category “influenza complication,” which previously comprised >40% of the original model query volume, underestimated ILINet data throughout the pH1N1 period (). Queries in the category “term for influenza” had elevated volume during the early months of the pH1N1 period; however, these queries comprised a small portion of the model volume (approximately 1%). Similarly, an additional analysis of regional-level models showed that the original model underestimated ILINet data in all ten U.S. regions as well as nationally (data not shown).
shows ILINet data and estimates from single-query models for the original-model queries [symptoms of flu], [symptoms of bronchitis], and [symptoms of pneumonia]. Prior to pH1N1, all three queries closely tracked ILINet data. During the pH1N1 pandemic, [symptoms of flu] continued to closely track ILINet data, whereas [symptoms of bronchitis] and [symptoms of pneumonia] clearly underestimated ILINet data, especially during pH1N1 Wave 2.
The relative volume of several updated model query categories changed during the pH1N1 period (); still, the overall model volume accurately estimated ILINet data throughout the pH1N1 period. During pH1N1 Wave 1, the relative volume for the “specific influenza symptom” category decreased by 28% (), and the relative volume for the “term for influenza” category increased by a factor of 2.5. During pH1N1 Wave 2, compared to pH1N1 Wave 1, the relative volume for the category “specific influenza symptom” decreased by a further 28%, and the relative volume for the category “general influenza symptoms” increased by 35%.
Category-level query volume before and during the pH1N1 pandemic in the updated United States GFT model.