The first purpose of this paper was to describe a protocol for monitoring secular trends as part of a 5-year, community-based RCT on smoke-free policies in a rural, tobacco-producing state. This paper presents a systematic, feasible protocol for identifying, operationalizing and monitoring secular events that might impact the outcomes of community-based RCTs. It is important to monitor secular trends because they may mask the effects of interventions tested by RCTs and threaten internal validity. The method for monitoring secular trend analysis is not well defined in the literature. The protocol presented in this paper may be helpful to researchers conducting community-based RCTs on a variety of health topics.
Secular trend monitoring begins with identification and selection of relevant categories. These categories should be selected based on literature review and community experience. Although there may be some overlap to those selected for this protocol, the categories selected must be individualized to the topic area. For example, investigators conducting a community-based RCT to test an intervention designed to reduce human immunodeficiency virus transmission will likely also be interested in monitoring community-wide wellness campaigns. However, instead of capturing articles on tobacco cessation or smoking prevalence rates, secular events related to safe sex seminars or sexually transmitted disease prevalence rates may be more appropriate.
The next step to monitoring secular trends is deciding on the frequency of monitoring. This decision should be made based on the types of media available in the area. For this project, a statewide newsletter is reviewed weekly; newspaper articles are monitored biweekly and other websites (i.e. health departments and schools), as well as Google searches, are conducted monthly.
The second purpose of this paper was to provide preliminary results of the first 2 years of secular trend monitoring. The research team counts the number of secular events per category, as well as compares percent change over time by study group (Treatment, Control I and Control II). Secular trends, or patterns of secular events, may be considered possible threats to internal validity. For example, among Treatment communities, there was an increase in tobacco use and cessation activities.
In the case of the smoke detector devices installed in school restrooms to address student smoking, the impact of this one event on community outcomes is unknown, but when considered together with other similar secular events of this category, this may curb youth tobacco use and focus the community's attention on other ways of addressing the tobacco problem. A reduction in the smoking rate may motivate the community to embrace policy development, as a greater percentage of non-smokers, compared with smokers, support smoke-free policies [16
]. This example illustrates that it will be important to analyze both individual secular events and secular trends at the close of the RCT to determine unique influences within and across communities. If Treatment communities are more successful than Controls at enacting and implementing smoke-free laws at the end of the 5-year study, it will be important to consider whether these secular trends played a role in the outcomes or if they were due primarily to the intervention.
At this time, the data have been largely analyzed at the level of study group (Treatment, Control I and Control II). However, at the completion of the study, secular events and trends will also be examined at the individual county level. Quantitative, descriptive data are valuable in assessing which events may be recurring secular trends. These data may also be used as control variables in future analyses.
Although these data are valuable in assessing secular trends, the overall analysis of the impact of secular trends as a potential threat to internal validity will be conducted qualitatively. Qualitative data may be more useful in describing the magnitude of possible effects on smoke-free policy. For example, a series of articles on smoking cessation classes and an article on an increase in the prevalence of adult smoking would both be coded as tobacco use and cessation activities. The qualitative analysis of these articles would be important to consider in differentiating whether the secular events would likely have a positive or negative effect on policy outcomes. Similarly, in the case of the health department that received a grant supporting school nurses, qualitative data are needed to determine the magnitude of impact that this might have on smoke-free law development. A $100
000 grant would not have nearly the same impact as one for $1
There are limitations to secular trend analysis. First, the monitor must use his or her judgment using a broad list of inclusion criteria to determine if a particular article might be relevant to the outcome of interest (i.e. smoke-free policy development in rural communities). With only a single monitor, there is potential for error. In future studies that monitor secular trends, it is recommended that two people independently code a sample of the articles to determine inter-rater reliability.
Second, due to the timing of the media searches, it is possible that certain relevant events will be missed. This is particularly possible for communities that do not have online newspapers. In addition, every 2 weeks, all stories available on the newspaper's main page are searched, but some relevant secular events may be missed. In future studies, it may be beneficial to search more frequently. In addition, other methods of identifying secular events may be considered. One possibility could be quarterly surveys of leaders in each study community.
Third, community events may fit into more than one category. Consider the previous example of the health department receiving a grant to fund school nurses. While this article was coded city/county infrastructure, it is also a wellness and an ‘education/school system’ article.
Another limitation to this data analysis is the sample size of secular events. Although there were 854 events captured overall, these events were divided by county type (Treatment, Control I or Control II) and category. The sample size must be taken into consideration when evaluating the percent change. For example, there was a 225.6% increase in wellness from Year 1 to Year 2 in Control I communities, but the actual number of events increased from 4 to 12.
This paper described a protocol for identifying and monitoring secular events in 40 study counties in rural Kentucky that are part of a larger 5-year RCT to test the effects of a stage-based, tailored community-based intervention on smoke-free policy outcomes. Secular trend monitoring is a feasible yet time-intensive method of collecting data on events that might confound the results of a community-based RCT and threaten internal validity. This paper described data from Years 1 and 2 in Treatment and Control groups and considered how these data may be used to interpret the effects of the intervention.