PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jgimedspringer.comThis journalToc AlertsSubmit OnlineOpen Choice
 
J Gen Intern Med. 2016 August; 31(8): 863–870.
Published online 2016 March 25. doi:  10.1007/s11606-016-3673-7
PMCID: PMC4945559

Change In Length of Stay and Readmissions among Hospitalized Medical Patients after Inpatient Medicine Service Adoption of Mobile Secure Text Messaging

Mitesh S. Patel, MD, MBA, MS,corresponding author1,2,3,4,5 Neha Patel, MD, MS,2,5 Dylan S. Small, PhD,3 Roy Rosin, MBA,1 Jeffrey I. Rohrbach, MSN,5 Nathaniel Stromberg, MS,5 C. William Hanson, MD,2,5 and David A. Asch, MD, MBA1,2,3,4,5

Abstract

Background

Changes in the medium of communication from paging to mobile secure text messaging may change clinical care, but the effects of these changes on patient outcomes have not been well examined.

Objective

To evaluate the association between inpatient medicine service adoption of mobile secure text messaging and patient length of stay and readmissions.

Design

Observational study.

Participants

Patients admitted to medicine services at the Hospital of the University of Pennsylvania (intervention site; n = 8995 admissions of 6484 patients) and Penn Presbyterian Medical Center (control site; n = 6799 admissions of 4977 patients) between May 1, 2012, and April 30, 2014.

Intervention

Mobile secure text messaging.

Main Measures

Change in length of stay and 30-day readmissions, comparing patients at the intervention site to the control site before (May 1, 2012 to April 30, 2013) and after (May 1, 2013 to April 30, 2014) the intervention, adjusting for time trends and patient demographics, comorbidities, insurance, and disposition.

Key Results

During the pre-intervention period, the mean length of stay ranged from 4.0 to 5.0 days at the control site and from 5.2 to 6.7 days at the intervention site, but trends were similar. In the first month after the intervention, the mean length of stay was unchanged at the control site (4.7 to 4.7 days) but declined at the intervention site (6.0 to 5.4 days). Trends were mostly similar during the rest of the post-intervention period, ranging from 4.4 to 5.6 days at the control site and from 5.4 to 6.5 days at the intervention site. Readmission rates varied significantly within sites before and after the intervention, but overall trends were similar. In adjusted analyses, there was a significant decrease in length of stay for the intervention site relative to the control site during the post-intervention period compared to the pre-intervention period (−0.77 days ; 95 % CI, −1.14, −0.40; P < 0.001). There was no significant difference in the odds of readmission (OR, 0.97; 95 % CI: 0.81, 1.17; P = 0.77). These findings were supported by multiple sensitivity analyses.

Conclusions

Compared to a control group over time, hospitalized medical patients on inpatient services whose care providers and staff were offered mobile secure text messaging showed a relative decrease in length of stay and no change in readmissions.

KEY WORDS: mobile secure text messaging, patient outcomes, length of stay, readmissions, smartphone, communication, residents, teaching hospital, inpatient, medical patients

INTRODUCTION

Residents spend up to one-third of their time communicating with other health care professionals,1 and inadequate communication can lead to errors and delays.24 Much of this communication has shifted mediums from handwritten notes and face-to-face conversations to mobile platforms such as email and text messaging.5 These technologies offer ease, but their effect on clinical outcomes is unknown.

The shift to mobile communication channels in the health care setting have been spurred by the rapid adoption of smartphones.5 However, many email clients and short message services (SMS) send patient health information through an unsecured cellular network, putting patient privacy at risk.6,7 Many hospitals still require providers to use online paging systems, which are also often not secure, have limited mobile access, and require either a synchronous phone call or face-to-face interaction in order to close the loop on the intended communication. Mobile secure text messaging may help to address these issues by encrypting messages and allowing for rapid, asynchronous two-way (or larger group) exchanges of information that may better fit within the clinical workflow.

Several studies suggest that when mobile secure text messaging is available, it is widely used and is perceived to improve communication.8,9 However, its clinical impact has not been examined.9 We hypothesized that the use of mobile secure messaging would improve the flow of communication within the hospital, thereby leading to faster coordination of care and shorter length of stay. If these discharges were appropriate, we would then hypothesize that there should no increase in readmissions. The objective of this study was to evaluate the association between inpatient medicine service adoption of mobile secure text messaging and changes in length of stay and readmissions among hospitalized medical patients.

METHODS

This study was approved by the University of Pennsylvania institutional review board. Informed consent was waived because the study posed minimal risk to patients.

Study Sample

The sample comprised adults admitted between May 1, 2012, and April 30, 2014, to two inpatient medicine floors at the Hospital of the University of Pennsylvania (intervention site) and two inpatient medicine floors at Penn Presbyterian Medical Center (control site) in Philadelphia. Each of these floors was a teaching service with teams of attending physicians, residents, and interns that admitted general medicine, gastroenterology, infectious disease, nephrology, and pulmonary patients. The two hospitals share the same house staff and electronic record, and are located in the same ZIP code less than a mile apart. Discharges against medical advice (n = 329), to hospice (n = 325), or by death (n = 60) were excluded from the sample, because while they may impact outcome measure values, they were not changing differentially between sites over time.

Intervention

Prior to the intervention, providers in both hospitals used a computer Internet browser to send one-way alphanumeric messages through a paging system. Providers were not able to directly acknowledge or reply to these text messages. Instead, providers either had to log into a computer to send a response or discuss via phone or face-to-face. This method was continued at the control site during the intervention period. In May 2013, all providers and staff (attending physicians, residents, nurses, pharmacists, social workers, clinical resource coordinators, and unit secretaries) on two inpatient medicine floors at the intervention site were instructed to use a smartphone-based secure text messaging application (Cureatr Inc., New York, NY, USA) to communicate patient information and to help facilitate medical decision making. Providers could use an iPhone or Android smartphone to access the application. Those without a smartphone were given an iPhone or iPod touch to use while working on the floor. The application was also accessible online from a computer browser. The secure messaging application encrypted text before it was sent via a cellular or wireless Internet connection, and decrypted text before it was viewed. Senders could see when their recipients read the message, and recipients could reply directly using the application. This technology was not offered at the control site, which continued to use the one-way paging system.

Main Outcome Measures

The main outcome measures were inpatient length of stay and 30-day readmissions. Length of stay was defined as the time from hospital admission to discharge either home or to another institution (e.g. skilled nursing facility). Readmissions were defined as unplanned readmissions, excluding scheduled visits such as inpatient elective surgery and chemotherapy and unplanned but anticipated admissions for childbirth and organ transplantation. Data on admissions to other hospitals were not available, and thus we were only able to assess readmissions to the hospital of the initial admission.

Data

Patient information including admissions, discharges, demographics, comorbidities, and readmissions was obtained using health insurance claims from the billing system at the University of Pennsylvania Health System. Utilization of mobile secure messaging was retrieved from the Cureatr platform, which reveals the number of messages sent and received by each role.

Statistical Analysis

We used a multiple time-series research design,1013 also known as difference-in-differences, to compare within-site pre- and post-intervention outcomes between sites where the secure messaging was offered and where it was not. While some opportunity for residual confounding remains, this approach reduces potential biases from unmeasured variables from three possible sources.11,14,15 First, a difference between hospitals that is stable over time cannot be mistaken for an effect of the intervention, because hospital fixed effects are used to compare each hospital with itself before and after the intervention. Second, changes affecting both hospitals similarly, such as technological improvements or pay-for-performance initiatives, cannot be mistaken for an effect, because the regression models use time fixed effects. Third, if the patient mix changes differently between hospitals, and if these changes are accurately reflected in the measured risk factors, this cannot be mistaken for an effect of the intervention, because the regression models adjust for these measured risk factors. In addition, if outcomes in the pre-intervention period are changing in parallel, it allows us to evaluate diverging trends in the post-intervention period that might indicate whether there was an association between the change in outcome and the intervention.

Similar to prior work,12,13 a multivariate logistic model was fit to the binary outcome of 30-day readmissions, and a multivariate linear regression model was fit to the continuous outcome of length of stay in days using the hospital admission as the unit of analysis. The models used data from May 1, 2012, to April 30, 2014, and were adjusted for age, gender, race/ethnicity, insurance status, and disposition. They were also risk-adjusted for patient comorbidity severity using the Charlson Comorbidity Index (CCI), which predicts 10-year mortality.16 We used calendar-month fixed effects by creating a binary variable for each of the 12 calendar months. Since there were 24 months during the study period, this ensured that a change due to seasonal factors (e.g. in July when new interns begin) could not be mistaken for an effect of the intervention. Each patient visit was attributed to the month in which the day of admission occurred. We created a binary time variable to indicate either the pre-intervention or post-intervention period and a binary site variable to indicate either the control or intervention site. The effect of the intervention was evaluated using interaction terms for each calendar month, time (post-intervention period), and site. Standard errors in the models were adjusted to account for clustering by patient.17,18 To assess the mean effect of the intervention in the post-intervention period, we exponentiated the mean of the monthly log odds ratios for the 30-day readmissions measure and computed a linear combination of monthly interaction estimates for length of stay.19,20 For both measures, a test of controls was conducted to test the null hypothesis of parallel trends during the pre-intervention period. We also tested the null hypothesis of parallel trends in patient comorbidity severity between the two sites before and after the intervention period using the CCI. An inability to reject the null hypothesis in this test supports the notion that any effects observed after the intervention were less likely to be due to differential shifts in comorbidity severity between the two sites. All hypothesis tests were two-sided and used a significance level of P <0.05. All statistical analyses were conducted using Stata version 12.1 software (StataCorp, College Station, TX, USA).

To evaluate the robustness of our model estimates, we performed several independent sensitivity analyses. First, we estimated the models using fixed effects by within-hospital location to adjust for variability that may have occurred within the same hospital but on different medicine floors. Second, we estimated the models using only each patient’s first admission to ensure that a subset of high utilizers was not driving the overall changes in outcomes. Third, while the main models used the CCI to adjust for patient disease severity, we also estimated the models adjusting for changes in the rates of patient comorbidities using a binary term for each condition. Fourth, for the outcome of length of stay, we estimated the model using a gamma regression, which may better fit data with positive skew.21 Fifth, we estimated the change in median length of stay using a quantile regression model.22,23 A change in median length of stay may indicate that changes are not just attributable to decreases in longer lengths of stay, but also to changes in shorter lengths of stay.

RESULTS

The study sample comprised 4977 patients with 6799 admissions at the control site and 6484 patients with 8995 admissions at the intervention site. Table Table11 shows patient characteristics and comorbidities, which were similar within sites before and after the intervention. A test of the CCI could not reject the null hypothesis of parallel trends in patient comorbidity severity between hospital sites before and after the intervention (−0.10; 95 % CI: − 0.23, 0.02; P = 0.11).

Table 1
Patient Characteristics

During the intervention period, there were 446,342 secure messages sent, of which 377,347 were individual messages and 68,995 were group (>1 recipient) messages. The highest volume was among residents (167,405 messages; 37.5 % of all messages) and nurses (177,184; 39.7 %), followed by social workers and clinical research coordinators (35,594; 8.0 %), attending physicians (34,926; 7.8 %), pharmacists (26,434; 5.9 %), and unit secretaries (4,799; 1.1 %).

Unadjusted monthly trends in mean length of stay are displayed by hospital site in Figure Figure1.1. During the pre-intervention period, the mean length of stay ranged from 4.0 to 5.0 days at the control site and from 5.2 to 6.7 days at the intervention site. A test of controls for the pre-intervention period could not reject the null hypothesis of parallel trends in length of stay at the two sites (0.01 days; 95 % confidence interval [CI], −0.73, 0.75, P = 0.98). In the first month after the intervention, the mean length of stay was unchanged at the control site (4.7 to 4.7 days) but declined at the intervention site (6.0 to 5.4 days). Trends were largely similar during the rest of the post-intervention period, ranging from 4.4 to 5.6 days at the control site and from 5.4 to 6.5 days at the intervention site.

Figure 1
Mean length of stay by hospital site. Data presented are mean length of stay in days by month before and after the intervention, which occurred after month 12. The solid black line represents the control site, Penn Presbyterian Medical Center. The dashed ...

Unadjusted monthly trends in the readmissions rate are displayed by hospital site in Figure Figure2.2. During the pre-intervention period, the readmission rates ranged from 8.0 to 15.1 % at the control site and 11.1 to 19.0 % at the intervention site. A test of controls for the pre-intervention period could not reject the null hypothesis of parallel trends in readmissions at the two sites (odds ratio [OR]: 1.11; 95 % CI, 0.72, 1.70, P = 0.65). During the post-intervention period, the readmission rates ranged from 9.5 to 15.5 % at the control site and 11.8 to 17.9 % at the intervention site.

Figure 2
Readmission rate by hospital site. Data presented are the percentage of patients discharged that were readmitted to the same hospital, by month, before and after the intervention, which occurred after month 12. The solid black line represents the control ...

In adjusted analyses, the annual mean length of stay was unchanged at the intervention site from before to after intervention, whereas it increased at the control site in the post-intervention period, representing a significant relative reduction associated with implementation of mobile secure text messaging (−0.77 days ; 95 % CI, −1.14, −0.40; P < 0.001). Findings were sustained in each of the sensitivity analyses, gamma regression model, and quantile regression model (Table (Table2).2). There was no significant difference in the odds of readmissions (OR, 0.97; 95 % CI: 0.81, 1.17; P = 0.77). Findings were sustained in each of the sensitivity analyses (Table (Table33).

Table 2
Adjusted Model Estimates for Change in Length of Stay
Table 3
Adjusted Model Estimates for Change in Readmission

DISCUSSION

It is important to recognize that changes to the medium of communication—for example, moving from face-to-face to telephone, or from telephone to mobile secure text messaging—change the effects of the messages. Because communication is so central to effective health care, the recent transformation of communication more generally has the potential to greatly affect clinical care, with the consequent need to evaluate new communication methods in health care just as we might evaluate new drugs, new insurance benefit designs, or new payment policies. To our knowledge, this is one of the first evaluations of patient care outcomes after the use of mobile secure text messaging.

We found that adoption of this mobile technology was associated with a significant relative reduction in length of stay after adjusting for time trends and patient demographics, comorbidities, insurance, and disposition. This result makes sense, because texting is perceived to facilitate communication, and we hypothesized that improved communication might accelerate care coordination. Based on the models, this intervention resulted in about a 0.77-day or 14 % relative decrease in length of stay.

The most significant comparative reduction in length of stay occurred immediately after the intervention. In the first month, mean length of stay was unchanged at the control site (from 4.7 to 4.7 days) but decreased at the intervention site (from 6.0 to 5.4 days). This observation of immediate effects suggest that the mechanism was immediate—perhaps, as postulated, because more effective communication among the care team reduced delays in medical decision making or helped expedite discharge plans. It should be noted that mean length of stay at the annual level was unchanged at the intervention site and increased at the control site. While we were unable to adjust for staff and hospital characteristics, there are several indications to suggest this change may have been attributable to the intervention. First, in the prior 12 months, mean length of stay differed between the two sites but changed similarly, as indicated by the parallel trends test. It was not until the first month after the intervention that these trends diverged. Second, aside from mobile secure text messaging, we know of no hospital or floor-based changes at either site occurring in May 2013 that could have accounted for such an effect on patient length of stay. Finally, these changes occurred before the time of resident turnover (June to July), and our model used calendar-month fixed effects, so these changes were likely not due to seasonal trends. Thus, while the absolute site-specific changes over time should be considered, it is important to interpret the findings according to the difference in changes in outcomes between the two sites over time.

If shorter lengths of stays meant that patients were discharged too early, we might expect to find an increase in readmissions. The observation that mobile secure text messaging is associated with substantially shorter length of stay but no increase in readmissions suggests a relatively unqualified benefit.2426 These results were consistent across multiple sensitivity analyses.

Our findings have important implications for hospitals and residency programs. Many hospitals still rely on paging systems that allow for only one-way alphanumeric messaging, have limited mobile capabilities, and may be less efficient within clinical workflow. Many providers may still use mobile channels for communication such as email and SMS text messages, but these channels are often not secure and not compliant with the standards set forth in the Health Insurance Portability and Accountability Act (HIPAA).27,28 Mobile secure text messaging can address many of these issues and allow for additional functionality to improve communication, including message acknowledgment, the ability to add or remove select individuals in a group message, or the ability to send an urgent response to a message. Similar to prior studies,8,9 we found high utilization of mobile secure messaging. The engagement of both physicians and staff may have created increased opportunities to improve communication towards more efficient workflow. Future studies to further understand the relationship between mobile secure text messaging and patient outcomes might use a prospective randomized intervention design and include qualitative analyses among key stakeholders.

This study is subject to several limitations. First, any observational study is susceptible to unmeasured confounders. However, the comparison of outcomes over time between the intervention and control sites diminished the potential bias from unmeasured confounding. While we were not able to address serial correlation in our model, the small number of groups and time periods suggests that this was less likely to bias our results.2931 Second, our findings are from hospitalized medical patients at a single health system, which may limit generalizability to other settings. Third, we did not have data to evaluate readmissions to hospitals other than those in this study. Fourth, length of stay and readmissions—while closely watched—are limited indicators of health care cost and quality. We were unable to adequately evaluate changes in mortality due to the low number of inpatient deaths in this patient sample. Finally, we evaluated outcomes associated with use of one mobile secure text messaging application. While many of the features of this technology are similar to those of other secure text messaging applications, more study is needed using other forms of this technology.

In conclusion, compared to a control group over time, hospitalized medical patients on inpatient services whose care providers and staff were offered mobile secure text messaging saw a relative decrease in length of stay and no change in readmissions.

ACKNOWLEDGMENTS

Dr. Patel had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the analysis. This study received no external funding, and no one other than the authors had any role in the design or conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. The authors received no financial compensation from nor do they have a financial interest in the mobile secure text messaging application described in this study. This study was approved by the University of Pennsylvania Instititutional Review Board

Compliance with Ethical Standards

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.

REFERENCES

1. Fletcher KE, Visotcky AM, Slagle JM, Tarima S, Weinger MB, Schapira MM. The composition of intern work while on call. J Gen Intern Med. 2012;27(11):1432–1437. doi: 10.1007/s11606-012-2120-7. [PMC free article] [PubMed] [Cross Ref]
2. The Joint Commission. Sentinel Event Data - Root Causes by Event Type. Available online at: http://www.jointcommission.org/Sentinel_Event_Statistics/. Accessed February 15, 2016.
3. Dunn AS, Markoff B. Physician-physician communication: what's the hang-up? J Gen Intern Med. 2009;24(3):437–439. doi: 10.1007/s11606-009-0913-0. [PMC free article] [PubMed] [Cross Ref]
4. Sutcliffe KM, Lewton E, Rosenthal MM. Communication failures: an insidious contributor to medical mishaps. Acad Med. 2004;79(2):186–194. doi: 10.1097/00001888-200402000-00019. [PubMed] [Cross Ref]
5. Horwitz LI, Detsky AS. Physician communication in the 21st century: to talk or to text? JAMA. 2011;305(11):1128–1129. doi: 10.1001/jama.2011.324. [PubMed] [Cross Ref]
6. Redelmeier DA, Detsky AS. Pitfalls with Smartphones in Medicine. J Gen Intern Med. 2013;28(10):1260–1263. doi: 10.1007/s11606-013-2467-4. [PMC free article] [PubMed] [Cross Ref]
7. Tran K, Morra D, Lo V, Quan SD, Abrams H, Wu RC. Medical students and personal smartphones in the clinical environment: the impact on confidentiality of personal health information and professionalism. J Med Internet Res. 2014;16(5) doi: 10.2196/jmir.3138. [PMC free article] [PubMed] [Cross Ref]
8. Wu R, Lo V, Morra D, Appel E, Arany T, Curiale B, Ryan J, Quan S. A smartphone-enabled communication system to improve hospital communication: Usage and perceptions of medical trainees and nurses on general internal medicine wards. J Hosp Med. 2015;10(2):83–89. doi: 10.1002/jhm.2278. [PubMed] [Cross Ref]
9. Przybylo JA, Wang A, Loftus P, Evans KH, Chu I, Shieh L. Smarter hospital communication: Secure smartphone text messaging improves provider satisfaction and perception of efficacy, workflow. J Hosp Med. 2014;9(9):573–578. doi: 10.1002/jhm.2228. [PMC free article] [PubMed] [Cross Ref]
10. Campbell DT, Stanley JC. Experimental and Quasi-Experimental Designs for Research. Dallas: Houghton Mifflin Co; 1963.
11. Dimick JB, Ryan AM. Methods for Evaluating Changes in Health Care Policy: The Difference-in-Differences Approach. JAMA. 2014;312(22):2401–2402. doi: 10.1001/jama.2014.16153. [PubMed] [Cross Ref]
12. Patel MS, Day S, Small DS, Howell JT, Lautenbach GL, Nierman EH, Volpp KG. Using Default Options Within the Electronic Health Record to Increase the Prescribing of Generic-Equivalent Medications: A Quasi-experimental Study. Ann Intern Med. 2014;161(10_Supplement):S44–S52. doi: 10.7326/M13-3001. [PubMed] [Cross Ref]
13. Patel MS, Volpp KG, Small DS, Hill AS, Even-Shoshan O, Rosenbaum L, Ross RN, Bellini L, Zhu J, Silber JH. Association of the 2011 ACGME Resident Duty Hour Reforms With Mortality and Readmissions Among Hospitalized Medicare Patients. JAMA. 2014;312(22):2364–2373. doi: 10.1001/jama.2014.15273. [PubMed] [Cross Ref]
14. Rosenbaum PR. Stability in the absence of treatment. J Am Stat Assoc. 2001;96:210–219. doi: 10.1198/016214501750333072. [Cross Ref]
15. Shadish WR, Cook TD, Campbell DT. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton-Mifflin; 2002.
16. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8. [PubMed] [Cross Ref]
17. Rogers WH. Regression standard errors in clustered samples. Stata Tech Bull Rep. 1993;3:88–94.
18. Williams RL. A note on robust variance estimation for cluster-correlated data. Biometrics. 2000;56:645–6. doi: 10.1111/j.0006-341X.2000.00645.x. [PubMed] [Cross Ref]
19. Newman SC. Biostatistical Methods in Epidemiology. New York: Wiley; 2001. p. 136.
20. Agresti A. An Introduction to Categorical Data Analysis. New York: Wiley; 2007. pp. 108–9.
21. Basu A, Manning WG, Mullahy J. Comparing alternative models: log vs. cox proportional hazard? Health Econ. 2004;13:749–765. doi: 10.1002/hec.852. [PubMed] [Cross Ref]
22. Koenker R. Quantile Regression. Cambridge University Press: 2005.
23. Lee AH, Fung WK, Fu B. Analyzing hospital length of stay: mean or median regression? Med Care. 2003;41:681–686. [PubMed]
24. Bueno H, Ross JS, Wang Y, Chen J, Vidán MT, Normand SL, Curtis JP, Drye EE, Lichtman JH, Keenan PS, Kosiborod M, Krumholz HM. Trends in length of stay and short-term outcomes among Medicare patients hospitalized for heart failure, 1993–2006. JAMA. 2010;303(21):2141–2147. doi: 10.1001/jama.2010.748. [PMC free article] [PubMed] [Cross Ref]
25. Kalra AD, Fisher RS, Axelrod P. Decreased length of stay and cumulative hospitalized days despite increased patient admissions and readmissions in an area of urban poverty. J Gen Intern Med. 2010;25(9):930–935. doi: 10.1007/s11606-010-1370-5. [PMC free article] [PubMed] [Cross Ref]
26. Kohlnhofer BM, Tevis SE, Weber SM, Kennedy GD. Multiple complications and short length of stay are associated with postoperative readmissions. Am J Surg. 2014;207:449–456. doi: 10.1016/j.amjsurg.2013.10.022. [PMC free article] [PubMed] [Cross Ref]
27. Health Insurance Portability and Accountability Act of 1996, Pub. L. No. 104–191
28. Pelletier M. To text or not to text? The Joint Commission Blog. April 10, 2015. Available online at: http://www.jointcommission.org/the_view_from_the_joint_commission/to_text_or_not_to_text/. Accessed 15 Feb 2016.
29. Angrist JD, Pischke JS. Mostly Harmless Econometrics. Princeton University Press; 2009.
30. Bertrand M, Duflo E, Mullainathan S. How much should we trust differences-in-differences estimates? Q J Econ. 2004;199:249–275. doi: 10.1162/003355304772839588. [Cross Ref]
31. Hansen CB. Generalized least squares inference in panel and multilevel models with serial correlation and fixed effects. J Econ. 2007;140:670–694. doi: 10.1016/j.jeconom.2006.07.011. [Cross Ref]

Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine