|Home | About | Journals | Submit | Contact Us | Français|
The Displaced New Orleans Residents Pilot Study was designed to examine the current location, well-being, and plans of people who lived in the City of New Orleans when Hurricane Katrina struck on 29 August 2005. The study is based on a representative sample of pre-Katrina dwellings in the city. Respondents were administered a short paper-and-pencil interview by mail, by telephone, or in person. The pilot study was fielded in the fall of 2006, approximately one year after Hurricane Katrina. In this paper, we describe the motivation for the pilot study, outline its design, and describe the fieldwork results using a set of fieldwork outcome rates and multivariate logistic models. We end with a discussion of the lessons learned from the pilot study for future studies of the effects of Hurricane Katrina on the population of New Orleans. The results point to the challenges and opportunities of studying this unique population.
Hurricane Katrina struck New Orleans, Louisiana, on the morning of August 29, 2005. The magnitude of the displacement that resulted was immense: the city's entire population of 455,000 was forced to leave the city and resettle, which some did temporarily and others permanently. The toll from the hurricane was enormous and continues to unfold. Many people had family members or friends who died or were injured, had homes that were severely damaged or destroyed, lost their jobs or businesses, and had their lives severely disrupted. Although this event occurred two years ago, research on many major topics of scientific and policy interest continues to be hampered by a lack of appropriate data.
A critical need in assessing the effects of the Hurricane Katrina and in planning a recovery is to obtain representative data on the whereabouts, status, health, and well-being of displaced residents. In this paper, we describe the Displaced New Orleans Residents Pilot Study (DNORPS), which was designed to collect such information for people who lived in the City of New Orleans when Hurricane Katrina struck. The pilot study was based on a representative sample of pre-Katrina dwellings in the City of New Orleans. Fieldwork focused on tracking respondents wherever they currently resided, including back to New Orleans. Respondents were administered a short paper-and-pencil interview by mail, by telephone, or in person. DNORPS was fielded in the fall of 2006, approximately one year after Hurricane Katrina.
The goal of DNORPS was to assess the feasibility of the study design and thereby to lay the groundwork for launching a major longitudinal study of displaced New Orleans residents. There is a compelling need for a full-scale, representative, panel study to investigate the effects of this major disaster—in particular, to examine trends and disparities in a variety of measures of well-being over the medium and long term and to investigate how individual, family, and contextual characteristics shape the successful or unsuccessful adjustment to this extraordinarily disruptive experience. The DNORPS results provide valuable insights for ways to successfully undertake a full-scale study, but are also interesting in their own right. In particular, the methods and results for DNORPS may be relevant to studying population effects in the aftermath of other natural or man-made disasters, and the results from this study may thus be of wider interest.
The paper is organized as follows. In the next section, we describe New Orleans before Hurricane Katrina and in the aftermath of the storm and provide an overview of current research on the demographic effects of Hurricane Katrina. We then describe the design of the Displaced New Orleans Residents Pilot Study, including the sampling plan, questionnaire, and fieldwork operations. The results of the pilot study are presented next, and focus on descriptive results and a multivariate analysis of outcomes using logistic regression models. Three main findings emerged from DNORPS. First, the main challenge in achieving a high response rate is to find sampled respondents. Second, among respondents who we successfully located and were able to contact, a very high percentage participated in the study. Finally, few systematic patterns of non-response are apparent.
The size, composition, and characteristics of the New Orleans population at the time of Hurricane Katrina shaped the effects of the hurricane on the city, and also strongly influence our ability to conduct survey research on this population.
At the beginning of the current decade, New Orleans was the 31st largest city in the country (U.S. Census Bureau 2000). The population of Orleans Parish in 2005 was 454,863, accounting for about one-third of the total population of 1,338,000 in the metropolitan area (U.S. Census Bureau 2005).1 According to the 2004 American Community Survey, the majority (69%) of the population in New Orleans was black. Whites accounted for 28% of the city's population and Asians 2%.
The population of New Orleans was declining long before Hurricane Katrina, with the city steadily losing population in recent decades. The population decreased by 18% (109,000 residents) between 1970 and 2000 (Brookings Institution 2005) and fell by a further 6% (30,000 residents) from 2000 to 2005 (U.S. Census Bureau 2005). Declining population in New Orleans has meant that many people may have been considering leaving the city and, after being forced to evacuate, may decide not to return. Lessons on the demographic consequences of other natural disasters, such as Hurricane Andrew, suggest that these events accelerated demographic shifts that were already underway (Solecki 1999). Any survey of the pre-Katrina population of New Orleans will therefore have to locate these movers in their new residential locations rather than waiting for—or expecting—them to return.
Prior to Hurricane Katrina, New Orleans experienced high rates of poverty, which, as elsewhere, were associated with crime, shortfalls in the provision of basic services such as health care and education, illiteracy, substandard housing, and lack of opportunity. Around 23% of New Orleans residents lived below the poverty line in 2004, which was ten percentage points higher than the national average; the median family income was only two-thirds of the national average (U.S. Census Bureau 2005). Blacks in New Orleans had a poverty rate of 35%, which was the highest among large cities in the country (U.S. Census Bureau 2000) and almost all of the extreme-poverty neighborhoods in New Orleans were predominantly black by the time of Hurricane Katrina. Rates of college education for blacks in the city were only about one-fourth of that for whites; and only two-thirds of black adults had at least a high school degree compared to 89% of whites. The city's unemployment rate in 2004 was nearly 12%—twice the national rate (U.S. Census Bureau 2005). In addition, the New Orleans public school system was among the worst in the country (Hill and Hannaway 2006). Finally, black and low-income families in New Orleans had far lower rates of homeownership than whites and higher-income families. Only 41% of blacks in New Orleans owned their homes, compared to 56% of white households (U.S. Census Bureau, 2000).
High rates of poverty and disadvantage among the pre-Katrina population of New Orleans are likely to have affected the displacement experience and fieldwork efforts to track sampled cases that moved. For example, the choice of where to migrate may have been constrained in possibly unexpected ways among poor families. Although we would expect poorer New Orleanians to have stayed closer to the city to minimize travel costs and because they lacked cars, limited evidence (e.g., Tizon and Smith 2005) suggests that many may instead have been forced to move further away from the city. Poor families may also have faced challenges in reestablishing connections with displaced friends and neighbors because they could not afford the cost of traveling back to the city regularly or at all. Another issue is that concentrated poverty neighborhoods in New Orleans are unattractive places to which to return, particularly for many poor people who experienced better neighborhood environments, job opportunities, schools, and amenities in their new locations. A final issue is that low rates of homeownership place a significant barrier for residents to return to the city because of the difficulty of finding rental housing in New Orleans. Of course, homeowners who are poor themselves face considerable challenges in either renovating or selling their houses in New Orleans.
Hurricane Katrina struck New Orleans, Louisiana, on the morning of August 29, 2005.2 The city suffered moderate damage from the hurricane itself, but enormous damage from extensive flooding caused by breaks in several levees. The population of New Orleans was reduced to fewer than several thousand people by the end of the first week of September 2005.
Several weeks after Hurricane Katrina, the levee breaches were closed and the floodwaters were drained from the city. By the end of September 2005, former residents were allowed to begin returning the city. Initially, only residents of areas that were not flooded were allowed back. However, as water was pumped out of the flooded areas and basic services and infrastructure were restored, residents from more of the affected areas were allowed to return. Still, the depth of the floodwaters and the duration of flooding meant that most housing in areas flooded with water depths of two feet or more was substantially damaged and, for the most part, was not habitable (McCarthy et al. 2006). Many residents who had fled the city and expected to return shortly were displaced for substantial periods; many still are.
The first reliable post-Katrina estimate of the New Orleans population was based on the American Community Survey and placed the city's population on January 1, 2006 at 158,000 (U.S. Census Bureau 2006a)—about one-third of the pre-Katrina population. By mid-2006, the population of New Orleans was estimated to have reached 223,000 (U.S. Census Bureau, 2007), about half of its previous size. A recent population estimate produced by the Brookings Institution and the Greater New Orleans Community Data Center (2008), and based on U.S. Postal Service counts of households receiving mail, suggests that the city's population was about 70% of its pre-Katrina size—or approximately 320,000 people—in late 2007. The composition of returned residents is unlikely to match the pre-Katrina population because the decision to return will be influenced by factors such as job opportunities and the availability of public services that have distinct effects on different groups. For instance, few families with children have returned to New Orleans—at the end of the 2005–06 school year, less than 5,000 children were enrolled in the city's public schools, compared to about 66,000 when Katrina struck (Pane et al. 2006).
Little is known about the current location of displaced New Orleans residents. In the initial period following Hurricane Katrina, there were several useful data sources about where displaced residents from New Orleans were living. In particular, information on the current location of evacuees was available from change-of-address forms filed with the U.S. Postal Service and from registrations with FEMA for aid. Analyses of these data showed that nearly 15% of evacuees from New Orleans relocated to distant cities in the East Coast, Midwest, and West Coast (Tizon and Smith 2005). However, the main destinations for displaced residents were suburban New Orleans, Houston, Dallas, Atlanta, and Baton Rouge. Unfortunately, the usefulness of these data sources waned when the change-of-address forms expired and FEMA aid came to an end.
In the medium term, the well-being of displaced New Orleans residents is likely to be shaped by several main factors, which center on their ability to reestablish permanency in their lives. Key aspects of permanency and well-being include finding housing, establishing stable living arrangements, securing employment, obtaining child care and enrolling children in school, accessing needed health care and social services, and managing health and behavioral problems. Limited evidence on the status of displaced persons from New Orleans across these various domains points to a situation of substantial initial disruption—but with almost no information about whether this disruption is continuing. Preliminary results from analyses based on the Current Population Survey indicate strong negative employment effects among displaced residents: the estimated unemployment rate among Katrina evacuees was 15%, with unemployment rates that were almost five times as high for black evacuees than for white evacuees (Bureau of Labor Statistics 2006).
The economic consequences of Hurricane Katrina are likely to be enormous and lasting. For most families who own their home, equity in their property represents their largest wealth component. Uninsured property losses due to flooding are therefore potentially devastating. Although New Orleans had among the highest participation rates in the flood insurance program in the country (Swenson 2006), even among the insured the average policy coverage in New Orleans ($152,000) was only equal to the median house price. This means that, for many New Orleanians, flood insurance was unlikely to be sufficient to cover the household contents or the loss of use of a home. Nearly half of the housing units in New Orleans were rental units, according to the 2000 Census. Renters may not have faced the same financial losses as homeowners, but they were forced to find new housing. Many poor residents were tenants who paid extremely low rents—few of these people can afford to return to New Orleans because of the higher rents that now prevail for reconstructed or undamaged dwellings in the city.
Existing literature on natural disasters and their population effects has focused largely on immediate impacts, such as survival, evacuation, job losses, health care, school enrollment, and related topics. A particularly large number of studies have examined the mental health effects of disasters, using a variety of survey approaches (for an introduction and overview see Norris et al., 2006). Less is known about the intermediate- and longer-term effects on broader measures of health and social and economic well-being. Furthermore, in analyzing the demographic effects of other natural disasters—such as Hurricane Andrew, which struck Florida in 1992— researchers have fielded new population surveys, after concluding that virtually all other data sources have significant shortcomings (Smith 1996; Smith and McCarty 1996).
Although much is known about the characteristics of New Orleans residents prior to Hurricane Katrina, there has been very little systematic data collected on evacuees (Briggs 2006; National Academy of Sciences 2007). Existing studies of the demographic effects of Hurricane Katrina fall into two categories: first, early and generally small-scale studies of evacuees and, second, larger-scale studies, including on-going national studies. Aggregate data on the current location of people displaced by Katrina have also been made available by FEMA and the U.S. Postal Service, but neither of these agencies will release individual-level data.
The first studies of evacuees include a number of surveys of shelter populations and evacuee lists and of early returnees to New Orleans. For example, Gallup, CNN, and USA Today conducted a survey about one month after Hurricane Katrina of 1,510 randomly sampled people who registered with the American Red Cross for assistance. Elliott and Pais (2006) used these data to examine differences in experiences in the aftermath of Hurricane Katrina by race and by socioeconomic status, and found additive effects of each on evacuation timing, employment, and stress. Another study, conducted by the Washington Post, Kaiser Family Foundation, and the Harvard School of Public Health, focused on the evacuee population in Houston shelters two weeks after Hurricane Katrina. This study documented the disadvantaged background of evacuees and their suffering in the immediate aftermath of the storm—during the evacuation and at shelters (Brodie et al. 2006). A third study, conducted by researchers at the Columbia University School of Public Health, fielded a survey among children and families in FEMA-sponsored trailer parks, hotels, and housing units in Louisiana in February of 2006 (Abramson and Garfield 2006) and conducted a similar study on Mississippi's Gulf Coast in August 2006 (Abramson, Garfield, and Redlener 2007). Both surveys found children to be suffering from high rates of chronic health conditions, mental health problems, and poor access to health care. Finally, a series of rapid population surveys were conducted in New Orleans by the city's Emergency Operation Center, to provide up-to-date estimates of the city's population in the initial months after the storm (Stone, Grant, and Weaver 2006).
The early studies were conducted rapidly and under very demanding circumstances, but provided useful information on the immediate aftermath of Hurricane Katrina. However, all of these studies were cross-sectional and often not representative of the population they sought to capture due, for example, to high (and often unknown) non-response rates and shortcomings with the listings from which the samples were drawn. But the most significant shortcoming of these early studies was that they did not produce representative data on all of the affected population. For example, to this day, there has been no systematic data collected on the evacuation experience, subsequent resettlement, and social, behavioral, and economic outcomes of more affluent population groups that were not evacuated to shelters and did not seek assistance from the Red Cross.
The second set of Katrina studies, which are larger in scale and include several nationally-representative data surveys, overcome many of the shortcomings of the early small-scale studies. Among the studies that focus specifically on the effects of Hurricane Katrina are the Hurricane Katrina Community Advisory Group, the Mississippi Community Study, the Louisiana Health and Population Survey, and the Kaiser Post-Katrina Baseline Survey. National studies include the American Community Survey and the Current Population Survey.
The Hurricane Katrina Advisory Group (2006) is a panel study of 1,043 adults from Alabama, Mississippi, and Louisiana who were selected from multiple sampling frames. This study focuses on the psychological well-being of adults. Although this was the first large-scale study launched after Katrina, it suffers from a number of shortcomings for addressing topics of interest to social scientists, such as migration and broader measures of social and economic well-being. Among these shortcomings are a response rate of approximately 27%, no information on characteristics of non-respondents, a total of only 168 respondents in New Orleans, and no contextual information on respondents' pre-Katrina neighborhood of residence (Hurricane Katrina Community Advisory Group 2006).
The Mississippi Community Study is collecting data on respondents sampled from a listing of all addresses in coastal areas of the state. This study focuses largely on mental health outcomes, although it is designed to examine the effects of contextual factors. There are a number of significant parallels between the DNORPS and the Mississippi Community Study, and the two studies will provide complementary information on adjacent areas.
The 2006 Louisiana Population and Health Survey collected cross-sectional data for parishes in Louisiana that were directly affected by Hurricane Katrina, including Orleans Parish. Basic demographic information was obtained for all respondents, including some broader measures of health status and social and economic well-being. The 2006 Kaiser Post-Katrina Baseline Survey (Kaiser Family Foundation 2007) is a cross-sectional survey that interviewed 1,504 randomly selected adults currently living in the New Orleans metropolitan area, including 901 in Orleans Parish, and collected information on a number of dimensions of well-being, including economic status, physical and mental health, and quality of life. These two studies provide complementary information to DNORPS—all three studies provide information on people who lived in New Orleans prior to Katrina and have subsequently returned to the city, although DNORPS alone collects information on previous New Orleans residents who have not returned while the Louisiana Population and Health Survey and the Kaiser Post-Katrina Baseline Survey collect data on new migrants to the city.
Finally, the American Community Survey (ACS) and the Current Population Survey (CPS) provide valuable data on the entire region affected by Katrina (U.S. Census Bureau 2006b; Bureau of Labor Statistics 2006). Both of these national surveys are fielded by the U.S. Census Bureau with overall response rates of 92% for the CPS and 98% for the ACS. Sample sizes in the CPS of approximately 700 respondents from New Orleans are too small to provide detailed estimates for the city, but sample sizes in the ACS of approximately 6,600 respondents from New Orleans are sufficiently large to identify and analyze displaced residents as well as the current population of the city. Both surveys will provide only a brief window into the whereabouts and status of displaced New Orleans residents: they each ask just a single migration question about place of residence one year previously and the CPS's special questions on Katrina migrants ended in October 2006 (Cahoon et al. 2006).
To begin to address the major research and policy questions that will likely remain unanswered by studies that have been conducted or are underway, we designed the Displaced New Orleans Residents Pilot Study. This pilot study was designed to provide key information and insights for launching a major panel study of displaced New Orleans residents. The results are also likely to be useful for surveys following major man-made or natural disasters in other settings. In this section, we describe DNORPS, providing an overview of the study design, questionnaire content, and fieldwork operations.
The design of DNORPS is based on a stratified, area-based probability sample of pre-Katrina dwellings in the City of New Orleans that provides representative information on pre-hurricane residents of New Orleans. A total of 344 cases were sampled for the study, with the sample size determined by the budget. These cases were selected at random from two separate lists of “named” and “unnamed” housing units using a systematic sampling algorithm that incorporated explicit stratification by flood depth.
Area-based sampling meant that dwelling units were selected at random from the entire geographic area of the study in a single-stage—in other words, dwellings were the primary sampling unit and there was no geographic clustering of cases. This design provides greater statistical power for a given sample size because design effects are minimized. The disadvantage of this design is higher costs per case for any in-person interactions due to the need for traveling greater distances compared to a clustered sample, and the inability to use statistical methods to identify the effects of unobserved cluster-level factors.
Knowing the addresses of all sampled cases allows us to geocode the sample and to attach area-based summaries at the block, block-group, tract, and zip-code level from the 2000 U.S. Census and from other data sources. This has two major benefits. First, it allows us to better understand non-response patterns. In particular, we can assign demographic and socioeconomic characteristics at a very fine geographic level (block and block-group) to all sampled cases and can then compare respondents and non-respondents along a number of key dimensions. Second, it allows us to study the effects of neighborhood contextual factors on outcomes and behaviors of interest—such as the effects of neighborhood-level recovery on the probability that a family decides to relocate back to New Orleans.
New Orleans was divided into three strata based on flood depth for sampling purposes. Flooding in New Orleans in the aftermath of Hurricane Katrina was most severe in the low-lying areas near Lake Pontchartrain and in the mid-city area, while neighborhoods situated on the natural levees on both sides of the Mississippi River had no flooding. Areas along several natural ridges that cross the city and along the shore of Lake Pontchartrain also escaped major flooding. Flood depth was directly tied to housing damage (McCarthy et al. 2006), and thus represents a crucial factor shaping residents' post-Katrina migration decisions (such as whether to return to the city) and financial and other dimensions of economic and social well-being.
Information on the depth of the floodwater was derived from remote sensing data from FEMA on the extent of flooding in New Orleans on 31 August 2005. The floodwater depth was calculated using a geographic information system that included detailed topographic information for the city, and was coded at the Census block level. The three flood depth strata, shown in Figure 1, were as follows:
A further goal was to achieve an even distribution of the sample within each stratum by three potentially important factors: geographic location (based on Census tract), racial composition (using the percent of the population at the block level that was black), and home owners versus renters (based on the block-level proportion of dwellings that were owner-occupied). To achieve an even distribution of cases by these factors we used an implicit stratification procedure, which was implemented by sorting cases according to these factors prior to applying a systematic sampling algorithm (see, for example, Levy and Lemeshow 1999).
The sample was drawn from a listing of all dwellings in the city of New Orleans created by Marketing Systems Group, Inc. (MSG). MSG develops and maintains an enhanced listing of all households in the U.S. based on records from the U.S. Postal Service and other sources. The specific listing that was used to draw the sample is an archived version from August 2005. We confirmed that the MSG listing file was comprehensive and complete by comparing it to data from the 2000 Census. MSG matches its listing to telephone directories, driver's license records, real estate transactions, and other sources to obtain, where possible, the name of the household head and a telephone number for each dwelling. For New Orleans, approximately half of all listed dwellings were “named.”
The final stage of stratification and sampling was to oversample dwellings from the named sample, under the assumption that these cases would be easier to locate. For similar reasons, DNORPS oversampled dwellings in the no-flood and low-flood strata, and undersampled dwellings in the high-flooding stratum. The oversampling rate was modest, however, and sample weights can be used to generate estimates for the entire population. Table 1 describes the final DNORPS sample.
DNORPS respondents completed a brief questionnaire that covered topics of central interest to the study. The questionnaire began with a confirmation of eligibility by verifying that the respondent did indeed reside at the sampled dwelling in New Orleans at the time of Hurricane Katrina. Next, a complete roster was obtained of all pre-Katrina household residents. Information was collected on these individuals' relationships with each other and their background characteristics, such as age, sex, race and ethnicity, place of birth, and education. The questionnaire asked about the evacuation and resettlement experience, including when each person left New Orleans, the initial evacuation destination, the evacuation location where they spent the most time, the current place of residence, and, if they currently resided in New Orleans, when they had moved back to the city. Information was also collected on the likelihood that each person will be living in New Orleans in one year's time. The questionnaire respondent was asked for further details about his or her own evacuation experience, including the type of accommodation they lived in for the first month after the storm, how long they lived in temporary housing, and the specific organizations to which they applied for aid.
The questionnaire asked about each person's pre-Katrina and current employment, marital, and health status. Other measures of well-being for each person include whether they had consulted a health professional for mental health problems and whether their lives were better, the same, or worse since the hurricane.
Information collected on housing characteristics includes the type of dwelling, whether the unit was owned or rented, and the extent of damage from Hurricane Katrina and the subsequent flooding. Respondents were also asked to report their household's total monthly income after taxes in the month preceding Hurricane Katrina.
Finally, the survey respondent was administered the K6 scale of non-specific psychological distress (Kessler et al. 2002; Kessler et al. 2003) which screens for anxiety and mood disorders in the previous 30 days. The K6 is widely used in the U.S. as a screener for mental illness, and has been used previously to examine psychological effects of Hurricane Katrina (Kessler et al. 2006).
It took respondents approximately 15 minutes, on average, to complete the ten-page questionnaire. Respondents received a check for $20.00 as an incentive for participating in the study.
Fieldwork for the Displaced New Orleans Residents Pilot Study began in mid-September 2006 and ended in November 2006. The goal was to complete interviews with an adult respondent in each sampled household through one of the following three modes: first, as a mail survey; next, through a telephone interview for cases that did not complete the mail survey; and, finally, through an in-person interview for respondents who were unable or unwilling to complete the survey by mail or telephone.
Fieldwork began by mailing a cover letter and questionnaire to all 344 cases, with an address correction requested. Problems with the postal system in New Orleans meant that, although we received 36 completed mail surveys, we received only seven address correction cards and 126 mailings returned as undeliverable. At the same time that questionnaires were mailed out, we began batch tracing of respondents using electronic databases such as Telematch, TransUnion, Death Audit Search, National Change of Address database, and Accurint, in order to obtain names (for the unnamed sample) and up-to-date telephone numbers and addresses. Names were obtained for 270 cases (78% of total) and telephone numbers were obtained for 230 of these cases, leaving 114 cases to be traced interactively by telephone or in person. Cases with a telephone number were sent to the RTI call center for an attempt at a telephone interview. Cases without a telephone number, or which could not be reached by telephone, were sent to one of four field interviewers located in New Orleans for on-the-ground tracing and in-person interviewing.
As fieldwork was ending, we decided to subsample pending cases and send only a random selection of cases to the field for in-person tracing in order to use remaining funds and time most effectively. A total of 29 cases were selected for this final fieldwork stage from the 154 then-pending cases. There were a total of 7 completes from these subsampled cases.
Fieldwork ended in mid-November 2006, when the budget was exhausted. At that time, there was a total of 113 cases for which tracing activities were still pending because we had not been able to obtain a confirmed name, telephone number, or new address (and had not determined the dwelling to be vacant at the time Hurricane Katrina struck). We had a further 45 cases for which we had located a respondent but had not successfully contacted him or her.
Our analysis of the fieldwork results for DNORPS is based, first, on describing outcome rates for the study and, second, on using multivariate logistic regression models to examine specific demographic, social, and economic factors at the neighborhood level that are associated with the outcomes for individual cases. The goal for this analysis is to document and explain the fieldwork results in order to understand the strengths and shortcomings of the pilot and to inform future studies of displaced New Orleans residents.
The detailed fieldwork results for DNORPS are summarized in Table 2. The upper part of the table shows the disposition of all cases released to the field, and the panel at the bottom shows several key outcome rates based on guidelines from the American Association for Public Opinion Research (AAPOR 2006). The fieldwork results are presented for each of the three strata separately as well as for the entire sample.
The top panel in Table 2 shows that a total of 147 interviews were completed for DNORPS. About one-quarter of interviews (36 cases) were completed by mail, while almost three-quarters (107 cases) were completed by telephone. For a handful of cases, questionnaires were completed by both mail and telephone (these cases are counted among those completed by telephone). Only four cases (less than three percent of the total) were completed through in-person interviews. This reflects the fact that on-the-ground fieldwork efforts were limited, and that a number of cases that were located in the field were transferred to the call center for completion as a telephone interview.
The next panel in Table 2 shows that out of the 344 cases released to the field, a total of 19 were determined to be ineligible. Reasons for ineligibility are that there were no surviving household members, the dwelling was vacant at the time of Hurricane Katrina, or that the case could not be classified as a pre-Katrina household from New Orleans.
The number of cases identified as vacant at the time of Hurricane Katrina is substantially smaller than we would expect based on data from other sources. In particular, results from the 2000 Census indicate that 12% of dwellings in New Orleans were vacant—and it is likely that vacancy rates rose by 2005 along with the modest decline in the city's population. The 2000 vacancy rates imply that approximately 344 × .12 = 41 sampled cases in DNORPS should have been vacant. However, only seven dwellings were identified as vacant at the time of Hurricane Katrina. The small number of vacant cases reflects the difficulty of determining the past occupancy status of a dwelling using fieldwork methods that centered on tracking cases by mail and telephone. Nevertheless, we were able to classify at least some cases as being vacant at the time of Hurricane Katrina, largely through on-the-ground fieldwork efforts. This result shows the value of having interviewers in the field as part of any larger-scale survey of this population.
The third panel in Table 2 shows cases that were not interviewed. The main reason for not completing an interview is that the case could not be located. We were unable to locate 113 cases, which is about one-third of all eligible cases—although note that some of these may in fact be ineligible, a fact that can only be determined by locating and questioning a respondent or informant. For virtually all of these cases, there remained leads that could be pursued by the fieldwork staff. In other words, few of these cases had reached a dead-end and hence, with additional time and resources, further tracking activities would be warranted.
The remaining 67 non-interviewed cases comprised one-fifth of eligible cases. These cases had been located by the fieldwork staff and were approximately evenly divided among three groups—refusals, unable-to-contact cases, and cases on which work had stopped. Unable-to-contact cases were finalized cases with an identified respondent and a current telephone number or address, but which we were unable to reach after numerous attempts. Additional attempts to contact these cases were judged to be unwarranted. Cases on which work had stopped include those that were not selected for the final fieldwork stage and those with an address outside of New Orleans but no telephone number (and which hence required an in-person visit).
The bottom panel of Table 2 presents outcome rates for the pilot study that were calculated based on guidelines established by AAPOR (2006). Overall, in DNORPS we located 65% of sampled cases. In the no-flood stratum, 76% of cases were located, but this dropped to 62% in the low-flood stratum and to 58% in the high-flood stratum. It was more difficult to locate respondents in areas that had flooded because a higher fraction of these respondents no longer resided in the sampled dwellings. Differences in rates of locating cases across strata are statistically significant, based on a Rao-Scott (1984) sample design-based F-test. Figure 2 shows the distribution of cases—which reflects the distribution of housing units and population across the city—and distinguishes cases that were located from those that were not located. No clear spatial patterns are visible in the map, except for the higher likelihood of locating a case in the non-flooded stratum.
The overall contact rate for DNORPS, calculated as the percent of eligible located cases that were successfully contacted, was 79%. There was little variation across strata in contact rates, which varied from 77% in the no-flood stratum, to 81% in the low-flood stratum, to 79% in the high-flood stratum. Differences in the contact rates across strata are not statistically significant.
The DNORPS cooperation rate, calculated as the percent of contacted cases that were interviewed, was 88% overall. This high cooperation rate corresponds directly to a low refusal rate of 12% overall, which was achieved even in the absence of extensive refusal-conversion efforts. Cooperation rates did not vary greatly by stratum, given the 12% refusal rate in the no-flood stratum, 8% rate in the low-flood stratum, and 15% rate in the high-flood stratum. Differences across strata in the cooperation rates were not statistically significant.
At the bottom of Table 2 we present two separate response rates—an observed response rate and an adjusted response rate. In both cases, these rates reflect the percentage of eligible cases that were successfully interviewed. The adjusted rate controls for the final fieldwork stage in which a subsample of cases were selected for additional tracking and interview attempts, while work stopped on the remaining cases. The adjusted response rate varies substantially by strata, from a high of 63% in the no-flood stratum to a low of 41% in the high-flood stratum. The overall observed response rate for the study was 45%, with the response rate in the no-flood stratum exceeding this rate by six percentage points (51%) and the response rate in the high-flood stratum falling six percentage points below this rate (39%). However, differences in the observed response rates across strata were not statistically significant.
The area-based sample design of the DNORPS allows us to conduct a multivariate logistic regression analysis of fieldwork outcomes, with covariates based on area-characteristics at the block and block-group level from the 2000 Census. Covariates are available for all cases—i.e., both responding and non-responding cases in the sample—and hence allow us to investigate whether there are systematic patterns for the various outcome rates according to demographic, social, and economic characteristics of the local-area of the dwelling.
The dependent variables for the regression analyses are fieldwork outcomes coded at the individual level that indicate whether an eligible case was located, a located case was contacted, or a contacted case cooperated (i.e., was interviewed). In addition, we examine the individual-level analog of the overall response rate by modeling whether or not an eligible case was successfully interviewed. Note that the probability of responding reflects the product of the first three outcomes (located, contacted, and cooperated) all occurring.
Covariates in the model include sample-design variables (flood-depth stratum and whether the case was named or unnamed), flood depth, and Census-based areal measures of demographic characteristics (median age and percent of population black), socioeconomic status (percent of employed adults with executive or professional occupations and median family income), residential stability (percent of dwellings classified as non-family and a residential stability factor score),3 and percent of dwellings vacant. All of the covariates were based on block or block-group information. Table 3 presents summary statistics for the model covariates.
We estimated logistic regression models using statistical techniques designed for analyzing complex survey data that adjust for stratification of the sample, with robust standard errors calculated using the jackknife method. The results, presented in Table 4, show the exponentiated parameter estimates and standard errors, along with indicators of statistical significance. Also shown are the goodness-of-fit test statistics for each model and the number of observations.
Only the first model, which estimates the probability of locating a respondent for an eligible case in the sample, fits the data according to the model's goodness-of-fit F-statistic. In other words, for the first model alone we find systematic differences in the outcome based on the covariates examined. Although some of the individual covariates are statistically significant in the other models, we cannot reject the hypothesis that the estimated parameters are jointly equal to one (i.e., have no effect on the outcome). There are two factors which drive this result. First, it reflects the modest sample size of the pilot, which leads to the standard errors for the estimated parameters being moderately large. Second, it points to the true absence of systematic patterns of nonresponse in these various fieldwork outcomes and to the genuinely small magnitudes of the estimated parameter effects.
For the model describing the probability of locating a case, there are several results worth noting. Although there are substantially lower rates of locating cases in the low-flood and high flood strata, compared to the no-flood stratum, these effects are not jointly statistically significant. Rather, the only statistically significant difference is between the no-flood and low-flood strata—but only at the .10 significance level. There are no statistically significant differences in locating named versus unnamed cases, which is due to an estimated parameter very close to one (rather than a large standard error) and points to the effectiveness of finding contact information for the unnamed sampled. Cases classified as “older” (based on the median age of residents in the New Orleans block of the dwelling) were located at higher rates. This may be because households in neighborhoods with older residents were more established and had stronger ties to the neighbors and to the local area. Cases in tracts with a higher fraction of non-family households had lower rates of being located. This reflects the greater likelihood of respondents living alone or with roommates and correspondingly higher transience levels and weaker ties to the dwelling, the neighborhood, and the city. Finally, note that none of the covariates describing race or socioeconomic status were statistically significant.
There were no statistically significant effects for any parameters in the model describing whether or not a located case was contacted. For the next model, describing whether or not a contacted case completes an interview, the effects of age and percent of population that is black are marginally significant (at the .10 significance level). Older age is associated with lower probability of cooperating while blacks appear to have higher cooperation rates. Finally, for the non-response model, the only statistically significant result is a finding of a lower probability of response in the high-flood stratum. Looking at this covariate across the models reveals that cases in the high-flood stratum have worse outcomes across the board; however, only when compounded across the outcome stages does this negative effect on the survey outcome of being in the high-flood stratum become statistically significant.
The goal of Displaced New Orleans Residents Pilot Study was to establish the feasibility of identifying a representative sample of pre-Katrina residents of New Orleans and to determine the likely success of efforts to track and interview this sample.
With regard to the first study goal, the pilot study demonstrated the feasibility of the sample design and highlighted a number of its features. These features include an area-based sample design that supports analyses of the effects of local area-based measures of demographic, socioeconomic, and housing characteristics on fieldwork results as well as substantive outcomes. With regard to the second study goal, the DNORPS results showed that sampled respondents once contacted could be successfully interviewed with cooperation rates approaching 90%, but that the main challenge was in locating respondents. Also, there were few systematic differences in fieldwork outcomes, by demographic, socioeconomic, and housing characteristics, across any of the fieldwork stages—with the exception of locating sampled cases.
The main challenge in improving the fieldwork results and in launching a full-scale study of this population is to locate a high fraction of sampled cases. Although the rate of successfully locating cases for DNORPS of 65% is reasonably high given the study constraints, it could be improved. Successfully locating a higher percentage of cases will involve two separate, but related, activities. One activity is to establish which cases are ineligible and, in particular, which dwellings were vacant at the time of Hurricane Katrina. Additional tracking and fieldwork activities are likely to be fruitful for determining the vacancy status at the time of Hurricane Katrina. For instance, these activities might include identifying and contacting public housing administrators, landlords, and rental property managers and asking them about occupancy status of sampled dwellings at the time of Hurricane Katrina. The second activity that will raise the proportion of eligible cases that are found is to conduct additional tracking activities. Fieldwork operations for the pilot study ended because of budget and time constraints rather than because all leads had been exhausted.
Overall, the results of the pilot provide a useful foundation for launching a larger-scale panel survey of displaced New Orleans to study social and economic well-being of this population over the coming years. The findings may also be relevant to studying the effects of man-made or natural disasters in other settings. Longitudinal studies are useful for providing an on-going assessment of the population effects of a disaster. They allow researchers to determine for whom the disaster is an experience from which they will never recover—and for whom ongoing support is essential—and to identify individuals for whom the disaster may have been a life-altering experience for the better.
I gratefully acknowledges the contributions of many colleagues in designing, implementing, and analyzing the Displaced New Orleans Residents Pilot Study, including Megan Beckett, Sandro Galea, Katherine Hall, Amelia Haviland, Anne Kenyon, Karol Krotki, Nicole Lurie, Adrian Overton, Kavita Patel, Christine Peterson, Michael Rendall, Ben Springgate, Greg Stone, Susan Twiddy, Joseph McMichael, Lily Trofimovich, Mark VanLandingham, Clayton Williams, and Julie Zissimopolous.
1The New Orleans metropolitan area comprises seven parishes, of which Orleans Parish had the largest pre-Katrina population. Orleans Parish boundaries correspond precisely with the incorporated limits of the City of New Orleans, and we use these two terms interchangeably.
2For more background information on Hurricane Katrina and its effects on New Orleans, see Comfort (2006).
3The residential stability factor score was based on the percentage of dwellings in multiple-unit structures, percent of dwellings that were owner-occupied, and percent of residents who lived in the same dwelling five years previously.