|Home | About | Journals | Submit | Contact Us | Français|
Background: The large-scale representative population surveys conducted by Germany’s Robert Koch Institute (RKI) contain questions pertaining to health and its determinants as well as the prevalence and frequency of outpatient services utilization. The same holds for the Socioeconomic Panel (SOEP, Sozio-ökonomisches Panel) and the Bertelsmann Healthcare Monitor (Gesundheitsmonitor) surveys. The purpose of this study is to examine the comparability of the instruments used in these surveys and their results.
Methods: The questions on outpatient care utilization examined in this study were taken from the public use files of the East-West Health Survey (Ost-West Survey; OW1991), the 1998 Federal National Health Survey (Bundesgesundheitssurvey; BGS1998), the 2003 Telephone Health Survey (TEL2003), and the 2009 German Health Update (Gesundheit in Deutschland aktuell GEDA2009). The study also used data from the 26 waves of the SOEP (1984–2009) and the 16 waves of the Bertelsmann Healthcare Monitor (2001–2009) studies.
Results: In the OW1991 and the BGS1998, questions on outpatient services utilization differ by the types of physicians inquired about. The four-week prevalence of contact with general practitioneers (GP) was 29% in the OW1991; the twelve-month prevalence in the BGS1998 was 69%. The OW1991 and the BGS1998 also surveyed participants on the number of physician contacts made during those reference periods (average number of contacts: 1.8 over the previous four weeks (OW1991) and 4.9 over the previous 12 months (BGS1998)). The TEL2003 inquires into the three-month prevalence of contact with private practice physicians in general (63%) as well as the number of contacts with primary care physicians over the previous twelve months (88% with at least one contact, average number of contacts: 4.6, range: 1–92). In the GEDA2009 survey, 88% of participants reported having contacted a physician at least once over the previous twelve months and an average of 6.1 contacts with all physicians working under contract with the German statutory health insurance (SHI) funds. The 2009 SOEP survey revealed a 28% three-month prevalence of contact with all types of physicians and an average of 3.6 contacts (among participants who had made at least one contact during this period). According to the Bertelsmann Health Monitor, the twelve-month prevalence of contact with GPs was 82%, with the average number of contacts being 5.0. The Bertelsmann Health Monitor also surveys participants on contacts made with four other types of physicians; the OW1991 and the BGS1998 ask about contacts made with over ten different types of physicians when examining the frequency of services use.
Conclusions: Not only do the target groups of the RKI surveys, the SOEP and the Bertelsmann Health Monitor differ; their questions on outpatient care utilization also differ in terms of examined reference period and types of physicians contacted by survey participants, question wording including clarifications (e.g., asking the participant to also consider contacts not made “in person” with physicians when answering a question), and response categories. Therefore, unlike the results of the surveys’ questions on inpatient care, the results of questions on the use of outpatient care services are not easily comparable, even those regarding contact with primary care physicians and GPs. The results of secondary analyses of German SHI claims data could be used to confirm the external validity of the surveys’ results.
Over the last two decades, the former German Federal Health Office and later the Robert Koch Institute (RKI) carried out numerous large-scale representative population surveys aimed at gathering information on health and its determinants as well as on the prevalence and frequency of out- and inpatient services utilization. Germany’s Socioeconomic Panel (SOEP) and Bertelsmann Healthcare Monitor also inquire the utilization of medical services. This study examines the comparability of the instruments employed in the surveys’ questionnaires to assess the utilization of medical services as well as the comparability of the surveys’ results.
In addition, since studies assessing disease prevalences or determinants of health and illness often reference to the RKI surveys and both the SOEP and Bertelsmann Healthcare Monitor surveys are frequently used as sources of reference in the social and health sciences, it is important to know the extent to which the results of these surveys may be considered as appropriate reference data.
To answer these questions, this study draws on the public use files of four representative population surveys conducted between 1991 and 2009 (for further information and download see http://www.rki.de/), data from the 16 waves of the Bertelsmann Healthcare Monitor (http://www.bertelsmann-stiftung.de) conducted between 2001 and 2009, and data from the 26 waves of the SOEP (http://www.diw.de) conducted between 1984 and 2009.
We first describe the survey characteristics and question sets. After that we have a detailed look on the questionnaires and specific questions to measure frequency of outpatient and inpatient care as well as other kind of utilization (e.g., prescription). At least to compare the results of the larger-scales population surveys we quantify frequencies of medical use depending on length of reference period, age and sex.
Already in the 1980s, two waves of the so-called “National Health Survey” were conducted with representative samples of the population of former West Germany. As part of the survey, participants underwent a standardized medical examination and were interviewed extensively on health-related topics. A third wave of the survey was conducted in 1990/91 in the former West German states and in 1991/92 in the former East German states using essentially the same methods (OW1991). Data from these surveys were meant to provide information on the prevalence of certain diseases, symptoms, risk factors, disorders, and health-related behaviors and living conditions in the German population. The population of the National Health Surveys included all Germans in former West Germany aged 25–69 and all Germans in former East Germany aged 18–79. Samples for the surveys were randomly selected with equal probability using a two-stage multi-stratified sampling procedure. The net samples of the National Health Surveys comprised 5,255 West Germans and 2,211 East Germans, with response rates of approximately 70% .
The 1998 Federal Health Survey (BGS1998) is a health-related survey and study of a representative sample of the resident population of Germany between the ages of 18 and 79. The primary goal of the survey was to obtain data for the country’s National Health Reporting System and to be able to draw comparisons with other European countries. Results from the 1998 survey would also be compared to results from previous surveys in order to identify trends in the distribution of diseases and risk factors within the German population. Questions asked as part of the survey pertained to subjective and objective health, the utilization of medical services, and a number of potential health determinants. The survey sample was obtained using a multi-stratified random sampling design. A total of 7,124 people completed the survey questionnaire and underwent a medical examination (response rate: 61%) , .
In continuation of the BGS1998, the Robert Koch Institute conducted the first nationwide representative Telephone Health Survey in Germany with a focus on chronic diseases (Tel2003). A total of 8,362 people participated in the survey (response rate: 52%). As in the BGS1998, participants were between the ages of 18 and 79. The multi-stage sampling method and the questions used in the survey were based on those used in the BGS1998 .
The German Health Update (GEDA2009) is a representative survey of the adult German-speaking population living in private households in Germany and having a landline phone. A total of 21,262 interviews were completed as part of the survey (response rate: 51%). The GEDA survey is regularly repeated as part of the RKI’s health monitoring program in order to observe trends in disease occurrence and in health and risk behaviors. It is also meant to provide the National Health Reporting System and health policymakers with timely information regarding health trends in the German population or subgroups thereof. The survey is a follow-up of the 2003 Telephone Health Survey and was conducted using the same methods .
Since 2001, the Bertelsmann Health Monitor has been conducted at regular intervals to survey insured individuals and private practice physicians on outpatient care. The main focus of the surveys conducted with the insured is to assess utilization of services, patient-centeredness in care delivery, patients’ expectations of the health care system, and patient satisfaction. Whereas the survey of the insured is conducted twice a year by mail, the physician survey is conducted once a year by phone. The sample used for the survey of the insured is drawn by the research company NFO Infratest Gesundheitsforschung from a panel of households that have agreed to participate in surveys. The access panel contains nearly 70,000 households with more than 160,000 household members . The response rates lay around 70 percent in all waves (see methodological reports on http://www.bertelsmann-stiftung.de/).
The Socioeconomic Panel (SOEP) is the largest and longest running (since 1984) multidisciplinary longitudinal study in Germany. The SOEP, commissioned by the German Institute for Economic Research (DIW) in Berlin, is carried out by the social research institute TNS Infratest Sozialforschung with more than 20,000 people and 1,000 households each year . Data collected through the SOEP provide information on income, employment status, education level and health. Since the survey measures the same people each year, it makes it possible to conduct particularly thorough analyses of both long-term societal trends and the life courses of specific groups. Although questions on health are not the main focus of the survey, the survey does regularly ask questions regarding health and the utilization of medical services. The households’ responses were more than 80 percent .
Table 1 (Tab. 1) compares the characteristics and question sets of the surveys examined in this study. Looking at the table, similarities are apparent not only in the sets of questions on services utilization, but also in the individual questions regarding potential determinants of utilization. There are, however, noticeable differences in the objectives of the surveys in terms of study population and design. For example, whereas the RKI surveys have a one-time cross-sectional design, the SOEP and Bertelsmann Health Monitor involve repeated panel surveys. In addition, the SOEP contains noticeably fewer questions on subjective health and on disabilities and limitations in daily living activities.
Looking at Table 2 (Tab. 2), clear differences can be seen in the surveys’ questions on the utilization of outpatient services. Only three out of six surveys ask participants when they last contacted a physician, and only the BGS1998 and the Health Monitor surveys ask about the reason for the last contact or for more details about the contact. In surveys containing questions on contact with a doctor during a specific time period, these periods range from four weeks to twelve months. There are also major differences in the questions regarding contacts with specific types of physicians. Whereas some of the surveys do not ask about contacts with specific physician groups at all, the types of physicians inquired about in the other surveys vary. Also, only a few surveys contain questions on the use of screening and prevention services.
The questions on outpatient utilization also differ in their wording with possible implications for the answer, e.g., “What kind of specialist did you demand and how often in the last 12 months?” (BGS98) or “How often do you visit a primary physician in the last 12 months?” (GEDA09) or “To what kind of specialist did you have contact in the last 12 months and how often?” It is unclear for the participants whether only direct contacts were meant or also practice consultations, e.g., for a drug prescription.
Greater similarities can be found in the surveys’ questions on the prevalence of hospitalization. All six studies asked whether there had been an inpatient stay during the previous twelve months. The only differences here were the way the surveys inquired into the frequency of inpatient care (i.e., the number of nights spent in a hospital or number of hospitalizations over the previous twelve months; see Table 3 (Tab. 3)). Questions regarding the utilization of other services (rehabilitation services, non-physician services, the use of prescribed or over-the-counter medications) will not be discussed here as they are not standard questions in the surveys analyzed here.
The East-West Survey (OW1991), the Federal Health Survey (BGS1998) and GEDA2009 survey all ask participants when the last time was that they had contacted a physician. The results of these surveys regarding whether a physician had been contacted within the previous four weeks differed by approximately 10 percentage points, with participant percentages varying from approximately 40 to 50 percent. Extending the reference period to 3 months, the same percentage point difference can be observed (63–72%). Age and sex-specific patterns of outpatient care utilization are also consistent; again, the surveys’ results differ by approximately 10 percentage points. Unlike the other surveys, the Bertelsmann Health Monitor does not provide any data on age or sex-specific use of outpatient care. It only inquires into the number of outpatient contacts made with different physicians over the previous twelve months (Table 4 (Tab. 4)).
The four-week prevalence of contact with GPs was 29% in the OW1991. According to the BGS1998, the twelve-month prevalence was 71%. Whereas the OW1991 reported an average of 1.8 contacts with GPs or primary care physicians over the previous 4 weeks, the BGS98 and the Bertelsmann Health Monitor each reported an average of 4.9 contacts over the previous 12 months and the 2003 Telephone Health Survey reported 4.0 contacts, though only for survey participants indicating that they have a regular primary care physician. There is marked variation in the reported number of physician contacts made by survey participants. The GEDA2009 and SOEP surveys only asked about the number of contacts made with physicians in general. The results were an average of 6.1 physician contacts made over a twelve-month period (GEDA2009) and an average of 3.6 contacts made over a three-month period (SOEP). While the Bertelsmann Health Monitor distinguishes between contacts made with four different types of physicians, the OW1991 and the BGS1998 ask about contacts made with over ten different types of physicians when examining the frequency of services use (Table 5 (Tab. 5)).
Even for identical questions on whether participants had stayed in a hospital over the previous twelve months, there were varying results, which ranged from around 9 to 15%. This variation can only partially be attributed to the different surveys populations since there are still noticeable differences even after stratification by age group. The patterns of age- and sex-specific inpatient care use are, however, the same in all surveys (Table 6 (Tab. 6)).
Due to inconsistency in the questions asked in the RKI surveys, these surveys are less suitable for showing changes utilization of outpatient care over time. The Bertelsmann Health Monitor and the Socioeconomic Panel, however, study the same questions over a longer time period. Nearly 95% of participants in the Bertelsmann Health Monitor reported having contacted their primary care physician at least once over the previous year, with an average total of 5 contacts during that period of time (Figure 1 (Fig. 1)). The proportion of participants reporting at least one contact in the last 12 months is nearly constant over the 16 waves, also the mean number of contacts to their primary care physician and the interquartil range of this item. In every wave about ten percent of the participants reported more than ten contacts with maxima between 50 and 100 contacts.
According to the results of the SOEP regarding inpatient hospital stays over the previous 12 months, the percentage of survey participants with at least one hospital stay remained a steady 11–12%. There was also a steady decline in the average number of overnight hospital stays among the group of participants who reported at least one hospital stay (Figure 2 (Fig. 2)). As with outpatient contacts, a small proportion of participants (≤10 percent) reported more than 30 night spent in hospital in the last 12 months, probably due to severe acute diseases or chronic and/or psychiatric illness.
This study examined the large-scale representative population and panel surveys whose data are readily accessible to the scientific community and whose results are therefore often used when estimating disease prevalences or assessing health care needs. The aim of the study was to compare the instruments used in these surveys and to explore differences in their results, either by providing a basic overview of these differences or by highlighting differences in the age- and sex-specific results. As this was a comparative study, it did not aim to determine the “actual” level or frequency of health services utilization nor the determinants of services use. (For such information, please refer to the original publications of these and other surveys , , , ). The scope of this study also did not include examining health services utilization by specific subgroups of the population, e.g., , ,  or by specific patient groups, e.g., , nor did it include multivariate modeling of health services use based on, for example, Andersen’s Behavioral Model as its theoretical background .
Overall, only a certain degree of comparability was found between the surveys. There are even substantial differences between the questions used in the different RKI surveys, which limits the comparability of their results. Not only do the target groups of the RKI surveys, the SOEP and the Bertelsmann Health Monitor differ; their questions on outpatient care utilization also differ in terms of examined reference periods and types of physicians contacted by survey participants, question wording including clarifications (e.g., asking the participant to also consider contacts not made “in person” with physicians when answering a question), and response categories. The effects of the different means of recruiting study participants (representative samples used in the RKI surveys vs. panel surveys used in the SOEP and Bertelsmann Health Monitor studies) also have to be taken into consideration. Therefore, unlike the results of the surveys’ questions on hospital admissions, the results of questions on the utilization of outpatient care are not easily comparable, even those regarding contacts with primary care physicians and GPs.
The fundamental question regarding the validity of self-reports on the utilization of medical services also needs to be raised, especially when it comes to questions regarding the prevalence of contact over a period of more than three months. Robra et al.  conducted an early study on comparing the results on outpatient services utilization from the EVaS Survey of outpatient care in Germany and the National Health Survey used in the German Cardiovascular Disease Prevention Study and found marked consistency between survey participants’ self-reports of frequency of in-person contacts with physicians and the data obtained from medical records. However, their study did not validate individual data from the two surveys; it only validated the results of extrapolations regarding the number of physician contacts made during a three-month period. The author is unaware of any more recent comparative studies in Germany that validate self-reported data through the use of external data sources.
International studies addressed the validity of self reported frequency of utilization. According to a somewhat dated review conducted by Harlow and Linet , which examined validation studies comparing data on the utilization of medical services obtained from questionnaires, there was good agreement between self-reports and hospital record data on hospital stays and operations. There was less agreement between data, however, when it came to contacts with dentists, specific medical services (e.g., x-rays) or medications. The study did not investigate the frequency of outpatient services utilization.
Walihan et al.  evaluated the accuracy of older Americans’ (aged 60+) self-reported utilization of health services (e.g., hospitalizations, utilization of emergency services, outpatient care visits, home care visits) and found that one-fourth of those who had one hospitalization did not report it. The same amount failed to report an emergency room visit. In terms of outpatient care visits, underestimations were the greatest when reporting on the number of visits; the degree of this underestimation was correlated with age and the frequency of outpatient visits. Another study found the degree of underestimation to be significantly higher for self-reports of medical services utilization by people with mental illnesses  and suggests that this underestimation is primarily due to recall bias and social desirability bias.
A review by Bhandari et al.  examined factors affecting the degree of underestimation. These included age, type of and reason for services use, the frequency of services use, and the extent of a perceived stigma associated with the use of certain services, such as mental health services or those for addiction disorders. Survey design (personal interview, phone interview, questionaire), question wording and understandability, the length of the reference period used for assessing services utilization, the order of the survey questions, and memory aids also have an impact on the validity of self-reported data. Based on the findings of these studies, it must be assumed that self-reported data on services utilization are of highly variable validity and therefore of limited comparability.
Given the mentioned difficulties associated with the validity of self-reports of medical services use, claims data collected by the German statutory health insurance (SHI) system on the utilization of medical services can be used for epidemiological studies in general and in particular for validating self-reported data. In recent years, these data have systematically been made available for use in health services research ,  and the larger health insurance funds have been establishing continuous (health data) reporting systems, e.g., , . However, these data also have their weaknesses when used for purposes other than their primary purpose associated with billing. These weaknesses include missing data on privately insured individuals or privately financed services. Outpatient sector data on the reasons for outpatient contacts and the number of contacts during a certain quarter are also missing since, in recent years, physicians contracting with the SHI funds have increasingly been paid lump sums in advance for outpatient services during a specific period rather than “per service” rendered.
There are also two other challenges that must be faced when choosing to combine selected claims SHI data with primary data rather than using it solely as routine data . First, there are numerous technical, organizational and data protection issues that must be dealt with , , . Also, many study participants will not consent to having their survey data and SHI data linked. The total of these “non-consenters” can easily reach up to 40% . Caution must therefore be taken when considering the potential use of linked personal data.
Future surveys and epidemiological studies will therefore not be able to forego sets of questions on services utilization and, when including such questions, should consider five steps to maximize data quality , taking into account a) cognitive status and age of the participants, b) recall time frame and frequency and type of utilization, c) an adequate questionnaire design, d) Mode of data collection and e) memory aids and probes. Beyond these considerations current and planned studies linking personal primary and secondary (claims) data are suitable for more accurate assessments of over- and underestimations resulting from self-reports of study participants .
The writing of this article was carried out as part of the scientific network project “Utilization of health-related services in Germany – theoretical approaches, methods and empirical results in medical sociology” (NWIN). The project is funded by the Deutsche Forschungsgemeinschaft (German Research Foundation, grant no.: JA 1849/1-1).
The author declares that he has no competing interests.