The patients had a mean age of 69 (SD 17) years and 225/504 (45%) were men. The diagnosis was cancer in 326 (65%), AIDS in 62 (12%), and other conditions in 116 (23%). The mean duration of disease was 83.5 (135.8) weeks, and the median performance status was 3 (corresponding to >50% of the day spent bedridden). The doctors had a median duration of medical practice of 16 years; 291/363 (80%) were men; 293/365 (80%) were board certified; and 255/345 (74%) rated themselves optimistic. A total of 114/358 (32%) specialised in general internal medicine, 71/358 (20%) in non-oncological internal medicine subspecialties, 61/358 (17%) in oncology, 55/358 (15%) in family or general practice, 27/358 (8%) in geriatrics, and 30/358 (8%) were surgeons or practised other specialties. In the past year, the doctors had had experience caring for a median of five patients with the same diagnosis and had referred a median of eight patients to a hospice. They had known the patient an average of 159 (308) weeks; had 11 (14) contacts in the previous three months; and had examined the patient 14 (29) days before.
Doctors' prognostic estimates
In only 18 of 504 patients did the doctor refuse to predict survival to us. Eighteen of the remaining 486 had missing dates of death, leaving 468 cases referred by 343 doctors for analysis of prognostic accuracy. The figure illustrates the extent of the error. The median observed patient survival was 24 days. The mean ratio of predicted to observed survival was 5.3. The correlation between predicted and observed survival was 0.28 (P<0.01). When an accurate prediction was defined as between 0.67 and 1.33 times the actual survival, 20% (92/468) of predictions were accurate, 63% (295/468) optimistic, and 17% (81/468) pessimistic. When an accurate prediction was defined as between 0.50 and 2.0 times the actual survival, 34% (159/468) of predictions were accurate, 55% (256/468) optimistic, and 11% (53/468) pessimistic. Death occurred within one month of the predicted date for 42% (195/468) of patients, at least one month before the predicted date in 46% (214/468), and at least one month after the predicted date in 13% (59/468) of patients.
The extent of prognostic error varied depending on both observed and predicted survival (table). The longer the observed survival (that is, the less ill the patient), the lower the error, and, conversely, the longer the predicted survival, the greater the error.
Factors associated with prognostic accuracy
Bivariate analyses of the trichotomous accuracy variable and patient attributes showed no important differences with respect to patients' age, sex, race, religion, or marital status. However, cancer patients were the most likely to have overoptimistic predictions (220/301 (67%) v 37/58 (64%) of AIDS patients and 56/109 (51%) of other patients) and the least likely to have overpessimistic predictions (39/301 (13%) v 13/58 (22%) and 29/109 (27%)); AIDS patients were the least likely to have correct predictions (8/58 (14%) v 60/301 (20%) of cancer patients and 24/109 (22%) of others; P<0.01).
Bivariate analyses of the doctor attributes showed no important differences with respect to sex, years in medical practice, board certification, self rated optimism, number of hospice referrals in past year, or number of medically similar patients in the past year. However, doctors in non-oncological medical subspecialties were the least likely to give correct estimates (8/79 (10%) v 11/30 (37%) doctors in surgery or other, 18/76 (27%) in family or general practice, 24/105 (23%) in oncology, and 30/180 (17%) in geriatric or general internal medicine), and oncologists were the least likely to be overpessimistic in their estimates (10/105 (9%) v 21/79 (27%) in other internal medicine subspecialties, 13/67 (19%) in family or general practice, 31/180 (17%) in geriatric or general internal medicine, and 4/30 (13%) in surgery or other; P<0.01).
Among the doctor-patient relationship variables (such as length of professional relationship, number of recent contacts, time since last examination), the interval since last examination was important: overpessimistic predictions were associated with the most recent examinations (7.5 days), overoptimistic predictions with the next most recent examinations (13.8 days), and the correct predictions with the longest interval since physical examination (19.5 days); P<0.05.
The trichotomous prognosis variable was regressed on patients' age, sex, race, diagnosis, duration of disease, and performance status and on doctors' experience, sex, optimism, board certification, specialty, related practical experience, duration of relationship, number of contacts, and interval since last examination (full results available on the BMJ's website). The model showed that doctors' prognostic accuracy was independent of most patient and doctor attributes. However, after other attributes were adjusted for, male patients were 58% less likely to have overpessimistic than correct predictions (odds ratio 0.42; 95% confidence interval 0.18 to 0.99). Doctors in the upper quartile of practice experience were 63% less likely to make optimistic rather than correct predictions (0.37; 0.19 to 0.74) and 78% less likely to make pessimistic rather than correct predictions (0.22; 0.08 to 0.61). Doctors with medical subspecialty training (excluding oncologists) were 3.26 times more likely than geriatricians and general internists to make pessimistic rather than correct predictions (3.26; 1.01 to 10.7). As the duration of the doctor-patient relationship increased, so too did the doctor's odds of making an erroneous prediction—for example, each one year longer that the doctor had known the patient resulted in a 12% increase in the odds of an overpessimistic prediction (1.12; 1.02 to 1.22). Also, as the interval since last physical examination increased, the odds of a doctor making a pessimistic rather than a correct prediction decreased; each day longer resulted in a 3% decrease in the odds (0.97; 0.94 to 0.99).