Ultrasonography has been considered the most operator dependent imaging technique. The paucity of studies on its validity, reliability, and sensitivity to change has largely contributed to this and has limited the development of multicentre and longitudinal ultrasonographic studies.
European rheumatologists highly experienced in musculoskeletal ultrasonography have comprised the faculty of the nine training courses on musculoskeletal ultrasonography organised in different European countries under the auspices of the EULAR Standing Committee for Education and Training since 1998. They have a teaching and research curriculum in this field. Many of them chair and organise ultrasonography training for rheumatologists in their countries.
For the last four years, the EULAR working group for musculoskeletal ultrasound has made an effort to standardise ultrasonographic scanning methods14
and diagnostic criteria and to develop reliability studies.
The first official ultrasound special interest group (SIG) met at OMERACT 7 (Asilomar, California) in May 2004. The principal activities of the ultrasonography SIG have been a systematic review of published reports and a consensus on preliminary pathological definitions of synovial hypertrophy, tenosynovitis, enthesopathy, and bone erosion.
The first Train the Trainers meeting was held in Berlin before the eighth EULAR sonography course organised by M Backhaus and W A Schmidt in June 2004. Fourteen teachers from that course participated in the present study, which had two main objectives: to assess the interscanner variability between the 14 examiners and to evaluate agreement in ultrasonographic diagnosis, with MRI findings as the gold standard, in four anatomical regions (shoulder, knee, wrist/finger, and ankle/toe) of four patients, respectively, with inflammatory rheumatic diseases.13
Before the study by Scheel et al
ultrasonographic interobserver reliability had only been tested between two examiners.4,5,7,8,9,10,11,12
Swen et al8
reported a good κ value (0.63) in the detection of rotator cuff full thickness tear. Middleton et al12
found a high agreement (92%) in the diagnosis of rotator cuff partial and full thickness tear. The κ values for ultrasonography detection of wrist synovitis, tenosynovitis, and erosions were from 0.73 to 0.89 according to Iagnocco et al
In the study by Szkudlarek et al
the overall agreement/κ values for the semiquantitative assessment of effusion, synovitis, power Doppler signal, and erosions in small joints of the hand and foot were 79%/0.48, 86%/0.63, 87%/0.55, and 91%/0.68, respectively. However, Filippucci et al10
reported higher κ values for the detection of effusion/synovitis and power Doppler signal (0.86 and 0.95, respectively) in the wrist and small joints of the hand and foot. In addition, the κ value for ultrasonographic identification of metacarpophalangeal erosions was 0.76 in the study by Wakefield et al
reported κ values of 0.90 and 0.71 for the detection of knee effusion and synovitis, respectively.
Although the results of the Train the Trainers interobserver study were moderate to good (overall κ for all examined joints
0.76), we organised the Teach the Teacher course four months later in order to re‐evaluate the interobserver reliability of the main periarticular and intra‐articular ultrasonographic diagnoses and reveal the principal disagreements between the participants by scanning patients together in real time.
Even though we showed a high level of overall agreement, our κ values were lower than those communicated in individual studies by some rheumatologists of the group.4,5,7,8,10,11
There may be several reasons for these differences. In previous reliability reports the two examiners worked at the same hospital and used the same machine, probably had a common ultrasonographic background, and usually reached consensus on scanning and diagnostic criteria before the study. However, in the present study as well as in the one by Scheel et al
the experts—despite meeting for a few days on several occasions in the past six years—work in different hospitals and countries and many were not familiar with the ultrasonographic equipment. The latter may explain the interobserver variability for power Doppler findings among participants. In addition, the examiners were unaware of the patient's clinical data and did not train together before the investigation—indeed, the aim of the study was to assess the interobserver reliability of the spontaneous ultrasonographic evaluation carried out by experts within the usual time spent on it in daily clinical practice. Nevertheless, our ultrasonographic interobserver reliability was similar to or better than that described in studies on MRI reliability in the detection of rotator cuff disorders8,16
or joint synovitis, erosions, and tenosynovitis,17
or on interobserver variability of the clinical examination of joint inflammation.12,18
Both clinical evaluation and MRI are widely considered to be the gold standard in clinical trials.
With regard to the second objective of our study, some issues should be explored. As Scheel et al13
reported, multiplanar and dynamic scans were not carried out by all the experts. Dynamic ultrasonography is very useful for detecting subtle musculoskeletal abnormalities such as small bone erosions, tendon tears, and minimum fluid within synovial recesses and tendon sheaths, and probably should be used for all musculoskeletal ultrasonographic studies. A more intensive training in standardisation of scanning methods is likely to improve the sensitivity and reliability of musculoskeletal ultrasonography.
Another point of interest is to identify which recesses of each joint should be scanned for detecting synovitis. As the sensitivity of ultrasonographic detection of synovitis has not yet been compared in the different joint recesses, most experts scan all of them, although it makes the examination longer. Future studies providing evidence of the more sensitive joint recesses for detecting intra‐articular inflammation would be very useful to shorten scanning time.
In addition, more accurate definitions of tendinosis, tendon partial tear, and complete tear—mainly rotator cuff lesions—based on validation studies of the ultrasonographic semiology are needed to improve interobserver agreement.
Finally, it was not easy to reach consensus among experts on the subjective diagnosis of pathological mild joint effusion, tenosynovitis, or bursitis versus normality. Physiological fluid in joint recesses, synovial sheath of tendons, and large bursae, as well as hypoechoic rims in joints that correspond to normal synovial fluid or articular cartilage, or both, are commonly detected with high resolution ultrasonography machines in normal subjects.19
Although in our study the experts used the same machine for scanning the same patient, their different ultrasonography backgrounds could have influenced the final diagnosis. Objective diagnostic criteria of pathological fluid within joints, tendon sheaths, and bursae are necessary to distinguish normality from mild pathology, independent of the ultrasonography machine used. This emphasises the relevance of the study by Schmidt et al
who determined standard reference values for musculoskeletal ultrasonography in a large series of healthy adults. Nevertheless, a rheumatological ultrasonography approach correlating ultrasonographic findings with clinical symptoms is always recommended.
Some limitations of our study should be mentioned. For example, κ values could not be calculated for each ultrasonographic diagnosis in all the regions because the observers in each group changed during the study. This is inconvenient from a statistical point of view. However, the main goal of the Teach the Teacher course was to work with as large a number of different experts as possible.
Further meetings of the EULAR/OMERACT musculoskeletal ultrasonography group for training in standardisation of scanning method, establishing definitions, quantifying ultrasonographic pathologies, and assessing reproducibility, sensitivity to change, and intermachine variability are necessary. These future exercises will contribute to the expanding use of musculoskeletal ultrasonography in clinical and research rheumatology to improve the evaluation of inflammatory activity and therapeutic response in patients with rheumatic diseases.