A quality assessment of randomized control trials of primary treatment of breast cancer.

The methodology of randomized control trials (RCTs) of the primary treatment of early breast cancer has been reviewed using a quantitative method. Sixty-three RCTs comparing various treatment modalities tested on over 34,000 patients and reported in 119 papers were evaluated according to a standardized scoring system. A percentage score was developed to assess the internal validity of a study (referring to the quality of its design and execution) and its external validity (referring to presentation of information required to determine its generalizability). An overall score was also calculated as the combination of the two. The mean overall score for the 63 RCTs was 50% (95% confidence interval [CI] = 46% to 54%) with small and nonstatistically significant differences between types of trial. The most common methodologic deficiencies encountered in these studies were related to the randomization process (only 27 of the 63 RCTs adopted a truly blinded procedure), the handling of withdrawals (only 26 RCTs included all patients in the analyses), the description of the follow-up schedule (only 12 RCTs reported adequately), the report of side effects (adequate information given in 33 RCTs), and the description of the patient population (satisfactory in 29 RCTs). Telephone calls to the principal investigators improved the quality scores by seven points on a scale of 100, indicating that some of the deficiencies lay in reporting rather than performance. There was evidence that quality has improved over time and that the increasing tendency of involving a biostatistician in the research team was positively associated with the improvement of the internal validity but not with the external.