# Increasing Generalizability of RCTs

Is your randomized controlled trial (RCT) generalizable to the general population?  This question is known as external validity and is a major issue for a number of treatments.  Sometimes, a treatment is very effective in an RCT, but less so in the real world.

One reason why this may be the case is that the sites selected for the RCT may not be representative of the general population who use the treatment.  To address this issue, a paper by Gheorghe et al. (2014) proposes using a generalizability index (GIx) to measure whether the sites selected for an RCT are representative of the general population.  Below I describe how to estimate the GIx.

GIx calculation

1. Identify site-specific variables that could influence cost effectiveness estimator. Some variables which may cause differences in site-specific clinical efficiency include the site’s capacity, clinical expertise, patient case-mix and specialization.   For instance, there are likely differences in cost-effectiveness based on whether an RCT is implemented in a teaching or non-teaching hospital.
2. Generate jurisdiction-wide estimates of these variables.  For all sites where the treatment could be implemented, measure the average values of the relevant variables form Step 1.
3. Summarize the “typical” value for the population of interest.  This could be the mean, median or other statistic representing the typical patient in the general population.
4. Rank sites in the study by each variable.   The rank will be based on the deviation from the “typical value”
5. Calculate and rank the (weighted) sum of ranks for each site across the included dimensions.  The authors describe this calculation in more detail: “The centre-level GIx will be given by the rank of its (weighted) sum of dimension ranks. Lower values of the GIx will correspond to the more ‘typical’ centres, whose characteristics are close to the typical values in the population of centres. Higher values of the GIx will correspond to centres that are more extreme in one or more dimensions. Weighting the variables may be desirable to incorporate evidence or belief about the relative influence of the included variables on cost-effectiveness estimates, such as from the variance – covariance matrix of a regression on the incremental net benefit.”

The GIx can serve two purposes.  First, it can identify sites that are most likely to be representative of the general population. To identify these sites, one could simply choose centers with low GIx scores.  Second, one could select sites from all quintiles of the GIx distribution in the case where site heterogeneity creates non-linear effects on cost-effectiveness.