Population-adjusted indirect comparisons (PAICs) include both matching-adjusted indirect comparisons (MAICs) and Simulated treatment comparisons (STCs). The key data requirement for these methods is that they have individual patient data (IPD) from at least one clinical trial. This means the methods are most useful for studied funded by the clinical trial sponsor or when IPD clinical trial are publicly available (for instance, see the Yale University Open Data Access (YODA) Project.
Standard approaches to comparing efficacy and safety across clinical trials when head-to-head trial data is not available often rely on indirect treatment comparisons (ITC, see Bucher et al. 1997) or network meta-analysis (NMA, see Dias et al. 2013). One key assumption for these methods to be valid are that there are no effect modifiers or the distribution of effect modifiers is identical across trials. This bias may be particularly large when there are sparse networks of trials in the indirect comparison.
This is a key advantage of PAIC, but there are also some disadvantages. A NICE DSU Technical Support Document (#18) from 2016 outlines some of these advantages and limitations. On the limitations side:
Both MAIC and STC can be used to carry out either an “anchored” indirect comparison, where there is a common comparator arm in each trial, or an “unanchored” indirect comparison, where there is a disconnected treatment network or single-arm studies. An unanchored MAIC or STC effectively assumes that absolute outcomes can be predicted from the covariates; that is, it assumes that all effect modifiers and prognostic factors are accounted for. This assumption is very strong, and largely considered impossible to meet.
Further, while conceptually PAICs should produce robust estimates in the presence of effect modifiers, there is limited evidence that accuracy materially improves. Finally, by using PAIC to reweight information from a given clinical trial, the analysis population may be more or less similar to the target population of an intervention.
The NICE DSU Technical report makes a number of recommendations to practitioners:
- Anchored preferred. Use anchored comparisons unless there is an absence of a common comparator.
- Demonstrate a need. PAIC should be used when one can show (i) that there are effect modifiers present and (ii) there are differences in the distribution of effect modifiers across trials.
- Effect modifiers. When using MAIC, all effect modifiers should be adjusted for to ensure balance and reduce bias, but no purely prognostic variables. This approach is recommended since increasing the number of matching variables may inflate the standard error due to over-matching.
- Use linear scale. NICE recommends that indirect comparisons must be carried out on the usual linear predictor scale
- Identifying a target population. Typically, clinical trials are conducted on a specific target population. Using PAICs reweight the trial sample in a way that could differ from the target population. Explicitly stating the target population of interest is important to verify how the reweighting moves the weighted sample closer or further from the target population.
- Clear reporting. Researchers should assess of covariate distributions, provide evidence that variables are effect modifiers, provide the distribution of weights (if applicable), and calculate appropriate measures of uncertainty.
For more details, see the entire NICE DSU Technical Support Document (#18) .