How do you design a good clinical trial? Clearly, the trial design depends on what question you are answering. For instance consider the dichotomy between an explanatory and pragmatic trial:
A pragmatic randomised trial is undertaken in the “real world” and with usual care and is intended to help support a decision on whether to deliver an intervention. An explanatory randomised trial is undertaken in an idealised setting, to give the initiative under evaluation its best chance to demonstrate a beneficial effect. These two approaches represent different attitudes to decision making on the usefulness of interventions.
The quotation above makes if seem like a pragmatic trial is always best. That is clearly not the case. Consider the case where there is a brand new technology which requires significant physician training in the real world. An explanatory trial could implement training as part of the protocol to insure that only trained doctors use it. A pragmatic trial could include physicians who were and were not trained and thus the treatment may be less successful in the real world. If one believes that in the long-run physicians will be trained to use the intervention, but in the short-run there could be problems, then an explanatory trial may be superior. If on the other hand, one is skeptical that physicians will actually pay attention to any training offered in the real world, a pragmatic trial may be more attractive. Again, both trials approaches are useful, but the best approach depends on the research question of interest (among other factors).
One tool to help researchers decide is the PRECIS-2 tool. The tool was designed by over 80 international trialists, clinicians, and policymakers.
PRECIS-2 has nine domains—eligibility criteria, recruitment, setting, organisation, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis—scored from 1 (very explanatory) to 5 (very pragmatic) to facilitate domain discussion and consensus.
PRECIS-2 is an update of PRECIS. PRECIS was widely cited but some limitations included that it didn’t use an explicit rating scale and it was not validated, in addition to some domain-specific critiques. What are the PRECIS-2 domains and how can one use it? Take a look at the graphic below.
More detail on each domain is after the jump.
-
- Eligibility criteria—Who is selected to participate in the trial? A highly pragmatic approach to eligibility criteria would be to include in the trial anyone with the condition of interest who is likely to be a candidate for the intervention if it was being provided in usual care for this condition. For instance, children, the elderly and those with comorbid conditions would be more likely to be included in a pragmatic trial
- Recruitment—How are participants recruited into the trial? The most extreme pragmatic approach to recruitment would be to do this in usual care so that only the people who attend a clinic with the condition of interest are recruited after they present on their own behalf without any overt recruitment effort. A highly pragmatic approach to trial design would also recruit from more than one clinic as an explicit way of increasing applicability of the trial results. Recruiting through usual appointments at a diverse range of clinics is likely to score at or close to 5 on the PRECIS-2 scale.
- Setting—Where is the trial being done? The most extremely pragmatic approach to setting would be to do the trial in an identical setting to which you intend the results to be applied. Such a trial is likely to score at or close to 5 on PRECIS-2. Even settings that seem rather restrictive could still be a highly pragmatic design choice if this setting is the usual care setting where patients are treated for the particular health condition. For example, if a trial was carried out in the most specialist intensive care units in the country and the intention of the trial was to support decision making in these highly specialised units, then the design choice with regard to setting is still pragmatic.
- . Organisation—What expertise and resources are needed to deliver the intervention? This domain has been included to encourage trialists to consider the match between how care is organised and delivered in the trial and how the intervention would be made available to patients in usual care…A highly pragmatic design would aim to slot the intervention into the usual organisation of care for the condition of interest, making use of no more than the existing healthcare staff and resources in that setting. A design like this is likely to score at or close to 5 on PRECIS-2.
- 5. Flexibility (delivery)—How should the intervention be delivered? The most pragmatic design approach to delivery flexibility would leave the details of how to implement the intervention up to providers, in other words, what happens in usual care. For example, the details of how to perform a surgical procedure could be left entirely to the surgeon, or how to deliver an educational programme is left to the discretion of the educator.
- Flexibility (adherence)—What measures are in place to ensure participants adhere to the intervention? A highly pragmatic design approach would allow for full flexibility in how end user recipients engage with the intervention. In usual care, health professionals encourage patients to take medication or follow therapy as best they can, and such encouragement would not count against a pragmatic design; if it also happens in usual care, allowing it in the trial is a pragmatic design decision. A trial with no special measures to enforce engagement or compliance would score at or close to 5 on PRECIS-2. On the other hand, a trial protocol that lays out methods to monitor and ensure patient compliance would score at or close to 1 on PRECIS-2.
- Follow-up—How closely are participants followed up? The most pragmatic position with regard to follow-up would be to have no more follow-up of recipients than would be the case in usual care. Indeed, the most extreme position is to have no follow-up contact at all with recipients and to obtain outcome data by other means (such as electronic medical records or other usual data to measure mortality or hospital admissions). As follow-up becomes more intense, the trial becomes more explanatory and the PRECIS-2 score will decrease.
- Primary outcome—How relevant is it to participants?
The choice of primary outcome is a crucial trial design decision, and a pragmatic approach would be to select an outcome that is of obvious importance from the patient’s perspective. Post-trial, an outcome selected using a pragmatic approach would also be relevant to commissioners of care, the people who decide whether to implement the intervention on the basis of its results. For example, an intervention that aims to reduce falls in elderly people living independently in the community should have as its primary outcome the number of falls in the elderly living independently in the community. This outcome has meaning to patients, their relatives and friends, healthcare professionals, and policymakers. Measures of, say, bone density, muscle strength, or functional ability are distant from the key question of whether the intervention prevents elderly people falling in their own homes. - Primary analysis—To what extent are all data included? Most trials are a superiority design so the most pragmatic approach with regard to the analysis would be to make no special allowance in the analysis for non-adherence, practice variability, etc. In other words, the pragmatic approach to the analysis would typically be an intention-to-treat analysis using all available data…Systematic exclusion of data from participants because, say, they were poorly adherent would make a trial more explanatory. Using all data but doing nothing to try and fill gaps caused by missing data would not in itself make trial more pragmatic or explanatory; missing data, especially if there is a lot of it, makes any conclusions more uncertain regardless of the design approach taken.
Source:
- Loudon, Kirsty, Shaun Treweek, Frank Sullivan, Peter Donnan, Kevin E. Thorpe, and Merrick Zwarenstein. “The PRECIS-2 tool: designing trials that are fit for purpose.” bmj350 (2015): h2147.