Surrogate Outcomes and the Prentice Criteria

Clinical trials aim to measure how different treatments affect outcomes of interest to patients.  In many cases, however, measuring the ultimate outcomes of interest proves logistically difficult.  For instance, measuring overall survival for some oncology patients may be impractical if survival for the standard of care is long since the required time and expense to fund a trial to capture the long-run OS could be prohibitive.  Instead surrogate measures (e.g., progression-free survival, time to progression, response rates) are often used. 

However, what makes a good surrogate outcome?  Is being correlated with the outcome of interest good enough?

A paper by Prentice (1989) gives a clear definition. He states that a surrogate variable is valid if and only if the true endpoint rate at any follow-up time is independent of treatment, given the preceding history of the surrogate variable).  Let the outcome of interest at time t be O_t, the surrogate endpoint S_t, and a drug treatment be D.  For simplicity, assume the drug D is administered at time 0 and D can take on any non-negative integer value where each integer is a different treatment.

What Prentice requires is that: P(O_t=o|S_t’, D) = P(O_t=o|S_t’) s.t. t>t’.

The actual Prentice paper assumes that the trial measures hazard ratios and requires that:

λ_T{t;S(t),D}= λ_T{t;S(t)}

Prentice also rules out the uninformative case where as surrogate provides no information.  For instance, let’s say that a surrogate is just random noise.  In that case, the hazard rate for the true outcome of interest would be completely independent of the surrogate outcome.  Prentice’s definition precludes this as he requires that:

λ_T{t;S(t)} ≠ λ_T(t).

All Prentice criteria are defined formally as:

1. f(O|D)≠f(O)à the drug must effect outcomes
2. f(S|D)≠f(S)à the drug must effect the surrogate
3. f(O|S) ≠f(O)à The surrogate affects the outcome
4. f(O|S,D) = f(O|S) à conditional on the surrogate, the outcome of interest is independent of drug treatment

So clearly surrogate outcomes can readily be used in clinical trials without a problem, right?  A paper by Berger says ‘not so fast’. 

“It is shown that [the Prentice] criterion alone ensures that an observed effect of the treatment on the true endpoint implies a treatment effect also on the surrogate endpoint, but contrary to popular belief, it does not ensure the converse, specifically that the observation of a significant treatment effect on the surrogate endpoint can be used to infer a treatment effect on the true endpoint.”

The reason for this is because the validity criterion may not be met.  The validity criterion requires that criterion 2 (i.e., the drug affects the surrogate) implies criterion 1 (i.e., the drug affects outcomes), but this may not be met.  Criterion 4 does not preclude the possibility that there are causal pathways from D to S that do not extend to the outcome of interest O.

One example from Buyse and Molenberghs (1998) shows that there can be cases where the drug has no effect on the outcome of interest, but the drug does have an effect on surrogate outcomes. In the example below, we see that 25% of people in both the control and experiment have T=0 (in my example that is O=0) and 25% in both groups have T=1 (in my example, that is T=0). However, we see that people that receive the experimental treatment have a 25% increase in surrogate values A and C but a 25% decease in surrogate values B and D. This is despite the treatment having no real affect on the true outcome of interest (T).

So, do we need to give up on surrogate outcomes?  The Berger (2004) paper provides some guidance.

One example of a common-sense approach for the use of surrogate endpoints in the context of cancer prevention trials is that ‘appropriate intermediate endpoints should (1) be integrally involved in the process of carcinogenesis so that modulation of expression correlates closely with disease course, (2) have different expression in normal versus preinvasive or at-pathway epithelium, (3) be easily and reproducibly measured from biologic specimens obtained during clinical trials, and (4) be able to be modulated by a successful chemopreventive intervention with minimal spontaneous fluctuation and remission’ [4]. If a surrogate endpoint satisfies all of these conditions, or analogous conditions to match the specific context in which it is to be used, then even without an iron-clad guarantee one may decide that it is ‘validated’ enough to warrant its use in practice.