How do you create a Markov Model for the effectiveness of pharmaceuticals? Below is an example from Briggs, Claxton and Sculpher’s book titled “Decision Modelling for Health Economic Evaluation.”
Example
The main characteristic of a Markov Model is that it defines different states and then defines transition probabilties between each state. Let us examine the case of someone with an HIV infection. There are 4 possible states in this simplified model:
- State A: cd4 levels are between 200 and 400
- State B: cd4 levels are less than 200
- State C: The individual has AIDS
- State D: Death
Transition Probabilities
Now we must find a baseline transition probability matrix. In this case, the baseline will be no treatment, but the baseline could also be one form of treatment to be compared to another. This table gives the transition probabilities. You can read the table as follows: A person in state A has a 72.1% chance of being in state A next period, a 20.2% chance of being in State B next period, a 6.7% chance of being in state C next period and a 1.0% chance of being in state D next period.
Now there is a drug treatment on the market which has a relative risk of 0.509. This means that the chance of moving to a worse state has decreased by about half. The book’s authors note that “Often it is assumed that baseline event probabilities should be as specific as possible to the location(s) and subgroup(s) of interest, but that the relative treatment effect is fixed.”
To get the new transition probabilties after treatment, we multiply the transition probabilities by 0.509 if the represent a worsening of the state. The remaining ‘extra’ probability is moved to the probability of transitioning to the current state. In this model, no one can transition to a better state (i.e., remission) so we do not have to worry about the relative risk of getting better.
Look at the first row of the transition probabilities matrix with the therapy. State A transition probabilities to states B, C, and D are multiplied by 0.509 to get the new transition probabilities (i.e., .202*.509 = .103; .067*.509=.034; 0.010*.509 = .005). The probability of staying in state A after treatment is just one minus the other probabilities (i.e., 1 – .103 – .034 – .004 = .858).
Survival
Now we want to find out how many people will survive. We can do this with a simple simulation. The baseline simulation is shown here and the simulation with the treatment is shown here. To get the future probabilities, simply multiply the transition probabilities by the people in each state. For instance, to find out how many people will be in state C in year 5, we need to look at year 4. We know that of the people in state A, 0.067 will go to state C; Of the people in state B, .407 will go to state C; of the people in state C, 0.750 will state in state C, and of the people in state D, 0.000 move into state C. Thus we calculate the number of people in state C in year 5 as: .27*.067+.23*.407+.34*.75+17*0 = 0.36.
We could also accomplish this with matrix algebra. The vector of people in each state is equal to [1 0 0 0]*Tn. This means that in year 0, we have 100% of people in state A. The transition matrix is represented by T and n is the number of years in the future we want to view.
In our analysis, we see that the baseline 20-year survival rate is only 3.2%, but with the treatment, this increases to 33.2%.
Cost
We can also determine the costs of the treatment and baseline. The treatment has the added expense of purchasing the drug for $2278. However, with the treatment fewer people are moving into the more expensive stages B and C. Thus there is a tradeoff.
We can see from the simulation, that the treatment is more expensive than the baseline. To calculate this, you simply multiply the proportion of people in each state by the cost in each state to get the cost per year. It is also important to discount the costs to get the expenses in terms of net present value. In this example, I used a 6% discount rate.
Markov with Memory
In general, Markov models are memoryless, meaning they do not care how long an individual has been in each state. It is possible to create ‘memory’ using tunnel stages. Let us examine the following example for disease X. In this model there are 5 stages, 1 having the disease, 3 remission stages, and death. An individual with disease X has a 60% chance of keeping the disease, a 20% chance of remission less than 1 year and a 20% chance of death. Of course, they have a 0% chance of being in remission for 1-2 years or >2 years after only 1 period. If a person does go into remission, we see that they have a 40% chance that the disease reoccurs, a 50% chance of getting to the remission for 1-2 years stage, and a 10% chance of death. By making these stage related to time, we have created a Markov model that simulates memory.
Summary
With Markov modelling, we can estimate the effect a drug has, both in terms of its health implications–such as survival rates and the number of people in each stage–as well as its cost implications. The key assumption is that the treatment effect is constant across all stages.