Dynamic Prediction

PSC Forum 6

November 9, 2023

Nicole Erler

Assistant Professor

Department of Biostatistics
Erasmus Medical Center, Rotterdam

nerler.com    N_Erler   NErler

Conflicts of Interest

Nothing to disclose.

Prediction

There are many reasons why we want to predict the future, not just for PSC patients.

The decisions the doctor (and the patient) need to take today, impact the patient’s future. And knowing what this future looks like — potentially under different scenarios — would, for example, allow a doctor to choose to best treatment or the best time for an intervention.

But also in a research context looking into the future is important to select the right patients for a particular study.

And maybe most importantly, the patients themselves, of course, want to know what to expect and what their daily life will look like.

Because unfortunately, none of us can actually look in to the future, we need prediction models.

Static Risk Calculators

There are a couple of established risk scores for PSC patients.

These classical prediction algorithms are built using patient and disease characteristics measured only once.

This is a limitation for multiple reasons. One of them being that when using data from a single time point, we cannot capture the disease dynamics.

Consider two patients: one with stable but borderline acceptable biomarker values, and the other recently diagnosed incidentally with initially excellent lab values, which have now declined to match the first patient’s.

These two patients likely have very different prognoses, yet the current models would provide them with the same risk estimates.

To advance our prediction models, we must acknowledge the dynamic nature of the disease. And so, our prediction models need to become dynamic as well.

From Static
 to Dynamic
   Prediction

How can we move from static to dynamic prediction?

The key to dynamic prediction is to model data over time.

Instead of using data from only one time point, we need data measured throughout the follow-up…

… data that describes how the outcome and covariates change over time.

And to capture that patients have different evolution over time, …

… we cannot rely on a single measurement per patient, but rather we need to model patient-specific trajectories.

A convenient way of achieving this is by using so-called random effects models. Random effects models are similar to standard regression models.

Random Effects Models

Linear Regression Model \[y_i = \mathbf x_i^\top \boldsymbol\beta + \varepsilon_i\]

\[\color{grey}{ \text{e.g.: }\log(\text{bili}_i) = \beta_0 + \beta_1 \text{age}_i + \beta_2\text{sex}_i + \beta_3 \text{time}_i + \ldots + \varepsilon_i}\]

Random Effects Model \[y_i(t) = \underset{\substack{\text{fixed effects}\\\text{(population level)}}}{\underbrace{\mathbf x_i(t)^\top \boldsymbol\beta}} + \bbox[#400020, 5pt]{\underset{\substack{\text{random effects}\\\text{(patient level)}}}{\underbrace{\mathbf z_i(t)^\top \mathbf b_i}}} + \varepsilon_i(t)\]

They involve modelling a response, such as the bilirubin or other biomarkers, based on various covariates like the patient’s sex, age, treatment type, comorbidities, and so on.

And because we now have measurements taken at different times, we also include this measurement time in the model.

But in addition, the random effects model incorporates patient-specific random effects.

These random effects account for the unique trajectory of each patient and describe how that trajectory differs from the population average trajectory.

Here you see the bilirubin values over time for six patients.

The dashed line shows the expected bilirubin based on the model, when we ignore the random effects part, and so it is the same for all patients.

Typically, such a population fit does not fit the trajectories of individual patients very well.

But when we add the random effects part, we get a different estimated trajectory for each patient, and we see that these fit the observed data much better.

Predictions from Random Effects Models

We can also estimate the patient-specific trajectory for a patient who was not part of the data that was used to estimate the model.

Based on the measurements available for this patient up until the current time, we can estimate this patient’s random effects. And combining this information with the covariate data, we obtain an estimate for how the trajectory will likely develop in the future.

This means we can use the model to obtain patient-specific dynamic predictions:

Here we see such a dynamic prediction of the log-transformed bilirubin for two patients. The vertical line marks the current time, left of it we have the observed values and the patient-specific model fit and on the right we see the predicted bilirubin with 95% confidence interval.

Over time, as more information becomes available, the trajectories are updated and begin to diverge.

However, starting at around 7 years, patient 2 starts to deteriorate, his bilirubin values increase, and the and the predicted values for the two patients become more similar again.

Joint Modelling

medication

AST

platelets

ALP

bilirubin

IBD

In practice, we are typically interested in multiple outcomes rather than just one: there are various relevant biomarkers, different aspects of quality of life, but also clinical outcomes such as treatment side effects, mortality, and so on.

Looking at each of these outcomes separately is a bit like looking at a puzzle, one piece at a time.

Unfortunately, that is what we’ve mostly been doing when using the traditional prediction models.

Joint Modelling

medication

AST

platelets

ALP

bilirubin

IBD

To make sound decisions, we must simultaneously consider and weigh all factors. We can create prediction models that account for multiple outcomes using the framework of joint models.

This approach makes sense because these outcomes are naturally interconnected. For instance, blood biomarkers and quality-of-life measures like pain, depression, and fatigue are interrelated.

By integrating these connections into our model, we enhance its accuracy and reliability.

Joint models also allow us to connect different types of outcomes.

For instance, we can link random effects models for continuous or categorical repeatedly measured variables with a model for a time-to-event outcome.

Using all of a patient’s past measurements, we can then estimate the patient’s risk of experiencing the event in the upcoming weeks, months or years.

And as before, these predictions are updated dynamically whenever new information becomes available and are patient-specific.

Modelling
the Patient’s
History

As a doctor, during a follow-up visit, you likely review the patient’s chart, prior lab measurements and patient-reported outcomes.

Why? Because it is essential to interpret the current data in the context of the past measurements.

Our dynamic prediction models inherently incorporate a patient’s history as they are trained on past measurements. But we can even go a step further and explicitly consider various aspects of a patient’s history.

In what I’ve shown you before when I was talking about joint modelling, you may have noticed that I indicated the hazard of the event being connected to the fitted trajectory of the biomarker and not the actually observed values.

Working with this fitted trajectory instead of the observed values has a number of advantages.

First, the estimated trajectory is known for all times.

The biomarker is only observed at a couple of times and standard models like the time-dependent Cox model, assume a step function which in most cases is not reasonable physiologically.

Second, the observed data are subject to short term variation and/or measurement error.

If you weigh yourself or take your blood pressure multiple times throughout the day, you will likely get varying measurements. Some of this variation can be explained; however, when it comes to longer-term predictions, …

… the more stable underlying value tends to be more predictive.

By using the estimated underlying trajectory we “filter out” the noise of the short term fluctuations, which, typically, leads to stronger associations.

Using the underlying trajectory, we can also calculate all sorts of different characteristics of the patient’s history.

In what we’ve seen so far, we assumed that the value of the biomarker at any time is related to other outcomes, for example the risk of an event, at the same time.

However, there may be a time lag in the effects. For instance, an increase in the bilirubin may not immediately cause itchiness; it may take a few days or weeks for symptoms to be come apparent.

In some instances, the rate of change can be predictive.

Rather than just considering a patient’s weight, we might find that the rate of weight loss is a better indicator of mortality risk.

Similarly, patients experiencing a rapid decline in quality of life may be more prone to deviate from their treatment plan than those with a gradual decline.

When considering toxic side-effects from treatment or other factors, the cumulative dose may be relevant.

We can include this in our joint model by calculating the area under the patient’s exposure level curve and connecting it to a time-to-event model.

And in that same context, it may be reasonable to assume that more recent exposure has a larger effect and we can add a weight to the cumulative effect to incorporate that.

Example

Surveillance of Prostate Cancer Patients

In our group, we are using joint modelling and dynamic prediction in several medical applications and I’d like to tell you a bit more about how we are using it for creating dynamic patient specific monitoring schedules for prostate cancer patients on AS.

Surveillance of Prostate Cancer Patients

These patients have been diagnosed with low grade cancer and are monitored regularly with biopsies to check if the cancer has progressed and an intervention is necessary.

To reduce the number of often unnecessary biopsies, we use repeated PSA measurements, which are a lot less invasive. Using a joint model, we can predict the risk of cancer progression.

Surveillance of Prostate Cancer Patients

Instead of having patients come in for the next biopsy according to a fixed schedule, we base the time of the next biopsy on the time when the predicted cumulative risk crosses a particular threshold.

For patients with a low risk, this may be a couple of years into the future.

Surveillance of Prostate Cancer Patients

For patients with higher or increasing PSA values, the estimated cumulative risk is increasing faster and will cross this threshold a lot sooner, resulting in the next biopsy being scheduled earlier.

Personalized Biopsy Schedules: Challenges

• Interval censoring

There are a couple of challenges that we had to tackle in our model.

First, the time of cancer progression is not directly observed. When a biopsy shows progression, we only know that it happened in the interval since the previous biopsy. Our event outcome is interval censored.

• Competing risk

Second, some patients start treatment even before their cancer has progressed and leave the active surveillance program. This constitutes a competing risk and has to be taken into account to avoid bias.

• Biopsy sensitivity < 100%

And third, biopsies aren’t perfect. They may result in false-negative results. This means that when we observe cancer progression, we cannot actually be sure that it happened in the previous interval, but it could have happened earlier and was just missed by the other biopsies.

Personalized Biopsy Schedules: Features

• Dynamic prediction of cancer progression risk

Besides the dynamic prediction of the cancer progression risk, our model has a couple of additional features.

• Multiple predictors, measured at different times

Besides the PSA, we actually use the core ratio, that is the proportion of cores used in the biopsy with cancerous cells, as a second predictor. But because this is only available when a biopsy is performed, those measurements are generally less frequent than the PSA measurements and they may be done at different times.

• Creation of a entire (tentative) biopsy schedule

Moreover, we can also obtain a whole schedule for future biopsies, by re-setting the risk to zero at the time of the planned biopsy and repeating the procedure.

• Patient- & time-specific risk thresholds
  ⇨ Minimize number of biopsies & detection delay

And we even made the risk threshold patient and time-dependent. It is chosen to minimize the expected number of biopsies the patient will undergo while limiting the expected delay in detection of cancer progression.

Possible Applications

Optimal patient-specific timing:

• time of the outcome measurement
• time of biomarker measurements
• time of an intervention

“What-if” analyses:

• expected outcomes under different (treatment) scenarios
• estimate causal effects

From Theory to Practice

Of course, any methodological advancement is only useful when it can be implemented in practice.

Similar to the existing online calculators, we can create interactive dashboards that bring up-to-date, patient-specific dynamic predictions into clinical practice and consultation rooms.

And translating the risk estimates into other, more praxis-oriented quantities, like the expected number of biopsies or intervals between visits, helps to communicate this risk to patients and facilitates shared decision-making.

In Summary: Dynamic Prediction …

… makes better use of the available information.

• Time as additional dimension of the data
• Model relevant features of the biomarker trajectories
• Filter out short term fluctuations and measurement error
• Patient-specific predictions
• Model complex & time-dependent structures between multiple markers and outcomes

In summary, dynamic prediction makes better use of the available data.

By working with repeated measurements we add a dimension to the data.

This allows us to

• model and incorporate the relevant features of the biomarker trajectories,
• filter out short term fluctuations and measurement error,
• obtain patient-specific predictions, and
• to model the complex and time-dependent structures that exist between multiple markers and outcomes.

Thank You for Your Attention