RANE - Rapid Assessment of Need for Evidence

Release version 1.0 25th July 2018

How to use this app

This is an R Shiny App which facilitates calculations of the value of research proposals in a timely manner. The inputs required in the app represent the minimum needed to understand the consequences of uncertainty and the need for further research. Full details of the approach used and applied examples using these methods are forthcoming. In the meantime click here for further details.

Users unfamiliar with value of information methods are encouraged to read the information in the 'How to estimate research value' tab. This section describes the value of information approach and how it applies to research funding in a resource constrained health care system.

To carry out an analysis click the 'Inputs' tab

This code has been produced under a GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007. This application comes with no warranty and should be utilised at the user's own risk.

Principles of research prioritisation

The following section outlines the principles of the assessments required when considering the need for additional evidence and the priority of proposed research. These assessments help inform the two questions which must be answered when considering whether to prioritize and commission research:

Are the expected health benefits of additional evidence sufficient to regard the research proposal as potentially worthwhile?
Should the proposal be prioritized over other research topics that could have been commissioned with the same resources?

Are the expected health benefits of additional evidence sufficient to regard the research proposal as potentially worthwhile?

How can health outcomes be improved?

Health outcomes can be improved by conducting research or implementing the findings of existing research. In order to understand the value of additional evidence it is necessary to distinguish between these two very different ways to improve health outcomes. It is also necessary to take account of the costs associated with research projects and the fact that minimum changes in outcomes may need to be observed before clinical practice will change.

How does conducting research improve health outcomes?

Additional evidence is valuable because it can improve patient outcomes by resolving existing uncertainty about the effectiveness of the interventions available. This helps inform treatment decisions for subsequent patients. For example, based on the balance of existing evidence a clinician might judge that a particular intervention is the most effective option, but there will be a chance that the other alternative interventions are in fact more effective. Therefore, when the existing evidence is uncertain there is a chance that one of the alternative interventions would have improved health outcomes to a greater extent. This means that there are adverse health consequences associated with uncertainty. A judgement about the level of uncertainty can come from a systematic review and meta-analysis, expert elicitation, extrapolation, meta-epidemiological study, or a combination of these sources. The level of uncertainty in the decision arises from the range of plausible values that the outcome can take. This is represented by the confidence interval (CI) or standard error around the estimate. A wide CI implies a large amount of uncertainty.

As an example, consider the evidence on the use of corticosteroids following traumatic brain injury (TBI) before the large definitive trial of CRASH. Before CRASH, a meta-analysis of 19 randomised controlled trials indicated that the effects of corticosteroids on death and disability were unclear. Taking death as an endpoint, we can start to understand the consequences of the uncertainty on mortality. The odds ratio for death was 0.93 in favour of the use of corticosteroids but with 95% CI from 0.71 to 1.18. This uncertainty means that every decision about the use of corticosteroids following TBI may not have been the most effective choice. In this case, there was a 74% chance that corticosteroids were effective and improved survival. However, there was a 26% chance that corticosteroids resulted in excess deaths per annum. This uncertainty can be translated into the consequences for patient outcomes, i.e. number of expected deaths per annum due to uncertainty, by combining the uncertain relative effect with an estimate of the baseline risk (derived from the control arms of the trials or from an alternative source) and multiplying by the incidence of TBI per year.

These expected health consequences can be interpreted as an estimate of the health benefits that could be gained each year if the uncertainty surrounding treatment choice could be resolved, i.e., it provides an expected upper bound on the health benefits of further research which would confirm whether corticosteroids following TBI increase deaths or reduce them. In effect, this is the value of reducing the 95% confidence interval to a single point. These potential expected benefits increase with the size of the patient population whose treatment choice can be informed by additional evidence and the time over which evidence about the effectiveness of these interventions is expected to be useful.

How does implementing the findings of existing evidence improve health outcomes?

In addition to funding research, it is also possible to improve health outcomes by ensuring that the treatment option that is expected to be best based on the findings of existing evidence is implemented into clinical practice. In fact, the improvements in health outcomes from implementing the findings of the current evidence base (implementation value) may be greater than the potential improvements in health outcomes through conducting further research.

What change in the primary endpoint is required to change practice?

The health benefits of conducting further research will only improve patient outcomes if the findings of the research change clinical practice. Again, it is important to recognise that there are many ways to influence implementation other than by conducting more research. However, concerns about implementation might influence research priority and the design of research. For example, if it is very unlikely that the findings of research will affect clinical practice and other mechanisms are unlikely to be effective at changing practice, then another area of research might be prioritised even though the expected benefits are smaller. Furthermore, if the research must demonstrate highly statistically significant results to be implemented this will influence design, cost and time taken for research to report. In some cases, larger clinical differences in effectiveness may be required before research would have an impact on practice. This will tend to reduce the potential benefits of further research as larger differences are less likely to be observed than smaller ones. The change in the primary endpoint required is called the minimum clinical difference (MCD)

Research costs imposed on the health system

Carrying out research consumes resources in the general health system e.g. doctors, nurses, and pharmacists whose time commitments are moved away from general patient care and reallocated to research projects. The general health budget may also bear the costs of administrative staff and equipment that is needed to carry out the research. Another important type of cost that is often borne by the health budget is the acquisition costs of the health technologies under investigation (e.g. drug or device costs). These costs must be taken into account to comprehensively understand the expected health benefits of research projects.

Should the research proposal be prioritized over other research topics that could have been commissioned with the same resources?

Since research funding bodies have limited resources, not all research proposals can be funded and so the benefits of some research projects must be foregone in order to fund others. Quantitative estimates of the health benefits of research projects are required to compare the benefits of funded research to the foregone benefits of research not funded. These assessments can help to inform which research projects represent “best buys” for the research funder. They allow decision makers to compare value across proposals make explicit judgements about the trade-offs between different outcomes. If the potential health benefits of research are not in generic health outcomes (such as QALYs), implicit extrapolations are required to link changes in primary outcomes to compare health outcomes.

Inputs required to estimate research value

Step 1: Primary outcome

Type of primary endpoint

The primary outcome measure or endpoint captures the most important aspects of health outcome in the research.

Express results in natural outcomes (e.g. heart attacks avoided) or in QALYs?

The benefits of research can be expressed in either natural outcomes or in Quality Adjusted Life Years (QALYs). Using QALYs requires more inputs but enables a comparison of the health benefits of further research and implementation efforts across diverse clinical areas.

Is the outcome a benefit (e.g. cure) or a harm (e.g. heart attack)?

For natural outcomes: the value of additional research is expressed in terms of ‘benefits gained’ or ‘harms avoided’ depending on whether this outcome is a benefit or harm.

Name of outcome e.g. heart attack

For natural outcomes: this will be used in reporting results.

Do the treatment costs depend on the primary outcome?

For binary outcomes: in some cases treatment costs will depend on whether the primary outcome occurs or not. For example, if a treatment is used to prevent disease progression then it will cease to be used (and its cost will no longer be incurred) if the individual progresses.

Number of possible states if the primary outcome does / does not occur (4 maximum)

For binary outcomes: the primary endpoint may be a scale or a composite outcome which is composed of a number of health states. If there are different levels of health and costs associated with these health states then these can be considered explicitly here. For example, if primary outcome is a composite of heart attack and stroke then there are two possible states if the primary outcome occurs and the costs and health consequences associated with these states must be explicitly considered.

Conditional on the primary outcome occurring, what is the probability of being in this state?

For binary outcomes: if the primary outcome is composed of distinct health states, different proportions of individuals may be expected to enter these health states. For example, if primary outcome is a composite of heart attack and stroke then conditional on the primary outcome occurring 40% and 60% of these individuals may be expected to experience heart attack and stroke respectively.

Patient time horizon / time in this state (years)?

For binary outcomes: if differential survival is considered then represents the expected survival time associated with each state. Otherwise this represents the patient time horizon considered for the decision i.e. how far into the future individual patient outcomes are modelled.

What is the health utility associated with this state?

For binary outcomes: this is a number which represents the health related quality of life associated with a state.

What are the disease related costs associated with this state?

For binary outcomes: these are the costs associated with a particular disease state, they do not include the costs of the treatment under consideration.

What is the health utility associated with the pre-transition health state?

For survival outcomes: this is a number which represents the health related quality of life associated with the pre-transition state.

What are the expected monthly disease related costs associated with the pre-transition health state?

For survival outcomes: in the same manner as for health utility changes in the expected survival must be linked to changes in disease related costs. These are the costs associated with a particular disease state, they do not include the costs of the treatment under consideration.

By how much is a one unit increase in the primary outcome expected to increase/decrease the health state utility?

For continuous outcomes: the effect of changes from baseline on changes in health related quality of life (utility) will depend on the severity of the disease and range of the outcome measure. 'Mapping' studies which use statistical methods to estimate the effect of a unit change in a natural outcome on utility provide this link.

By how much is a one unit increase in the primary outcome expected to increase/decrease monthly disease related costs?

For continuous outcomes: changes in the primary outcome may also be expected to result in changes in disease related costs. These are the costs associated with a particular disease state, they do not include the costs of the treatment under consideration.

How long is the treatment effect expected to last? (months)

For continuous outcomes: the scale of the health gains and disease related costs associated with changes in the primary outcome will depend on the expected treatment effect duration. Estimates of treatment effect duration exist for few outcomes so in practice with will require expert opinion to inform this.

Step 2: Interventions

How many treatment options are under consideration? (Maximum of 4)

There may be a number of relevant treatment options for a given indication. This app currently allows for up to 4 options to be considered.

Current level of utilisation (%)

Some estimate of the current level of utilisation of the interventions in clinical practice is required to establish the value of changing practice if the results of new research suggest a change. It can also be used to establish whether there is greater value from encouraging the implementation of what existing evidence suggests is the most effective intervention rather than conducting new research. The utilisation of all treatments must sum to 100%.

Choose method of entering baseline probability of outcome

An estimate of event rate with the baseline treatment is required. This is used to obtain an estimate of the absolute effect of the interventions on the primary outcome by applying the relative measure of effect to the baseline risk. There are two options for entering this data. 1) Upper and lower 95% range: this may come from discussion with an expert and/or from a confidence interval reported in the literature. 2) Number of events and number at risk: this may come from an observational study or control arm of an RCT

Scale for relative effect

An estimate of the relative effectiveness of the intervention is required for the primary outcome, along with an estimate of its uncertainty. This can be expressed with a 95% confidence interval in terms of an odds ratio (binary), relative risk (binary), risk difference (binary), hazard ratio (survival) or mean difference (continuous).

Minimum clinical difference (MCD)

Specifying a MCD required to change clinical practice is one way to incorporate differences in costs, adverse events or other considerations which are not captured in the primary endpoint. The MCD for a particular treatment is always defined relative to a current standard of care (baseline treatment). The MCD is defined as the 'the minimum improvement in the primary endpoint which is required for the new treatment to be considered worthwhile'. For example, a larger MCD may be required if the new treatment is more expensive than the current standard of care. How the MCD is expressed differs between binary, continuous and survival endpoints.

Binary: Absolute change in probability of primary endpoint e.g. an MCD of 0.02 implies that the probability of death must decrease by at least 2% for the new treatment to be worthwhile relative to the current standard of care.
Continuous: Natural units of primary endpoint e.g. an MCD of 2 implies that the Mini-Mental State Examination (MMSE) must be 2 points higher for patients given the new treatment for it to be considered worthwhile relative to the current standard of care.
Survival: Months of survival in the origin state e.g. an MCD of 1 implies that the new treatment must be associated with an expected increase in progression free survival relative to the current standard of care for it to be considered worthwhile.

Treatment costs over patient time horizon

For binary outcomes: here Treatment costs are assumed to be the same for all individuals treated; regardless of health outcomes (see below). If treatment costs accrue over multiple years they should be discounted to present value.

Treatment costs over patient time horizon if the primary outcome occurs / does not occur

For binary outcomes: in some cases treatment costs will depend on the primary outcome, for example; intensive preventative treatment may be administered continuously until an event occurs (e.g. heart attack). Once the event has occurred the preventative treatment is halted and these treatment costs are no longer incurred.

Treatment costs per month

For continuous or survival outcomes: treatment costs incurred each month.

Are individuals always treated until progression under this treatment?

For survival outcomes individuals may be treated until progression or there may be a maximum duration of treatment.

Step 3: Proposed research

Type of research

The value of research can be calculated for either full research (e.g a randomised controlled trial (RCT)) or feasibility studies. The inputs required for the analysis will depend on the type of study chosen.

Full research: In contrast to feasibility studies which facilitate full research this type of research aims to address clinical questions directly. For example RCTs which aim to reduce uncertainty about relative effects.
Feasibility study: If there are uncertainties about whether a full trial is possible, a short feasibility study can be carried out to assess the possibility of future full research. If the feasibility study is successful, researchers have the option to carry out the follow up trial.

Probability of feasibility research leading to follow-up study

For feasibility studies: the motivation is that there is uncertainty about whether a full trial is possible. If the feasibility study shows that the full trial is not possible, the research budget spent on the feasibility study will have no impact on health outcomes. For this reason, the likelihood of a feasibility study leading to the full trial is an important determinant of its value.

Expected duration of research

Some assessment of the duration of time for the proposed research to be conducted and for the results of the research to report is required since the value of research declines the longer it takes to report. This might be informed by an assessment of sample size, recruitment rates, or historical experience from conducting similar types of studies. For feasibility studies: estimates of duration for both the feasibility study and the full trial are required.

Costs of the research to funder

These costs are the costs of research which are directly borne by the research funder. For feasibility studies: estimates of research funder costs for both the feasibility study and the full trial are required.

Costs of the research imposed on health system

These costs are the costs of research which fall on the general health system. Carrying out research consumes valuable resources from the general health care budget e.g. additional treatment costs and health professionals whose time commitments are moved away from general patient care and reallocated to research projects. For feasibility studies: estimates of research health system costs for both the feasibility study and the full trial are required.

Length of time for which the new evidence would be valuable

The information generated by new research will not be valuable indefinitely because other changes occur over time. For example, over time new and more effective interventions become available which will eventually make those currently available obsolete. This means that new information about effectiveness is only relevant for a specific amount of time. A judgement about the length of time that the evidence from the proposed research might be valuable is required to estimate the expected benefits over an appropriate time horizon.

Discount rate

Discounting should be used to reflect the fact that resources committed today could be invested at a real rate of return to free up more resources in the future.

Incidence per annum

An estimate of the number of individuals facing the uncertain choice between alternative interventions is required in order to establish the size of the benefits to the target population.

Opportunity cost of health system expenditure

Increasing treatment costs will be associated with health opportunity costs. These are the improvement in health that would have been possible if any additional resources required had, instead, been made available for other health care activities.

Select appropriate values and then proceed to the 'Step 2' tab

See the 'Input information' tab for detail on how to interpret the inputs

Primary outcome measure

Type of primary endpoint

Express results in natural outcomes or in Quality Adjusted Life Years (QALYs)?

Is the outcome a benefit or a harm?

Name of outcome

Do the treatment costs depend on the primary outcome?

Yes

Number of possible states if the primary outcome occurs (4 maximum)

Number of possible states if the primary outcome does not occur (4 maximum)

Outcomes if primary endpoint occurs

State 1.1

Conditional on the primary outcome occurring, what is the probability of being in this state?

Patient time horizon / time in this state (years)?

What is the health utility associated with this state?

What are the disease related costs associated with this state?

Is an increase in the primary outcome expected to be associated with an increase or decrease in health state utility?

By how much is a one unit increase in the primary outcome expected to increase/decrease the health state utility?

What is the health utility associated with the pre-transition health state?

State 1.2

Conditional on the primary outcome occurring, what is the probability of being in this state?

Patient time horizon / time in this state (years)?

What is the health utility associated with this state?

What are the disease related costs associated with this state?

Is an increase in the primary outcome expected to be associated with disease related cost increases or decreases?

By how much is a one unit increase in the primary outcome expected to increase/decrease monthly disease related costs?

What are the expected monthly disease related costs associated with the pre-transition health state?

State 1.3

Conditional on the primary outcome occurring, what is the probability of being in this state?

Patient time horizon / time in this state (years)?

What is the health utility associated with this state?

What are the disease related costs associated with this state?

How long is the treatment effect expected to last? (months)

State 1.4

Conditional on the primary outcome occurring, what is the probability of being in this state?

Patient time horizon / time in this state (years)?

What is the health utility associated with this state?

What are the disease related costs associated with this state?

Outcomes if primary endpoint does not occur

State 2.1

Conditional on the primary outcome occurring, what is the probability of being in this state?

Patient time horizon / time in this state (years)?

What is the health utility associated with this state?

What are the disease related costs associated with this state?

State 2.2

Conditional on the primary outcome occurring, what is the probability of being in this state?

Patient time horizon / time in this state (years)?

What is the health utility associated with this state?

What are the disease related costs associated with this state?

State 2.3

Conditional on the primary outcome occurring, what is the probability of being in this state?

Patient time horizon / time in this state (years)?

What is the health utility associated with this state?

What are the disease related costs associated with this state?

State 2.4

Conditional on the primary outcome occurring, what is the probability of being in this state?

Patient time horizon / time in this state (years)?

What is the health utility associated with this state?

What are the disease related costs associated with this state?

Select appropriate values and then proceed to the 'Step 3' tab

See the 'Input information' tab for detail on how to interpret the inputs

How many treatment options are under consideration? (Maximum of 4)

Baseline treatment

Name of baseline treatment

Current level of utilisation (%)

Treatment costs over patient time horizon

Treatment costs over patient time horizon if the primary outcome occurs

Treatment costs over patient time horizon if the primary outcome does not occur

Treatment costs per month

Are individuals always treated until progression under this treatment?

What is the maximum number of months that this treatment will be given?

Intervention 1

Name of intervention

Current level of utilisation (%)

Treatment costs over patient time horizon

Treatment costs over patient time horizon if the primary outcome occurs

Treatment costs over patient time horizon if the primary outcome does not occur

Treatment costs per month

Are individuals always treated until progression under this treatment?

What is the maximum number of months that this treatment will be given?

Intervention 2

Name of intervention

Current level of utilisation (%)

Treatment costs over patient time horizon

Treatment costs over patient time horizon if the primary outcome occurs

Treatment costs over patient time horizon if the primary outcome does not occur

Treatment costs per month

Are individuals always treated until progression under this treatment?

What is the maximum number of months that this treatment will be given?

Intervention 3

Name of intervention

Current level of utilisation (%)

Treatment costs over patient time horizon

Treatment costs over patient time horizon if the primary outcome occurs

Treatment costs over patient time horizon if the primary outcome does not occur

Treatment costs per month

Are individuals always treated until progression under this treatment?

What is the maximum number of months that this treatment will be given?

Baseline treatment

Choose method of entering baseline probability of outcome

Number of events

Number at risk

Select a plausible 95% range for the baseline probability of outcome

No baseline input required for continuous outcomes

Type of survival distribution

Lambda (scale) parameter for baseline treatment (natural scale)

Gamma (shape) parameter for baseline treatment (natural scale)

Intervention 1

Scale for relative effect

Compared to the baseline, is it expected that this intervention is either always better, always worse, or is there uncertainty about which is better on the primary outcome?

Select a plausible 95% range for the odds ratio. The lower bound is set to 1

Select a plausible 95% range for the risk ratio. The lower bound is set to 1

Select a plausible 95% range for the risk difference. The lower bound is set to 0

Select a plausible 95% range for the odds ratio. The upper bound is set to 1

Select a plausible 95% range for the risk ratio. The upper bound is set to 1

Select a plausible 95% range for the risk difference. The upper bound is set to 0%

Select a plausible 95% range for the odds ratio

Select a plausible 95% range for the risk ratio

Select a plausible 95% range for the risk difference

Choose method of entering mean difference

Expected mean difference

Standard error for mean difference

Select a plausible 95% range for the mean difference

Compared to the baseline, is it expected that this intervention is either always better, always worse, or is there uncertainty about which is better on the primary outcome?

Select a plausible 95% range for the mean difference. The lower bound is set to 0

Select a plausible 95% range for the mean difference. The upper bound is set to 0

Select a plausible 95% range for the hazard ratio

Compared to the baseline, is it expected that this intervention is either always better, always worse, or is there uncertainty about which is better on the primary outcome?

Select a plausible 95% range for the hazard ratio. The lower bound is set to 1

Select a plausible 95% range for the hazard ratio. The upper bound is set to 1

Minimum clinical difference (MCD)

Intervention 2

Scale for relative effect

Select a plausible 95% range for the odds ratio

Select a plausible 95% range for the risk ratio

Select a plausible 95% range for the risk difference

Compared to the baseline, is it expected that this intervention is either always better, always worse, or is there uncertainty about which is better on the primary outcome?

Select a plausible 95% range for the odds ratio. The lower bound is set to 1

Select a plausible 95% range for the risk ratio. The lower bound is set to 1

Select a plausible 95% range for the risk difference. The lower bound is set to 0

Select a plausible 95% range for the odds ratio. The upper bound is set to 1

Select a plausible 95% range for the risk ratio. The upper bound is set to 1

Select a plausible 95% range for the risk difference. The upper bound is set to 0%

Choose method of entering mean difference

Expected mean difference

Standard error for mean difference

Select a plausible 95% range for the mean difference

Compared to the baseline, is it expected that this intervention is either always better, always worse, or is there uncertainty about which is better on the primary outcome?

Select a plausible 95% range for the mean difference. The lower bound is set to 0

Select a plausible 95% range for the mean difference. The upper bound is set to 0

Select a plausible 95% range for the hazard ratio

Compared to the baseline, is it expected that this intervention is either always better, always worse, or is there uncertainty about which is better on the primary outcome?

Select a plausible 95% range for the hazard ratio. The lower bound is set to 1

Select a plausible 95% range for the hazard ratio. The upper bound is set to 1

Minimum clinical difference (MCD)

Intervention 3

Scale for relative effect

Select a plausible 95% range for the odds ratio

Select a plausible 95% range for the risk ratio

Select a plausible 95% range for the risk difference

Compared to the baseline, is it expected that this intervention is either always better, always worse, or is there uncertainty about which is better on the primary outcome?

Select a plausible 95% range for the odds ratio. The lower bound is set to 1

Select a plausible 95% range for the risk ratio. The lower bound is set to 1

Select a plausible 95% range for the risk difference. The lower bound is set to 0

Select a plausible 95% range for the odds ratio. The upper bound is set to 1

Select a plausible 95% range for the risk ratio. The upper bound is set to 1

Select a plausible 95% range for the risk difference. The upper bound is set to 0%

Choose method of entering mean difference

Expected mean difference

Standard error for mean difference

Select a plausible 95% range for the mean difference

Compared to the baseline, is it expected that this intervention is either always better, always worse, or is there uncertainty about which is better on the primary outcome?

Select a plausible 95% range for the mean difference. The lower bound is set to 0

Select a plausible 95% range for the mean difference. The upper bound is set to 0

Select a plausible 95% range for the hazard ratio

Compared to the baseline, is it expected that this intervention is either always better, always worse, or is there uncertainty about which is better on the primary outcome?

Select a plausible 95% range for the hazard ratio. The lower bound is set to 1

Select a plausible 95% range for the hazard ratio. The upper bound is set to 1

Minimum clinical difference (MCD)

Select appropriate values and then press 'Run analysis'

See the 'Input information' tab for detail on how to interpret the inputs

Proposed research

Type of research

Expected duration of research (years)

Cost of research to funder

Costs of research imposed on health system

Probability of feasibility research leading to follow-up study

Expected duration of feasibility research (years)

Expected duration of follow-up research (years)

Costs of feasibility research to funder

Costs of follow-up research to funder

Costs of feasibility research imposed on health system

Costs of follow-up research imposed on health system

Time over which evidence would be valuable (years)

Other inputs

Discount rate (%)

Incidence per annum

Currency used in analysis

Calculate the value of reconsidering the evidence?

Yes

Opportunity cost of health system expenditure

Click once and go to the Results tab

Click once, and go to the Results tab. The calculation to reconsider the evidence can take up to 10 minutes to report.

Headline results

What is the value of changing practice based on what we currently know about the treatments?

The table below displays the expected outcomes for each treatment:

The table below displays the expected treatment costs associated with each treatment option:

What are the expected health consequences of the remaining uncertainty?

The average over this range of outcomes provides an estimate of the health benefits that could potentially be gained each year if the uncertainty in the decision could be resolved.

What is the value of the proposed research?

The value of the feasibility trial depends on the health effect of the full trial and the likelihood that the full trial will be commissioned and report.

Health impact of full trial

As the potential full trial is expected to have no health impact, there is no value in the proposed feasibility trial.

Value of feasibility research

Whether this research represents good value to the health system depends on the value of the other potential uses of these resources.

Due to the opportunity costs imposed on the health system by this research, this research design is expected to have negative net health consequences.

About this software

This app was developed using R Shiny by David Glynn supervised by Claire Rothery and Karl Claxton. This research was funded by the UK National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme as project number 16/29/01 and through a PhD studentship to David Glynn, funded by the Centre for Health Economics, University of York. The views expressed in this application are those of the authors and not necessarily those of the NIHR HTA Programme. Any errors are the responsibility of the authors.

The source code is available on GitHub at https://github.com/david-glynn.