Chapter 14 Sensitivity analyses
An important step of any Bayesian analysis is to evaluate whether the results are strongly affected by the choice of priors and/or to the assumptions made.
14.1 Choice of LCM
Regarding the sensitivity of our results to the assumptions made, this could be assessed by comparing the results from the LCM where a given assumption was made vs. the ones from a LCM where the assumption was relaxed. Of course, to fully “isolate” the effect of a given assumption, the same priors would have to be used in both LCM. This is, however, not always possible since the “relaxed” LCM will often need a larger number of informative priors to make it identifiable.
In the end, providing the results from an “alternative” LCM could help the readers understand what was the influence of the model chosen on the presented results. As author, you would probably want to use the LCM that is the most “biologically relevant” as your main model.
14.2 Perturbing informative priors
Whenever informative priors are used in a LCM, the sensitivity of the LCM to the choice of priors should be assessed. This can be done by running the same LCM, but with “perturbed” and usually “more diffuse” priors. Johnson et al. (2019) recommended, for instance, to move the mode of the prior distribution of a given parameter by a few percentage-points and to increase the width of its prior distribution. Then, the results from the LCM using the scientific literature-derived priors can be compared to those of the LCM using perturbed priors to assess how sensitive to the choice of priors the results are. The amount and direction by which the mode will be moved and the level of increase of the width of the distribution will depend on the problem at hand.
For instance, in the preceding sections we used the Romano et al. (2006) results, to inform the priors for the Se and Sp of a US exam conducted between 28-45 days post-insemination:
-Se = 0.90 (95CI: 0.85, 0.95)
-Sp = 0.95 (95CI: 0.90, 0.97)
We thus came up with these prior distributions for US’ Se and sp.
Thus, a possibility could be to move down the Se and Sp modes by 10 percentage points (i.e., mode for Se:0.80 and mode for Sp:0.85) and to increase the with of the distribution by moving the 2.5th percentiles by 30 percentage-points (Se 2.5th percentile: 0.55; Sp 2.5th percentile: 0.60). Thus:
Then, the results from the LCM with the informative and perturbed priors could be compared.
For instance, in exercise 3 we compared two conditionally independent diagnostic tests (PAG and US) in three populations (herd #1, herd #2, and herd #3). For that model, though the model is identifiable, we could use the Romano et al. (2006) informative priors on US’ Se and Sp. Then, we could run the same LCM, but with the perturbed US’ Se and Sp priors suggested above. If we do that, we would obtain the following results:
Table. Results from a 2 tests, 3 populations LCM using different priors (literature-based vs. perturbed informative priors on US’ Se and Sp). The priors differing between LCM are indicated in bold.
| Parameter | Romano et al. 2006 priors | Perturbed priors | |||
|---|---|---|---|---|---|
| Prior | Posterior | Prior | Posterior | ||
| Prev1 | Beta(1.0, 1.0) | 0.558 (0.496, 0.618) | Beta (1.0, 1.0) | 0.556 (0.494, 0.617) | |
| Prev2 | Beta(1.0, 1.0) | 0.523 (0.440, 0.604) | Beta (1.0, 1.0) | 0.520 (0.435, 0.603) | |
| Prev3 | Beta(1.0, 1.0) | 0.559 (0.452, 0.662) | Beta (1.0, 1.0) | 0.557 (0.449, 0.657) | |
| Se US | Beta (100, 12) | 0.925 (0.893, 0.954) | Beta (13.6, 4.1) | 0.931 (0.890, 0.970) | |
| Sp US | Beta (100, 6.2) | 0.977 (0.956, 0.990) | Beta (14.0, 3.3) | 0.981 (0.957, 0.994) | |
| Se PAG | Beta (1.0, 1.0) | 0.996 (0.981, 1.00) | Beta (1.0, 1.0) | 0.996 (0.980, 1.00) | |
| Sp PAG | Beta (1.0, 1.0) | 0.981 (0.934, 0.999) | Beta (1.0, 1.0) | 0.977 (0.922, 0.999) |
In this specific example, we can see that the results are virtually unchanged. Thus, we could say that our initial analysis was quite robust to the choice of prior distributions. Personally, we would present the LCM with the Romano et al. (2006) priors as our “main” model. If we do have valid information from the literature about one or many of the unknown parameters, then why not using them?