Once the final model was estimated, a predicted probability of PTSD was generated for each respondent from model coefficients. A receiver-operating characteristic (ROC) curve was calculated from these predicted probabilities (Zou et al. 2007) and area under the ROC curve (AUC) was calculated to quantify overall prediction accuracy (Hanley & McNeil, 1983). We then evaluated sensitivity and positive predictive value among the 5% of respondents with highest predicted probabilities to determine how well the model implies that subsequent PTSD could be predicted if the model was applied in the immediate aftermath of a future disaster. Sensitivity was the proportion of observed PTSD cases found among the 5% of respondents with highest predicted probabilities. Positive predictive value was the prevalence of PTSD among this 5% of respondents. We used the method of replicated 10-fold cross-validation with 20 replicates (i.e. 200 separate estimates of model coefficients) to correct for the over-estimation of prediction accuracy when both estimating and evaluating model fit in a single small sample (Smith et al. 2014).
