paperKB
coga / coga-kb
Help
Sign in

Chunk #25 — Results — Strength and consistency of overall model predictions

Source
Post-traumatic stress disorder associated with natural and human-made disasters in the World Mental Health Surveys.
Embedded
yes

Text

Although the small sample size precluded estimating model coefficients separately in each survey, we could compare overall model fit in subsamples by calculating individual-level predicted probabilities from model 4 with 20 replicates of 10-fold cross-validation, estimating subsample ROC curves from these predicted probabilities, and calculating AUC based on these curves. Estimated AUC based on 20 replicates of 10-fold cross-validated predictions was 0.63 in the total sample and 0.48–0.75 in subsamples defined by respondent’s sex, age, and education. These are weak to intermediate levels of overall classification accuracy (Roemer et al. 1998). However, the 5% of respondents with highest predicted probabilities of PTSD included a substantial proportion (44.5%) of all disaster-related PTSD (sensitivity) in the total sample. This is nine times the concentration of risk expected by chance (Table 4). Subgroup sensitivities among this 5% of respondents with highest predicted risk ranged from 56.4% among men to 22.4% among respondents with low-average/low education. Positive predictive value (the proportion of predicted positives who met criteria for PTSD) among the 5% of respondents with higher predicted risk was 20.4% in the total sample and between 39.5% among respondents with high-average/high education to 3.9% among respondents with low-average/low education (Fig. 1).