Alert - This a masked draft version for internal use only
3. Main Analyses Country Comparison
Load Data
Code
# load data for analysisload("../data/wrangled_data/dt_ana_full.RData")
Analysis Strategy
We evaluate whether the predictive structure for violent intent differs across samples (countries; with a focal interest in the Jihadist sample) using two complementary strategies:
a multilevel model that separates within- from between-country effects and, in a second step, allows country-specific slopes;
a two-step meta-analysis that fits the same model within each country, then pools coefficient estimates and tests heterogeneity.
Why both?
The multilevel model performs one-step partial pooling (stabilizing noisy country estimates via shrinkage), while the meta-analysis gives two-step, fully transparent country-by-country estimates with classic heterogeneity statistics (Q, τ², I²). Agreement between the two strengthens credibility.
Prepare Data
We use group-mean centering (suffix _cw) for individual-level predictors to estimate within-country relationships, and include country means (grand-mean centered) to estimate between-country differences. This avoids conflating contextual differences with individual-level effects.
This model allows country-specific baselines for violent intent while estimating common within-country slopes for the centered predictors and between-country effects for country means.
We now allow country-specific slopes for the individual-level predictors. This addresses the question: do predictors of violent intent vary across samples? If so, we should see non-zero variance in the random slopes and improved model fit.
mod_ml_ri_ml <-update(mod_ml_ri, REML =FALSE)mod_ml_rs_ml <-update(mod_ml_rs, REML =FALSE)model_comp <-anova(mod_ml_ri_ml, mod_ml_rs_ml) # LRT for added random slopesdf_model_comp <-as.data.frame(model_comp)df_model_comp %>%kable(., caption ="Comparing fixed and random slope model") %>%kable_styling(full_width = F, latex_options =c("hold_position", "scale-down"))
Comparing fixed and random slope model
npar
AIC
BIC
logLik
-2*log(L)
Chisq
Df
Pr(>Chisq)
mod_ml_ri_ml
9
33896.46
33961.70
-16939.23
33878.46
NA
NA
NA
mod_ml_rs_ml
18
33594.34
33724.83
-16779.17
33558.34
320.1144
9
0
Model fit improves drasticially with random slopes (not suprisingly given the many different countries). We now assess how the Jihadist sample differs from the other samples.
The following shrinkage (partial-pooling) contrasts, for each country and each predictor, the stand-alone country OLS (orinary least squares) slope with the multilevel Best Linear Unbiased Prediction (BLUP) slope which comes from the random-slopes model.
Hollow circle (OLS): the slope estimated by fitting a separate linear model within that country (no pooling).
Filled circle (BLUP): the country’s slope implied by the multilevel model = fixed effect + random deviation (i.e., after partial pooling).
Segment connecting them: the amount and direction of shrinkage from OLS → BLUP.
Long segments = noisier country estimates (often small n) get pulled more toward the overall mean slope.
Short segments = precise country estimates change little.
OLS
For each country, we fit a simple regression model just on that country’s data. The slope we get is the country-specific OLS slope.
Properties:
Uses only that country’s sample.
Very unbiased, but high variance if the country has a small sample size (noisy, unstable).
BLUB
The multilevel model’s estimate of a country’s slope = the global fixed slope plus that country’s random deviation.
Properties:
Shrinks noisy country slopes toward the overall mean slope (partial pooling).
Countries with large N or strong signal stay close to their OLS.
Countries with small N or noisy estimates get pulled toward the overall slope.
To compare effects across countries transparently, we refitted the same country-level linear model using grand-mean standardized predictors (suffix _gmz), extracted per-country coefficients and standard errors. We now meta-analyze them with a random-effects model (REML). This yields an overall average effect and heterogeneity statistics (Q, τ², I²).
Code
meta_results <- country_effects %>% tidyr::nest(data =c(country, estimate, std.error)) %>%# one row per term dplyr::mutate(.results = purrr::map(data, ~ metafor::rma(yi = .x$estimate,sei = .x$std.error,method ="REML",slab = .x$country )) ) %>% dplyr::select(term, .results)
Code
# 4) Forest + Baujat (meta-analytic echo), adapted to generic slopes/SEs# Helper: draw ggplot-style forest make_forest_generic <-function(rma_obj,main_title ="Meta-analysis",order_by =NULL,show_prediction_interval =TRUE,study_labels =NULL,highlight ="Jihadist", # <- NEW: what to highlight (character pattern(s) or logical vector)highlight_color ="red") { # <- NEW: highlight color# robust SE: prefer $sei if present, otherwise sqrt(vi) sei_vec <-if (length(rma_obj$sei) >0) as.numeric(rma_obj$sei) elsesqrt(as.numeric(rma_obj$vi)) wi <-1/ (sei_vec^2) slab_vec <-if (!is.null(study_labels)) study_labels else rma_obj$slab# build highlight flag (case-insensitive substring match OR logical vector) hl_flag <-rep(FALSE, length(slab_vec))if (is.logical(highlight) &&length(highlight) ==length(slab_vec)) { hl_flag <- highlight } elseif (is.character(highlight) &&length(highlight) >0) {for (pat in highlight) hl_flag <- hl_flag |grepl(pat, slab_vec, ignore.case =TRUE) } df <-tibble(slab = slab_vec,yi =as.numeric(rma_obj$yi),sei = sei_vec,ci_l = yi -qnorm(.975) * sei_vec,ci_u = yi +qnorm(.975) * sei_vec,w = wi,hl =ifelse(hl_flag, "highlight", "other") )if (!is.null(order_by) && order_by %in%names(df)) { df <-arrange(df, .data[[order_by]]) }# overall pooled effect (+ CI + optional PI) overall <-tibble(slab ="Overall (REML)",yi =as.numeric(rma_obj$b),ci_l =as.numeric(rma_obj$ci.lb),ci_u =as.numeric(rma_obj$ci.ub) )# prediction interval if available pi_band <-NULLif (isTRUE(show_prediction_interval)) { pr <-tryCatch(predict(rma_obj, transf =NULL), error =function(e) NULL)if (!is.null(pr) &&!any(is.na(c(pr$pi.lb, pr$pi.ub)))) { pi_band <-c(pr$pi.lb, pr$pi.ub) } } g <-ggplot(bind_rows(df %>%select(-w, -sei), overall),aes(x = yi, y =reorder(slab, yi))) +geom_vline(xintercept =0, linetype =2, linewidth =0.4) +annotate("rect",xmin = overall$ci_l[1], xmax = overall$ci_u[1],ymin =-Inf, ymax =Inf,alpha =0.06) + {if (!is.null(pi_band))annotate("rect",xmin = pi_band[1], xmax = pi_band[2],ymin =-Inf, ymax =Inf,alpha =0.04)} +# Error bars + points ONLY for study rows, colored by highlight flaggeom_errorbarh(data = df,aes(xmin = ci_l, xmax = ci_u, color = hl),height = .15) +geom_point(data = df,aes(color = hl, size =pmin(w /max(w, na.rm =TRUE), 1)),show.legend =FALSE) +# pooled vertical + CIgeom_vline(xintercept = overall$yi[1], linewidth =0.4) +geom_vline(xintercept = overall$ci_l[1], linetype ="dashed", linewidth =0.3) +geom_vline(xintercept = overall$ci_u[1], linetype ="dashed", linewidth =0.3) +scale_color_manual(values =c(other ="black", highlight = highlight_color), guide ="none") +labs(x ="Effect (95% CI)",y =NULL,title = main_title,subtitle =paste0("Pooled = ",formatC(overall$yi[1], digits =3, format ="f")," [", formatC(overall$ci_l[1], digits =3, format ="f"),", ", formatC(overall$ci_u[1], digits =3, format ="f"), "]",if (!is.null(pi_band)) paste0(" | PI [",formatC(pi_band[1], digits =3, format ="f"), ", ",formatC(pi_band[2], digits =3, format ="f"), "]" ) else"" )) +theme_bw() +theme(panel.grid.minor =element_blank())return(g)}# Helper: draw funnel (generic) with highlightingmake_funnel_generic <-function(rma_obj,main_title ="Funnel plot",refline =0,levels =c(90, 95, 99),grayscale =c("white", "gray55", "gray75"),legend_pos ="topright",highlight ="Jihadist", # <- NEWhighlight_color ="red") { # <- NEW# Ensure we have yi/vi/sei yi <-as.numeric(rma_obj$yi) vi <-if (!is.null(rma_obj$vi)) as.numeric(rma_obj$vi) else (as.numeric(rma_obj$sei))^2 sei <-sqrt(vi) slab_vec <- rma_obj$slab# highlight flag (same rules as forest) hl_flag <-rep(FALSE, length(slab_vec))if (is.logical(highlight) &&length(highlight) ==length(slab_vec)) { hl_flag <- highlight } elseif (is.character(highlight) &&length(highlight) >0) {for (pat in highlight) hl_flag <- hl_flag |grepl(pat, slab_vec, ignore.case =TRUE) }# Trim-and-fill on the fitted model tf <-tryCatch(trimfill(rma_obj), error =function(e) rma_obj)# PET / PEESE regressions pet <-tryCatch(rma(yi = yi, sei = sei, mods =~ sei), error =function(e) NULL) peese <-tryCatch(rma(yi = yi, sei = sei, mods =~I(sei^2)), error =function(e) NULL) reml_b <-as.numeric(rma_obj$b)funnel(tf, level = levels, shade = grayscale, refline = refline,legend = legend_pos,main =paste0( main_title, "\n","REML = ", formatC(reml_b, digits =2, format ="f"),if (!is.null(pet)) paste0(", PET = ", formatC(pet$b[1], digits =2, format ="f")) else"",if (!is.null(peese)) paste0(", PEESE = ", formatC(peese$b[1], digits =2, format ="f")) else"",", n = ", length(yi) ))# pooled markerabline(v = reml_b, lty ="dashed")points(x = reml_b, y =0, cex =1.4, pch =17)# PEESE curve + pointif (!is.null(peese) &&length(peese$b) >=2) { sei_grid <-seq(0, max(sei, na.rm =TRUE), length.out =1000) vi_grid <- sei_grid^2 yi_grid <-as.numeric(peese$b[1]) +as.numeric(peese$b[2]) * vi_gridlines(x = yi_grid, y = sei_grid)points(x =as.numeric(peese$b[1]), y =0, cex =1.3, pch =5) }# PET line + point + CI segmentif (!is.null(pet) &&length(pet$b) >=2) {abline(a =-as.numeric(pet$b[1]) /as.numeric(pet$b[2]),b =1/as.numeric(pet$b[2]))points(x =as.numeric(pet$b[1]), y =0, cex =1.2)segments(x0 =as.numeric(pet$ci.lb[1]), y0 =0,x1 =as.numeric(pet$ci.ub[1]), y1 =0,lty ="dashed") }# OVERDRAW highlighted studies in red so they popif (any(hl_flag, na.rm =TRUE)) {points(x = yi[hl_flag], y = sei[hl_flag], pch =19, col = highlight_color, cex =1.2) }invisible(list(trimfill = tf, PET = pet, PEESE = peese))}# Driver: pass highlight through# Driver: iterate meta_results and produce Forest + Funnel with section headersplot_meta_suite <-function(meta_results, add_funnel =TRUE,highlight ="Jihadist", highlight_color ="red") { terms <-unique(meta_results$term)for (tm in terms) { mobj <- meta_results %>%filter(term == tm) %>%pull(.results) %>% .[[1]] pretty <- dplyr::recode(tm,"obsessive_passion_gmz"="Obsessive passion","moral_neutralization_gmz"="Moral neutralization","perceived_discrimination_gmz"="Perceived discrimination",.default = tm )# Emit a second-level markdown header so Quarto renders itcat("\n\n## ", pretty, "\n\n")print(make_forest_generic(mobj,main_title =paste0("Forest: ", pretty),order_by =NULL,show_prediction_interval =TRUE,highlight = highlight,highlight_color = highlight_color))if (add_funnel)make_funnel_generic(mobj,main_title =paste0("Funnel: ", pretty),highlight = highlight,highlight_color = highlight_color) }}# Run the meta suite (set add_funnel = FALSE if you don’t want funnels)plot_meta_suite(meta_results, add_funnel =TRUE)
Obsessive passion
Warning: `geom_errobarh()` was deprecated in ggplot2 4.0.0.
ℹ Please use the `orientation` argument of `geom_errorbar()` instead.