Is meta-analysis necessary?

Question

I have a large dataset that covers 5 countries. I plan to build a prediction model using this dataset. I would like to compute a ROC curve for each country, and then one overall ROC curve for all countries. To go about this, I plan to either:

1. Build 5 individual prediction models for each of the five countries by partitioning the data according to country. Then, compute 5 ROC curves from the 5 models. Next, using a meta analysis approach, form one summary ROC curve to represent all the countries.

2. Build 1 prediction model using the entire dataset, which includes a model variable corresponding to country. Compute 5 different ROC curves for each of the 5 countries by partitioning the test data i.e. only use test samples from country X to compute the country X ROC curve. Next, use the entire test dataset to compute the overall ROC curve for all the countries.

Please let me know which of the two above approaches would be optimal. Thank you.

Answer 1

In my experience, it is generally better to use the full dataset to build a single prediction model rather than splitting the data and fitting separate models by country. Pooling improves statistical efficiency, yields more stable predictions, and results in narrower prediction intervals due to reduced estimation uncertainty.

Fitting separate models per country (approach 1) forfeits shared information, increases the risk of overfitting, especially in countries with smaller sample sizes, and prevents borrowing strength across groups.

A single global model (approach 2), including country as a fixed effect, allows performance to be assessed within each country by computing ROC curves on the relevant test subsets. An overall ROC curve can also be computed from the full test set.

A more flexible, and often better, approach is to use a mixed-effects model with random intercepts for country, and potentially random slopes (which allow for the effects of covariates to differ between countries). Mi9xed effects models allow partial pooling, improving generalisability while capturing country-specific variation.

Finally, note that ROC curves assess discrimination. If outcome prevalence varies by country, a proper scoring rule such as the Brier score, and, for visual assessment, calibration plots, may also be more informative.