Does Machine Learning Outperform Traditional Scorecards? Evidence from a Benchmark Credit Risk Dataset

Machine learning (ML) methods are increasingly promoted as the basis for "next-generation" risk models, yet their added value over traditional, interpretable techniques on standard-scale credit portfolios remains debated, with direct implications for model validation and audit practice. This study benchmarks a nonlinear ensemble method against a classical logistic regression scorecard on a well-established public credit dataset.

We used the Statlog German Credit Data (UCI Machine Learning Repository), comprising 1,000 loan applicants (700 good, 300 bad) described by 20 financial and socio-demographic attributes. A Gradient Boosting Classifier (300 trees, depth 3) was compared against a logistic regression baseline within an identical preprocessing pipeline (one-hot encoding, feature scaling). Performance was assessed via a 70/30 stratified holdout and 5-fold cross-validation, using AUC-ROC, accuracy, and Brier score; feature importance was extracted from the ensemble model.

Both models achieved comparable discriminative power (5-fold cross-validated AUC: 0.780 ± 0.023 for logistic regression versus 0.780 ± 0.026 for gradient boosting; holdout AUC 0.794 versus 0.766). The logistic regression baseline showed a marginally better Brier score (0.162 versus 0.177). Checking-account status, credit amount, loan duration, and applicant age were the dominant risk drivers in both models.

On this benchmark, a nonlinear ML model provided no measurable accuracy gain over a transparent scorecard, suggesting that, at limited sample sizes typical of many institutional credit books, model complexity should not be assumed superior. We argue that internal audit and risk governance frameworks should require systematic benchmarking of ML risk models against simple, explainable baselines before deployment, reserving ML's added value for settings with larger or alternative data sources.