Modern Data Science in Credit

The final module of this track asks the question every bank’s data science team has been asked by every executive since roughly 2016: why are we still using logistic regression from the 1960s? The answer is not “inertia.” Credit is the domain where machine learning’s usual playbook meets four constraints it doesn’t face anywhere else — and the interesting engineering happens exactly at those constraints.

Why credit is the hard mode of ML

The feedback loop is glacial and censored. A recommender system learns from millions of clicks per day. A credit model books a loan today and learns whether it was a mistake in 18–24 months — and only for the loans it approved (Module 2’s reject inference problem, permanently unsolved). Iteration speed, ML’s superpower, is mostly unavailable.

The world drifts under the model. Macro cycles, competitive shifts, product changes — the data-generating process is non-stationary (the EDA post’s regime problem). A model fit on a benign period encodes benign-period physics; 2020 broke every model in production within weeks, through no fault of the fitting procedure.

Every decision must be explainable — individually, by law. Not “the model is 84% accurate” but “your application was declined because of X, Y, Z” — a specific, truthful statement about one person, required by regulation (below). This kills any architecture whose decisions can’t be decomposed.

Errors are asymmetric and correlated. A bad movie recommendation costs a shrug. Bad credit decisions cost principal, cluster in downturns (Module 10), and — worst — can systematically harm protected groups, which is both morally serious and legally existential.

XGBoost for default prediction: what actually works

Gradient-boosted trees are the workhorse challenger to the scorecard, and on tabular credit data they reliably find what logistic regression on hand-binned WOE features misses: interactions (thin file × high utilization), non-linearities beyond the binning, and signal in messy alt-data fields. Typical honest gains: 2–6 points of AUC/Gini over a well-built scorecard — real money at portfolio scale, but far from the revolution the conference talks promised, because a good scorecard already captures most of the signal in bureau data.

The practical pipeline, distilled:

• Features still matter more than the algorithm. Everything from DS Module 2 — point-in-time correctness, availability lags, ratio hygiene — applies unchanged. XGBoost finds interactions; it cannot un-leak a leaky feature.
• Use monotonic constraints. XGBoost lets you force “higher utilization ⇒ never lower predicted risk” per feature. This single flag buys back most of the explainability gap with scorecards, prevents the model from memorizing artifacts (“borrowers at exactly 87% utilization are fine”), costs almost nothing in AUC, and is the first thing a validator will ask about.
• Calibrate after you rank. Boosted trees rank well but their raw probabilities are often mis-scaled — and if you trained on a balanced sample, the base rate is wrong entirely (Zmijewski’s lesson from Module 3, returned). Isotonic or Platt calibration on a holdout maps scores to honest PDs — which is what pricing, provisioning (Module 9’s IFRS 9), and capital (Module 10) actually consume.
• Validate out-of-time, not just out-of-sample. A random holdout shares the training period’s regime; the honest test is the next period. (The full argument is DS Module 5.)

Neural networks earn their complexity in credit only where the data stops being a flat table: sequences of transactions (an LSTM/transformer over open-banking cash flows), text, or images of documents. On bureau-style tabular data, boosted trees plus good features remain stubbornly competitive, and everything below about explainability gets harder.

Interpretability: from black box to reason codes

The regulatory requirement is not “understand the model in general” — it’s “explain this decision.” The tool that industrialized this is SHAP (Shapley additive explanations): for one prediction, it allocates the gap between the model’s output and the average output across the features, using the Shapley value from cooperative game theory — the only allocation satisfying a small set of fairness axioms (features that contribute equally get equal credit; contributions sum exactly to the gap).

The output is an additive decomposition per applicant. That’s precisely the structure a scorecard gives you for free — which is the deep irony of this module: the 1960s scorecard was born explainable; SHAP is how the 2020s model buys that property back. Feel the decomposition:

The widget’s model is a glass-box additive score, so its bars are its exact explanation — for an XGBoost model, TreeSHAP computes the same style of decomposition efficiently and exactly. Note the bottom row: sort the positive (risk-increasing) contributions, take the top few, map them to standard phrases — that is, almost verbatim, how adverse action reason codes are generated in production. Push utilization to 90% and watch it claim the #1 reason slot.

Two supporting tools complete the kit: partial dependence plots (global shape of one feature’s effect — where you check monotonicity and sane behavior at the edges) and LIME (local linear approximations — useful, but unstable enough that SHAP has largely displaced it in credit). The notebook builds all of them on a real boosted model.

SR 11-7: the discipline of not trusting your own models

The Fed’s 2011 guidance on model risk management is the document that governs how models live inside banks, and its core idea deserves more respect than its reputation as paperwork: models are wrong in ways their builders are structurally unable to see, so institutions must build organized distrust. The regime:

• Development standards: documented purpose, theory, data lineage, assumptions, and known limitations — written down before results make everyone fall in love with the model.
• Independent validation: a separate team with the authority and skill to say no — “effective challenge.” They re-derive, stress, benchmark against simpler models (is the XGBoost really beating the scorecard out-of-time?), and probe exactly the corners the developers avoided.
• Ongoing monitoring: performance, stability, and drift tracking in production (DS Module 6 is entirely about this), with thresholds that trigger review rather than vibes.
• Inventory and governance: every model registered, risk-tiered, owned, and periodically re-validated. “We forgot that model was running” is a finding.

ML raises the stakes on each point: more hyperparameters and data dependencies to document, harder effective challenge (the validator needs SHAP fluency, not just econometrics), faster and quieter failure modes in production. If your organization can’t validate it, it doesn’t matter how well it ranks.

Fair lending: the constraint that defines the field

US law (ECOA/Regulation B, and the Fair Housing Act for mortgages) prohibits credit discrimination on race, color, religion, national origin, sex, marital status, age, and receipt of public assistance. Two legal theories matter for modelers:

• Disparate treatment — using a protected characteristic, directly or through an obvious stand-in. No model touches these features; that part is easy.
• Disparate impact — a facially neutral practice that disproportionately harms a protected group without adequate business justification. This is the hard one, because ML is a proxy-finding machine: given enough features, it will reconstruct protected characteristics from zip code, shopping patterns, device type, or the interaction of a dozen innocent variables — no malice required, just optimization.

The working discipline: test outcomes for group disparities (approval rates, error rates, pricing) even though — awkwardly — you often must infer group membership to run the test, since you’re barred from collecting it; search for less-discriminatory alternatives (a model with slightly lower AUC and materially lower disparity is often the legally and ethically required choice); and justify every feature’s business rationale in writing, especially alt-data features. Adverse action notices close the loop: every declined applicant gets specific principal reasons — which is why the SHAP-to-reason-code pipeline above is a compliance requirement, not a UX nicety.

Alternative data — open-banking cash flows, payroll, rent and utility payments — is genuinely the most promising frontier: it scores thin-file borrowers the bureau can’t see, often expanding fair access. It also multiplies proxy risk and drift risk simultaneously. The technology is the easy part.

The track, closed

Twelve modules ago, credit risk was one equation: EL = PD × LGD × EAD. Everything since has been ways of earning the right to a number in it — scorecards and ratios, curves and bonds, options and intensities, matrices and factor models and copulas, and finally machine learning under the twin disciplines of validation and fairness. The synthesis worth keeping: the models got fancier; the questions never changed. Who might not pay? How much would that cost? Do those events cluster? And can you explain your answer — to a regulator, to a validator, and to the person you declined?

Where this connects

• The scorecard this module keeps benchmarking against is Module 2; the sample-bias trap in calibration is Module 3’s oldest lesson.
• Monitoring, drift, and retraining cadence — the production half of this story — is DS Module 6; honest evaluation is DS Module 4; causal reading of model attributions is DS Module 8.
• Provisioning and capital consume these PDs downstream: Module 9 and Module 10.

Try it yourself

The notebook below is the full pipeline on a synthetic bureau-style dataset: scorecard baseline (logistic on WOE features), XGBoost with and without monotonic constraints, out-of-time evaluation (AUC/KS/Gini), isotonic calibration with reliability plots, TreeSHAP — global importance, dependence plots, and single-applicant waterfalls — and an adverse-action reason-code generator built from the SHAP output. The closing exercise runs a simple disparate-impact test on an inferred group label and asks you to find the less-discriminatory alternative model.

Get new posts by email

One email per new article. No spam, no upsells, unsubscribe anytime.