Feature Engineering for Credit & Risk

Module 1 was about not fooling yourself when you look at financial data. This one is about the step where most of a credit model’s performance is actually won or lost: turning raw records into features. The algorithm matters far less than people think — a logistic regression on excellent features beats gradient boosting on lazy ones, and every experienced credit modeler has watched that upset happen.

Throughout, the running example is the standard credit setup: a loan-month panel (one row per loan per month) with balances, payments, limits, and delinquency status, from which we predict default over the next 12 months.

First, fix the observation design

Before any feature exists, three design decisions define what a row is — get these wrong and no feature can save you:

1. Snapshot date — the moment you pretend to stand at. Every feature must be computable from data available strictly before this date (the look-ahead rule from Module 1, now load-bearing).
2. Performance window — how far forward you look for the outcome (default within 12 months? 24?). Longer windows mean more positives but staler relationships — and you silently lose the most recent year(s) of data, which can’t be labeled yet.
3. The bad definition — 90+ days past due is conventional, but whatever you choose, the model predicts exactly that, not “default” in the abstract.

Ratios: the workhorse transformation

Raw magnitudes are almost never good credit features: a $5,000 balance is alarming on a $6,000 limit and trivial on a $100,000 one. Dividing fixes three problems at once — it removes scale (so the feature transfers across small and large borrowers), it encodes domain knowledge (utilization = dependence on borrowed money), and it makes distributions far more stable over time as portfolios grow and inflation moves.

The classics earn their keep: utilization (balance/limit), payment ratio (payment/minimum due), debt-to-income, months-of-cash-cushion. The credit-models track runs on the corporate versions — coverage, leverage, turnover.

Ratio hygiene, learned the hard way:

• Zero and near-zero denominators. A closed card with a $1 limit produces a utilization of 5,000. Winsorize the ratio (not the raw inputs) at a sensible cap, and add an indicator for “denominator was degenerate.”
• Both-zero is meaningful. 0/0 utilization is an unused facility — that’s information, not a missing value. Give it its own encoding rather than imputing.
• Sign flips. Negative equity, negative income adjustments — a ratio that crosses zero changes meaning entirely. Split it into sign and magnitude, or bin it (WOE handles this gracefully below).

Lags and rolling windows: putting time into a row

A single month’s snapshot wastes the panel’s real asset: trajectory. The toolkit, in increasing order of sophistication:

Lags — the value $k$ months ago: utilization_lag3, dpd_lag1. Cheap, interpretable, and they let a pointwise model see change: the pair (now, 6 months ago) is a crude derivative.

Rolling aggregates — statistics over a trailing window: mean(utilization, 6m), max(days_past_due, 12m), count(payments_missed, 24m). Three window-design rules:

• Match the window to the behavior’s memory. Delinquency signals decay fast (3–6 month windows); spending habits are slow (12–24 months). When in doubt, generate several lengths and let variable selection arbitrate.
• Max and min often beat mean. max(dpd, 12m) asks “how bad did it ever get?” — for risk, the worst moment is usually more predictive than the average one.
• Recency-weight when the window is long. An exponentially weighted mean respects that last month matters more than eleven months ago.

Trend features — the direction: utilization_now − utilization_6m_ago, or a regression slope over the window. Two borrowers at 60% utilization are different animals if one came from 30% and the other from 90%. Rising utilization plus falling payment ratio is the classic pre-delinquency signature — each feature is mediocre alone and strong together.

Categoricals: the case for WOE encoding

One-hot encoding, the ML default, behaves badly on credit data: high-cardinality fields (region, industry, product code) explode into sparse dummies; rare categories get coefficients fitted on a handful of loans; and the first unseen category in production throws an error at scoring time.

Credit modeling’s house answer is Weight of Evidence encoding — replace each category (or numeric bin) with the log-odds ratio it carries:

The same transformation that builds scorecards in credit-models Module 2 is, from the data science side, a supervised encoder: one numeric column out regardless of cardinality, missing values become just another category, and the encoded scale is already in log-odds — exactly what a logistic model wants to consume linearly.

For continuous variables, WOE needs bins first, and binning is where judgment enters: too few bins waste signal, too many produce noisy WOE estimates on thin cells, and a validator will (rightly) reject a utilization variable whose risk pattern zigzags. Build the intuition by hand:

Notice the trade-off as you move the cuts: pushing an edge into the sparse high-utilization tail makes that bin’s WOE dramatic and unstable — the diagnostic flags it. The standard practice is monotonic binning: choose edges so WOE moves in one direction, accepting slightly lower in-sample IV in exchange for a pattern that generalizes and can be explained to a regulator in one sentence.

Missingness, interactions, and the checklist

Three closing habits. First, per Module 1: missing means something in credit data — encode indicators or give missingness its own WOE bin; never silently impute the mean. Second, interactions: thin-file × high-utilization is riskier than either alone; in scorecard world you encode these as segment-specific bins (or separate scorecards — FICO’s segmentation from credit-models Module 2 is exactly this idea), while tree-based models find them automatically at the cost of explaining them later. Third, write the feature dictionary as you go — name, formula, window, availability lag, missing-value rule. Model validation will demand it, production monitoring will depend on it, and future-you will not remember what util_wavg_6m_v3 meant.

The pre-modeling checklist:

1. Is every feature computable strictly before the snapshot date, at its true availability lag?
2. Are all ratios capped, with degenerate denominators flagged?
3. Do rolling windows end at $t-1$ with documented lengths?
4. Are WOE mappings fitted out-of-fold, with smoothing on thin cells?
5. Is every binned variable’s WOE pattern monotone (or defensibly not)?
6. Does the feature dictionary exist and match the code?

Where this connects

• The WOE/IV machinery is shared with credit-models Module 2 — same math, two vocabularies: data scientists say “supervised encoding,” credit modelers say “scorecard construction.”
• Which features survive selection — and how to measure their power honestly (AUC, KS, Gini) — is Module 4 of this track (classification metrics done right).
• Whether these hand-built features still matter in the gradient-boosting era is the opening question of credit-models Module 12: XGBoost finds interactions, but the regulator still asks what each input means.

Try it yourself

The notebook below generates a synthetic loan-month panel (5,000 loans, 36 months, with engineered-in default behavior), then builds the full feature set from this post: ratios with hygiene, lags, rolling means/maxes with correct boundary alignment, trend slopes, and WOE encoding with out-of-fold fitting — and quantifies the leakage you get when the boundary and out-of-fold rules are broken, by comparing in-sample vs. out-of-time AUC for the clean and leaky versions of the same features. Every section is runnable top to bottom.

Get new posts by email

One email per new article. No spam, no upsells, unsubscribe anytime.