Step 02 · Prediction Engine
In progressSolvenz is the same composite as in the Analytics Engine cases. The scenario is illustrative; the industry figures are sourced. The Prediction Engine is in active development; this case describes the workflow it is being built to run.
The weekend the treasurer spent interrogating Thursday’s outflow ended well. Instead of matching a competitor’s forty basis points across the book, Solvenz made a targeted counter-offer to a few thousand high-balance, short-tenure accounts and kept the rest of its deposit base at the old rate. But the Monday committee that approved it also left a question on the table that nobody could answer: that analysis explained a run after it had started. What would Solvenz know, next time, at 16:39?
The speed of deposit flight has changed what that question is worth. On 9 March 2023, Silicon Valley Bank saw more than $40 billion leave in a single day, with management expecting 00 billion more the next. Solvenz is not SVB: its depositors are not one tightly networked industry and its book is not concentrated in uninsured balances. But the lesson for any asset-liability committee is general. By the time an outflow appears in a daily digest, the cheapest moment to respond has already passed.
What the treasurer wanted was a rate-sensitivity model: for every savings account, the probability that it moves most of its balance within a month of a comparable competitor move. He had asked for exactly that two years earlier. The BI team scoped it at nine months, and the estimate was not padding. At a regulated lender a model is not a spreadsheet; it falls under model risk management, which US supervisory guidance describes as sound development, independent validation with effective challenge, and ongoing monitoring. Data extraction, feature engineering, candidate models, a validation report, a monitoring plan - each one a hand-off between the treasury, a data scientist the bank shared with three other departments, and a model-risk function with its own queue. The project had been approved, deprioritized and quietly shelved.
The model the committee can challenge
The Monday after the rate event, the treasurer typed the request he had once written as a nine-page brief:
Predict, for every savings account, the probability that it moves more than half its balance out within 30 days of a competitor raising a comparable rate. Train on the rate events in our history, including last month’s. Explain the drivers.
The Prediction Engine treated it as what it is: a classification problem with a rare target that only occurs around specific events. Solvenz’s history contained three of them - two earlier competitor moves and last month’s. That shape decided the test before any model was trained. The engine trained on the two earlier events and held out the most recent one entirely, because the only honest question is whether a model predicts a run it has never seen.
Features were assembled from all four systems the Analytics Engine already reads: balance band, tenure and balance volatility from the core ledger; salary credits, recent transfers to other banks and digital-channel activity from the transaction store; products held and loan relationships from the CRM. It trained two candidates - a regularised logistic model, the kind a risk function finds easy to explain, and a gradient-boosted model - and reported both side by side.
On the held-out event, the boosted model ranked accounts so that the riskiest 10 percent contained 58 percent of the balance that actually left. The logistic model captured 49 percent; random outreach, by definition, 10. The engine reported calibration as well as ranking - in the top decile it predicted 31 percent would leave and 27 percent did - and listed the drivers in plain terms: short tenure, no salary credit, balance above €100,000, recent transfers out to other institutions, and no lending relationship.
Share of departing balance captured
- Scored on last month's rate event, which the model never trained on.
- Calibration, top decilepredicted 31% leave · observed 27%
Composite scenario; figures illustrative. Training events: two earlier competitor rate moves. Test event: the most recent one.
Then came the part that used to take a quarter. The engine’s output is itself most of a validation pack: the specification, the exact queries that built each feature, the train and test split, held-out metrics, calibration, feature importances and a reproducible run. The head of model risk still did her job, and did it properly. She rejected one feature - a marketing “rate-seeker” flag that had itself been derived from past outflows - as leakage, and asked for a re-run without it. The re-run took twenty minutes and cost the model two points of accuracy, which is the true number and the one that went into production. That exchange is what effective challenge is supposed to look like. It took a day.
Standing predictions
Scores now refresh nightly. The treasurer’s standing rule is one sentence: text me if the balance-weighted at-risk total rises more than fifteen percent week on week. Relationship managers receive their thirty most at-risk clients every Monday, which turns a retention call into a conversation held before a competitor’s email rather than after it.
The chief risk officer took the same route on the lending side. Her channel analysis in the Analytics Engine had shown one origination channel deteriorating twice as fast as another; now she asked for the forward-looking version: the probability that each active loan reaches 30 days past due within the next 90 days. The engine validated out of time on the two most recent quarters and reported something a pooled score would have hidden: the model ranked the newest loan vintage less well than older ones. That is what drift looks like when it is measured rather than averaged away, and it went into the model’s monitoring plan with a threshold attached. Collections now calls the highest band at five days past due instead of waiting for thirty.
Most recent vintage, band 5: 14.9%Drift reported, not averaged away
Composite scenario; figures illustrative. Collections now calls band 5 at five days past due rather than thirty.
What the treasury has now
Six weeks later a different competitor raised its rate by twenty-five basis points. The committee did not meet on Monday to find out who was leaving. The at-risk list existed on Thursday at 17:00 - about 2,100 accounts, sorted by relationship value - and counter-offers went to the ones worth keeping on Friday morning. Outflow over the next fortnight was a fraction of the previous event’s.
The data scientist the bank used to share across departments stopped being the gate every model had to pass through. She now reviews specifications, owns the monitoring thresholds, and spends her time on the questions that genuinely need a specialist. The model-risk function reviews more models than it used to, faster, with better documentation than the hand-built ones ever had.
The Analytics Engine let Solvenz see a run while it was happening. The Prediction Engine lets it see the next one coming. What neither can do on its own is decide how much each counter-offer should be worth - a question with a liquidity ratio, a budget and a fairness rule attached, and a best answer that can be computed.
Sources
- Board of Governors of the Federal Reserve System, Review of the Federal Reserve’s Supervision and Regulation of Silicon Valley Bank (2023): outflows above $40B on 9 March, 00B more expected the next day.
- Federal Reserve SR 11-7, Supervisory Guidance on Model Risk Management (2011): development, validation with effective challenge, ongoing monitoring.