The Lending Club
Credit Book

Business result

+7.35¢returned per dollar invested, against +6.55¢ for funding everything

Model quality

0.683ROC-AUC on 2015, scored exactly once

Loans monitored

988,585issued 2016 to 2018, replayed as 36 monthly batches

Input drift

43 to 60%of features drifted in every single month

Part 00Orientation

This is a guide to a complete machine learning system: not only a model, but the machinery around it that makes the model trustworthy enough to act on. It assumes you know roughly what a classifier is and nothing beyond that.

The project answers one question. An investor on a peer-to-peer lending platform sees a list of loans. Each shows an amount, an interest rate, a grade the platform assigned, and a profile of the borrower. Some will be repaid with interest. Some will default. Which should the investor fund?

That sounds like a classification problem, and for the first ten minutes it is. The interesting parts sit elsewhere. Most of the columns in the raw data would let you cheat. The obvious metric is worthless here. A default does not lose all your money. The right answer turns out not to be "reject bad loans" at all. And once the model is deployed you cannot tell whether it still works, because the answer arrives three years late.

How to read this

The shape of the system

Everything below hangs off this diagram. The left column is the reproducible pipeline that turns a raw CSV into a deployable artifact. The right side is what happens to that artifact once it exists.

%%{init:{'theme':'base','themeVariables':{'primaryColor':'#EAF1EF','primaryTextColor':'#101C19','primaryBorderColor':'#0E6E62','lineColor':'#5C6B66','secondaryColor':'#F1F4F3','tertiaryColor':'#FCFDFC','fontFamily':'IBM Plex Mono, monospace','fontSize':'12px'}}}%%
flowchart TB
  CSV[("Raw CSV
2.26M loans · 1.68 GB")]
  subgraph PIPE["Reproducible pipeline · dvc repro"]
    direction TB
    A["ingest
keep only allowlisted columns"]
    B["prepare
row-level features · label · maturity filter"]
    C["split
by issue date · schema validation"]
    D["train
3 model families · time-based CV · MLflow"]
    E["evaluate
calibrate · tune policy · score test once"]
    F["package
model + calibrator + threshold + economics"]
    A --> B --> C --> D --> E --> F
  end
  subgraph OPS["Operations"]
    direction TB
    G["FastAPI + Docker
/predict"]
    H["monitoring
36 monthly batches · drift"]
    I["trigger
persistent drift, then retrain"]
    J["Prefect flow
full retrain"]
    K["promotion gate
judged in dollars"]
    H --> I --> J --> K
  end
  CSV --> A
  F --> G
  F --> H
  K -.->|"only if better"| F

Part 01Why the metric is money

What peer-to-peer lending is

Lending Club connected borrowers who wanted a personal loan with investors who wanted to lend. A borrower applies. The platform runs its own underwriting, approves or rejects, and for approved loans assigns a grade from A to G with an interest rate to match. Grade A loans are the safest and pay the least interest. Grade G loans are the riskiest and pay the most.

The approved loan is then listed. Investors browse the listings and choose which to put money into. A loan runs for a fixed term, and this project covers 36-month loans. During that term the borrower makes a fixed monthly payment called the installment. If they pay to the end, the investor gets their principal back plus interest. If they stop paying, the loan is eventually charged off, and the investor keeps whatever was recovered.

Accuracy measures the wrong thing

About 86% of the loans in this dataset were repaid. So a model that predicts "never defaults" for every loan is 86% accurate and completely useless, because it gives the investor no way to tell loans apart.

This is the standard problem with imbalanced classification. When one outcome is much more common than the other, accuracy rewards you for ignoring the rare one, and the rare one is the point.

Even a good accuracy number would not answer the investor's question. Consider two mistakes. Fund a loan that defaults and you lose part of your money. Decline a loan that would have repaid and you lose the interest you would have earned. Those are not equally bad, and how bad each one is depends on the loan's amount and interest rate. A classification metric treats every error as one unit of wrong. An investor counts dollars. So the project's headline metric is realized return per dollar invested.

What a default actually costs

A default does not lose all your money. When a Lending Club loan is charged off, the borrower has usually made payments for a while, and some further amount is recovered afterwards. Measured on this dataset's training loans:

Both numbers matter. Assume a default loses 100% of the money and you will be far too cautious, declining profitable loans. Assume a repaid loan pays the full contractual interest and you will overestimate every return by about 17%.

The bridge from probability to dollars

A classifier outputs a probability. An investor needs a decision. Expected value connects them:

E[return] = (1 - p) x gain_if_repaid  -  p x loss_if_default

where   gain_if_repaid = (installment x term - amount) x 0.8288
        loss_if_default = 0.3657 x amount
        p               = the model's predicted default probability

This one equation explains why the rest of the project looks the way it does. The decision multiplies by the predicted probability, so the probability has to be right in value, not merely ranked correctly. A model that confidently says 2% when the truth is 20% wrecks the arithmetic even if it ranks every loan in the correct order. That requirement drives the choice of training metric and forces an extra calibration step, both covered in part 04.

Part 02The data, and what was thrown away

The source is accepted_2007_to_2018Q4.csv: 2,260,702 loans, 1.68 GB, 151 columns, from a public Kaggle mirror of Lending Club's own release. The file's SHA-256 is recorded in the project so anyone can confirm they have the same data.

Of those 2.26 million loans, the project uses 621,022. Understanding why three quarters were discarded is most of the work of this part.

Filter one: only loans with a known outcome

A loan's loan_status can be Fully Paid, Charged Off, Default, Current, In Grace Period, or various stages of Late. Only the first three are final. A loan still marked Current has no label, since nobody yet knows how it ends, so it cannot be used for supervised training.

The target is defined accordingly. Charged Off and Default become 1, Fully Paid becomes 0, and everything else is dropped.

Filter two: only loans old enough to have finished

This filter is the subtle one, and getting it wrong produces a classic mistake called survivorship or maturity bias.

The dataset is a snapshot taken at the end of 2018. A 36-month loan issued in June 2018 has only been running six months by then. It cannot possibly be Fully Paid yet. The only way it has a final status is if it ended early, which for a young loan almost always means it was charged off.

Keep recent loans and your labelled sample is quietly poisoned. The recent portion contains defaults but almost no successes. A model trained on it learns that recent loans are catastrophically risky, which is an artifact of the calendar rather than anything about credit.

The project measured this rather than assuming it:

The cutoff is therefore December 2015. Everything issued afterwards is excluded from training, and later put to a different use entirely as simulated production traffic for drift monitoring in part 08.

Why the split is by date

A random train/test split would let the model learn from 2013 loans and be scored on 2010 loans. That is impossible in production, since you cannot train on the future, and it makes results look better than they are, because the model gets to see the economic conditions of the period it is being tested on.

Splitting by issue date simulates reality. Train on the past, predict the future. It also makes the test harder and the resulting number smaller, which is the point.

What a loan looks like to the model

Twenty-nine raw columns are permitted as inputs. A few of them are replaced by derived versions, and the final set the model receives is 30 columns, which encoding then expands into 73 numeric features. The derived features are worth naming, because each one encodes a piece of domain judgement:

Missing values: three kinds, three treatments

Beginners are usually taught to fill missing values with the median. That is right about a third of the time. This project distinguishes three cases.

Ordinary missing. The value exists but was not recorded. Fill with the median learned from the training fold only.

Structurally missing. mths_since_last_delinq is blank when the borrower has never been delinquent. That is not absent data, it is the most informative value in the column. Filling it with the median would claim the borrower was delinquent an average amount of time ago, which is false. Instead it gets a sentinel far outside the real range, 999, which tree models can split on cleanly.

Missing as a signal. For three columns, whether the value is missing predicts default on its own. emp_length_years is 4.3% missing, and those loans default at 18.2% against 12.4%. pub_rec_bankruptcies is 0.8% missing, 24.5% against 12.5%. revol_util is 0.1% missing, 16.9% against 12.6%. These get the median and an extra 0/1 column recording that they were missing.

Part 03Four ways to cheat

Data leakage is when information reaches the model that it would not have at the moment of the real decision. It is the most common way a machine learning project produces an impressive number that means nothing. This project defends against four distinct kinds.

1. Target leakage, or the columns that describe the future

The raw file has 151 columns. Most of them are recorded after the investor's decision:

Including recoveries would produce a model with near-perfect accuracy, because a non-zero recovery is a default. It would also be worthless, since at decision time that column is always empty.

2. Preprocessing leakage, the invisible kind

This one catches careful people. Suppose you fill missing incomes with the median income. Compute that median over the whole dataset before splitting, and the median now contains information from your test loans, which then flows into the training data. The same applies to percentile caps, category frequencies and scaling factors.

The effect is small per statistic and invisible in the code. It reliably inflates results.

The project solves this structurally by dividing every transformation into two layers:

%%{init:{'theme':'base','themeVariables':{'primaryColor':'#EAF1EF','primaryTextColor':'#101C19','primaryBorderColor':'#0E6E62','lineColor':'#5C6B66','fontFamily':'IBM Plex Mono, monospace','fontSize':'12px'}}}%%
flowchart LR
  R[Raw loan row] --> LA
  subgraph LA["LAYER A: safe before the split"]
    direction TB
    A1["parse ' 36 months' to 36"]
    A2["FICO low and high to midpoint"]
    A3["issue date minus credit line"]
    A4["amount over income"]
  end
  LA --> SP{{"train / validation / test
split by date"}}
  SP --> LB
  subgraph LB["LAYER B: must be fitted"]
    direction TB
    B1["median imputation"]
    B2["99.5th percentile cap"]
    B3["category frequencies"]
    B4["standard scaling"]
  end
  LB --> M["73 features, then the model"]

This works because Pipeline makes the correct behaviour automatic instead of a matter of discipline. When cross-validation calls fit on a fold, every learned statistic inside the pipeline is recomputed from that fold's training rows alone. You cannot forget.

3. Temporal leakage, or training on the future

Part 02 covered this: all splits are by issue date. It applies to cross-validation too. The folds expand forwards in time, as Exhibit 2 shows, and a fold never validates on a year earlier than its training data.

4. Evaluation leakage, or reusing the test set

Look at the test result, adjust something, look again, and the test set has become a tuning set. Its number no longer estimates performance on unseen data. It estimates performance on data you have been quietly optimising against.

The project's rule is that 2015 is scored exactly once, at the very end. Everything that shapes the model, including hyperparameters, the calibrator and the funding threshold, is chosen on 2014.

That rule started as a convention maintained by reading the code carefully. An audit later flagged it as the weakest link, and it was made structural:

Part 04Modelling and honest validation

Start with baselines you have to beat

Before any model, the project establishes what "no model" looks like. There are two baselines. The first is Lending Club's own grade, turned into a classifier that predicts each grade's historical default rate. This is the real competitor, since the platform already did risk assessment, and a model that cannot beat it adds nothing. The second is funding everything, which is the return an investor gets by buying every listed loan.

Without these, a return of +7.35¢ per dollar is a number floating in space. With them it becomes a claim about doing better than the obvious alternatives.

The metric that gets optimised

The logic chain is short. The funding rule multiplies by p, so p has to be accurate in value, so the training metric has to punish miscalibration. That gives log loss.

Class imbalance, and what was deliberately not done

With 14% defaults, the standard advice is to oversample the minority class with SMOTE or set class_weight="balanced". The project does neither, and this is one of its more instructive decisions.

The two models

Logistic regression is the interpretable baseline. It fits a weighted sum of the features and squashes it into a probability. It needs scaled inputs so its penalty applies evenly, and one-hot encoded categories, and it cannot represent interactions unless you build them by hand.

LightGBM is a gradient-boosted decision tree ensemble. It builds hundreds of small trees, each correcting the previous ones' errors. It finds interactions automatically, ignores feature scale, and handles the mix of numeric and categorical data well. It is also far more capable of overfitting, which is why the validation design matters.

Hyperparameters are chosen by randomised search, meaning 20 random combinations rather than the full grid. A full grid over 7 settings would be thousands of fits. Random sampling finds a nearly as good configuration for a fraction of the compute, because only a few of the settings matter much.

Calibration, or making 0.2 mean 20%

Models trained for ranking are often miscalibrated. They separate risky from safe correctly while the numbers themselves are systematically off. Since the profit equation consumes the number, this has to be fixed.

The fix is isotonic regression, a flexible monotone mapping from raw score to observed frequency, fitted on held-out 2014 loans. Monotone means it never reorders anything. It only stretches and compresses the scale.

Results, and an honest reading of them

A first draft of this project stopped there and declared LightGBM the winner. An audit called that out, for a reason worth absorbing.

The gap between LightGBM and logistic regression is 0.00173 log loss. The spread between folds is 0.026, fifteen times larger, because 2011, 2012 and 2013 are genuinely different years. Comparing two averages tells you nothing when the year-to-year variation dwarfs the difference you are claiming.

Both models see identical folds, so that variation is common to both, and the right comparison is paired, fold by fold:

LightGBM does win every fold, which is a real result. But the variation in its margin is as large as the margin itself, and it received nearly seven times the hyperparameter search budget, which biases the comparison in its favour. The honest summary is that it is consistently ahead by a little, not decisively better. Per-fold scores are kept in the report so anyone can check rather than take it on trust.

On the 2015 test year, scored once: log loss 0.3992, ROC-AUC 0.6834, Brier 0.1206.

Exhibit 4Which features drive the predictions (SHAP)

Part 05From probability to decision

The finding that reshaped the project

Apply the expected-value rule, funding whenever E[return] is above zero, and it funds 99.88% of all loans.

The reason is that Lending Club priced the loans. A grade-G borrower pays 25% or more in interest precisely because they are likely to default, and the rate is set so that the expected value is positive anyway. At these rates almost every loan is worth funding in isolation.

So the deployed rule is a tuned threshold: fund when the raw risk score is below a cut-off chosen to maximise return per dollar. That cut-off, selected on the second half of 2014, is 0.14, which corresponds to a calibrated default risk of about 14.6%.

%%{init:{'theme':'base','themeVariables':{'primaryColor':'#EAF1EF','primaryTextColor':'#101C19','primaryBorderColor':'#0E6E62','lineColor':'#5C6B66','fontFamily':'IBM Plex Mono, monospace','fontSize':'12px'}}}%%
flowchart TB
  L["Loan listing
29 raw fields"] --> V{"valid?
fields consistent
with each other"}
  V -->|no| X["HTTP 422
rejected, not scored"]
  V -->|yes| P["pipeline
73 features"]
  P --> S["raw risk score"]
  P --> C["calibrated probability"]
  S --> T{"score below 0.14 ?"}
  C --> E["expected return in $
(1-p)·gain - p·loss"]
  E --> T2{"expected return above 0 ?"}
  T -->|no| D1["DECLINE
score_above_threshold"]
  T -->|yes| T2
  T2 -->|no| D2["DECLINE
negative_expected_return"]
  T2 -->|yes| F["FUND
score_below_threshold"]

Comparing fairly with equal selectivity

Any selection policy can look better by being pickier. Fund only the safest 5% of loans and your return per dollar will look wonderful, and you will have deployed almost no capital, which is not a usable strategy.

So every comparison is also run at matched selectivity. Force the model to fund exactly the same share of loans as the grade A to B rule, then compare returns. That isolates whether the model is better at choosing from whether it is simply more conservative.

Read the last two rows against the grade rule. At essentially the same selectivity, the model earns more and defaults less.

Is the edge real, or luck?

A point estimate cannot tell you. A gap of 0.002 per dollar could easily be noise on a different set of loans. So each comparison is bootstrapped: resample the test loans with replacement 1,000 times, recompute both returns each time, and look at the distribution of the difference.

Read that last row honestly. LightGBM's edge over a plain logistic regression is small, with an interval that nearly touches zero. The gap that matters is model against no model, which is four times larger.

Exhibit 6Calibration and the profit curve

Part 06Serving: the bundle

Why shipping "the model" is not enough

A trained estimator is only one of four things needed to reproduce a funding decision, and they were produced at different stages from different data:

Ship these separately and production drifts out of sync with the report, because someone updates the model and forgets the threshold. Bundled into one artifact, the service either has a complete, consistent decision or it has nothing.

Training and serving skew, and the one function that prevents it

The API asks callers for raw listing fields. It never asks for fico_mid or credit_history_months. A caller cannot reasonably be expected to know what those mean, and if they computed them slightly differently, say by dividing credit history days by 30 instead of 30.44, the model would silently score garbage.

So both the training pipeline and the API call the same function, derive_features(). Skew is not merely unlikely, it is structurally impossible.

Validating input like a loan, not like a form

An early version of the API accepted anything structurally well-formed. An audit found three ways to get a confident answer out of nonsense, all since fixed:

That last row is the interesting one. North Dakota genuinely never appears in the training window. Rejecting it would turn a legitimate application into a client error. So states validate against the 51 real USPS codes rather than against the training data, and a real state the model has not seen becomes a business event for the drift monitor to report.

The container

The Docker image is multi-stage. A builder stage installs dependencies, then only the resulting environment and the application code are copied into a clean runtime image, so the build tools never ship. It runs as a non-root user, declares a HEALTHCHECK, and carries 36 packages rather than the roughly 200 the training environment needs. MLflow, DVC, SHAP and the plotting libraries all sit in a separate optional dependency group that the image never installs.

Part 07Automation and the promotion gate

Retraining on a schedule is dangerous without a gate

Models go stale, so teams schedule retraining. The failure mode is that a bad training run reaches production silently. The pipeline succeeded, every task was green, and the model is worse.

%%{init:{'theme':'base','themeVariables':{'primaryColor':'#EAF1EF','primaryTextColor':'#101C19','primaryBorderColor':'#0E6E62','lineColor':'#5C6B66','fontFamily':'IBM Plex Mono, monospace','fontSize':'12px'}}}%%
flowchart TB
  T["retrain: ingest, prepare, split, train, evaluate"] --> CB["candidate bundle"]
  CB --> SC["score candidate on 2014 H2
return per dollar"]
  CH["champion, currently live"] --> SC2["score champion on the same loans"]
  SC --> G{"candidate minus champion
above 0.0005 ?"}
  SC2 --> G
  G -->|yes| PR["PROMOTE
candidate becomes live"]
  G -->|no| RJ["REJECT
champion stays"]

Continuous integration on data nobody has

CI runners have no access to a 1.68 GB dataset. Running only unit tests would prove that every part works, but not that the parts fit together.

The solution is synthetic loans with planted risk. A generator produces loans whose default probability genuinely depends on grade, FICO and loan-to-income. CI then runs the real code end to end over them, cleaning, labelling, splitting, preprocessing, training, calibrating, choosing a policy, bundling and serving over HTTP, then asserts the model reaches AUC above 0.60 on the planted signal. It takes under two seconds and catches wiring mistakes that unit tests cannot see.

Part 08Monitoring when labels arrive late

This is the most transferable idea in the project. A 36-month loan funded today reveals whether it was a good decision in three years, and waiting is not a monitoring strategy.

The setup

The 988,585 loans issued between 2016 and 2018, deliberately excluded from training, are replayed as 36 monthly production batches. Each is scored by the deployed bundle exactly as the API would score it.

Why the visible labels are a trap

Some of those loans do have final outcomes, and it is tempting to score the model on them. That would be a serious mistake.

A loan issued in June 2018 can only have finished by the 2018Q4 snapshot if it ended early, either paid off ahead of schedule or charged off in its first months. The loans you can see are the least representative ones.

Nothing about the model or the world changed to produce that swing. Only the amount of elapsed time did. Charted on a dashboard it would look exactly like a model failing and then recovering. So label coverage is reported as context and never used as a quality metric.

What gets watched instead

Two things are visible immediately: the inputs, meaning whether incoming applications still resemble training data, and the model's own output, meaning whether the score distribution is moving.

Exhibit 8Three years of monitoring

Measuring drift so the months stay comparable

From a number to a decision

A drift figure on a dashboard is not yet a decision. The trigger turns the monthly reports into one of three outcomes: do nothing, retrain, or call an engineer.

%%{init:{'theme':'base','themeVariables':{'primaryColor':'#EAF1EF','primaryTextColor':'#101C19','primaryBorderColor':'#0E6E62','lineColor':'#5C6B66','fontFamily':'IBM Plex Mono, monospace','fontSize':'12px'}}}%%
flowchart TB
  B["monthly batch"] --> Q{"data quality:
missing jump, or
unseen category?"}
  Q -->|yes, and new| I["ALERT: investigate
an engineer, not a retrain"]
  Q -->|already known| SU["suppressed
reported once, not monthly"]
  B --> D{"drift share above 0.30
or prediction drift above 0.15?"}
  D -->|no| OK["clean, reset the run"]
  D -->|yes| R{"2 consecutive
months?"}
  R -->|no| W["wait, one month
is often an artifact"]
  R -->|yes, first time| RT["ALERT: retrain"]
  R -->|yes, already fired| SU2["suppressed
one alert per episode"]

Persistence is required because a single month is often an artifact, whether a marketing push, a holiday, or a product change that reverts. Requiring the same breach in consecutive months trades a month of delay for far fewer false alarms. Retraining is not free either, since it consumes the promotion gate and every model swap carries risk.

The two alert types are separate because retraining fixes a world that has moved. It does not fix a column that suddenly arrives empty, and firing "retrain" at a broken feed would train the next model on the same broken data. Different problem, different owner, different urgency.

What the monitoring found

The headline is that a model trained on 2007 to 2013 was already operating on a materially shifted population by January 2016. The two investigate alerts are real findings: addr_state=ND and home_ownership=ANY, values that never appear in the training window. Neither breaks the service, and both are worth knowing about.

Part 09Fairness, and one open decision

The dataset contains no protected attributes, no race, sex or age, so disparate treatment cannot be measured directly. What can be measured is whether outcomes differ across addr_state, a well documented proxy for race and income in the US.

A model card that merely asserts "state may be a proxy" is box-ticking. This one measures it.

Finding 1: funding rates differ sharply by state

Against an overall funding rate of 66.3%, the spread runs from 47.8% in Nevada to 74.0% in Massachusetts, a gap of 26 points.

Finding 2: a big gap is not automatically a problem

Some groups genuinely are riskier, and a funding gap that reflects real risk is the model doing its job. The question is whether the gap is earned. Comparing the correlation between each group's mean predicted score and its actual default rate:

* Only three groups, so a correlation of 1.00 across three points is nearly meaningless. Shown for contrast, not as evidence.

purpose spreads far wider than addr_state, with small-business loans funded at 14% and credit-card refinancing at 82%, and that spread is almost entirely explained by real differences in default. addr_state spreads less and is explained half as well.

Finding 3: the model is not equally right about each state

Among loans the policy funded, the realized default rate ranges from 5.8% in Oregon to 14.3% in Arkansas, against 10.0% overall. The stated 14.6% risk bar means something materially different depending on where the borrower lives.

Finding 4: the feature earns none of it

addr_state carries just 1.6% of total SHAP weight. Refitting the selected model without it entirely:

A difference of 0.00001 log loss, with AUC fractionally better without it.

This is the only entry in the project's 80-decision log marked Open, deliberately. A model card that documents a known issue with measurements and an explicit recommendation is more credible than one with nothing open, because it shows someone looked.

Part 10The MLOps toolkit

Every tool here earns its place by removing a specific failure mode. The useful question is not what a tool does, but what goes wrong without it.

Data and computation

Modelling

Reproducibility

Quality

Serving and operations

Part 11The adversarial audit

When the project was finished, it was reviewed again from scratch as if by a stranger, deliberately looking for reasons to reject it. That review is arguably the most useful artifact in the repository, because it shows what "finished" misses.

What it confirmed was sound

No target leakage. No preprocessing leakage. Time-based splits correct. Class imbalance handled appropriately. Metrics matched to the business problem. No sign of overfitting or an unrealistically good result. Seeds set, dependencies pinned, no hardcoded secrets.

The three critical findings

Two things the fixes uncovered

Fixing C1 led to the discovery that the negative expected returns were real data rather than only hostile input, namely the 1,015 internally inconsistent loans described in part 06. And the new validation immediately failed one of the project's own tests, which had been quietly scoring an impossible loan for weeks.

Deliberately not fixed

Regenerable charts are committed to git. They render in the README on GitHub, and removing them for cleanliness would break what makes the repository readable at a glance.

The API has no authentication. Adding token auth to a portfolio demo is theatre, since it would look like security without being any, so it is documented as a stated non-goal instead.

Part 12Running it yourself

Requires Python 3.11 or 3.12, uv, and about 4 GB of free disk. Docker optional.

# 1. install. The "train" extra carries MLflow, DVC, Evidently, SHAP
uv sync --extra train --extra dev

# 2. check the install. Needs no data, takes about 10 seconds
uv run pytest                     # 118 tests

# 3. get the data
#    Download accepted_2007_to_2018Q4.csv from the Kaggle dataset
#    wordsforthewise/lending-club and place it at:
#      data/raw/accepted_2007_to_2018Q4.csv

# 4. rebuild every published number from the raw CSV (about 30 min)
uv run dvc repro

Then explore:

uv run mlflow ui --backend-store-uri sqlite:///mlflow.db   # every tuning run
uv run uvicorn lending_club.serving.app:app --reload       # localhost:8000/docs

# retrain end to end, with the promotion gate (about 100s)
PREFECT_API_URL= PREFECT_SERVER_ALLOW_EPHEMERAL_MODE=true uv run python flows/retrain.py

# replay 2016 to 2018 as monthly production batches (about 4 min)
uv run python -m lending_club.monitoring.batches
uv run python -m lending_club.monitoring.drift

One genuine reproducibility bug was found late. The README originally said uv sync --extra dev, which installs neither DVC nor MLflow nor Evidently, so dvc repro, the one command the whole project rests on, would have failed immediately for anyone following the instructions.

ReferenceGlossary