Git + DVC: versioning code and data together

In Articles 1 and 2 you got comfortable versioning code with Git. But there's a problem Git alone can't solve well, and it's one every ML practitioner runs into sooner or later: Git is genuinely bad at handling data and model files.

This article introduces DVC, Data Version Control, the tool built specifically to fill that gap, and walks through how Git and DVC work together as a team. We'll build a complete heart disease classification project and version both the code, with Git, and the datasets and models, with DVC.

By the end you'll understand why Git struggles with data, how DVC and Git split the work between them, the full DVC workflow (tracking data, remote storage, pipelines, experiments), and how to reproduce any past version of your project, code and data and model together, with a single pair of commands.

What this article covers

Part 1Why Git can't version data

Git was built for text, not gigabytes

Git was designed to track source code: small text files where it can store line-by-line differences efficiently. Hand it a 50 GB dataset or a 2 GB trained neural network and it falls apart.

The workarounds, and why they fail

"I'll just add data to .gitignore." Now your data isn't versioned at all. Six months later you can't reproduce the model your client approved, because nobody knows which version of the data trained it.

"I'll store data on a shared drive, with folder names like data_v2_final." You're right back in the manual-versioning nightmare Git was supposed to fix, just for data instead of code.

"I'll use Git LFS." Better, but limited. It doesn't handle ML pipelines, experiments, metrics tracking, or flexible remote storage (S3, GCS, Azure, SSH) the way DVC does.

What you actually need

A system that versions large data and model files without bloating Git, links a specific data version to a specific code version so they always match, stores the large files somewhere cheap, lets you reproduce any past experiment exactly, and works alongside Git rather than against it.

That system is DVC.

Part 2What DVC is, and how it works with Git

DVC, Data Version Control, is an open-source command-line tool that brings Git-like versioning to data and ML models. It's built to sit on top of Git, not replace it.

The key insight: Git tracks pointers, DVC tracks data

Here's the trick that makes the whole system work. When you tell DVC to track a large file, say heart.csv, it moves the actual file into a hidden cache and creates a tiny text file called heart.csv.dvc in its place. That .dvc file is a few lines long and contains a hash of your data plus its location. Git tracks the tiny pointer file. DVC tracks the actual heavy data.

The division of labor

When you check out an old Git commit, the .dvc pointer files come along with it automatically. Then dvc checkout reads those pointers and restores the exact matching data. Code and data stay in sync, always.

Part 3The mental model: pointers, cache, and remotes

Three locations matter in DVC. Once these click, the rest of the tool makes sense.

The workspace is your project folder, where you actually see and edit files. The DVC cache is a hidden local store, .dvc/cache, holding content-addressed copies of every version of your data, the local equivalent of .git. The DVC remote is cloud or network storage where you back up and share that cache, the equivalent of GitHub for your data.

The parallel to Git is deliberate and worth holding onto:

Part 4Installing and initializing DVC

Install DVC

# most common
pip install dvc

# with support for a specific remote
pip install "dvc[s3]"       # Amazon S3
pip install "dvc[gdrive]"   # Google Drive
pip install "dvc[gs]"       # Google Cloud Storage
pip install "dvc[azure]"    # Azure Blob Storage
pip install "dvc[ssh]"      # SSH/SFTP

# or all of them
pip install "dvc[all]"

dvc --version
# Output: 3.67.1 (DVC ships new releases often — expect a newer number)

Initialize DVC in a Git repository

DVC needs an existing Git repo to sit on top of. The order is always Git first, DVC second.

# create and enter the project
mkdir heart_disease_classifier
cd heart_disease_classifier

# Git first
git init

# then DVC
dvc init

What dvc init creates:

.dvc/
├── .gitignore       ← tells Git to ignore the DVC cache
├── config           ← DVC configuration (remotes, settings)
.dvcignore           ← like .gitignore, but for DVC

# DVC stages these for you — commit them with Git
git status

Output

Changes to be committed:
        new file:   .dvc/.gitignore
        new file:   .dvc/config
        new file:   .dvcignore

git commit -m "Initialize DVC"

You now have a project tracked by both Git, for code, and DVC, for data.

Part 5Tracking your first dataset

We'll use the Heart Disease dataset, a classic from the UCI Machine Learning Repository: 303 patients, 13 features (age, cholesterol, blood pressure, and so on), and a binary target for the presence of heart disease.¹

# create a data directory, and place heart.csv inside it
mkdir data
# columns: age, sex, cp, trestbps, chol, fbs, restecg, thalach,
# exang, oldpeak, slope, ca, thal, target

age,sex,cp,trestbps,chol,fbs,restecg,thalach,exang,oldpeak,slope,ca,thal,target
63,1,3,145,233,1,0,150,0,2.3,0,0,1,1
37,1,2,130,250,0,1,187,0,3.5,0,0,2,1
41,0,1,130,204,0,0,172,0,1.4,2,0,2,1
...

Track it with DVC

dvc add data/heart.csv

Output

To track the changes with git, run:

        git add data/heart.csv.dvc data/.gitignore

To enable auto staging, run:

        dvc config core.autostage true

Three things just happened: the real content of heart.csv moved into the DVC cache, a tiny pointer file data/heart.csv.dvc got created, and data/heart.csv was added to a local .gitignore so Git leaves the real file alone.

The pointer file itself:

cat data/heart.csv.dvc

outs:
- md5: a1b2c3d4e5f6789012345678abcdef00
  size: 11328
  hash: md5
  path: heart.csv

That hash is a fingerprint of your exact data. Change even one value and the hash changes with it.

Commit the pointer with Git

git add data/heart.csv.dvc data/.gitignore
git commit -m "Track heart disease dataset with DVC"

Now Git knows about the pointer, and DVC knows about the data. They're linked.

Part 6Setting up remote storage

The DVC cache lives locally. To back up your data and share it with teammates, you push it to a remote.

Option A — a local directory (good for learning)

# a local folder standing in for the cloud — fine for practice
mkdir -p /tmp/dvc-remote
dvc remote add -d myremote /tmp/dvc-remote

Option B — Amazon S3

dvc remote add -d myremote s3://my-bucket/heart-disease-data
# credentials via the AWS CLI or environment variables

Option C — Google Drive

dvc remote add -d myremote gdrive://your-folder-id

The -d flag sets it as the default remote.

# the remote config lives in .dvc/config — commit it
git add .dvc/config
git commit -m "Configure DVC remote storage"

Push the data

dvc push

Output

1 file pushed

The data is safely backed up now. A teammate who clones the Git repo runs dvc pull to get the exact same data.

# a teammate's workflow
git clone <your-repo-url>
cd heart_disease_classifier
dvc pull          # pulls the actual data down from the remote

Part 7Building a reproducible DVC pipeline

This is where DVC earns its keep. Instead of running scripts by hand and hoping you remember the order, you define a pipeline, a graph of stages that DVC runs in the right order and re-runs only when something relevant changes.

Step 7.1 — the project code

src/preprocess.py, cleans and splits the data:

"""preprocess.py — Load, clean, and split the heart disease data."""
import pandas as pd
import yaml
from sklearn.model_selection import train_test_split
import os

with open("params.yaml") as f:
    params = yaml.safe_load(f)

test_size = params["preprocess"]["test_size"]
random_state = params["preprocess"]["random_state"]

df = pd.read_csv("data/heart.csv")
df = df.drop_duplicates()
df = df.dropna()

X = df.drop(columns=["target"])
y = df["target"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=test_size, random_state=random_state, stratify=y
)

os.makedirs("data/processed", exist_ok=True)
X_train.to_csv("data/processed/X_train.csv", index=False)
X_test.to_csv("data/processed/X_test.csv", index=False)
y_train.to_csv("data/processed/y_train.csv", index=False)
y_test.to_csv("data/processed/y_test.csv", index=False)

print(f"Preprocessing complete. Train: {len(X_train)}, Test: {len(X_test)}")

src/train.py, trains the model:

"""train.py — Train a Random Forest on the processed data."""
import pandas as pd
import yaml
import joblib
import os
from sklearn.ensemble import RandomForestClassifier

with open("params.yaml") as f:
    params = yaml.safe_load(f)

n_estimators = params["train"]["n_estimators"]
max_depth = params["train"]["max_depth"]
random_state = params["train"]["random_state"]

X_train = pd.read_csv("data/processed/X_train.csv")
y_train = pd.read_csv("data/processed/y_train.csv").values.ravel()

model = RandomForestClassifier(
    n_estimators=n_estimators,
    max_depth=max_depth,
    random_state=random_state,
    n_jobs=-1,
)
model.fit(X_train, y_train)

os.makedirs("models", exist_ok=True)
joblib.dump(model, "models/model.pkl")
print(f"Model trained with {n_estimators} trees, max_depth={max_depth}")

src/evaluate.py, evaluates and writes metrics:

"""evaluate.py — Evaluate the model and save metrics."""
import pandas as pd
import joblib
import json
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

X_test = pd.read_csv("data/processed/X_test.csv")
y_test = pd.read_csv("data/processed/y_test.csv").values.ravel()

model = joblib.load("models/model.pkl")
y_pred = model.predict(X_test)

metrics = {
    "accuracy": accuracy_score(y_test, y_pred),
    "precision": precision_score(y_test, y_pred),
    "recall": recall_score(y_test, y_pred),
    "f1": f1_score(y_test, y_pred),
}

with open("metrics.json", "w") as f:
    json.dump(metrics, f, indent=4)

print("Metrics:", metrics)

Step 7.2 — a parameters file

DVC wants hyperparameters kept in a params.yaml file rather than hardcoded, which is what makes experiments trackable.

preprocess:
  test_size: 0.2
  random_state: 42

train:
  n_estimators: 100
  max_depth: 5
  random_state: 42

Step 7.3 — define the pipeline stages

Each stage declares its dependencies (-d), parameters (-p), and outputs (-o):

# stage 1: preprocess
dvc stage add -n preprocess \
  -d src/preprocess.py -d data/heart.csv \
  -p preprocess.test_size,preprocess.random_state \
  -o data/processed \
  python src/preprocess.py

# stage 2: train
dvc stage add -n train \
  -d src/train.py -d data/processed \
  -p train.n_estimators,train.max_depth,train.random_state \
  -o models/model.pkl \
  python src/train.py

# stage 3: evaluate
dvc stage add -n evaluate \
  -d src/evaluate.py -d models/model.pkl -d data/processed \
  -M metrics.json \
  python src/evaluate.py

This generates a dvc.yaml describing the whole pipeline:

stages:
  preprocess:
    cmd: python src/preprocess.py
    deps:
    - data/heart.csv
    - src/preprocess.py
    params:
    - preprocess.random_state
    - preprocess.test_size
    outs:
    - data/processed
  train:
    cmd: python src/train.py
    deps:
    - data/processed
    - src/train.py
    params:
    - train.max_depth
    - train.n_estimators
    - train.random_state
    outs:
    - models/model.pkl
  evaluate:
    cmd: python src/evaluate.py
    deps:
    - data/processed
    - models/model.pkl
    - src/evaluate.py
    metrics:
    - metrics.json:
        cache: false

The -M flag marks metrics.json as a metrics file, which is what lets DVC compare it across runs.

Step 7.4 — run the pipeline

dvc repro

Output

Running stage 'preprocess':
> python src/preprocess.py
Preprocessing complete. Train: 242, Test: 61

Running stage 'train':
> python src/train.py
Model trained with 100 trees, max_depth=5

Running stage 'evaluate':
> python src/evaluate.py
Metrics: {'accuracy': 0.85, 'precision': 0.86, 'recall': 0.84, 'f1': 0.85}

Generating lock file 'dvc.lock'

DVC writes dvc.lock, a record of the exact hashes of every input and output from this run. That's what guarantees reproducibility later.

Step 7.5 — the useful part of dvc repro

Run it again with nothing changed:

dvc repro

Stage 'preprocess' didn't change, skipping
Stage 'train' didn't change, skipping
Stage 'evaluate' didn't change, skipping

DVC checks what actually changed and skips everything else. Now change a hyperparameter in params.yaml, say max_depth: 10, and run it again:

dvc repro

Stage 'preprocess' didn't change, skipping
Running stage 'train':           ← only re-runs what's actually affected
...
Running stage 'evaluate':
...

Preprocessing gets skipped, since nothing it depends on changed; only training and evaluation re-run. On a large pipeline, that dependency-aware skipping is a real time saver, confirmed directly: changing max_depth in this exact setup skips preprocessing and re-runs only the two downstream stages, every time.

Step 7.6 — commit the pipeline

git add dvc.yaml dvc.lock params.yaml src/ .gitignore metrics.json
git commit -m "Add DVC pipeline: preprocess -> train -> evaluate"

# push the data/model outputs to the DVC remote
dvc push

Part 8Versioning data and code together

Here's the core workflow that ties Git and DVC into one system. Every change gets committed to both.

1. [change code, data, or params]
2. dvc repro             ← rebuild the affected part of the pipeline
3. git add [code, dvc.yaml, dvc.lock, .dvc files]
4. git commit -m "..."   ← version the pointers and the code together
5. dvc push               ← back up the actual data and models
6. git push                ← back up the code and pointers

git push and dvc push go together. Git pushes the lightweight pointers and code; DVC pushes the heavy data and models. Do only one and a teammate ends up with a broken, half-synced state.

Part 9Experiments, metrics, and params

DVC has real support for ML experimentation built in.

Comparing metrics across versions

dvc metrics show

Path           accuracy   precision   recall    f1
metrics.json   0.85       0.86        0.84      0.85

Tracking parameters

dvc params diff

Path          Param                  HEAD    workspace
params.yaml   train.max_depth        5       10

Quick experiments with dvc exp

The dvc exp commands let you try variations without cluttering up Git history:

# change a parameter on the fly
dvc exp run --set-param train.n_estimators=200

# try a different depth
dvc exp run --set-param train.max_depth=8

# compare everything
dvc exp show

 Experiment          n_estimators   max_depth   accuracy   f1
 ─────────────────────────────────────────────────────────────
 workspace           200            8           0.88       0.87
 main                 100            5           0.85       0.85
 ├── exp-a1b2c        200            5           0.87       0.86
 ├── exp-c3d4e        100            8           0.86       0.85
 └── exp-e5f6g        200            8           0.88       0.87

This table is one of the most genuinely useful things DVC gives you for ML work: which hyperparameters produced which results, at a glance.

# promote the best one to your workspace
dvc exp apply exp-e5f6g

# then commit it properly
git add dvc.lock params.yaml metrics.json
git commit -m "Adopt best config: 200 trees, max_depth=8 (acc 0.88)"
dvc push
git push

Part 10Full heart disease project walkthrough

Let's run the whole thing as one continuous sequence, as if starting from scratch.

git init
dvc init
git commit -m "Initialize Git and DVC"

mkdir data
# place heart.csv in data/
dvc add data/heart.csv
git add data/heart.csv.dvc data/.gitignore
git commit -m "Add heart disease dataset (v1, 303 rows)"

# configure and push to remote
mkdir -p /tmp/dvc-remote
dvc remote add -d myremote /tmp/dvc-remote
git add .dvc/config
git commit -m "Configure DVC remote"
dvc push

mkdir src
# create src/preprocess.py, src/train.py, src/evaluate.py, params.yaml as shown above

# define the pipeline stages (same three dvc stage add commands as Part 7)

dvc repro

git add dvc.yaml dvc.lock params.yaml src/ .gitignore metrics.json
git commit -m "Add full ML pipeline, baseline accuracy 0.85"
dvc push

# replace the data file with the new version
# (copy the new, larger heart.csv into data/)

# tell DVC the tracked file changed
dvc add data/heart.csv

To track the changes with git, run:
        git add data/heart.csv.dvc

# the pointer's hash changed — commit the new pointer
git add data/heart.csv.dvc
git commit -m "Update dataset to v2 (added 500 records)"

# re-run the pipeline on the new data
dvc repro

Running stage 'preprocess':     ← data changed, so everything reruns
...
Running stage 'train':
...
Running stage 'evaluate':
Metrics: {'accuracy': 0.89, ...}    ← more data, better accuracy

git add dvc.lock metrics.json
git commit -m "Retrain on v2 data, accuracy improved to 0.89"
dvc push
git push

git tag -a v1.0 -m "Baseline model on 303-record dataset, acc 0.85"
git tag -a v2.0 -m "Model on expanded 800-record dataset, acc 0.89"
git push --tags

Part 11Switching between versions: the payoff

This is where all the setup pays for itself. Say your manager asks: can you reproduce the exact model and data from the v1.0 release you showed the client? With Git alone, you'd only get the code back. With Git and DVC together, you get everything, code, data, and model, in exact sync.²

# step 1: check out the old code AND pointer files, via Git
git checkout v1.0

At this point, your .dvc and dvc.lock files describe the v1.0 state, but the actual data/heart.csv sitting on disk is still the v2 version. Pointers and real files are mismatched.

# step 2: tell DVC to restore the data matching these pointers
dvc checkout

M       data/heart.csv          ← restored to the 303-record version
M       data/processed/
M       models/model.pkl        ← the original v1 model is back

The whole workspace, code, the exact 303-row dataset, and the original trained model, is now precisely as it was at the v1.0 release. Run it, inspect it, demo it.

dvc metrics show
# accuracy: 0.85 — the original v1 numbers, reproduced exactly

Back to the latest:

git checkout main
dvc checkout

The mental model to keep: git checkout switches the pointers and code. dvc checkout makes the actual data and models match those pointers. Run them as a pair whenever you're time-traveling.

The complete Git + DVC cheat sheet

SETUP
─────────────────────────────────────────────
git init                          Initialize Git
dvc init                          Initialize DVC (needs Git first)
dvc remote add -d myremote <url>  Set default remote storage

TRACKING DATA
─────────────────────────────────────────────
dvc add data/file.csv             Start tracking a data file
git add data/file.csv.dvc         Commit the pointer (with Git)
dvc add data/file.csv             Re-run after data changes

PIPELINES
─────────────────────────────────────────────
dvc stage add -n <name> ...       Define a pipeline stage
dvc repro                         Run the pipeline (changed stages only)
dvc dag                           Visualize the pipeline graph

SYNCING
─────────────────────────────────────────────
dvc push                          Upload data/models to remote
dvc pull                          Download data/models from remote
git push                          Upload code + pointers
git pull                          Download code + pointers

EXPERIMENTS & METRICS
─────────────────────────────────────────────
dvc exp run --set-param k=v       Run a quick experiment
dvc exp show                      Compare experiments in a table
dvc exp apply <exp-id>            Adopt an experiment's results
dvc metrics show                  Show current metrics
dvc metrics diff                  Compare metrics vs. last commit
dvc params diff                   Compare params vs. last commit

TIME TRAVEL (always run as a pair)
─────────────────────────────────────────────
git checkout <tag/commit>         Switch code + pointers
dvc checkout                      Restore matching data + models

What goes where

Summary

Git and DVC now form one complete versioning system for ML work. Git alone falls short on data and models because it was built for text, not binaries; DVC picks up exactly that slack. The pointer model is the whole trick: Git tracks tiny .dvc files, DVC tracks the heavy data behind them. A cache stores things locally, a remote backs them up to the cloud. Pipelines, via dvc repro, make your work reproducible and dependency-aware. Experiments and metrics, via dvc exp show, let you compare model versions honestly. And time travel, git checkout paired with dvc checkout, restores any past code-plus-data-plus-model combination on demand.

From here, worth exploring: CML (Continuous Machine Learning) for running training and posting metrics automatically on every pull request, DVC Studio for a team-wide dashboard of experiments, data registries via dvc import for sharing versioned datasets across projects, and a proper model registry for promoting and deploying specific versions to production.

You can now version an entire ML project end to end, every line of code and every byte of data, and reproduce any past state on demand. That's the actual foundation reproducible machine learning is built on.