MLOps for ML practitioners · Article 6

Flask for ML deployment: just enough to serve your model

How Flask actually works, and exactly what to write to turn a trained model into an API other programs can actually call.

Part of the MLOps series · read Article 5 first · next up: FastAPI and Pydantic

Article 5 used Flask to wrap a heart disease model in an API, but skated past how Flask actually works. This article fills that gap. By the end, you'll understand Flask well enough to take any trained model and expose it as a web service, something other programs, or people, can send data to and get predictions back from.

We're not covering everything Flask can do; it's a full web framework and you could write a whole book on it. We're covering exactly the parts an ML practitioner needs to deploy a model, and nothing more. This is the "just enough Flask" article.

What this article covers

#Topic
1Why does an ML model need a web framework at all?
2What is Flask and how does it work?
3The request-response cycle, the core idea
4Routes, methods, and endpoints
5Handling input: JSON requests
6Sending output: JSON responses
7The complete ML serving pattern
8Testing your API
9Production notes: Gunicorn, and why app.run() isn't enough

Part 1Why does an ML model need a web framework?

You've trained a model. It lives in a model.pkl file. You can load it in a Python script and call model.predict(). So why bring Flask into it at all?

The problem: a model file isn't usable by others

Think about who actually needs to use your model: a website where users enter their health data and see a risk score, a mobile app sending sensor readings and expecting predictions back, another team's software that needs your fraud-detection score, a business dashboard pulling live predictions.

None of these are Python scripts running on your laptop. They're separate programs, often on different machines, sometimes written in entirely different languages. They can't just import joblib and load your .pkl file. They need a standard way to ask your model a question and get an answer.

The restaurant analogy

Picture your trained model as a skilled chef locked in a kitchen. The chef cooks brilliantly, but customers can't walk in and talk to the chef directly, that would be chaos. What you need is a waiter and a service window: customers place an order (send data) at the window, the order goes back to the chef (your model), and the finished dish (the prediction) comes back out.

Flask is that waiter and service window. It stands between the outside world and your model, accepts requests in a standard format (HTTP), passes the data through, and sends the prediction back, so any program from anywhere can use your model.

What "serving a model" means

Turning a model file into a live, callable service is called serving, or deploying, the model. Flask is one of the simplest tools in Python for doing this.

Before Flask model.pkl usable only inside a Python script isolated, one machine only After Flask Any client web, mobile, other code HTTP prediction Flask + model accessible to the whole world
Fig. 1 — the same model file, before and after Flask stands in front of it

Part 2What Flask is, and how it works

Flask is a lightweight Python web framework. "Lightweight" means it gives you the essentials for a web service without forcing much structure on you, which suits ML APIs well, since most only need a couple of endpoints.

What Flask actually does

At its core, Flask does one thing: it listens for incoming web requests and runs your Python function to respond to them. You write functions, tell Flask "when a request hits this address, run this function," and it handles the messy networking underneath: listening on a port, parsing the HTTP request, sending the response back.

The smallest Flask app there is

from flask import Flask

# 1. Create the Flask application object
app = Flask(__name__)

# 2. Define a route: "when someone visits '/', run this function"
@app.route("/")
def home():
    return "Hello, my model is alive!"

# 3. Start the server
if __name__ == "__main__":
    app.run()

Run it with python app.py, open http://localhost:5000 in a browser, and you'll see the message. Every Flask app, ML ones included, is built on these same three steps.

Step 1 app = Flask(__name__)

Creates the application. __name__ is a special Python variable telling Flask where the app lives, so it can find files relative to it. You'll write this line, essentially unchanged, in every Flask app you ever build.

Analogy: opening the restaurant and hanging up the "we're open" sign.

Step 2 @app.route("/")

A decorator, the @ symbol, connecting a web address to a function: "when a request comes to /, run the function below it." / is the root address, the homepage.

Analogy: posting a menu that says "if a customer asks for the homepage, the waiter does this."

Step 3 app.run()

Starts the web server, which begins listening for requests. Without this line, nothing happens at all.

Analogy: unlocking the door and letting customers in.

Part 3The request-response cycle

Everything in Flask comes down to one pattern: a request comes in, your code runs, a response goes out. Understand this cycle and you understand Flask.

CLIENT FLASK SERVER 1. REQUEST POST /predict — here's patient data 2. Flask matches the address to your function and runs it 3. Your function calls model.predict(), builds an answer 4. RESPONSE prediction: 1, probability: 0.78
Fig. 2 — the request-response cycle every Flask API runs on

For an ML API specifically: a client sends patient data to your prediction address, Flask receives it and runs your prediction function, that function feeds the data to the model and gets a prediction, Flask sends it back. Everything else in this article is detail layered on top of this one loop.

A quick word on HTTP methods

A request carries a method that signals intent. You only need two for ML work: GET, "give me something," used for simple checks like "are you alive?" and what your browser sends when you visit a URL; and POST, "here's some data, do something with it," used for predictions, since the client is sending the patient's measurements. GET is asking the waiter "are you open?" POST is handing over your order with all the details. Predictions use POST, because the client needs to send input features to the model.

Part 4Routes, methods, and endpoints

A route, or endpoint, is a specific address in your API paired with the function that handles it. An ML API typically needs just two or three.

from flask import Flask

app = Flask(__name__)

# a health-check endpoint (GET) — confirms the API is running
@app.route("/health", methods=["GET"])
def health():
    return "OK"

# a prediction endpoint (POST) — does the actual ML work
@app.route("/predict", methods=["POST"])
def predict():
    return "prediction goes here"

@app.route("/health", methods=["GET"]) defines an endpoint at /health that only responds to GET; @app.route("/predict", methods=["POST"]) defines one at /predict that only responds to POST. The methods=[...] list is what restricts this. Leave it off, and Flask assumes GET only.

Why bother with a /health endpoint? In real deployments, monitoring systems and container orchestrators, Kubernetes, Docker's own health checks, periodically ping it to confirm your service is alive. It's a tiny endpoint, but a genuine professional standard, exactly what Article 5 used.

Analogy: /health is the "are you open?" sign customers can check. /predict is the actual service counter where the real work happens.

Part 5Handling input: JSON requests

For ML, clients send input as JSON, a simple, universal text format that looks like a Python dictionary. One patient's data:

{
  "age": 63,
  "sex": 1,
  "cp": 3,
  "trestbps": 145,
  "chol": 233
}

Reading JSON in Flask

Flask gives you a request object holding everything about the incoming request:

from flask import Flask, request

app = Flask(__name__)

@app.route("/predict", methods=["POST"])
def predict():
    # get the JSON data the client sent
    data = request.get_json()

    # now 'data' is a plain Python dictionary
    age = data["age"]
    chol = data["chol"]

    return f"Received age={age}, chol={chol}"

The key line is data = request.get_json(): it takes the raw JSON and converts it into a normal Python dictionary, which you handle like any other dictionary from there.

Analogy: the waiter takes the customer's written order (JSON) and reads it out loud in a form the chef understands (a Python dictionary).

You do need from flask import request to use this. The request object is automatically available inside your route functions, representing whichever request is currently coming in.

Part 6Sending output: JSON responses

Send predictions back as JSON too. Flask's jsonify function does this cleanly, turning a Python dictionary into a proper JSON response.

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/predict", methods=["POST"])
def predict():
    data = request.get_json()

    # (model prediction logic would go here)
    prediction = 1
    probability = 0.78

    # convert a Python dict into a JSON response
    return jsonify({
        "prediction": prediction,
        "probability": probability,
        "interpretation": "Heart disease likely"
    })

The key line is return jsonify({...}): build a Python dictionary with your results, and jsonify turns it into JSON any client can read.

Analogy: the chef plates the finished dish (jsonify) so it arrives presented the way the customer expects, rather than handed over in a messy pot.

Why not just return {...}? Modern Flask does auto-convert a plain dictionary to JSON, so that often works too. But jsonify is explicit, handles edge cases like content-type headers correctly, and makes your intent clear. Use it.

A real gotcha: NumPy types

ML models often return NumPy types, numpy.int64, numpy.float64, instead of plain Python ones, and jsonify genuinely fails on these.1 Fix it by converting explicitly:

prediction = int(model.predict(input_df)[0])        # numpy.int64 → int
probability = float(model.predict_proba(input_df)[0][1])  # numpy.float64 → float

This is exactly why Article 5 wrapped the model's outputs in int(...) and float(...). Now you know why: it's what makes them JSON-friendly.

Part 7The complete ML serving pattern

Now assemble the whole thing into the standard pattern you'll reuse for nearly every model you deploy.

"""app.py — The standard Flask pattern for serving an ML model."""
from flask import Flask, request, jsonify
import joblib
import pandas as pd

# ── 1. Create the app ──────────────────────────────────
app = Flask(__name__)

# ── 2. Load the model ONCE at startup ──────────────────
# This runs once when the server starts, NOT on every request.
model = joblib.load("models/model.pkl")

# The exact feature names/order the model expects
FEATURES = ["age", "sex", "cp", "trestbps", "chol", "fbs", "restecg",
            "thalach", "exang", "oldpeak", "slope", "ca", "thal"]

# ── 3. Health check endpoint ───────────────────────────
@app.route("/health", methods=["GET"])
def health():
    return jsonify({"status": "healthy"})

# ── 4. Prediction endpoint ─────────────────────────────
@app.route("/predict", methods=["POST"])
def predict():
    # a) Read the incoming JSON
    data = request.get_json()

    # b) Arrange it into the model's expected format
    input_df = pd.DataFrame([[data[f] for f in FEATURES]], columns=FEATURES)

    # c) Get the prediction (convert numpy types to Python types)
    prediction = int(model.predict(input_df)[0])
    probability = float(model.predict_proba(input_df)[0][1])

    # d) Send the result back as JSON
    return jsonify({
        "prediction": prediction,
        "probability": round(probability, 4),
        "interpretation": "Heart disease likely" if prediction == 1
                          else "Heart disease unlikely"
    })

# ── 5. Start the server ────────────────────────────────
if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

The design decision that matters most: load the model once

model = joblib.load("models/model.pkl") sits at the top level, outside any function. That's deliberate. The right way, shown above, loads the model once when the server starts; every request afterward reuses it, fast. Load it inside the predict() function instead, and it reloads from disk on every single request, adding real delay and wasting memory. Don't do that.

Analogy: you hire the chef once, when the restaurant opens, not fresh for every single order. Loading the model per request is like firing and rehiring the chef for every customer.

Why host="0.0.0.0"?

This came up in Article 5, and it matters especially inside Docker. app.run() with no host defaults to 127.0.0.1, meaning the API is reachable only from the same machine, or the same container. Nothing outside can reach it. host="0.0.0.0" tells Flask to accept requests from any network interface, making it reachable from outside the container or machine.

Analogy: 127.0.0.1 is a private intercom that only works inside your own house. 0.0.0.0 is a public phone line the outside world can actually call.

Part 8Testing your API

Once your app is running (python app.py), you've got a few ways to test it.

Option A — curl

# health endpoint
curl http://localhost:5000/health

# a prediction, sending JSON with -d
curl -X POST http://localhost:5000/predict \
  -H "Content-Type: application/json" \
  -d '{"age": 63, "sex": 1, "cp": 3, "trestbps": 145, "chol": 233, "fbs": 1, "restecg": 0, "thalach": 150, "exang": 0, "oldpeak": 2.3, "slope": 0, "ca": 0, "thal": 1}'

-X POST sets the method; -H "Content-Type: application/json" tells the server what's coming; -d '{...}' is the actual JSON payload.

Option B — Python, the requests library

How another Python program would call your API:

import requests

response = requests.post(
    "http://localhost:5000/predict",
    json={
        "age": 63, "sex": 1, "cp": 3, "trestbps": 145, "chol": 233,
        "fbs": 1, "restecg": 0, "thalach": 150, "exang": 0,
        "oldpeak": 2.3, "slope": 0, "ca": 0, "thal": 1
    }
)

print(response.json())
# {'prediction': 1, 'probability': 0.78, 'interpretation': 'Heart disease likely'}

The json={...} argument sends your dictionary as JSON automatically; response.json() reads the reply back into a Python dictionary.

Option C — the browser, GET endpoints only

For GET endpoints like /health, just visit http://localhost:5000/health directly. POST endpoints can't be tested this way; typing a URL always sends GET.

Part 9Production notes: why app.run() isn't enough

Something beginners often miss: run python app.py and Flask may print a warning like this:

WARNING: This is a development server. Do not use it in a production deployment.

Flask's built-in server is meant for development and testing only. It's single-threaded and isn't built to handle real traffic; enough simultaneous users will overwhelm it.2

The fix: a production server, Gunicorn

For real deployment, run your Flask app through a production-grade server like Gunicorn, short for Green Unicorn.3 Gunicorn runs multiple worker processes to handle many requests at once.

pip install gunicorn

# run with 4 worker processes
# "app:app" means: in app.py, use the object named 'app'
gunicorn --workers 4 --bind 0.0.0.0:5000 app:app

--workers 4 runs 4 parallel processes handling concurrent requests; --bind 0.0.0.0:5000 listens on port 5000, reachable from outside; app:app follows the pattern filename:flask_object_name.

app.run() 1 process requests wait in line gunicorn --workers 4 worker worker worker worker 4 requests handled at once
Fig. 3 — app.run() is one waiter; Gunicorn is a full waitstaff

Analogy: app.run() is a single waiter serving one table at a time, fine for practicing at home. Gunicorn is a full waitstaff serving many tables at once, what you actually need once the restaurant is open to the public.

Connecting back to Docker

Article 5's Dockerfile ended with CMD ["python", "app.py"] for simplicity. For production, swap that line for Gunicorn:

# development (Article 5)
CMD ["python", "app.py"]

# production
CMD ["gunicorn", "--workers", "4", "--bind", "0.0.0.0:5000", "app:app"]

Add gunicorn to requirements.txt too. Everything else in the Dockerfile stays the same. One line takes your containerized model from "works for testing" to "ready for real traffic."

Summary

You now know enough Flask to deploy any ML model. Models need it to turn an unusable file into something any program can call, the waiter and service window. Flask itself runs on a three-step pattern: create the app, define routes, run the server, all built on the same request-response cycle every API follows. GET handles checks, POST handles predictions. request.get_json() reads input; jsonify() sends output, and numpy types need wrapping in int()/float() first. The full serving pattern loads the model once at startup, never per request. You can test with curl, Python's requests library, or a browser for GET endpoints, and you know why app.run() alone isn't enough in production, and how Gunicorn fixes that.

ToolRoleArticle
Git / GitHubVersion and share code1, 2
DVCVersion data and models3
FlaskWrap the model in an API6 (this one)
DockerPackage the API and environment4, 5

The natural flow: write model code, versioned with Git; track datasets and trained models with DVC; wrap the model in an API with Flask; package the whole thing into a portable container with Docker, running identically anywhere, served in production by Gunicorn.

Worth exploring next: FastAPI, a modern alternative to Flask with automatic input validation and interactive docs, increasingly the default for ML APIs, which is exactly where this series goes next; Pydantic, for validating incoming requests so bad input fails cleanly instead of crashing; proper request validation and error handling; batch predictions for handling many records in one request; and authentication, protecting your API with keys or tokens before it's public.

You can now take a trained model and turn it into a live, callable service, the last bridge between "I built a model" and "people can actually use it."