Everything so far has described data you already have. This article is about making a confident guess about data you don't, using a smaller piece of data you do. That's inferential statistics, and it's really where data science starts.
Say you're a data scientist at Netflix. There are 260 million subscribers around the world, and your product team walks up and asks: what's the average number of hours our users watch per week?
Netflix keeps a record every single time anyone hits play, pause, or stop, for every user. So here's a fair question: why bother sampling when the full data is right there? If all anyone cared about was the past, this wouldn't even be a debate. Every subscriber's complete history is already sitting in a database. Add it all up, divide by the number of people, and that's the actual average. Not an estimate, not a guess, the real number. When you have everyone's data like that, it's called a census, and there's no reason to sample when you can just do the math on everyone.
So why do companies still sample? Two real reasons.
The first one is bigger than it sounds: most of these decisions aren't really about the past at all, they're about what happens next. Will people watch more if Netflix shifts toward longer movies instead of short episodes? What will next quarter look like? What will someone who hasn't even signed up yet probably do? None of that is sitting in any log file, because it hasn't happened yet. The best you can do is take your most recent data and use it as a stand-in for a future that doesn't exist yet.
The second reason is just practical. Every time you want a quick number, running it against tens of billions of raw events takes time and costs money. Doing that over and over, every time someone wants to check something, gets expensive fast. So instead, people pull a sample, get a fast answer, and only run the full calculation when the exact number actually matters.
Here's what actually happens in practice: you grab a sample, ask a few thousand people, crunch the numbers, and make a confident call about everyone, along with an honest range of how wrong you might be. That's the whole idea behind inferential statistics. You're taking a value you can't directly see and figuring it out from a smaller piece you can.
And that's basically the line between two roles: an analyst tells you what already happened. A data scientist uses that to figure out what's likely to happen next, and what to actually do about it.
Before any theory, here are five real situations big companies deal with using exactly what this article covers. Four of these, Netflix, Uber, Swiggy, and Stripe, already have complete logs of everything that's happened. The sampling in each case isn't about missing data, it's about something still in the future, still happening right now, or just too expensive to calculate on the full dataset every single time. The fifth, Spotify, is genuinely different: there's no log anywhere that could answer their question, no matter how good their data collection is.
Netflix wants the true average daily watch time across all subscribers, to help decide how much to invest in longer movies versus short episodes going forward. That decision is about future spending, and future viewing behavior obviously has no logs yet. So instead of crunching the full historical dataset every time someone wants to check this, Netflix takes a sample of 1,000 users, looks at their daily watch time, and uses that to estimate the average for everyone, along with how confident it can be. Hold onto that number, 1,000 people turns out to be more than enough, once the confidence interval is built later in this article.
The question: "We saw an average of 1.8 hours a day in our sample. What's the real average across all users, and how sure are we?"
Uber runs millions of trips a day in Mumbai. To set driver incentives and pricing for next month, a month that hasn't happened yet, they need a good estimate of what trip times will look like. So they take a sample of 2,000 recent trips, the most representative slice available right now, and build a confidence interval around it. That interval becomes their best guess for what's coming.
The question: "Our sample shows an average trip of 22 minutes. What range can we confidently say covers the real average for all Mumbai trips?"
Swiggy advertises "delivery in 30 minutes or less." The ops team wants to check, right now, whether that's actually true, one city at a time. Since deliveries keep happening every single minute, there's no such thing as a finished population to check, it's always still rolling in. Starting with Bangalore, they take the 500 most recent orders and use that to estimate how the city is doing right now.
The question: "Is our real average delivery time actually under 30 minutes? How confident can we be?"
Spotify's growth team wants to know what percentage of their 350 million free users would upgrade to Premium if they saw a special discount offer. Here's the thing: there's genuinely no way to answer this from data already on hand, because nobody has ever seen this exact offer before. There's no record anywhere of how the full 350 million would react to something they've never seen. The only way to actually find out is to show it to some people and watch what happens. So they run the offer for 10,000 free users and track how many convert.
The question: "We saw 12% conversion in our sample. What's the real conversion rate across all free users, and how much uncertainty is there?"
Stripe processes payments for millions of small businesses. Their risk team wants the true average transaction amount for businesses with under 10 employees, so they can set thresholds for catching fraud. A fraud threshold is naturally about the future, its whole job is to catch unusually large transactions before they clear, transactions that haven't happened yet. So it has to be built using recent data as a stand-in for what's coming. They sample 4,000 recent transactions from businesses that fit the criteria.
The question: "What's the confidence interval around the 90th percentile of transaction values, the cutoff above which a transaction counts as unusually large?"
Notice the pattern across all five: the population is either something that hasn't happened yet, still happening right now, or just too expensive to check in full every time. A manageable sample can be measured right now. And the population value has to be figured out from that sample, honestly, with the uncertainty stated up front. Let's build the toolkit to actually solve these.
Two words that show up constantly from here on: population and sample.
The population is the entire group actually being asked about: every Netflix subscriber, every Uber trip ever taken in Mumbai, every Swiggy order in India. The sample is the smaller piece of that group actually looked at: the 1,000 Netflix users surveyed, the 2,000 Uber trips pulled from the logs, the 500 Swiggy orders reviewed.
Think of a chef tasting soup from a massive pot. Nobody drinks the whole pot to check if it needs more salt, one spoonful tells you everything you need to know. That spoonful is the sample. The pot is the population.
| Term | Symbol | What it means |
|---|---|---|
| Population mean | μ (mu) | The real average across the whole population |
| Population std dev | σ (sigma) | The real spread across the whole population |
| Sample mean | x̄ (x-bar) | The average calculated from a sample |
| Sample std dev | s | The spread calculated from a sample |
| Sample size | n | How many data points are in the sample |
A simple trick to keep these straight: Greek letters (μ, σ) belong to the population, because they're almost never actually known. Latin letters (x̄, s) belong to the sample, because those are the ones actually calculated. And that's really the whole game of inferential statistics in one line: use x̄ and s, the things measurable, to make a smart guess about μ and σ, the things that aren't.
Not every sample is equally good. A bad sample can wreck a conclusion, no matter how sophisticated the math applied afterward.
Simple random sampling. Everyone in the population has an equal shot at being picked. Spotify could give every free user a random number, sort those numbers, and grab the top 10,000, nobody more or less likely to be picked than anyone else.
Stratified sampling. Split the population into groups first (called strata), then sample from each group separately. Swiggy might split users into Metro, Tier-2, and Tier-3 cities, then sample 60% from Metro, 30% from Tier-2, and 10% from Tier-3, matching how the real population is actually split, so no group gets lost in the noise. It's a bit like splitting a crowd into students, professionals, and retirees, and picking randomly within each group instead of just grabbing from the crowd as a whole. The final mix ends up looking like the real crowd.
Systematic sampling. Pick every k-th item off a list. If Uber has a million trips logged today and wants a sample of 2,000, grab every 500th trip.
Convenience sampling, use with caution. Sampling whoever's easiest to reach. Surveying only users who happened to click a feedback pop-up isn't representative, since those users tend to already have strong opinions.
The one rule that matters more than any formula in this article: a sample has to actually represent the population. A biased sample gives biased conclusions, no matter how sophisticated the statistics applied afterward. Garbage in, garbage out.
No sample is perfect. A sample mean (x̄) will almost never land exactly on the true population mean (μ). That gap has a name: sampling error.
This isn't a mistake, it's just natural randomness. Take 100 different samples, and you'd get 100 slightly different x̄ values, some close to μ, some further off. That variation, in how x̄ bounces around from sample to sample, is basically the entire thing inferential statistics exists to deal with.
Say Netflix doesn't just take one sample of 1,000 users. Say they take 500 separate samples, each with 1,000 users. Every one of those samples gives a slightly different average watch time. So here's the question: if all 500 of those averages were plotted, what would that picture actually look like?
That's exactly the question the Central Limit Theorem answers, and the answer genuinely surprises most people the first time they see it.
Take a bunch of random samples of size n from any population, no matter what shape that population is, and the averages of those samples will form a normal, bell-shaped curve, as long as n is big enough, usually 30 or more does the trick. It doesn't matter if the original data is skewed, lopsided, or a strange shape. The averages of samples pulled from it will always settle into a bell curve.
That bell curve has two useful properties: its center lands right on μ, the true population mean, and its spread equals σ/√n, a quantity important enough to get its own name, the Standard Error.
Imagine a city where people's daily coffee habit is spread out completely evenly between 0 and 6 cups, not bell-shaped at all, just flat. Some people drink 0, some drink 6, everything in between is equally likely.
Now grab 500 random groups of 40 people each, and calculate the average coffee consumption for each group. Even though the original data was totally flat, those 500 averages will bunch up into a nice bell curve centered around 3 cups, the real population average. That's the Central Limit Theorem doing its thing, the same "flat distribution becomes a bell curve once you average" pattern the dice-rolling example showed back in Day 10.
Because it means you get to borrow everything already known about the normal distribution, which is a lot, to make solid statements about a sample average, no matter what the raw data underneath it actually looks like. This is the bridge that makes confidence intervals and hypothesis tests possible. Without CLT, basically none of the rest of this article would work.
Rather than just asserting it, here's the claim checked directly: draw a heavily skewed population, take 1,000 samples of size 50 from it, and look at what happens to the distribution of the sample means.
The population is badly skewed, most gaps are short, a handful are very long. But the 1,000 sample means land in a clean bell curve, centered almost exactly on the true population mean, with a spread that matches the σ/√n formula almost perfectly. That's the Central Limit Theorem, delivered exactly as promised.
The Standard Error answers one simple question: on average, how far off is a sample mean likely to be from the real population mean?
The takeaway in plain terms: bigger sample means smaller SE means a more precise estimate. Smaller sample means bigger SE means a less precise estimate. Taste one spoonful of soup, and there's not much confidence about the whole pot, that's a big SE. Taste twenty spoonfuls from different spots, and there's real confidence, a small SE. More sample, better estimate, simple as that.
SE keeps shrinking as n grows, but slower and slower each time. Going from 100 to 1,000 users doesn't make an estimate 10 times more precise, just about 3 times more precise, because of that square root sitting in the formula.
Say a sample of 1,000 Netflix users gives x̄ = 1.8 hours a day. Walking up to the product team and saying "the average watch time is exactly 1.8 hours" is overselling it. That's a point estimate, and it's basically never going to be exactly right.
What should be said instead is something like: "I'm 95% confident the real average is somewhere between 1.75 and 1.85 hours a day." That's a confidence interval, an honest range that accounts for the fact that this comes from a sample, not the whole population.
A confidence interval is a range of values, built from a sample, believed to contain the true population number, along with a stated level of confidence.
Three pieces go into this: x̄, the sample mean, sits right in the middle. z*, the critical value, decides how wide a net to cast based on how confident the estimate needs to be. And s/√n, the standard error, accounts for sample size and spread.
Honestly, this is the single most misunderstood idea in all of statistics. It does not mean "there's a 95% chance the true mean is somewhere in this specific interval." It does mean: "if this whole sampling process were repeated 100 times, and a fresh confidence interval built each time, about 95 of those 100 intervals would actually contain the true population mean."
Picture it like fishing with a net sized for "95% confidence." Cast that net 100 times, and it catches the fish, the true mean, about 95 of those times. Any single cast might miss. But the net itself is built to work 95% of the time. The true mean never moves, it's fixed, it's just unknown. It's the interval that moves around every time the sample changes. A 95% CI just means 95% of those moving intervals will happen to land on the fixed true mean.
The confidence level decides the critical value (z*):
| Confidence Level | z* (Critical Value) | What it means |
|---|---|---|
| 90% | 1.645 | A narrower net, a bit less sure |
| 95% | 1.960 | The usual go-to |
| 99% | 2.576 | A wider net, very sure |
Want to be 90% sure of catching the fish? A medium net does the job. Want 99%? A much bigger net is needed. More confidence always means a wider interval, there's no way around it. Extra certainty can't be bought without giving up some precision in return.
That whole "z* × (s/√n)" piece has its own name too: the Margin of Error, or MoE. It's literally the "±" part of the estimate.
There are two flavors of critical value, and the rule is simpler than most people make it sound. Use the z-distribution only when the true population standard deviation σ is actually known, which in real life almost never happens. Use the t-distribution whenever σ is unknown and it's being estimated with s from a sample instead, which is, basically, always.
So why do so many textbooks say "just use z once n is 30 or more"? Because once n gets large enough, the t-distribution starts looking almost identical to the z-distribution anyway, by n=100 the difference is basically nothing. It's a shortcut people use, not a hard rule. The safest habit is to always use t when σ is unknown, no matter how big the sample is. In Python, scipy.stats.t.ppf() gives the right answer every time, and it automatically starts behaving like z once the sample gets big.
Worth keeping these two ideas separate: CLT, needing n ≥ 30, is about whether the shape of the sampling distribution looks normal. Whether to use z or t is a completely different question, about how well σ is known.
This is one of the most useful, and most skipped, questions in data science. Before collecting any data, the answer to "how many data points do I actually need" should already be known.
Too few, and the confidence interval ends up so wide it's useless ("we're 95% sure the conversion rate is somewhere between 3% and 45%," not exactly actionable). Too many, and time, money, and computing power get spent on data that wasn't needed in the first place.
Two tools help here: a rough minimum floor from CLT, and an actual formula that gives the exact n needed for a chosen precision.
This is the baseline. For CLT to actually kick in and allow normal-distribution-based intervals safely: n ≥ 30 is fine for data that's only moderately skewed. n ≥ 100 to 150 is safer once data is heavily skewed, think revenue, transaction sizes, or delivery times with the occasional huge outlier. For proportions, n × p̂ ≥ 10 and n × (1 − p̂) ≥ 10 should both hold, a check with its own name: the success-failure condition.
These are floors, not targets. The real answer to "how big should my sample be" comes from how precise the estimate actually needs to be.
The confidence interval formula can be flipped around. Instead of starting with n and computing the interval, start with the precision wanted and solve for the n needed.
Start with Margin of Error (E) = z* × (σ / √n), and rearrange for n: √n = z* × σ / E, so n = (z* × σ / E)². That's the sample size formula for a mean:
z* is the critical value based on how confident the estimate needs to be (1.96 for 95%). σ is the best guess at the population standard deviation, from a pilot sample, an old study, or domain knowledge. E is the biggest margin of error acceptable, which is really a business call, not a math one.
Think of E as a "how wrong can I afford to be" budget. Estimating average delivery time and fine being off by ±5 minutes? Set E = 5. Need it tighter, say ±1 minute? Set E = 1, just expect to need a much bigger sample to get there.
Estimating something like a conversion rate instead of a mean uses this version instead:
Here p̂ is the estimated proportion. With genuinely no idea what it might be, p̂=0.5 is the safe, conservative choice, since it's the value that makes p̂(1−p̂) as large as possible, giving the biggest, safest sample size recommendation.
A few things jump out here. Precision is expensive: Netflix needs 865 people for ±3 minutes of precision, but jumping to 7,779 people only buys ±1 minute, that's 9 times more data for just 3 times better precision. The square root is doing its thing again, and it means tightening precision gets disproportionately expensive.
Spotify's case shows why a good guess for p̂ actually matters. They expect around 12% conversion. Going in blind with the conservative p̂=0.5 would have meant planning for 9,604 people instead of just 4,057, more than double what was actually needed. A quick pilot study, or even just a rough prior guess, can save a lot of unnecessary sampling.
Higher confidence costs too. Going from 95% to 99% confidence on Netflix's estimate pushes the required sample from 865 to 1,493, about a 70% jump. More certainty is never free, it's always paid for somewhere, and here that payment is in sample size.
And σ is really the wildcard in all of this. Without already knowing the population standard deviation, running a small pilot study first, collecting 30 to 50 data points, calculating s from those, and using that to size the real sample, is completely standard practice.
| Question | What it tells you |
|---|---|
| Are you estimating a mean or a proportion? | Which formula to use |
| What margin of error can you live with? (E) | Your precision needs, a business call |
| What confidence level do you actually need? (90/95/99%) | Your certainty needs, also a business call |
| What's your best guess at σ or p̂? | Pilot study, past data, or a safe default |
A practical note: in the real world, sample size is often capped by budget, time, or how much data can actually be collected. If the formula says n=10,000 but only 2,000 is affordable, plug 2,000 back into the CI formula and see what margin of error would actually result. Then have an honest conversation about whether that's good enough for the decision at hand.
The classical formula used so far, x̄ ± z* × s/√n, quietly assumes two things: the data is roughly normal, or n is big enough for CLT to kick in, and the thing being estimated is a mean, not a median, a max, a correlation, or anything more complicated.
So what happens when a confidence interval is needed for the median delivery time (Swiggy's case)? Or the 90th percentile of transaction sizes (Stripe)? Or some ratio, like revenue per trip divided by cost per trip? Or a model metric like AUC or RMSE?
Proportions, like a conversion rate, do have their own classical formula, used for Spotify shortly. But for everything else on that list, there's no clean formula waiting. The classical approach just doesn't stretch that far. This is exactly where Bootstrap comes in.
Bootstrap is a resampling trick that builds a confidence interval for basically any statistic, without needing a special formula or assuming anything about how the data is shaped.
The core idea in one line: resampling from the real population isn't possible, but resampling from a sample is, and treating that sample as if it were the population does the job. In practice, the original sample gets resampled thousands of times (with replacement), the statistic of interest gets calculated on every single resample, and the spread of all those results becomes the answer.
Say there's a sample of 200 Swiggy delivery times, and a 95% confidence interval is needed for the median. Start with the original 200 data points. Pull a bootstrap sample: randomly grab 200 values from the original sample, with replacement (some values might get picked more than once, others not at all, totally fine). Calculate the median of that bootstrap sample. Repeat that a whole lot of times, usually somewhere between 1,000 and 10,000. Sort all those bootstrap medians. Grab the value at the 2.5th percentile and the value at the 97.5th percentile. That range is the 95% Bootstrap confidence interval.
Think of it like this: there's no access to the ocean, that's the population. What's available is a bucket of water, the sample. Bootstrap's move: keep scooping from the bucket, measure each scoop, and however spread out those scoops turn out to be tells you how uncertain the estimate really is.
Not every resample looks like the original 200 points, some values get picked two or three times, others don't show up at all, but each one gives a slightly different median, and the spread of those 10,000 medians becomes the confidence interval.
| Feature | Classical CI (z/t) | Bootstrap CI |
|---|---|---|
| Needs data to look a certain way | Yes, normal or large n | No, though the sample still needs to be representative |
| Works for any statistic | Only cleanly for means | Yes: median, percentile, ratio, anything |
| How you compute it | One clean formula | Needs a computer, but it's fast |
| How intuitive it feels | Kind of abstract | Feels natural once it clicks |
| Works with small samples? | Not great | Works reasonably well |
Bootstrap isn't a shortcut that cuts corners. If anything it's more honest, since it makes fewer assumptions about the data in the first place.
Time to go back to the five problems from the start and solve every single one, with real Python code.
Problem: a sample of 1,000 Netflix users shows an average daily watch time of 108 minutes (1.8 hours), with a standard deviation of 45 minutes. Build a 95% confidence interval for the real average across all Netflix users.
Netflix can say, with 95% confidence, that the real average daily watch time across all subscribers falls somewhere between 1.75 and 1.85 hours. Both ends sit comfortably under 2 hours, useful context for a team deciding where to put content money.
For a decision this big, it's a completely fair question: why settle for 1,000 people when the full viewing history of all 260 million subscribers is already sitting in a database? If the goal is purely the most precise possible number for last week's average, querying everyone wins, no contest. There's no statistical reason to prefer a sample when the whole population is right there and affordable to check. So what's actually going on here?
The confidence interval already built above answers this, once the same math gets pushed out to n=260,000,000 to see what happens.
Going from 1,000 people to the full 260 million tightens the margin of error by roughly 510 times. But look at what that actually buys: going from knowing the average within about 2.8 minutes, down to knowing it within a third of a second. Nobody is deciding whether to greenlight more two-hour movies or more 22-minute episodes based on knowing the average down to the nearest third of a second. Once a sample is already tight enough to answer the real question, do long-form and short-form viewers actually behave differently, and by roughly how much, querying the entire population just buys precision the decision has no use for.
There's a second, more practical reason too. A decision like this almost never comes from a single query. A team digging into this will slice the data by genre, region, device, subscription length, testing a bunch of angles before settling on a recommendation. Each of those slices runs in seconds against a sample of 1,000. Run the same thing against the full event log, closer to hundreds of billions of individual play, pause, and stop events, not a tidy 260 million rows, and that same exploration becomes slower and pricier to repeat a dozen times over. Sampling lets teams move fast, confirming the final number against the complete dataset once, right before it goes into a decision.
But here's the point really worth sitting with. Even a perfect, zero-error average across all 260 million subscribers only shows what happened under the content mix Netflix already has. It can't say what would happen if Netflix actually shifted investment toward longer content, because nobody's made that change yet. That's a forward-looking question about cause and effect, and no historical average, no matter how exactly calculated, can answer that on its own. The confidence interval built here is a solid, honest starting point, it shows where things stand today, but it's one input into the decision, not the whole decision. Figuring out "will people watch more if we change the mix" is exactly what experiments and hypothesis testing are for, and that's exactly where the next article picks up.
Problem: Uber samples 2,000 trips in Mumbai. The sample average is 22 minutes, with a standard deviation of 12 minutes. Build a 99% confidence interval for the real average trip time.
Uber can be 99% confident the real average trip time in Mumbai sits between 21.3 and 22.7 minutes. That interval is tight mostly because n=2,000 is large, and it's precise enough to set driver incentives and surge pricing with real confidence.
Problem: Swiggy has a sample of 500 orders from Bangalore. The ops team wants a 95% confidence interval for the median delivery time, not the mean, since delivery times tend to be skewed and the median better reflects what a typical customer actually experiences.
The median (28.34) is the number to trust here, not the mean (31.33), since a handful of very late deliveries drag the mean upward. The bootstrap interval says the real median likely sits between 27.7 and 29.1 minutes, comfortably inside Swiggy's 30-minute promise. Trusting the mean instead would have wrongly suggested Swiggy was breaking its own SLA.
Problem: Spotify shows a discount offer to 10,000 free users, and 1,200 of them convert to Premium, that's 12%. What's the 95% confidence interval for the real conversion rate across everyone?
For proportions, the formula shifts slightly: p̂ = sample proportion = x/n, SE = √(p̂(1−p̂)/n), CI = p̂ ± z* × SE.
Spotify can be 95% confident the real conversion rate lands between 11.4% and 12.6%. Scaled to their full 350 million free users, that's roughly 40 to 44 million potential new Premium subscribers, a number worth actually building revenue forecasts around.
Problem: Stripe samples 4,000 transactions from small businesses. The risk team needs a 95% confidence interval for the 90th percentile of transaction values, so they can set a fraud threshold. Anything above that threshold gets extra scrutiny.
The real 90th percentile likely sits between $1,091 and $1,261. Stripe's risk team can safely set the fraud-review trigger at $1,261, catching unusually large transactions without flagging a bunch of perfectly normal ones. None of this would have been possible with the classical formula, since that one only works cleanly for means. This is bootstrap earning its keep.
Here's how the big ideas from this article connect.
Sampling is what makes huge problems manageable, and a representative sample is what makes the resulting conclusions trustworthy. Figure out sample size before collecting anything, using n=(zσ/E)² for a mean and n=z²p̂(1−p̂)/E² for a proportion, since tighter margins need disproportionately more data. The Central Limit Theorem is what makes all of this possible in the first place: sample averages end up normally distributed once the sample is big enough, no matter what the original data looks like. Standard Error (s/√n) measures how precise an estimate is. Confidence intervals are the honest way to report a number, a range, not a single value, with a stated level of certainty. Bootstrap hands you a confidence interval for pretty much any statistic, without needing a special formula for each one. And more confidence always means a wider interval, certainty is never free, it's always paid for in precision.
The next article, Hypothesis Testing End to End, goes one step further. Instead of just estimating values, it moves into testing actual claims: did this new recommendation algorithm actually increase watch time? Is the new checkout flow really faster than the old one? Did this marketing campaign genuinely move the needle on conversion?
That article covers p-values, null and alternate hypotheses, t-tests, z-tests, and the whole decision-making framework data scientists use to make evidence-based calls, and all of it builds directly on the same CLT foundation covered here.
Interview-style questions, on sampling, CLT, confidence intervals, and Bootstrap, the kind that actually come up at places like Google, Meta, Uber, and Amazon.
They measure two different things. Standard deviation (s or σ) describes how spread out the individual data points are, the variety inside the dataset itself. Standard error (SE = s/√n) describes how much the sample mean itself would bounce around across different samples, how precise the estimate is, not how spread out the raw data is. SE shrinks as the sample grows, since √n grows, but s doesn't shrink at all: collecting more data doesn't make people more similar to each other, it just makes the estimate of their average more precise. A classic trap: doubling the sample size doesn't cut SE in half, it cuts it by a factor of √2 ≈ 1.41. Cutting SE in half actually requires quadrupling the sample size.
Sampling bias happens when a sample isn't actually representative, because certain people are more or less likely to end up in it than others. Even perfect statistics applied afterward can't fix wrong conclusions from a biased sample, the classic "garbage in, garbage out" problem. Survivorship bias looks only at users still active today to figure out "what makes people stay," completely ignoring everyone who already left. Voluntary response bias comes from surveys that only reach people who chose to respond, like clicking a pop-up, over-representing people with strong opinions. Coverage bias happens when, say, Swiggy only samples delivery times from Metro cities but tries to make claims about all of India, leaving out how Tier-2 and Tier-3 cities actually behave. The fix is good sampling design from the start: stratified sampling so every relevant group is represented, randomized assignment wherever possible, and always knowing where the data actually came from.
Build a 95% confidence interval and check whether 6 falls inside it. SE = 2.1/√200 = 0.148, MoE = 1.96 × 0.148 = 0.291, CI = [4.91, 5.49] minutes. The PM's claimed 6 minutes falls outside this interval, so their claim doesn't line up with what the data shows at 95% confidence. There's strong evidence the real average is actually below 6, probably somewhere between 4.9 and 5.5. This is basically the setup for a formal hypothesis test too, covered in the next article, but the confidence interval alone already gives a clear, easy-to-explain answer.
Three big ones. Random sampling: data points need to be drawn randomly and independently from the population, if this breaks (say, only sampling users from one city), the CI doesn't represent the population it claims to. Approximate normality of the sampling distribution: guaranteed by CLT once n is large enough (30+ for mild skew, more for heavy skew), for small samples from skewed data, CLT hasn't kicked in yet and the interval can end up inaccurate, Bootstrap is the better move here. And σ needs to be reasonably well estimated: the formula swaps in s for σ, fine for large n, but adding extra uncertainty when n is small, which is exactly why the heavier-tailed t-distribution exists. If random sampling is broken, no statistical trick fixes it, the problem lives in how the data was collected. If normality is broken, Bootstrap is the way out.
Both should be centered around roughly the same value, since both x̄'s should land close to μ thanks to CLT, but the n=1,000 interval is far narrower. SE scales with 1/√n: at n=100, SE=s/10; at n=1,000, SE=s/31.6, about 3.16 times smaller. With s=15: at n=100, MoE=1.96×1.5=2.94, a CI about 5.9 units wide. At n=1,000, MoE=1.96×0.47=0.93, a CI about 1.9 units wide. Both are technically valid 95% CIs, in repeated sampling, 95% of either type would contain μ. But the n=1,000 version is genuinely more useful, since it's far more precise, which is exactly why planning sample size matters: a technically valid CI that's 6 units wide might just be too vague to actually drive a real decision.
After pulling a data point to include in a bootstrap sample, it gets put right back before pulling the next one, so the same data point could get picked zero times, once, or several times within a single resample. This is on purpose, and it's essential to how Bootstrap works: the whole point is to simulate what would happen drawing many different samples from the real population, and since that's not actually possible, treating the one available sample as a stand-in and sampling with replacement from it is exactly what introduces the needed variation. Sampling without replacement instead would just reshuffle the same original data every time, zero variation, a useless, collapsed interval. One fun fact: on average, each bootstrap resample ends up containing about 63.2% of the original data points, some values show up more than once, some don't show up at all, and that number comes straight out of the probability math behind sampling with replacement.
This is a proportion confidence interval, let the math answer it directly. n=500, p̂=0.68, SE=√(0.68×0.32/500)=0.0209, MoE=1.96×0.0209=0.041, CI=[0.639, 0.721]=[63.9%, 72.1%]. The answer: yes, and there's a number to back it. With 95% confidence, the real percentage of users who like this feature is somewhere between 63.9% and 72.1%. Since that entire range sits above 50%, there's solid statistical evidence that a genuine majority of users, not just the ones surveyed, actually prefer this feature. The important habit is giving more than just the raw 68%, an honest range, translated into something actually actionable.
With simple random sampling, everyone in the population has an equal shot at being picked, drawing blindly from the whole pool. It's easy to set up, but by pure chance it can under-represent smaller subgroups. With stratified sampling, the population gets split into meaningful groups first, then sampled from each group, either proportionally (to match the real population's makeup) or equally (so every group has enough people to analyze on its own). Go with simple random sampling when the population is fairly uniform, or the sample is big enough that subgroups will naturally show up in roughly the right proportions anyway. Go with stratified sampling when solid estimates for specific subgroups are needed (Swiggy wanting separate delivery numbers for Metro, Tier-2, and Tier-3 cities), when a small but important subgroup might get missed entirely by chance, or when tighter overall estimates matter, since stratified sampling tends to give more precise CIs than SRS for the same total n.