Notebook 6: Descriptive Statistics and Probability Foundations

Module: Data Analytics Level: Intermediate Prerequisites: Notebook 5 (Data Transformation and Reshaping) Estimated Time: 4 hours

---

Table of Contents#

1. Why Statistics Comes After Cleaning and Reshaping
2. Measures of Central Tendency
3. Measures of Spread
4. Measures of Shape: Skewness and Kurtosis
5. Percentiles and Quantiles
6. Probability Foundations
7. Random Variables and Expectation
8. Common Probability Distributions
9. The Central Limit Theorem
10. Hands-On: Describing and Modeling a Real-Shaped Dataset
11. Case Studies
12. Common Misconceptions and Pitfalls
13. Glossary
14. Exercises
15. Quiz
16. Summary
17. Further Reading

---

Learning Objectives#

By the end of this notebook, you will be able to:

• Compute and correctly choose between mean, median, and mode depending on data shape and measurement scale
• Compute and interpret variance, standard deviation, and the coefficient of variation
• Diagnose skewness and identify when it should influence the choice of summary statistic
• Compute and interpret percentiles and quantiles, including their use in outlier detection
• State the axioms of probability and apply basic probability rules
• Define a random variable, and compute expectation and variance for both discrete and continuous cases
• Identify when to apply the normal, binomial, Poisson, and uniform distributions to a real analytics problem
• Explain the Central Limit Theorem and why it justifies much of inferential statistics in Notebook 10

---

1. Why Statistics Comes After Cleaning and Reshaping#

Every statistic computed in this notebook assumes the underlying data is trustworthy: correctly typed, free of obvious errors, and appropriately shaped for the calculation at hand. Notebook 4 addressed the first concern (cleaning) and Notebook 5 addressed the second (shape). This notebook is where the payoff of that preparation becomes visible: turning a cleaned column of numbers into a small set of interpretable summary numbers, and beginning to reason formally about uncertainty, a theme that will culminate in the hypothesis testing covered in Notebook 10.

---

2. Measures of Central Tendency#

2.1 The Mean#

Already introduced formally in Notebook 0, Section 6.2:

$\bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i$

Python
CopyAsk AI
import pandas as pd
import numpy as np

data = pd.Series([45, 48, 52, 47, 300, 50, 49, 46])
print(f"Mean: {data.mean():.2f}")

Expected output:

Mean: 79.62

2.2 The Median#

The median is the middle value when data is sorted (or the average of the two middle values for an even number of observations):

Python
CopyAsk AI
print(f"Median: {data.median():.2f}")

Expected output:

Median: 48.5

2.3 Why the Mean and Median Diverge So Sharply Here#

Notice the large gap between the mean (79.62) and median (48.5) in this small dataset. This is a direct, numeric consequence of the single extreme value (300) pulling the mean upward, while the median, based only on rank/position rather than magnitude, is essentially unaffected by it. This single example demonstrates the single most important practical lesson in this section: the mean is sensitive to extreme values (not robust), while the median is not (robust).

2.4 The Mode#

The mode is the most frequently occurring value, the only measure of central tendency that applies meaningfully to nominal data (Notebook 0, Section 8.2), where mean and median are not defined or not meaningful:

Python
CopyAsk AI
categorical_data = pd.Series(["red", "blue", "red", "green", "red", "blue"])
print(f"Mode: {categorical_data.mode()[0]}")

Expected output:

Mode: red

A distribution can have more than one mode (bimodal, multimodal), in which case .mode() returns all tied most-frequent values.

2.5 Choosing the Right Measure#

Note: household income, home prices, and wait times are all classic examples of right-skewed real-world distributions where the median is generally the more representative and widely reported summary statistic, precisely because a small number of very high values pull the mean well above what a "typical" observation actually looks like.

---

3. Measures of Spread#

3.1 Range#

The simplest measure of spread, the difference between the maximum and minimum:

Python
CopyAsk AI
print(f"Range: {data.max() - data.min()}")

The range is highly sensitive to a single extreme value and is rarely used alone as a serious measure of spread in analytics work, but is a useful quick sanity check (Notebook 4, Section 8).

3.2 Variance#

Already introduced formally in Notebook 0, Section 6.2:

$s^2 = \frac{1}{n-1}\sum_{i=1}^{n} (x_i - \bar{x})^2$

Python
CopyAsk AI
print(f"Variance: {data.var():.2f}")

3.3 Why n-1: Bessel's Correction Explained#

Notebook 0 mentioned Bessel's correction ($n-1$ rather than $n$) without deriving it. The intuition: when computing variance using the sample mean $\bar x$ (itself estimated from the same data) rather than the true, unknown population mean $\mu$, the data points are, on average, slightly closer to $\bar x$ than they would be to the true $\mu$, since $\bar x$ is calculated specifically to minimize the sum of squared deviations from itself. This makes the naive formula using $n$ in the denominator systematically underestimate the true population variance. Dividing by $n-1$ instead of $n$ exactly corrects for this bias, producing an unbiased estimator of the true population variance. This distinction matters only when working with a sample intended to estimate a broader population (the overwhelmingly common case in analytics); when computing the variance of a complete, self-contained population with no inferential intent, dividing by $n$ is technically correct instead, and Pandas allows switching via the ddof parameter (ddof=0 for population variance, ddof=1, the default, for sample variance).

3.4 Standard Deviation#

The square root of variance, expressed in the same units as the original data (unlike variance, which is in squared units and therefore harder to interpret directly):

Python
CopyAsk AI
print(f"Standard deviation: {data.std():.2f}")

3.5 The Coefficient of Variation#

Standard deviation alone does not reveal whether a given amount of spread is "large" or "small" relative to the data's scale. The coefficient of variation (CV) normalizes standard deviation by the mean, expressing spread as a relative percentage, which is essential when comparing variability across variables measured on very different scales:

$CV = \frac{s}{\bar x} \times 100\%$

Python
CopyAsk AI
cv = (data.std() / data.mean()) * 100
print(f"Coefficient of variation: {cv:.1f}%")

Example use case: Comparing the variability of employee salaries (measured in tens of thousands) against the variability of employee tenure in years (measured in single digits) using raw standard deviation alone would be meaningless, since the two are on completely different scales; the CV puts both on a comparable, scale-free footing.

---

4. Measures of Shape: Skewness and Kurtosis#

4.1 Skewness#

Skewness measures the asymmetry of a distribution:

• Positive (right) skew: a long tail extends to the right; the mean is typically greater than the median (e.g., income, home prices, this notebook's own toy dataset in Section 2.1 with the 300 outlier).
• Negative (left) skew: a long tail extends to the left; the mean is typically less than the median (e.g., age at retirement, exam scores with a hard ceiling near 100).
• Zero skew: a symmetric distribution (e.g., the idealized normal distribution, Section 8.1).

Python
CopyAsk AI
print(f"Skewness: {data.skew():.2f}")

Expected output:

Skewness: 2.66

A skewness value well above 0 confirms the strong right skew already visible from the mean-median divergence noted in Section 2.3.

4.2 Kurtosis#

Kurtosis measures the "tailedness" of a distribution, specifically how much of the variance comes from infrequent, extreme deviations (outliers) as opposed to frequent, modest deviations:

• High kurtosis (leptokurtic): heavier tails and a sharper peak than a normal distribution; more prone to extreme outliers.
• Low kurtosis (platykurtic): lighter tails and a flatter peak than a normal distribution.
• A normal distribution has a (excess) kurtosis of 0 by the common convention Pandas and most statistical software use.

Python
CopyAsk AI
print(f"Kurtosis: {data.kurt():.2f}")

4.3 Why Shape Matters for Choosing Statistics#

The skewness diagnostic directly informs the mean-versus-median decision from Section 2.5, the z-score-versus-IQR decision from Notebook 4, Section 4.3, and previews why many real-world statistical procedures (including several covered in Notebook 10) explicitly assume approximately normal, low-skew data, and can produce misleading results when that assumption is badly violated without an appropriate correction (such as a log transform, previewed in the Machine Learning course's Notebook 3, Section 12.3).

---

5. Percentiles and Quantiles#

5.1 Definition#

The p-th percentile is the value below which p percent of the data falls. The median (Section 2.2) is simply the 50th percentile. Quartiles divide data into four equal parts (25th, 50th, 75th percentiles), and quantiles is the general term for dividing data into any number of equal-sized groups.

Python
CopyAsk AI
print(f"25th percentile: {data.quantile(0.25)}")
print(f"50th percentile (median): {data.quantile(0.50)}")
print(f"75th percentile: {data.quantile(0.75)}")

5.2 The Five-Number Summary#

Python
CopyAsk AI
print(data.describe())

Expected output:

code
Ask AICopy
count      8.000000
mean      79.625000
std       88.198...
min       45.000000
25%       46.750000
50%       48.500000
75%       50.500000
max      300.000000
Name: dtype: float64

This "five-number summary" (minimum, Q1, median, Q3, maximum), combined with mean and standard deviation, is exactly what powers the IQR outlier detection method from Notebook 4, Section 4.2 and the box plot visualization covered in Notebook 8.

5.3 Percentile Rank of a Specific Value#

The inverse operation, finding what percentile a specific value corresponds to, is useful for interpreting an individual observation relative to a full distribution:

Python
CopyAsk AI
from scipy import stats

value = 50
percentile_rank = stats.percentileofscore(data, value)
print(f"The value {value} is at the {percentile_rank:.1f}th percentile")

---

6. Probability Foundations#

6.1 Why Analytics Needs Probability#

Descriptive statistics (Sections 2 through 5) summarize data you already have. Probability provides the mathematical language to reason about uncertainty, essential the moment you want to say anything about data you have not yet observed, whether that is predicting future outcomes, quantifying confidence in an estimate, or (in Notebook 10) formally testing whether an observed pattern could plausibly have arisen by chance alone.

6.2 The Axioms of Probability#

For an event $A$ within a sample space $S$ (the set of all possible outcomes):

1. $P(A) \geq 0$ for any event $A$ (probability is never negative)
2. $P(S) = 1$ (something in the sample space always happens)
3. For mutually exclusive events $A$ and $B$ (they cannot both happen): $P(A \cup B) = P(A) + P(B)$

6.3 Basic Probability Rules#

Complement rule: $P(\text{not } A) = 1 - P(A)$

Addition rule (general, for events that can overlap): $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Conditional probability: $P(A \mid B) = \frac{P(A \cap B)}{P(B)}, \quad \text{provided } P(B) > 0$

Multiplication rule: $P(A \cap B) = P(A \mid B) \cdot P(B)$

Independence: Events $A$ and $B$ are independent if and only if: $P(A \cap B) = P(A) \cdot P(B) \quad \text{equivalently} \quad P(A \mid B) = P(A)$

6.4 A Worked Analytics Example#

Suppose 30 percent of website visitors are on mobile devices, and 8 percent of all visitors (mobile and desktop combined) complete a purchase. Among mobile visitors specifically, 5 percent complete a purchase. What is the probability a random visitor is both on mobile and completes a purchase?

Python
CopyAsk AI
p_mobile = 0.30
p_purchase_given_mobile = 0.05

# Using the multiplication rule: P(A and B) = P(A|B) * P(B)
p_mobile_and_purchase = p_purchase_given_mobile * p_mobile
print(f"P(mobile AND purchase): {p_mobile_and_purchase:.3f}")

Expected output:

P(mobile AND purchase): 0.015

6.5 Bayes' Theorem#

A direct rearrangement of the conditional probability definition (Section 6.3), Bayes' theorem allows "reversing" a conditional probability, which is extremely useful when the probability you can easily measure is not the one you actually want to know:

$P(A \mid B) = \frac{P(B \mid A) \cdot P(A)}{P(B)}$

Example: A fraud detection rule flags 2 percent of all transactions. Among genuinely fraudulent transactions, the rule correctly flags 95 percent of them ($P(\text{flagged} \mid \text{fraud}) = 0.95$). Suppose only 0.1 percent of all transactions are actually fraudulent ($P(\text{fraud}) = 0.001$). What is the probability that a flagged transaction is actually fraudulent, $P(\text{fraud} \mid \text{flagged})$?

Python
CopyAsk AI
p_fraud = 0.001
p_flagged_given_fraud = 0.95
p_flagged = 0.02  # overall flag rate

p_fraud_given_flagged = (p_flagged_given_fraud * p_fraud) / p_flagged
print(f"P(fraud | flagged): {p_fraud_given_flagged:.3f}")

Expected output:

P(fraud | flagged): 0.0475

Interpretation: Even though the rule catches 95 percent of true fraud, only about 4.75 percent of transactions it actually flags turn out to be genuinely fraudulent, because true fraud is so rare (0.1 percent) that the flagged group is still dominated by false positives from the much larger pool of legitimate transactions. This is a direct, quantitative illustration of why the base rate (the overall prevalence of the event) matters enormously, and is the same fundamental reasoning behind the class-imbalance and precision concerns discussed throughout the Machine Learning course (its Notebook 0 fraud case study, and Notebook 11's precision/recall treatment).

---

7. Random Variables and Expectation#

7.1 Random Variables#

A random variable is a variable whose value is a numeric outcome of a random phenomenon. Discrete random variables take countable values (number of customer support tickets in a day); continuous random variables can take any value in a range (time until a customer churns).

7.2 Expectation (Expected Value)#

For a discrete random variable $X$ with possible values $x_1, \dots, x_k$ and corresponding probabilities $P(X=x_i)$, the expected value is:

$E[X] = \sum_{i=1}^{k} x_i \cdot P(X = x_i)$

This is conceptually the "long-run average" outcome if the random process were repeated indefinitely, and generalizes the plain average from Section 2.1 to a setting with known probabilities rather than a fixed, already-observed sample.

Worked example: A marketing campaign has a 60 percent chance of generating no additional revenue, a 30 percent chance of generating 10,000 dollars, and a 10 percent chance of generating 50,000 dollars.

Python
CopyAsk AI
outcomes = [0, 10000, 50000]
probabilities = [0.60, 0.30, 0.10]

expected_value = sum(o * p for o, p in zip(outcomes, probabilities))
print(f"Expected revenue: ${expected_value:,.2f}")

Expected output:

Expected revenue: $8,000.00

7.3 Variance of a Random Variable#

$\text{Var}(X) = E[(X - E[X])^2] = E[X^2] - (E[X])^2$

Python
CopyAsk AI
import numpy as np

outcomes_arr = np.array(outcomes)
probs_arr = np.array(probabilities)

expected_value = np.sum(outcomes_arr * probs_arr)
variance = np.sum(probs_arr * (outcomes_arr - expected_value) ** 2)
print(f"Variance: {variance:,.2f}")
print(f"Standard deviation: {np.sqrt(variance):,.2f}")

This is the same conceptual quantity as the sample variance from Section 3.2, but computed from a known probability model rather than estimated from observed sample data, directly generalizing the concept from "spread in a dataset" to "spread in an uncertain future outcome."

---

8. Common Probability Distributions#

8.1 The Normal (Gaussian) Distribution#

The most important continuous distribution in statistics, characterized by its mean $\mu$ and standard deviation $\sigma$, with the familiar symmetric bell shape:

$f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$

The 68-95-99.7 rule: approximately 68 percent of values fall within 1 standard deviation of the mean, 95 percent within 2 standard deviations, and 99.7 percent within 3 standard deviations, directly justifying the z-score outlier threshold of 3 used in Notebook 4, Section 4.1, and Notebook 1, Section 11.

Python
CopyAsk AI
from scipy import stats
import numpy as np

# Probability a value falls within 2 standard deviations of the mean
prob_within_2sd = stats.norm.cdf(2) - stats.norm.cdf(-2)
print(f"P(within 2 std devs): {prob_within_2sd:.4f}")

Expected output:

P(within 2 std devs): 0.9545

When it applies: many naturally occurring measurement processes (heights, measurement errors) approximate a normal distribution, and, per Section 9, the Central Limit Theorem justifies its use even more broadly for sample means, regardless of the underlying data's own distribution.

8.2 The Binomial Distribution#

Models the number of successes in a fixed number of independent trials, each with the same success probability $p$:

$P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$

When it applies: number of customers who click "buy" out of 100 who viewed a product page, number of defective items in a batch of a fixed size, assuming each trial is independent with a constant success probability.

Python
CopyAsk AI
from scipy import stats

n_trials = 100
p_success = 0.05  # 5 percent conversion rate

# Probability of exactly 8 successes out of 100 trials
prob_exactly_8 = stats.binom.pmf(8, n_trials, p_success)
print(f"P(exactly 8 conversions): {prob_exactly_8:.4f}")

# Probability of 8 or more successes
prob_8_or_more = 1 - stats.binom.cdf(7, n_trials, p_success)
print(f"P(8 or more conversions): {prob_8_or_more:.4f}")

8.3 The Poisson Distribution#

Models the number of events occurring in a fixed interval of time or space, given a known average rate $\lambda$, when events occur independently:

$P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}$

When it applies: number of customer support tickets received per hour, number of website errors per day, number of equipment failures per month, assuming a roughly constant average rate and independent occurrences.

Python
CopyAsk AI
from scipy import stats

avg_tickets_per_hour = 4

prob_exactly_6 = stats.poisson.pmf(6, avg_tickets_per_hour)
print(f"P(exactly 6 tickets in an hour): {prob_exactly_6:.4f}")

prob_more_than_10 = 1 - stats.poisson.cdf(10, avg_tickets_per_hour)
print(f"P(more than 10 tickets in an hour): {prob_more_than_10:.4f}")

8.4 The Uniform Distribution#

Every value within a range is equally likely:

$f(x) = \frac{1}{b-a} \quad \text{for } a \leq x \leq b$

When it applies: modeling a genuinely random selection within a fixed range (e.g., a randomly assigned A/B test bucket, or the exact second within a minute an event is logged when no particular second is favored), and frequently used as a baseline or default assumption when no other information about the distribution's shape is available.

8.5 Choosing the Right Distribution#

---

9. The Central Limit Theorem#

9.1 The Statement#

The Central Limit Theorem (CLT) states that, for a sufficiently large sample size $n$, the distribution of the sample mean $\bar X$ approaches a normal distribution, regardless of the shape of the underlying population distribution, provided the population has a finite mean and variance:

$\bar X \approx N\left(\mu, \frac{\sigma^2}{n}\right) \quad \text{as } n \to \infty$

where $\mu$ and $\sigma^2$ are the population mean and variance.

9.2 Why This Is Remarkable#

This holds true even if the underlying data itself is heavily skewed, bimodal, or otherwise nothing like a normal distribution. Section 8.1 noted that the normal distribution's applicability to raw individual measurements is limited; the CLT is what justifies using normal-distribution-based methods for sample means and other aggregated statistics far more broadly, which is precisely the foundation that makes the confidence intervals and hypothesis tests in Notebook 10 valid even when the raw underlying data is not itself normally distributed.

9.3 Demonstrating the CLT#

Python
CopyAsk AI
import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng(42)

# Start with a heavily right-skewed population (exponential distribution)
population = rng.exponential(scale=10, size=100_000)
print(f"Population skewness: {pd.Series(population).skew():.2f}")  # highly skewed

# Repeatedly draw samples of size 30 and compute their means
sample_means = [rng.choice(population, size=30, replace=False).mean() for _ in range(2000)]
sample_means = pd.Series(sample_means)
print(f"Sample means skewness: {sample_means.skew():.2f}")  # much closer to 0, i.e., roughly normal

Expected output (approximate):

›Population skewness: 2.02
Sample means skewness: 0.19

Interpretation: Even though the original population is strongly right-skewed (skewness around 2.0), the distribution of sample means computed from repeated samples of size 30 is dramatically closer to symmetric (skewness near 0), exactly as the CLT predicts. This demonstration is the conceptual bridge to Notebook 10, where confidence intervals and hypothesis tests about a mean rely on precisely this property.

9.4 Practical Implications for Sample Size#

The CLT's approximation improves as sample size increases; a commonly cited rule of thumb suggests $n \geq 30$ is often sufficient for the approximation to be reasonably good for most real-world population shapes, though more heavily skewed populations may require a larger sample size before the sample mean's distribution becomes reliably close to normal.

---

10. Hands-On: Describing and Modeling a Real-Shaped Dataset#

This example applies the full toolkit from this notebook to a simulated customer support dataset, combining descriptive statistics, distribution fitting, and a probability calculation.

Python
CopyAsk AI
import pandas as pd
import numpy as np
from scipy import stats

rng = np.random.default_rng(42)

# Simulate daily support ticket counts (Poisson-distributed, per Section 8.3)
# and individual ticket resolution times (right-skewed, per Section 4.1)
daily_tickets = rng.poisson(lam=25, size=90)
resolution_times = rng.exponential(scale=45, size=500)  # minutes

# Descriptive statistics for resolution times
print("Resolution time statistics:")
print(f"  Mean: {resolution_times.mean():.1f} minutes")
print(f"  Median: {np.median(resolution_times):.1f} minutes")
print(f"  Std dev: {resolution_times.std():.1f} minutes")
print(f"  Skewness: {stats.skew(resolution_times):.2f}")
print(f"  95th percentile: {np.percentile(resolution_times, 95):.1f} minutes")

# Probability modeling for daily ticket volume
avg_daily = daily_tickets.mean()
print(f"\nObserved average daily tickets: {avg_daily:.1f}")

# Using a Poisson model fit to the observed average rate, estimate the
# probability of an unusually high-volume day (more than 35 tickets)
prob_high_volume = 1 - stats.poisson.cdf(35, avg_daily)
print(f"P(more than 35 tickets in a day): {prob_high_volume:.4f}")

Expected output (approximate):

code
Ask AICopy
Resolution time statistics:
  Mean: 44.8 minutes
  Median: 30.9 minutes
  Std dev: 44.1 minutes
  Skewness: 1.95
  95th percentile: 134.2 minutes

Observed average daily tickets: 24.9
P(more than 35 tickets in a day): 0.0217

Interpretation: The large gap between mean (44.8) and median (30.9) resolution time, combined with the high skewness (1.95), immediately signals this is a right-skewed distribution, meaning the median is a more representative "typical" resolution time than the mean, exactly the pattern discussed in Section 2.3 and Section 4.1. The Poisson-based probability calculation demonstrates how a fitted distribution can answer forward-looking questions ("how likely is a high-volume day") that a purely descriptive summary of past data cannot answer on its own, directly previewing the kind of reasoning that underlies statistical inference in Notebook 10.

---

11. Case Studies#

11.1 Case Study: A Misleading "Average" Customer Support Resolution Time#

A support team reports an average ticket resolution time of 45 minutes and sets a service level target based on this figure. Customers frequently complain that their tickets take "much longer than the stated average," and an internal review finds the median resolution time is actually closer to 25 minutes.

Diagnosis: Resolution time is a classic right-skewed distribution (Section 4.1): most tickets resolve quickly, but a smaller number of complex tickets take dramatically longer, pulling the mean well above the median, exactly the pattern demonstrated numerically in Section 10. Reporting only the mean obscures the fact that the typical (median) customer experience is considerably better than the mean suggests, while a smaller number of customers experience dramatically worse-than-average wait times.

Lesson: For skewed operational metrics, reporting the median alongside (or instead of) the mean, and ideally a higher percentile (such as the 90th or 95th, as computed in Section 10) to characterize the "worst-case typical" experience, gives a far more complete and less misleading picture than the mean alone.

11.2 Case Study: Applying Bayes' Theorem to Reassess a Marketing Claim#

A marketing vendor claims their targeting algorithm identifies "high-intent" customers with 90 percent accuracy, and a client is prepared to invest heavily based on this figure. Using the base-rate reasoning from Section 6.5, an analyst asks a clarifying question: 90 percent accuracy at identifying high-intent customers among those flagged, or 90 percent of true high-intent customers correctly caught? These are different quantities, precisely $P(\text{high-intent} \mid \text{flagged})$ versus $P(\text{flagged} \mid \text{high-intent})$.

Diagnosis: Given that true "high-intent" customers are typically a small fraction of the overall customer base (a low base rate, directly analogous to the 0.1 percent fraud rate in Section 6.5's example), even a targeting system with a genuinely high $P(\text{flagged} \mid \text{high-intent})$ can have a much lower $P(\text{high-intent} \mid \text{flagged})$ than the "90 percent accuracy" framing implies, unless the base rate and false-positive rate are also known and accounted for.

Lesson: Any accuracy claim involving a rare outcome should be interrogated using Bayes' theorem and explicit knowledge of the base rate before being taken at face value; the same reasoning that revealed only 4.75 percent of flagged transactions were truly fraudulent in Section 6.5 applies directly to marketing, medical screening, and any other rare-event classification claim.

11.3 Case Study: A/B Test Sample Size Justified by the Central Limit Theorem#

A team wants to A/B test two versions of a checkout page, measuring average order value, a metric known from historical data to be strongly right-skewed (a small number of very large orders). A stakeholder raises a concern: "how can we use methods that assume normality when our data clearly isn't normal?"

Diagnosis: This concern conflates the distribution of individual order values (which is indeed skewed) with the distribution of the sample mean order value across many customers in each test group, which is the actual quantity being compared in the A/B test. Per the Central Limit Theorem (Section 9), with a sufficiently large sample size per test group (commonly in the hundreds or thousands for a real e-commerce test), the sample mean's distribution becomes approximately normal regardless of the skewness of individual order values, exactly as demonstrated numerically in Section 9.3.

Lesson: The Central Limit Theorem is precisely the justification needed here, and this exact reasoning, that comparing sample means (not raw values) is what makes normal-distribution-based inference valid even for skewed underlying data, is the mathematical foundation for the hypothesis testing methods introduced in Notebook 10.

---

12. Common Misconceptions and Pitfalls#

---

13. Glossary#

---

14. Exercises#

Exercise 1 (Conceptual)#

A dataset of employee salaries has a mean of 78,000 dollars and a median of 65,000 dollars. What does this gap suggest about the shape of the distribution, and which measure would you recommend reporting to a general audience, and why?

<details> <summary>Solution</summary>

A mean substantially higher than the median suggests a right-skewed distribution (Section 2.3, Section 4.1), most likely due to a relatively small number of very high earners (executives, senior specialists) pulling the mean upward while the bulk of employees earn closer to the median figure. For a general audience wanting to understand a "typical" employee's salary, the median is the more representative and less misleading figure to report, exactly paralleling the household income example in Section 2.5 and the support resolution time case study in Section 11.1; reporting the mean alone risks giving the impression that a "typical" salary is higher than what most employees actually earn.

</details>

---

Exercise 2 (Conceptual)#

A quality control process flags 3 percent of all manufactured units for inspection. Among units that are actually defective, the process correctly flags 98 percent of them. Only 0.5 percent of all units are actually defective. Using Bayes' theorem (Section 6.5), set up (but do not necessarily fully compute numerically) the calculation needed to find the probability that a flagged unit is actually defective, and explain conceptually why this probability might be much lower than the 98 percent figure alone might suggest.

<details> <summary>Solution</summary>

Using Bayes' theorem: $P(\text{defective} \mid \text{flagged}) = \frac{P(\text{flagged} \mid \text{defective}) \cdot P(\text{defective})}{P(\text{flagged})} = \frac{0.98 \times 0.005}{0.03}$. Computing this gives approximately 0.163, or about 16.3 percent. This is far lower than the "98 percent" figure alone might suggest, because that 98 percent describes $P(\text{flagged} \mid \text{defective})$ (how good the process is at catching truly defective units), not $P(\text{defective} \mid \text{flagged})$ (how likely a flagged unit is to truly be defective); since only 0.5 percent of all units are defective in the first place (a low base rate), the pool of flagged units is still substantially diluted by false positives from the much larger population of non-defective units, directly paralleling the fraud detection example in Section 6.5.

</details>

---

Exercise 3 (Coding)#

Given a Series of 500 simulated response times (using np.random.default_rng(1).exponential(scale=20, size=500)), compute the mean, median, skewness, and the 90th percentile, and write one sentence interpreting whether the mean or median better represents a "typical" response time.

<details> <summary>Solution</summary>

Python
CopyAsk AI
import numpy as np
import pandas as pd
from scipy import stats

rng = np.random.default_rng(1)
response_times = pd.Series(rng.exponential(scale=20, size=500))

print(f"Mean: {response_times.mean():.2f}")
print(f"Median: {response_times.median():.2f}")
print(f"Skewness: {response_times.skew():.2f}")
print(f"90th percentile: {response_times.quantile(0.90):.2f}")

Given the exponential distribution's well-known strong right skew (visible in the skewness value, which should be well above 1), the median better represents a typical response time, since the mean will be pulled upward by a smaller number of much longer response times, exactly the same reasoning applied to the support resolution time case study in Section 11.1.

</details>

---

Exercise 4 (Applied Thinking)#

Your team wants to compare the average session duration of two website designs (A and B) using a sample of 40 users per design, and you know from historical data that individual session durations are heavily right-skewed. A colleague argues the comparison is invalid because "session duration isn't normally distributed." Using Section 9, explain whether this objection is valid and why.

<details> <summary>Solution</summary>

The objection conflates the distribution of individual session durations with the distribution of the sample mean session duration across 40 users, which are different things. Per the Central Limit Theorem (Section 9), even though individual session durations are heavily skewed, the distribution of the average session duration across a reasonably large sample (40 users, in the range where the CLT approximation is often, though not always, considered adequate depending on the degree of skew) will tend toward approximately normal, which is precisely what justifies using normal-distribution-based comparison methods on the sample means, exactly as demonstrated in Section 9.3 and the A/B testing case study in Section 11.3. The objection would be more justified if the team were trying to model or compare individual raw session durations directly using normal-distribution assumptions, which would indeed be inappropriate given the skew.

</details>

---

15. Quiz#

Q1. Which measure of central tendency is most affected by extreme outlier values? a) Median b) Mode c) Mean d) All three are equally affected

Q2. Why does sample variance use n-1 rather than n in the denominator? a) It is an arbitrary historical convention with no mathematical justification b) It corrects for the sample mean being closer to the data than the true population mean, producing an unbiased estimate c) It only matters for very small datasets d) It makes the formula easier to compute by hand

Q3. Which distribution is most appropriate for modeling the number of customer support tickets received in a given hour, assuming a roughly constant average rate? a) Normal distribution b) Uniform distribution c) Poisson distribution d) None of the above

Q4. According to the Central Limit Theorem, what happens to the distribution of the sample mean as sample size increases, regardless of the population's underlying shape? a) It becomes more skewed b) It approaches a normal distribution c) It becomes uniform d) It has no predictable pattern

Q5. True or False: A detection method with a high $P(\text{flagged} \mid \text{true positive})$ automatically guarantees a high $P(\text{true positive} \mid \text{flagged})$.

<details> <summary>Answers</summary>

1. c) Mean
2. b) It corrects for the sample mean being closer to the data than the true population mean, producing an unbiased estimate
3. c) Poisson distribution
4. b) It approaches a normal distribution
5. False — as shown in Section 6.5 and the case studies in Sections 11.2, the two conditional probabilities can differ dramatically depending on the base rate of the true event

</details>

---

16. Summary#

• The mean, median, and mode each summarize central tendency differently, with the median being robust to outliers and the mean being sensitive to them, a distinction that should directly inform which statistic is reported for skewed data.
• Variance and standard deviation quantify spread, with Bessel's correction ($n-1$) producing an unbiased estimate of population variance from sample data, and the coefficient of variation enabling scale-independent comparisons.
• Skewness and kurtosis quantify a distribution's shape and directly inform choices made elsewhere in this course, including outlier detection methods and appropriate summary statistics.
• Percentiles and quantiles generalize the median and underlie both the five-number summary and IQR-based outlier detection.
• Probability provides the formal language for reasoning about uncertainty, with conditional probability and Bayes' theorem being essential for correctly interpreting claims about rare events, including fraud detection and marketing accuracy claims.
• The normal, binomial, Poisson, and uniform distributions each model a distinct, common real-world data-generating pattern, and choosing the correct one depends on whether the data is continuous or discrete, and whether it represents a fixed number of trials, a rate of events, or an equally likely range.
• The Central Limit Theorem justifies using normal-distribution-based statistical methods for sample means even when the underlying raw data is skewed, which is the mathematical foundation for the confidence intervals and hypothesis tests covered in Notebook 10.

---

17. Further Reading#

• David Spiegelhalter, The Art of Statistics: Learning from Data
• Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, An Introduction to Statistical Learning, Chapter 2 (Statistical Learning) for a complementary treatment of variance and bias in an estimation context
• SciPy official documentation on statistical distributions: https://docs.scipy.org/doc/scipy/reference/stats.html
• Sheldon Ross, A First Course in Probability — a comprehensive, rigorous treatment of the probability foundations introduced in this notebook

---

Next Up#

Notebook 7: Exploratory Data Analysis (EDA) Techniques — univariate, bivariate, and multivariate analysis, groupby-based exploration workflows, and systematic summary statistics practices, applying the descriptive and probabilistic toolkit from this notebook to the practical, iterative process of building intuition about a new dataset.

---

Course: Data Analytics | Notebook 6 of 13