Notebook 0: Introduction to Machine Learning

Module: Machine Learning Level: Beginner Prerequisites: Basic Python, basic algebra Estimated Time: 3–4 hours

---

Table of Contents#

1. What Is Machine Learning?
2. A Brief History
3. The Formal Definition
4. Mathematical Formulation
5. Types of Machine Learning
6. Hands-On: Your First Three Models
7. When Should You Use Machine Learning?
8. The End-to-End ML Workflow
9. Case Studies
10. Common Misconceptions & Pitfalls
11. Glossary
12. Exercises
13. Quiz
14. Summary
15. Further Reading

---

Learning Objectives#

By the end of this notebook, you will be able to:

• State the formal definition of Machine Learning and explain each component
• Express a learning problem mathematically (hypothesis space, loss function, empirical risk)
• Distinguish Supervised, Unsupervised, Semi-Supervised, Self-Supervised, and Reinforcement Learning, with math notation for each
• Train and evaluate a regression model, a classification model, and a clustering model in Python
• Apply a decision framework for "should I use ML here?"
• Walk through the full ML lifecycle used in production systems
• Analyze three real-world ML case studies end-to-end
• Recognize the most common beginner mistakes before you make them

---

1. What Is Machine Learning?#

Traditional software follows the paradigm:

Rules (written by a human) + Data  →  Output

A tax calculator, a shipping-cost calculator, and a login system are all traditional software: a human encodes the exact logic, and the computer executes it deterministically.

Machine Learning inverts this:

Data + Output (examples)  →  Rules (learned automatically)

Instead of a human writing if age > 65 and income < 20000: high_risk = True, we give the algorithm thousands of past loan applications along with their actual outcomes (defaulted / paid back), and the algorithm infers the pattern — often a pattern too subtle, high-dimensional, or non-linear for a human to hand-code.

This is not a new idea — it's a shift in where the logic comes from. In traditional programming, logic comes from a domain expert's head. In ML, logic comes from the statistical structure of data.

💡 Analogy: Traditional programming is like giving someone a recipe. Machine Learning is like giving someone 10,000 example dishes and having them reverse-engineer the recipe themselves.

---

2. A Brief History#

Understanding where ML came from clarifies why the field is structured the way it is today.

This course focuses on the classical, pre-deep-learning toolkit — the algorithms in the 1957–2010 range in this table. These remain the default choice for structured/tabular data and are the foundation you need before deep learning makes sense.

---

3. The Formal Definition#

The most widely cited formal definition comes from Tom Mitchell (1997):

"A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E."

Let's unpack this with a concrete example — a spam filter:

"Learning" happens when, after processing more labeled emails (E), the model's accuracy (P) on new, unseen emails improves at the task (T).

This T/E/P framing is useful throughout the course — whenever you approach a new algorithm, ask: what is T, what is E, and how is P measured?

---

4. Mathematical Formulation#

This section introduces the notation used throughout the rest of the course. Don't worry about memorizing it now — you'll see it repeatedly in Notebooks 1–13.

4.1 The Dataset#

A dataset $D$ consists of $n$ examples:

$D = \{(x^{(1)}, y^{(1)}), (x^{(2)}, y^{(2)}), \dots, (x^{(n)}, y^{(n)})\}$

where:

• $x^{(i)} \in \mathbb{R}^d$ is the feature vector for example $i$ (a $d$-dimensional vector of inputs)
• $y^{(i)}$ is the label/target for example $i$
  • For regression: $y^{(i)} \in \mathbb{R}$ (a real number, e.g., house price)
  • For classification: $y^{(i)} \in \{1, 2, \dots, K\}$ (a class, e.g., spam=1, not-spam=0)
• In unsupervised learning, there is no $y^{(i)}$ — only $\{x^{(1)}, \dots, x^{(n)}\}$

4.2 The Hypothesis Space#

A model (or hypothesis) is a function $h$ that maps inputs to predictions:

$h_\theta: \mathbb{R}^d \rightarrow \mathcal{Y}$

where $\theta$ represents the model's learnable parameters (e.g., the weights in a linear regression), and $\mathcal{Y}$ is the output space (real numbers for regression, class labels for classification).

The hypothesis space $\mathcal{H}$ is the set of all functions the model could represent by varying $\theta$. For example, in linear regression:

$\mathcal{H} = \{h_\theta(x) = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \dots + \theta_d x_d \mid \theta \in \mathbb{R}^{d+1}\}$

Choosing an algorithm (linear regression vs. decision tree vs. neural network) means choosing a different hypothesis space — this is one of the most important decisions in ML, and it's why Notebooks 3–10 cover so many different model families.

4.3 The Loss Function#

A loss function $L(y, \hat{y})$ quantifies how wrong a single prediction $\hat{y} = h_\theta(x)$ is compared to the true label $y$. Common choices:

• Squared Error (regression): $L(y, \hat{y}) = (y - \hat{y})^2$
• 0/1 Loss (classification): $L(y, \hat{y}) = \mathbb{1}[y \neq \hat{y}]$ (1 if wrong, 0 if correct)
• Log Loss / Cross-Entropy (classification): $L(y, \hat{y}) = -[y \log(\hat{y}) + (1-y)\log(1-\hat{y})]$

4.4 Empirical Risk Minimization#

"Training a model" means finding the parameters $\theta$ that minimize the average loss over the training data — this quantity is called the empirical risk:

$\hat{\theta} = \arg\min_{\theta} \frac{1}{n}\sum_{i=1}^{n} L\big(y^{(i)}, h_\theta(x^{(i)})\big)$

This single equation is the mathematical heart of almost every supervised learning algorithm in this course. What changes between algorithms is:

1. The form of $h_\theta$ (linear? tree-based? kernel-based?)
2. The loss function $L$
3. The optimization method used to find $\hat\theta$ (closed-form solution? gradient descent? greedy splitting?)

You will see this exact equation reappear, specialized, in Notebooks 3 (Linear Regression), 4 (Logistic Regression), and 6 (Decision Trees).

⚠️ Important distinction: Minimizing loss on the training data (empirical risk) is not the real goal — the real goal is minimizing loss on new, unseen data (generalization). This tension is exactly what Notebook 1 (Statistical Learning Foundations) is about.

---

5. Types of Machine Learning#

5.1 Supervised Learning#

Learn a mapping $h_\theta: X \rightarrow Y$ from labeled examples $(x^{(i)}, y^{(i)})$.

Regression — $y \in \mathbb{R}$ is continuous. $\text{Example: } h_\theta(\text{sqft}, \text{bedrooms}, \text{location}) = \text{predicted price}$

Classification — $y \in \{1, \dots, K\}$ is categorical. $\text{Example: } h_\theta(\text{email text features}) = \text{spam or not spam}$

5.2 Unsupervised Learning#

Learn structure from unlabeled data $\{x^{(1)}, \dots, x^{(n)}\}$ — there is no $y$.

Clustering — partition data into groups $C_1, \dots, C_k$ such that points within a group are more similar to each other than to points in other groups.

Dimensionality Reduction — find a lower-dimensional representation $z = f(x)$, $z \in \mathbb{R}^{d'}$ with $d' \ll d$, that preserves most of the important structure in $x$.

5.3 Semi-Supervised Learning#

A mix: a small labeled set $D_L = \{(x^{(i)}, y^{(i)})\}$ plus a large unlabeled set $D_U = \{x^{(j)}\}$. Common when labeling is expensive (e.g., medical imaging — a radiologist's time is costly, but raw scans are plentiful).

5.4 Self-Supervised Learning#

The model generates its own labels from the structure of the data itself (e.g., predict a masked word from surrounding words). This is the foundation of modern large language models — mentioned here for completeness, but outside the scope of this classical-ML course.

5.5 Reinforcement Learning#

An agent interacts with an environment over discrete time steps. At each step $t$, the agent observes a state $s_t$, takes an action $a_t$, and receives a reward $r_t$. The goal is to learn a policy $\pi(a|s)$ that maximizes cumulative future reward:

$\pi^* = \arg\max_{\pi} \; \mathbb{E}\Big[\sum_{t=0}^{T} \gamma^t r_t\Big]$

where $\gamma \in [0,1]$ is a discount factor that weighs future rewards against immediate ones. This is a fundamentally different setup from supervised/unsupervised learning — there's no fixed dataset, just ongoing interaction.

5.6 Comparison Table#

---

6. Hands-On: Your First Three Models#

Below are three minimal, complete, runnable examples using scikit-learn — one for each of regression, classification, and clustering. The goal here is not to understand every parameter yet (that comes in later notebooks) — it's to see the shared pattern: .fit() then .predict().

6.1 Regression: Predicting a Continuous Value#

Python
CopyAsk AI
from sklearn.datasets import make_regression
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

# 1. Generate synthetic data: y = f(X) + noise
X, y = make_regression(n_samples=200, n_features=1, noise=15, random_state=42)

# 2. Split into train/test (see Notebook 1 for why this matters)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# 3. Train (this is the "empirical risk minimization" step from Section 4.4)
model = LinearRegression()
model.fit(X_train, y_train)

# 4. Predict & evaluate
y_pred = model.predict(X_test)
print(f"Learned equation: y = {model.coef_[0]:.2f} * x + {model.intercept_:.2f}")
print(f"Mean Squared Error on test set: {mean_squared_error(y_test, y_pred):.2f}")

Expected output (approximate):

Mathematical Formulation
Explain FormulaCopy
Learned equation: y = 42.31 * x + -0.85
Mean Squared Error on test set: 198.47

6.2 Classification: Predicting a Category#

Python
CopyAsk AI
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# 1. Load a real dataset: tumor features -> malignant/benign
data = load_breast_cancer()
X, y = data.data, data.target

# 2. Split into train/test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# 3. Train
clf = LogisticRegression(max_iter=5000)
clf.fit(X_train, y_train)

# 4. Predict & evaluate
y_pred = clf.predict(X_test)
print(f"Test Accuracy: {accuracy_score(y_test, y_pred):.2%}")

Expected output (approximate):

Test Accuracy: 97.37%

6.3 Clustering: Finding Structure With No Labels#

Python
CopyAsk AI
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
import numpy as np

# 1. Generate synthetic data with 3 natural groups (notice: no y at all)
X, true_groups = make_blobs(n_samples=300, centers=3, random_state=42)

# 2. Train (note: .fit() takes only X, no labels)
kmeans = KMeans(n_clusters=3, random_state=42, n_init=10)
kmeans.fit(X)

# 3. Inspect the discovered structure
print(f"Cluster centers found:\n{np.round(kmeans.cluster_centers_, 2)}")
print(f"First 10 assigned clusters: {kmeans.labels_[:10]}")

Expected output (approximate):

code
Ask AICopy
Cluster centers found:
[[ 4.7   2.01]
 [-2.6   8.9 ]
 [-6.9  -6.85]]
First 10 assigned clusters: [1 1 0 2 0 2 1 0 2 1]

🔎 Notice the pattern: All three examples follow model.fit(...) → model.predict(...). This consistent API is one reason scikit-learn is the standard tool for classical ML — you'll reuse this exact pattern in every notebook from here on.

---

7. When Should You Use Machine Learning?#

7.1 Decision Framework#

Ask these questions, roughly in order:

1. Can a simple rule or lookup table solve this? → If yes, use that. It will be cheaper, faster, and fully explainable.
2. Do you have enough relevant historical data? → ML needs examples to learn from. A rule of thumb: dozens of examples per feature at an absolute minimum; hundreds to thousands is far safer.
3. Is the relationship between inputs and outputs too complex to hand-code? → This is where ML earns its keep (e.g., "does this image contain a cat?").
4. Can you tolerate imperfect, probabilistic predictions? → ML models are never 100% accurate. If a single wrong answer is catastrophic and unrecoverable (e.g., a parachute release mechanism), pure ML without safeguards is risky.
5. Will the patterns change over time? → If yes, ML models can be retrained on fresh data; static hand-coded rules would need constant manual updates.

7.2 A Simple Flow#

Architecture & Data Flow
Copy
                 ┌───────────────────────────┐
                 │ Can a fixed rule solve it? │
                 └─────────────┬─────────────┘
                          yes  │  no
                    ┌──────────┴──────────┐
                    ▼                     ▼
             Use plain code      Do you have labeled/
                                 relevant historical data?
                                          │
                                 yes ─────┼───── no
                                  │              │
                                  ▼              ▼
                          Use Machine        Collect data first,
                          Learning           or use rules/heuristics
                                             as a placeholder

7.3 Worked Examples#

---

8. The End-to-End ML Workflow#

This is the roadmap for the entire course — each stage below corresponds to specific notebooks.

Architecture & Data Flow
Copy
 1. Problem Definition
        │
 2. Data Collection
        │
 3. Data Preprocessing & Feature Engineering ───────► Notebook 2
        │
 4. Train/Test Split ───────────────────────────────► Notebook 1
        │
 5. Model Selection & Training ─────────────────────► Notebooks 3–4, 6–10
        │
 6. Model Evaluation ────────────────────────────────► Notebooks 5, 11
        │
 7. Hyperparameter Tuning ───────────────────────────► Notebook 12
        │
 8. Final Evaluation on Held-Out Test Set
        │
 9. Deployment & Monitoring ─────────────────────────► Notebook 13
        │
 10. Retraining Loop (as new data arrives) ───────────┘
        └──────────────────────────────────────────────┘

8.1 Stage Details#

1. Problem Definition — Translate a business/research question into a T/E/P statement (Section 3). "Reduce customer churn" becomes "predict, for each customer, the probability they cancel within 30 days, using their usage history."
2. Data Collection — Gather examples. Quality and relevance matter more than sheer volume.
3. Preprocessing — Handle missing values, encode categories, scale numeric features (Notebook 2).
4. Splitting — Separate data so you can honestly estimate performance on unseen examples (Notebook 1).
5. Model Selection — Choose a hypothesis space $\mathcal{H}$ appropriate to the problem (Notebooks 3–10).
6. Evaluation — Measure performance with the right metric for the task (Notebooks 5, 11) — accuracy is often the wrong choice, as you'll see.
7. Tuning — Search over hyperparameters to improve generalization (Notebook 12).
8. Deployment — Serve predictions in production; monitor for data drift and degrading performance (Notebook 13).

---

9. Case Studies#

9.1 Case Study: Netflix Recommendation Engine#

Task (T): Predict how likely a user is to watch and enjoy a given title. Experience (E): Billions of viewing events — what users watched, how long, what they rated, what they searched for. Performance (P): Measured via engagement metrics (watch-through rate) and, historically, the famous 2006 "Netflix Prize" used RMSE on predicted star ratings.

Approach: Netflix combines collaborative filtering (unsupervised structure: users who liked similar things in the past tend to like similar things in the future) with supervised models that use features like genre, actors, time of day, and device type to rank a personalized list of titles for the homepage.

Why ML, not rules? With tens of thousands of titles and hundreds of millions of users, a hand-coded rule ("if you watched X, recommend Y") cannot scale or personalize at this level of nuance. The patterns are latent — hidden combinations of taste that emerge only statistically from data.

9.2 Case Study: Credit Card Fraud Detection#

Task (T): Classify a transaction as fraudulent or legitimate, in real time. Experience (E): Historical transactions labeled as fraud/not-fraud (though fraud is typically <1% of all transactions — a class imbalance problem, covered in Notebook 11). Performance (P): Not plain accuracy (a model that always predicts "legitimate" would be 99% accurate but catch zero fraud!) — instead, precision and recall on the fraud class matter most.

Approach: Gradient boosted trees (Notebook 6) are industry-standard here because they handle mixed numeric/categorical features well and produce calibrated probability scores that can be thresholded based on business risk tolerance.

Why ML, not rules? Fraud patterns evolve constantly as fraudsters adapt; a static rule list ("flag if amount > $5,000") is trivially circumvented and quickly outdated. ML models can be retrained regularly as new fraud patterns emerge.

9.3 Case Study: Predictive Maintenance in Manufacturing#

Task (T): Predict whether a machine component will fail within the next 7 days. Experience (E): Sensor time-series data (vibration, temperature, pressure) leading up to historical failure events. Performance (P): Recall on the "will fail" class is prioritized (missing a failure is expensive), balanced against false-alarm rate (too many false alarms erode trust in the system).

Approach: Feature engineering (Notebook 2) extracts statistics from raw sensor streams (rolling averages, rate of change), then a classifier (Random Forest or Gradient Boosting, Notebook 6) predicts failure probability.

Why ML, not rules? A simple threshold ("alert if temperature > 90°C") misses gradual multi-signal degradation patterns that only become visible when combining several sensors — exactly the kind of complex, multivariate pattern classical rules struggle with.

---

10. Common Misconceptions & Pitfalls#

---

11. Glossary#

---

12. Exercises#

Exercise 1 (Conceptual)#

For each scenario below, state the Task (T), Experience (E), and Performance measure (P): a) A weather app predicting tomorrow's high temperature b) A social media platform deciding whether a post violates community guidelines

<details> <summary>💡 Solution</summary>

a)

• T: Predict tomorrow's high temperature for a given location
• E: Historical weather records (temperature, humidity, pressure, etc.) paired with actual next-day temperatures
• P: Mean Absolute Error (or RMSE) between predicted and actual temperature

b)

• T: Classify a post as "violates guidelines" or "does not violate guidelines"
• E: A dataset of posts labeled by human moderators
• P: Precision and recall on the "violates" class (accuracy alone would be misleading, since most posts are compliant)

</details>

---

Exercise 2 (Conceptual)#

Explain, in your own words, why a model that achieves 99.9% accuracy might still be a bad model. Use the credit card fraud case study to justify your answer.

<details> <summary>💡 Solution</summary>

If fraud makes up only 0.1% of transactions, a model that predicts "not fraud" for every single transaction achieves 99.9% accuracy while catching zero fraud — making it useless for the actual business goal. This illustrates why the right metric, chosen based on the class balance and business cost of errors, matters more than raw accuracy. Notebook 11 covers this in depth via precision, recall, and F1-score.

</details>

---

Exercise 3 (Coding)#

Using the load_wine dataset from sklearn.datasets, train a LogisticRegression classifier and print its test accuracy. Follow the same four-step pattern from Section 6.2.

<details> <summary>💡 Solution</summary>

Python
CopyAsk AI
from sklearn.datasets import load_wine
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

data = load_wine()
X, y = data.data, data.target

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

clf = LogisticRegression(max_iter=5000)
clf.fit(X_train, y_train)

y_pred = clf.predict(X_test)
print(f"Test Accuracy: {accuracy_score(y_test, y_pred):.2%}")
# Expected: roughly 94-100% depending on random_state

</details>

---

Exercise 4 (Applied Thinking)#

Pick a product or app you use daily. Identify one place it likely uses Machine Learning, and specify whether it's Supervised, Unsupervised, or Reinforcement Learning — and why.

<details> <summary>💡 Sample Solution (answers will vary)</summary>

Spotify's "Discover Weekly" playlist likely combines:

• Unsupervised learning (clustering users/songs with similar listening patterns — collaborative filtering)
• Supervised learning (predicting the probability a user will skip vs. finish a song, trained on historical skip/listen labels)

There's no single correct answer here — the goal is to practice mapping real products to the taxonomy from Section 5.

</details>

---

13. Quiz#

Test your understanding before moving to Notebook 1.

Q1. In Mitchell's formal definition of ML, what does "P" stand for? a) Prediction b) Performance measure c) Parameters d) Probability

Q2. Which of the following is an example of unsupervised learning? a) Predicting house prices from square footage b) Classifying emails as spam or not spam c) Grouping customers into segments with no predefined categories d) Predicting a patient's risk of disease from lab results

Q3. What is the "empirical risk" in the context of ERM (Empirical Risk Minimization)? a) The financial risk of deploying a model b) The average loss of the model across the training dataset c) The variance of the model's predictions d) The probability that a model will overfit

Q4. Why is accuracy often a poor metric for fraud detection? a) Accuracy is too hard to compute for large datasets b) Fraud detection isn't a classification problem c) Class imbalance means a trivial "always predict legitimate" model gets high accuracy while catching no fraud d) Accuracy only works for regression problems

Q5. True or False: A model with very high training accuracy is guaranteed to perform well on new data.

<details> <summary>💡 Answers</summary>

1. b) Performance measure
2. c) Grouping customers into segments with no predefined categories
3. b) The average loss of the model across the training dataset
4. c) Class imbalance means a trivial "always predict legitimate" model gets high accuracy while catching no fraud
5. False — high training accuracy can indicate overfitting; true performance must be measured on held-out/unseen data (Notebook 1)

</details>

---

14. Summary#

• ML flips the traditional programming paradigm: instead of hand-coding rules, we learn rules from data.
• Mitchell's formal definition frames every ML problem in terms of Task, Experience, and Performance measure.
• Every supervised learning problem can be expressed mathematically as Empirical Risk Minimization: finding parameters $\theta$ that minimize average loss over training data.
• The five major paradigms are Supervised, Unsupervised, Semi-Supervised, Self-Supervised, and Reinforcement Learning.
• ML is the right tool when patterns are too complex to hand-code, sufficient relevant data exists, and probabilistic answers are acceptable — not a default choice for every problem.
• The full ML lifecycle spans problem definition through deployment and monitoring — this course follows that lifecycle notebook by notebook.
• Real systems (Netflix, fraud detection, predictive maintenance) combine multiple ML paradigms and require careful metric selection, not just "make accuracy high."

---

15. Further Reading#

• Tom Mitchell, Machine Learning (1997) — the textbook that formalized the T/E/P definition
• Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning — the classic in-depth reference for the math in Section 4
• Aurélien Géron, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow — practical companion to the coding sections in this course
• scikit-learn official documentation: https://scikit-learn.org/stable/

---

Next Up#

Notebook 1: Statistical Learning Foundations — bias-variance tradeoff, overfitting/underfitting, train/test/validation splits, and cross-validation. This is where we formally answer: why doesn't minimizing training loss guarantee a good model?

---

Course: Machine Learning | Notebook 0 of 13