Notebook 0: Introduction to Data Analytics

Module: Data Analytics Level: Beginner Prerequisites: Basic Python, basic algebra Estimated Time: 3–4 hours

---

Table of Contents#

1. What Is Data Analytics?
2. Data Analytics vs. Data Science vs. Machine Learning
3. The Four Types of Analytics
4. The Analytics Workflow
5. The Modern Analytics Toolkit
6. Mathematical Foundations You Will Use Throughout This Course
7. Hands-On: Your First Analysis
8. Data Types and Measurement Scales
9. Case Studies
10. Common Misconceptions and Pitfalls
11. Glossary
12. Exercises
13. Quiz
14. Summary
15. Further Reading

---

Learning Objectives#

By the end of this notebook, you will be able to:

• Define data analytics and distinguish it from data science and machine learning
• Identify which of the four types of analytics (descriptive, diagnostic, predictive, prescriptive) a given business question calls for
• Walk through the end-to-end analytics workflow used in real organizations
• Identify the core tools in the modern analytics stack and what each is used for
• Apply basic vector and summary-statistic notation that will recur throughout the course
• Distinguish between the four measurement scales and explain why the distinction affects which statistics are valid
• Perform a first, complete, small-scale exploratory analysis in Python

---

1. What Is Data Analytics?#

Data analytics is the process of examining raw data to draw conclusions, identify patterns, and support decision-making. Where Notebook 0 of the Machine Learning course described building systems that automatically learn predictive rules from data, this course is about something broader and, in most organizations, more foundational: understanding what the data actually says, cleaning it into a usable state, and communicating findings that humans can act on.

A useful way to frame the distinction: machine learning asks "can I build a system that automatically predicts $y$ from $x$ in the future?" Data analytics asks "what happened, why did it happen, and what should we do about it?" Both fields share a common toolkit (statistics, Python, the same underlying data), but analytics puts far more weight on description, explanation, and communication, while ML puts more weight on automated, repeatable prediction.

Example: A retailer noticing sales dropped 12 percent last quarter is a data analytics question — what happened, in which regions, for which products, and why. Building a system that automatically flags at-risk customers before they churn is a machine learning question — a repeatable, automated prediction applied to new customers going forward.

---

2. Data Analytics vs. Data Science vs. Machine Learning#

These three terms are used inconsistently across the industry, but a widely used rough distinction is useful for orienting yourself in this course and in the separate Machine Learning course:

In practice these fields overlap heavily and the boundaries are blurry — a data analyst who builds a churn-prediction dashboard is doing work that shades into data science, and a machine learning engineer who explores a dataset before modeling is doing data analytics. This course focuses specifically on the analytics core: high-performance manipulation, cleaning, vector math, exploratory data analysis, and statistics — the skills that underpin all three fields.

---

3. The Four Types of Analytics#

A widely used framework (sometimes attributed to Gartner) divides analytics into four types, roughly ordered by increasing complexity and increasing business value:

3.1 Descriptive Analytics: "What happened?"#

Summarizes historical data to understand past performance. Examples: total revenue last month, average customer age, number of website visits per day. This is almost always the starting point of any analysis and is covered in depth in Notebook 6 (Descriptive Statistics).

3.2 Diagnostic Analytics: "Why did it happen?"#

Digs into historical data to find causes and correlations behind an observed pattern. Examples: why did revenue drop in the Northeast region, why did the bounce rate spike on mobile devices last week. This relies heavily on the exploratory data analysis and correlation techniques covered in Notebooks 7 and 9.

3.3 Predictive Analytics: "What is likely to happen?"#

Uses historical patterns to forecast future outcomes. Examples: predicting next quarter's revenue, estimating the probability a customer will churn. This is where analytics begins to overlap substantially with the Machine Learning course's supervised learning content, though simpler predictive analytics (trend extrapolation, basic regression) is well within the scope of this course.

3.4 Prescriptive Analytics: "What should we do about it?"#

Recommends specific actions based on predictive insights, often incorporating optimization or simulation. Examples: recommending an optimal pricing strategy, suggesting which customers to target with a retention offer. This is the most advanced and least commonly implemented type of analytics in most organizations, since it depends on mature descriptive, diagnostic, and predictive capabilities already being in place.

3.5 Comparison Table#

Key idea: Most real analytics work is descriptive and diagnostic. This course spends the majority of its time here deliberately, since these skills are both the most immediately applicable and the necessary foundation for predictive and prescriptive work.

---

4. The Analytics Workflow#

This is the roadmap for the entire course, mirroring the ML workflow from the Machine Learning course's Notebook 0 but adapted for analytics-first work.

Architecture & Data Flow
Copy
 1. Define the Business Question
        |
 2. Collect / Access the Data
        |
 3. Data Cleaning & Preparation ----------------> Notebook 4
        |
 4. Data Transformation & Reshaping -------------> Notebook 5
        |
 5. Exploratory Data Analysis (EDA) -------------> Notebooks 6, 7, 9
        |
 6. Visualization ------------------------------> Notebook 8
        |
 7. Statistical Analysis / Hypothesis Testing ---> Notebook 10
        |
 8. Communicate Findings & Recommend Action
        |
 9. Monitor & Iterate
        +-----------------------------------------+

4.1 Stage Details#

1. Define the Business Question — Every analysis should start with a specific, answerable question (not "look at the data and see what you find," but "why did customer retention drop in Q3 among users acquired through paid search?").
2. Collect / Access the Data — Pulling data from databases, files, APIs, or spreadsheets. This course assumes data is already accessible and focuses on what happens next.
3. Data Cleaning — Handling missing values, duplicates, inconsistent formats, and outliers (Notebook 4). Industry surveys of data professionals consistently report that cleaning and preparation consumes a large share of total project time, often cited as around 60 to 80 percent, though exact figures vary by survey and role.
4. Transformation — Reshaping data into the structure needed for analysis: pivoting, aggregating, merging multiple sources (Notebook 5).
5. Exploratory Data Analysis — Systematically examining variables and relationships to build intuition before drawing conclusions (Notebooks 6, 7, 9).
6. Visualization — Communicating patterns visually, both for your own exploration and for presenting to stakeholders (Notebook 8).
7. Statistical Analysis — Formally testing whether observed patterns are statistically meaningful or likely due to chance (Notebook 10).
8. Communicate Findings — Translating technical results into a clear narrative and actionable recommendation for a non-technical audience.
9. Monitor & Iterate — Good analytics is rarely a one-time exercise; dashboards and recurring reports need to be maintained and questioned as new data arrives.

---

5. The Modern Analytics Toolkit#

This course focuses on the Python-based stack (NumPy, Pandas, Matplotlib/Seaborn, SciPy/statsmodels), which together form the dominant open-source toolkit for data analytics as of this writing.

---

6. Mathematical Foundations You Will Use Throughout This Course#

6.1 Data as Vectors and Matrices#

A single column of numeric data (e.g., 1,000 customers' ages) can be represented as a vector:

$\mathbf{x} = (x_1, x_2, \dots, x_n)$

A full dataset with $n$ rows (observations) and $d$ columns (variables) is naturally represented as an $n \times d$ matrix:

$X = \begin{pmatrix} x_{11} & x_{12} & \dots & x_{1d} \\ x_{21} & x_{22} & \dots & x_{2d} \\ \vdots & \vdots & \ddots & \vdots \\ x_{n1} & x_{n2} & \dots & x_{nd} \end{pmatrix}$

This is exactly the same dataset notation used in the Machine Learning course (Notebook 0, Section 4.1), reflecting the fact that "a table of numbers" is the shared underlying object beneath both fields. Notebook 1 of this course covers how NumPy represents and operates on exactly this structure efficiently.

6.2 Summary Statistics Notation#

Two quantities will recur constantly starting in Notebook 6:

Mean (average): $\bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i$

Variance (average squared deviation from the mean): $s^2 = \frac{1}{n-1}\sum_{i=1}^{n} (x_i - \bar{x})^2$

The $n-1$ divisor (rather than $n$) is called Bessel's correction, used when computing variance from a sample to estimate the variance of the full population without systematic bias; Notebook 6 explains this in depth.

6.3 Why Vectorized Notation Matters for Performance#

Writing $\bar x = \frac{1}{n}\sum x_i$ mathematically corresponds, in code, to two very different implementations:

Python
CopyAsk AI
# Loop-based (slow): explicitly repeats the summation one element at a time
total = 0
for x in data:
    total += x
mean = total / len(data)

# Vectorized (fast): the same mathematical operation, computed via
# optimized, compiled array operations rather than a Python-level loop
import numpy as np
mean = np.mean(data)

Both compute the exact same mathematical quantity $\bar x$, but the vectorized version, covered in depth in Notebook 1, can be dramatically faster on large datasets because the underlying computation happens in optimized, compiled code rather than the Python interpreter's comparatively slow loop mechanism. This performance gap is the central justification for "high-performance data manipulation" as a named topic in this course.

---

7. Hands-On: Your First Analysis#

This example performs a complete, if simple, descriptive and diagnostic analysis: load data, compute summary statistics, and identify a pattern.

Python
CopyAsk AI
import pandas as pd
import numpy as np

# Simulated retail sales data
data = pd.DataFrame({
    "region": ["North", "South", "East", "West", "North", "South", "East", "West"],
    "quarter": ["Q1", "Q1", "Q1", "Q1", "Q2", "Q2", "Q2", "Q2"],
    "revenue": [125000, 98000, 143000, 110000, 118000, 76000, 151000, 108000]
})

# Descriptive analytics: what happened overall?
print("Overall summary statistics:")
print(data["revenue"].describe())

# Diagnostic analytics: why might overall performance have changed?
pivot = data.pivot_table(values="revenue", index="region", columns="quarter")
pivot["change"] = pivot["Q2"] - pivot["Q1"]
pivot["pct_change"] = (pivot["change"] / pivot["Q1"] * 100).round(1)

print("\nRevenue by region, quarter-over-quarter:")
print(pivot)

Expected output:

code
Ask AICopy
Overall summary statistics:
count         8.000000
mean     116125.000000
std       22811.406...
min       76000.000000
25%      106500.000000
50%      114000.000000
75%      126750.000000
max      151000.000000
Name: revenue, dtype: float64

Revenue by region, quarter-over-quarter:
quarter      Q1      Q2  change  pct_change
region
East     143000  151000    8000         5.6
North    125000  118000   -7000        -5.6
South     98000   76000  -22000       -22.4
West     110000  108000   -2000        -1.8

Interpretation: The descriptive step (.describe()) shows overall revenue is fairly consistent, but the diagnostic step (the pivot table) reveals that this hides a real story: the South region dropped sharply (-22.4 percent) while East grew, and North and West were roughly flat. A purely descriptive summary would have missed this entirely — this is exactly why diagnostic analytics (Section 3.2) and the exploratory techniques in Notebook 7 matter.

---

8. Data Types and Measurement Scales#

Understanding what kind of data you have determines which statistics and visualizations are valid — a mistake at this stage propagates through the entire analysis.

8.1 Quantitative vs. Categorical#

• Quantitative (numeric) data: represents a measurable quantity. Subdivided into:
  • Discrete: countable, often integer values (number of purchases, number of children)
  • Continuous: can take any value in a range (height, temperature, revenue)
• Categorical (qualitative) data: represents categories or labels rather than quantities (color, country, product type)

8.2 The Four Measurement Scales#

8.3 Why This Distinction Matters#

The classic illustrative mistake: computing the mean of a nominal variable encoded as numbers (e.g., zip codes, or a "region" column encoded as 1=North, 2=South, 3=East, 4=West) produces a number that is mathematically well-defined but semantically meaningless — there is no real sense in which "region 2.5" exists, since the encoding is nominal, not ratio. This exact mistake was previewed in the Machine Learning course's Data Preprocessing notebook regarding categorical encoding, and it applies with equal force in pure analytics work: always check whether a numeric-looking column is actually nominal or ordinal before applying arithmetic summary statistics to it.

Python
CopyAsk AI
# A common and subtle mistake:
region_codes = pd.Series([1, 2, 3, 4, 1, 2])  # 1=North, 2=South, 3=East, 4=West
print(region_codes.mean())  # Produces a number, but it is meaningless

Expected output:

2.1666666666666665

This number is computable but has no real-world interpretation, since "region" is a nominal variable and averaging its arbitrary numeric codes is not a valid operation.

---

9. Case Studies#

9.1 Case Study: A Dashboard That Answered the Wrong Question#

A subscription company builds a dashboard showing overall monthly active users (MAU) trending steadily upward, and leadership treats this as confirmation the product is healthy. A deeper diagnostic analysis six months later reveals that new user acquisition was masking a steadily worsening churn rate among existing users — the descriptive metric (MAU) was accurate but insufficient to answer the real underlying question, which was about the health of the existing customer base rather than gross size.

Diagnosis: This is a direct illustration of the difference between descriptive and diagnostic analytics (Sections 3.1 and 3.2). A single aggregate descriptive number, however accurate, can obscure an important underlying pattern that only becomes visible through segmentation (e.g., new versus existing users, examined separately, as in the retail hands-on example in Section 7).

Lesson: Descriptive dashboards should be treated as a starting point for questions, not a final answer, and should generally be paired with the ability to drill down into subgroups, a theme that recurs throughout the exploratory data analysis notebooks later in this course.

9.2 Case Study: Simpson's Paradox in University Admissions#

A well-documented historical case (the University of California, Berkeley graduate admissions data from 1973) found that, in aggregate, a smaller percentage of female applicants were admitted than male applicants, suggesting possible bias. When the data was broken down by academic department, women were found to be admitted at an equal or higher rate than men in most individual departments; the aggregate disparity arose because women disproportionately applied to more competitive departments with lower overall admission rates for everyone.

Diagnosis: This is a classic example of Simpson's Paradox — a trend that appears in aggregated data can reverse or disappear when the data is broken into meaningful subgroups. It is one of the most important cautionary results in all of applied statistics and directly motivates the emphasis on segmentation and multivariate exploration in Notebook 7.

Lesson: Aggregate statistics can be actively misleading, not just incomplete, when an important subgroup variable (in this case, department) is correlated with both the grouping variable of interest (gender) and the outcome (admission rate). Always ask what subgroup structure might be driving an aggregate pattern before drawing conclusions from it.

9.3 Case Study: Vectorization Turning a Multi-Hour Job Into a Multi-Second One#

A retail analytics team originally computed a "days since last purchase" feature for 5 million customer transaction records using a Python for loop that iterated row by row, taking several hours to run nightly and repeatedly delaying morning reports. Rewriting the same calculation using vectorized Pandas and NumPy operations (subtracting two datetime columns directly across the entire dataset at once) reduced the runtime to under a minute.

Diagnosis: The original approach paid the overhead of the Python interpreter's loop mechanism 5 million separate times. The vectorized approach, following the same principle introduced in Section 6.3, pushes the actual computation down into NumPy's underlying compiled code, which processes the entire array in one optimized operation.

Lesson: This exact performance gap — the difference between looping in Python and vectorizing with NumPy or Pandas — is one of the most consequential practical skills in professional data analytics, and is the primary subject of Notebooks 1 and 3.

---

10. Common Misconceptions and Pitfalls#

---

11. Glossary#

---

12. Exercises#

Exercise 1 (Conceptual)#

Classify each of the following variables by measurement scale (nominal, ordinal, interval, or ratio), and justify your answer: a) Customer satisfaction survey response: "very dissatisfied," "dissatisfied," "neutral," "satisfied," "very satisfied" b) Number of items in a customer's shopping cart c) Year a company was founded

<details> <summary>Solution</summary>

a) Ordinal. The categories have a clear, meaningful order (dissatisfaction to satisfaction), but the "distance" between adjacent categories is not necessarily equal or numerically defined.

b) Ratio. Zero items is a meaningful "true zero" (no items at all), and arithmetic operations like "twice as many items" are meaningful.

c) Interval, arguably bordering on a special case. Calendar year has equal spacing (one year is one year) but no true zero in the sense that "year 0" does not represent an absence of time; ratios like "founded twice as recently" are not meaningful even though differences (age in years, computed as current year minus founding year) are meaningful and are, in fact, ratio-scaled once converted to an age or duration.

</details>

---

Exercise 2 (Conceptual)#

A company reports that overall customer satisfaction increased this year. Using the concepts from Section 9.1 and 9.2, list two different reasons this aggregate claim might be misleading even if the underlying number is computed correctly.

<details> <summary>Solution</summary>

Two plausible reasons: (1) The aggregate increase could mask a subgroup reversal (Simpson's Paradox, Section 9.2) — for example, satisfaction could have decreased within every individual customer segment, but a shift in the mix of segments (e.g., more responses this year from a segment that has always reported higher satisfaction) could produce an increase in the overall aggregate despite no segment actually improving. (2) The metric could be a case of a descriptive number obscuring a diagnostic problem (Section 9.1) — for example, satisfaction among long-tenured customers could be falling while a surge of new customers, who tend to rate more generously before experiencing any friction, pulls the aggregate average up, masking a retention risk that would only become visible by segmenting new versus existing customers.

</details>

---

Exercise 3 (Coding)#

Using the retail sales DataFrame from Section 7, add a new column showing each region's Q1 revenue as a percentage of total Q1 revenue across all regions. Which region contributed the largest share?

<details> <summary>Solution</summary>

Python
CopyAsk AI
import pandas as pd

data = pd.DataFrame({
    "region": ["North", "South", "East", "West", "North", "South", "East", "West"],
    "quarter": ["Q1", "Q1", "Q1", "Q1", "Q2", "Q2", "Q2", "Q2"],
    "revenue": [125000, 98000, 143000, 110000, 118000, 76000, 151000, 108000]
})

q1_data = data[data["quarter"] == "Q1"].copy()
total_q1 = q1_data["revenue"].sum()
q1_data["pct_of_total"] = (q1_data["revenue"] / total_q1 * 100).round(1)
print(q1_data[["region", "revenue", "pct_of_total"]])

Expected output shows East contributing the largest share of Q1 revenue (143,000 out of a total of 476,000, roughly 30 percent), which aligns with East also being the strongest-growing region in the Section 7 diagnostic analysis.

</details>

---

Exercise 4 (Applied Thinking)#

Pick a metric you encounter regularly (app usage time, monthly expenses, a fitness tracker statistic). Identify one way this metric could be misleading in aggregate form, and describe what subgroup breakdown might reveal a more complete picture, drawing on Section 9's case studies.

<details> <summary>Solution</summary>

Answers will vary. A sample answer: "Average daily step count" could look stable or improving in aggregate while masking a pattern where step count is rising sharply on weekends (e.g., due to recreational activity) and falling on weekdays (e.g., due to a more sedentary work routine), a pattern only visible by breaking the aggregate down by day of week rather than looking at the overall daily average — directly paralleling the retail region breakdown in Section 7 and the subgroup-reversal risk highlighted in Section 9.2.

</details>

---

13. Quiz#

Q1. Which type of analytics answers the question "why did this happen"? a) Descriptive b) Diagnostic c) Predictive d) Prescriptive

Q2. Which measurement scale has a true zero point, allowing meaningful ratio statements like "twice as much"? a) Nominal b) Ordinal c) Interval d) Ratio

Q3. What does Simpson's Paradox demonstrate? a) Larger sample sizes always produce more accurate results b) A trend in aggregated data can reverse or disappear when broken into subgroups c) Categorical data cannot be visualized d) Descriptive statistics are always misleading

Q4. Why is vectorization generally faster than an equivalent Python for loop for numeric operations on arrays? a) Vectorized code uses less memory in every case b) The computation is pushed into optimized, compiled code instead of the Python interpreter's loop mechanism c) Loops are not capable of numeric operations d) Vectorization always changes the underlying mathematical result

Q5. True or False: Computing the mean of a column of nominal category codes (e.g., 1=North, 2=South, 3=East, 4=West) produces a meaningful summary statistic.

<details> <summary>Answers</summary>

1. b) Diagnostic
2. d) Ratio
3. b) A trend in aggregated data can reverse or disappear when broken into subgroups
4. b) The computation is pushed into optimized, compiled code instead of the Python interpreter's loop mechanism
5. False — nominal codes have no inherent numeric meaning, so their mean is not a meaningful statistic, per Section 8.3

</details>

---

14. Summary#

• Data analytics is the process of examining data to understand what happened, why it happened, and what to do about it, and overlaps with but is distinct from data science and machine learning.
• The four types of analytics (descriptive, diagnostic, predictive, prescriptive) form a natural progression of increasing complexity and business value, with descriptive and diagnostic work forming the bulk of this course.
• The analytics workflow runs from defining a business question through cleaning, transformation, exploration, visualization, statistical testing, and communication.
• Data as vectors and matrices, and vectorized computation, are the shared mathematical and performance foundation underlying every notebook in this course.
• The four measurement scales (nominal, ordinal, interval, ratio) determine which statistics and operations are valid for a given variable, and misapplying them is a common, subtle analytical error.
• Aggregate statistics can actively mislead, as shown by Simpson's Paradox and the dashboard case study, making subgroup analysis and healthy skepticism of top-line numbers essential habits.

---

15. Further Reading#

• Wes McKinney, Python for Data Analysis — the standard reference for Pandas, written by its original author
• David Spiegelhalter, The Art of Statistics: Learning from Data — an accessible treatment of statistical reasoning and common pitfalls, including Simpson's Paradox
• Pandas official documentation: https://pandas.pydata.org/docs/
• NumPy official documentation: https://numpy.org/doc/

---

Next Up#

Notebook 1: NumPy Fundamentals and Vector Math — arrays, dtypes, broadcasting, vectorized operations, and the linear algebra basics that underpin efficient data manipulation throughout the rest of this course.

---

Course: Data Analytics | Notebook 0 of 13