∇GradientSchool
Intermediate
15 min read
#Classification#Scikit-Learn#Logistic Regression#Confusion Matrix#ROC-AUC

Hands-On Classification with Scikit-Learn

End-to-end classification pipeline: Breast Cancer diagnostic dataset, stratified splitting, LogisticRegression, Confusion Matrix, Precision-Recall, ROC-AUC curves, and deployment inference.

Hands-On Classification with Scikit-Learn

1. Setup & Environment Imports#

We import numpy and pandas for dataset manipulation, scikit-learn for real-world diagnostic dataset access, data splitting, scaling, modeling, and evaluation metrics, and matplotlib/seaborn for confusion matrix and ROC curve visualizations.

🐍 Python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Scikit-Learn modules
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
 accuracy_score,
 precision_score,
 recall_score,
 f1_score,
 confusion_matrix,
 classification_report,
 roc_curve,
 auc
)

# Serialization
import joblib

# Plot styling
%matplotlib inline
sns.set_theme(style="whitegrid")

2. Real-World Diagnostic Dataset Ingestion#

We use the canonical Breast Cancer Wisconsin Diagnostic Dataset from sklearn.datasets.

  • Feature Space: 30 continuous dimensional features computed from digitized fine needle aspirate (FNA) images of breast masses (e.g., mean radius, texture, perimeter, area, smoothness).
  • Target Classes: Binary target where 0 = Malignant (Cancerous) and 1 = Benign (Non-Cancerous).
🐍 Python
# Load dataset
cancer_data = load_breast_cancer()

# Construct DataFrame
df = pd.DataFrame(cancer_data.data, columns=cancer_data.feature_names)
df['target'] = cancer_data.target

print("Dataset Matrix Shape:", df.shape)
print("\nFirst 5 records:")
print(df.head())

# Inspect target class distribution
class_counts = df['target'].value_counts()
print("\nClass Distribution:")
print(f"• Benign (Class 1): {class_counts[1]} samples ({class_counts[1]/len(df)*100:.1f}%)")
print(f"• Malignant (Class 0): {class_counts[0]} samples ({class_counts[0]/len(df)*100:.1f}%)")

X = df.drop('target', axis=1)
y = df['target']

3. Stratified Train-Test Splitting#

In clinical and high-stakes classification, maintaining exact class proportions across partitions is essential.

Always pass stratify=y during train-test splitting. Without stratification, rare positive or malignant cases could randomly skew into only one partition, compromising training stability and validation reliability.

🐍 Python
# Stratified 80/20 train-test partition
X_train, X_test, y_train, y_test = train_test_split(
 X, y,
 test_size=0.2,
 random_state=42,
 stratify=y
)

print(f"Training split: {X_train.shape[0]} samples (Benign: {np.mean(y_train==1)*100:.1f}%)")
print(f"Testing split: {X_test.shape[0]} samples (Benign: {np.mean(y_test==1)*100:.1f}%)")

4. Feature Standardization#

Linear classifiers like Logistic Regression and distance-based estimators are sensitive to feature scales. Because geometric radius ([5,30][5, 30]) differs significantly in magnitude from area ([100,2500][100, 2500]), features must be standardized to prevent unscaled variables from dominating gradient updates.

🐍 Python
scaler = StandardScaler()

# Fit strictly on training data
X_train_scaled = scaler.fit_transform(X_train)

# Transform test data using frozen training statistics
X_test_scaled = scaler.transform(X_test)

print("Standardization complete. Feature scaling verified.")

5. Model Fitting & Logistic Regression Analysis#

We fit a regularized LogisticRegression classifier. The estimator uses L2 (Ridge) penalty to minimize cross-entropy loss (Log-Loss):

L(w)=−1n∑i=1n[yilog⁡(p^i)+(1−yi)log⁡(1−p^i)]+12C∥w∥22\mathcal{L}(\mathbf{w}) = -\frac{1}{n} \sum_{i=1}^n \left[ y_i \log(\hat{p}_i) + (1-y_i) \log(1 - \hat{p}_i) \right] + \frac{1}{2C} \|\mathbf{w}\|_2^2

🐍 Python
model = LogisticRegression(max_iter=1000, random_state=42)
model.fit(X_train_scaled, y_train)

print("Model converged successfully.")

# Feature importance analysis (magnitude of weights)
importance_df = pd.DataFrame({
 'Feature': X.columns,
 'Coefficient': model.coef_[0]
}).sort_values(by='Coefficient', ascending=False)

print("\nTop 5 Influential Features (Positive Log-Odds for Benign):")
print(importance_df.head())

6. Discrete Predictions vs. Posterior Probabilities#

Classification models output both discrete predicted class labels (y^∈{0,1}\hat{y} \in \{0, 1\}) and continuous posterior probability confidence vectors (P(y=1∣x)∈[0,1]P(y=1|\mathbf{x}) \in [0, 1]).

🐍 Python
# Class label predictions (Default threshold tau = 0.5)
y_pred = model.predict(X_test_scaled)

# Probability estimates
y_prob = model.predict_proba(X_test_scaled)[:, 1]

# Inspect sample outputs
results_df = pd.DataFrame({
 'Actual Ground Truth': ['Benign' if v == 1 else 'Malignant' for v in y_test[:5]],
 'Predicted Label': ['Benign' if v == 1 else 'Malignant' for v in y_pred[:5]],
 'Posterior Probability (Benign)': y_prob[:5].round(4)
})

print("\nSample Test Predictions:")
print(results_df)

7. Multi-Angle Model Evaluation#

In diagnostic scenarios, relying on accuracy alone is insufficient. We compute a comprehensive suite of classification metrics.

Classification Report (Accuracy, Precision, Recall, F1)#

🐍 Python
print("=" * 60)
print("Classification Report:")
print(classification_report(y_test, y_pred, target_names=['Malignant (0)', 'Benign (1)']))
print("=" * 60)

acc = accuracy_score(y_test, y_pred)
prec = precision_score(y_test, y_pred)
rec = recall_score(y_test, y_pred)
f1 = f1_score(y_test, y_pred)

print(f"Overall Accuracy: {acc:.4f}")
print(f"Precision Score: {prec:.4f} (Positive Predictive Value)")
print(f"Recall Score: {rec:.4f} (Sensitivity / True Positive Rate)")
print(f"F1 Score: {f1:.4f} (Harmonic Mean of Precision and Recall)")

Confusion Matrix Heatmap Analysis#

The confusion matrix cross-tabulates actual vs. predicted labels into True Positives, False Positives, True Negatives, and False Negatives:

🐍 Python
cm = confusion_matrix(y_test, y_pred)

plt.figure(figsize=(7, 5))
sns.heatmap(
 cm,
 annot=True,
 fmt='d',
 cmap='Blues',
 xticklabels=['Predicted Malignant', 'Predicted Benign'],
 yticklabels=['Actual Malignant', 'Actual Benign']
)
plt.title('Confusion Matrix: Diagnostic Evaluation')
plt.xlabel('Predicted Label')
plt.ylabel('Ground Truth Label')
plt.tight_layout()
plt.show()

print("\nConfusion Matrix Breakdown:")
print(f"• True Negatives (Malignant correctly identified): {cm[0][0]}")
print(f"• False Positives (Malignant incorrectly called Benign): {cm[0][1]}")
print(f"• False Negatives (Benign incorrectly called Malignant): {cm[1][0]}")
print(f"• True Positives (Benign correctly identified): {cm[1][1]}")

Receiver Operating Characteristic (ROC) & AUC Curve#

The ROC curve demonstrates the trade-off between True Positive Rate (Sensitivity) and False Positive Rate (1−Specificity1 - \text{Specificity}) across all classification thresholds:

🐍 Python
fpr, tpr, thresholds = roc_curve(y_test, y_prob)
roc_auc = auc(fpr, tpr)

plt.figure(figsize=(8, 5))
plt.plot(fpr, tpr, color='#2563eb', lw=2.5, label=f'ROC Curve (AUC = {roc_auc:.3f})')
plt.plot([0, 1], [0, 1], color='#94a3b8', lw=1.5, linestyle='--', label='Random Classifier (AUC = 0.50)')

plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate (1 - Specificity)')
plt.ylabel('True Positive Rate (Recall / Sensitivity)')
plt.title('Receiver Operating Characteristic (ROC) Curve')
plt.legend(loc='lower right')
plt.grid(True, linestyle=':', alpha=0.6)
plt.tight_layout()
plt.show()

print(f"Computed ROC-AUC Score: {roc_auc:.4f}")

8. Model & Scaler Artifact Persistence#

Persist both the fitted classifier and preprocessor artifacts to disk using joblib.

🐍 Python
joblib.dump(model, 'classification_model.pkl')
joblib.dump(scaler, 'classification_scaler.pkl')

print("Serialized artifacts saved: classification_model.pkl and classification_scaler.pkl")

9. Production Inference on New Patient Data#

Simulate deploying the model inside an API microservice to evaluate a new, unseen patient diagnostic record.

🐍 Python
# Load production artifacts
loaded_model = joblib.load('classification_model.pkl')
loaded_scaler = joblib.load('classification_scaler.pkl')

# Simulate incoming patient test record (1x30 vector)
raw_patient_record = X_test.iloc[[0]]

# Apply frozen production scaler
scaled_record = loaded_scaler.transform(raw_patient_record)

# Generate prediction and calibrated confidence
predicted_class = loaded_model.predict(scaled_record)[0]
confidence = loaded_model.predict_proba(scaled_record)[0][predicted_class]

diagnosis = "Benign (Non-Cancerous)" if predicted_class == 1 else "Malignant (Cancerous)"

print("Production Inference Result:")
print(f"• Diagnostic Classification: {diagnosis}")
print(f"• Model Confidence: {confidence*100:.2f}%")

10. Pipeline Engineering Summary#

In this classification masterclass, we implemented:

  1. Diagnostic Ingestion: Loaded multidimensional Wisconsin FNA biopsy data.
  2. Stratification: Guaranteed identical class distributions across train and test partitions.
  3. Scaling: Prevented feature magnitude bias with StandardScaler.
  4. Model Optimization: Trained regularized LogisticRegression for probability calibration.
  5. Comprehensive Diagnostics: Evaluated Precision, Recall, F1, Confusion Matrices, and ROC-AUC.
  6. Production Deployment: Serialized preprocessor and estimator for automated scoring.
Knowledge Checkpoint

Classification with Scikit-Learn Checkpoint

Q1.What mathematical function maps linear combinations of features $z = w^T x + b$ into probabilities $(0, 1)$ in Logistic Regression?
AThe Sigmoid (logistic) function: $\sigma(z) = \frac{1}{1 + e^{-z}}$
BThe ReLU function: $\max(0, z)$
CThe Step function: $z > 0 ? 1 : 0$
DThe Linear Identity function: $f(z) = z$
Q2.What splitting criterion does a Decision Tree Classifier use to evaluate node purity?
AGini Impurity or Shannon Information Gain (Entropy)
BMean Squared Error (MSE)
CCosine Similarity
DEuclidean Distance
Q3.How does Support Vector Machine (SVM) find the optimal decision hyperplane?
ABy maximizing the margin (geometric distance) between the separating hyperplane and the closest support vectors of each class.
BBy averaging all positive instances together.
CBy applying k-nearest neighbor voting.
DBy minimizing tree depth.
Track Your Learning

Finished studying this notebook?

Mark this guide as completed to update your course progress roadmap.

Previous Lesson
Hands-On Regression with Scikit-Learn
Next Lesson
The Ultimate Guide to ML Evaluation Metrics