Mathematical and algorithmic deep dives: Linear & Polynomial Regression, Logistic Regression, CART Trees, Random Forests, Gradient Boosting (XGBoost/LightGBM/CatBoost), SVMs, KNN, Naive Bayes, and Multi-Layer Perceptrons.
Deep mathematical mechanics of linear and polynomial regression: Ordinary Least Squares (OLS) closed-form matrix derivation, gradient descent convergence, Gauss-Markov theorem, and polynomial basis expansion.
Internal mathematical formulation of logistic regression: sigmoid activation function derivations, log-odds logit mapping, Maximum Likelihood Estimation via Binary Cross-Entropy, and multiclass Softmax.
In-depth CART algorithmic mechanics: mathematical formulations of Gini Impurity vs Shannon Entropy, recursive binary partitioning, cost-complexity pruning (ccp_alpha), and tree variance properties.
Variance reduction through ensemble aggregation: bootstrap sampling mathematics, random feature subspace projection, out-of-bag (OOB) error estimation, and MDI vs permutation feature importance.
Mathematical foundations of boosting: gradient descent in functional space, pseudo-residuals, learning rate shrinkage, and algorithmic architectural comparisons of XGBoost, LightGBM, and CatBoost.
Rigorous mathematical treatment of SVMs: geometric margin maximization, KKT conditions, primal-to-dual Lagrangian derivation, soft-margin slack variables, and Mercer's condition for non-linear kernels.
Geometric mechanics of instance-based learning: Minkowski distance metrics, mathematical proof of the curse of dimensionality, decision boundary geometry, and spatial search with KD-Trees and Ball-Trees.
First-principles derivation of probabilistic classification: Bayes' theorem, conditional independence assumptions, Maximum Likelihood estimation, Laplace smoothing, and Gaussian/Multinomial/Bernoulli models.
Foundational neural network architecture: forward propagation tensor algebra, non-linear activation functions, multivariate matrix calculus chain rule derivation for backpropagation, and weight initialization.