Machine Learning Technical Interview Masterclass

Focus: Supervised Learning Theory, Algorithm Assumptions, Evaluation Metrics, and Production Scenarios Level: Beginner to Advanced Domain: Machine Learning Engineering & Data Science

---

Table of Contents#

1. Fundamentals of Machine Learning Paradigms
2. Core Algorithms: Regression & Classification
3. Evaluation Metrics Selection Matrix
4. The Bias-Variance Tradeoff & Overfitting
5. Data Preprocessing & Data Leakage
6. System Design & Scenario-Based Case Studies
7. High-Impact Technical Interview Strategies

---

1. Fundamentals of Machine Learning Paradigms#

Q1: What are the mathematical differences between Supervised, Unsupervised, Semi-Supervised, and Reinforcement Learning?#

Answer:

• Supervised Learning: Learns a mapping function $f: \mathcal{X} \rightarrow \mathcal{Y}$ from labeled dataset $\mathcal{D} = \{(x_i, y_i)\}_{i=1}^N$ to minimize empirical risk $\sum \mathcal{L}(f(x_i), y_i)$. Used for Regression ($y \in \mathbb{R}$) and Classification ($y \in \{1, \dots, C\}$).
• Unsupervised Learning: Discovers latent data structure $P(X)$ from unlabeled dataset $\mathcal{D} = \{x_i\}_{i=1}^N$ (e.g., Clustering via K-Means/DBSCAN, Dimensionality Reduction via PCA/t-SNE).
• Semi-Supervised Learning: Combines a small labeled set $\mathcal{D}_L$ with a large unlabeled set $\mathcal{D}_U$ ($|\mathcal{D}_U| \gg |\mathcal{D}_L|$) using smoothness, cluster, or manifold assumptions.
• Reinforcement Learning: An agent interacts with an environment modeled as a Markov Decision Process (MDP) $(S, A, P, R, \gamma)$ to learn a policy $\pi(a \mid s)$ that maximizes cumulative expected discounted reward $\mathbb{E}\left[\sum_{t=0}^\infty \gamma^t R_t\right]$.

---

2. Core Algorithms: Regression & Classification#

Q2: What are the core assumptions of Ordinary Least Squares (OLS) Linear Regression?#

Answer:

1. Linearity: The relationship between dependent and independent variables is linear in parameters: $y = X\beta + \epsilon$.
2. Strict Exogeneity: The expected error given inputs is zero: $\mathbb{E}[\epsilon \mid X] = 0$.
3. No Multicollinearity: The design matrix $X$ has full column rank ($\text{rank}(X) = p$), meaning features are not linearly dependent.
4. Homoscedasticity: Error terms have constant variance: $\text{Var}(\epsilon_i \mid X) = \sigma^2$.
5. No Autocorrelation: Residuals are pairwise uncorrelated: $\text{Cov}(\epsilon_i, \epsilon_j \mid X) = 0, \forall i \neq j$.
6. Normality: Errors are normally distributed $\epsilon \sim \mathcal{N}(0, \sigma^2 I)$ for valid statistical hypothesis testing ($t$-test, $F$-test).

Q3: Why is Logistic Regression classified as a Generalized Linear Model (GLM)?#

Answer: Logistic Regression fits a linear boundary $\eta = w^T x + b$, but passes it through a non-linear link function (the Logit function, inverse of Sigmoid $\sigma(\eta) = \frac{1}{1 + e^{-\eta}}$) to model the log-odds of binary class probabilities:

$\ln\left(\frac{P(y=1 \mid x)}{1 - P(y=1 \mid x)}\right) = w^T x + b$

Parameters are estimated by maximizing the Bernoulli Log-Likelihood via gradient ascent (Cross-Entropy Loss minimization).

Q4: Compare Decision Trees vs. Random Forests vs. Gradient Boosted Trees (GBDT).#

---

3. Evaluation Metrics Selection Matrix#

Q5: Why is Accuracy deceptive on imbalanced datasets?#

Answer: On a dataset with $99\%$ negative samples (e.g., Credit Card Fraud), a zero-rule model predicting all zeros attains $99\%$ accuracy while having $0\%$ recall on fraud.

code
Ask AICopy
Metric Matrix for Classification:
├── True Positive Rate (Recall / Sensitivity) = TP / (TP + FN)
├── Precision (Positive Predictive Value) = TP / (TP + FP)
├── F1-Score (Harmonic Mean) = 2 * (Precision * Recall) / (Precision + Recall)
└── Specificity (True Negative Rate) = TN / (TN + FP)

Harmonic Mean Property: The harmonic mean heavily penalizes extreme asymmetry. If $\text{Precision}=1.0$ and $\text{Recall}=0.0$, the arithmetic mean is $0.50$, but the harmonic $F_1 = 0.0$, accurately reflecting that the model failed.

Q6: When should you prioritize Precision vs. Recall?#

• Prioritize Precision (Minimize False Positives):
• Spam Filtering: Legitimate important emails must not be relegated to Spam.
• Content Recommendation: Recommended items must be relevant.
• Prioritize Recall (Minimize False Negatives):
• Cancer / Pathological Diagnostics: Missing an active malignant tumor is catastrophic.
• Fraud Detection: Failing to intercept a high-dollar fraudulent transfer incurs direct financial loss.

Q7: Compare MAE vs. MSE vs. RMSE vs. $R^2$ in Regression.#

---

4. The Bias-Variance Tradeoff & Overfitting#

Q8: Derive the mathematical decomposition of Expected Prediction Error.#

Answer: For true relationship $y = f(x) + \epsilon$ with noise $\epsilon \sim \mathcal{N}(0, \sigma^2)$ and estimator $\hat{f}(x)$:

$\mathbb{E}\left[(y - \hat{f}(x))^2\right] = \underbrace{\left(\text{Bias}\left[\hat{f}(x)\right]\right)^2}_{\text{Approximation Error}} + \underbrace{\text{Var}\left(\hat{f}(x)\right)}_{\text{Sensitivity to Sample Variance}} + \underbrace{\sigma^2}_{\text{Irreducible Noise}}$

Architecture & Data Flow
Copy
Error
 |
 | \ / Total Error
 | \ /
 | \ Optimal Zone / Variance
 | \ | / /
 | \ | / /
 | Bias \ | / /
 | \ \ | / /
 |____\_______v___v___v/____________________
 Model Complexity

Q9: What are 5 distinct strategies to combat Overfitting?#

1. Regularization: Add L1 (Lasso, $\lambda \sum |w_i|$) for sparsity or L2 (Ridge, $\lambda \sum w_i^2$) for weight shrinkage.
2. Cross-Validation: Utilize $K$-Fold cross-validation to guarantee performance consistency across folds.
3. Pruning / Depth Constraints: Limit maximum tree depth (max_depth), minimum split samples (min_samples_split).
4. Ensembling: Aggregate multiple independent models via Bagging (Random Forest).
5. Data Augmentation / Resampling: Expand training volume to reduce model variance.

---

5. Data Preprocessing & Data Leakage#

Q10: What is Data Leakage and how do you prevent it in production pipelines?#

Answer: Data Leakage occurs when information from outside the training partition (test set, out-of-fold validation set, or future temporal data) unintentionally influences the model fitting stage.

Common Sources & Remedies:

• Global Preprocessing: Scaling or imputing before splitting $\rightarrow$ Remedy: Wrap transformations inside Scikit-Learn Pipeline.
• Target Leakage: Including features calculated using information not available at inference time (e.g., using Total_Duration_Of_Call to predict whether a customer will cancel their account during that call) $\rightarrow$ Remedy: Perform strict feature point-in-time auditing.
• Temporal Leakage: Randomly shuffling time-series data $\rightarrow$ Remedy: Use TimeSeriesSplit (Walk-Forward Validation).

---

6. System Design & Scenario-Based Case Studies#

Q11: High-Cardinality Dataset Scenario#

Question: You have a dataset with 5,000,000 rows and 800 features. Training a Random Forest is taking 4 hours per run. How do you redesign the pipeline?

Answer:

1. Feature Selection:

• Remove zero/near-zero variance features.
• Compute Pearson/Spearman correlation matrix and drop collinear pairs ($r > 0.90$).
• Run LightGBM with tree-based importance on a $10\%$ subsample to filter the top 100 features.

1. Algorithm Switch: Transition from CPU Random Forest to LightGBM (histogram-based splits, GPU acceleration) or XGBoost (tree_method='hist').
2. Data Types Optimization: Downcast floats (float64 $\rightarrow$ float32) and categorical integers (int64 $\rightarrow$ int16/category).

Q12: Medical Triage Classifier Optimization#

Question: An emergency room AI system flags high-risk cardiac patients. The clinical director states: "We cannot afford to miss a single high-risk patient, even if it means running extra tests on moderate-risk patients." How do you calibrate your pipeline?

Answer:

1. Metric Focus: Optimize for Recall (Sensitivity) on the High-Risk class ($y=1$).
2. Threshold Tuning: Shift the classification decision threshold $\tau$ downward from the default $0.50$ to $0.15-0.20$ along the Precision-Recall curve.
3. Loss Function Modification: Implement cost-sensitive learning via class weights or asymmetric focal loss penalizing False Negatives significantly higher than False Positives.

---

7. High-Impact Technical Interview Strategies#

1. Structure Every Response: Begin with the high-level definition $\rightarrow$ present the mathematical formulation $\rightarrow$ contrast edge cases $\rightarrow$ conclude with practical Scikit-Learn implementation details.
2. State Assumptions Explicitly: When discussing algorithms, immediately state prerequisites (e.g., "Linear regression assumes independent observations, homoscedasticity, and lack of multicollinearity...").
3. Tie Metrics to Business ROI: Connect metrics directly to business outcomes (e.g., "In loan underwriting, a False Positive leads to customer attrition, while a False Negative causes capital write-offs...").