Supervised learning trains on labeled data (input-output pairs). Unsupervised learning finds patterns in unlabeled data with no predefined outputs.

Deep Dive: In supervised learning every training example has a correct answer (label). The algorithm learns to map inputs to outputs by minimizing prediction error. Examples: classification (spam/not spam) regression (predicting house prices). In unsupervised learning data has no labels. The algorithm discovers hidden structure: clustering groups similar items dimensionality reduction compresses features anomaly detection finds outliers. There is also semi-supervised learning (small labeled dataset + large unlabeled dataset) and self-supervised learning (labels generated from the data itself as in language model pretraining). Choosing the right paradigm depends on whether labeled data is available and how expensive it is to obtain.

Real-World: A credit card fraud detection system: training on historical transactions labeled as 'fraud' or 'legitimate' is supervised learning. Discovering clusters of unusual spending behavior without predefined fraud labels is unsupervised (anomaly detection). Real production systems often use both — unsupervised to surface suspicious patterns supervised to classify confirmed cases.

⚠ Common Mistakes: Thinking unsupervised learning is always worse because it has no labels — it is simply solving a different problem. Confusing clustering (unsupervised) with classification (supervised). Underestimating the cost and effort of labeling data for supervised learning at scale.

🏭 Production Scenario: A retail company tried to build a supervised product recommendation model but had insufficient labeled purchase-intent data. Switching to unsupervised collaborative filtering (clustering users by purchase history) produced better recommendations in production without requiring explicit labels.

Follow-up questions: What is semi-supervised learning? What is self-supervised learning as used in GPT? When is unsupervised learning preferred over supervised?

// ID: ML-BEG-001  ·  DIFFICULTY: 2/10  ·  ★★☆☆☆☆☆☆☆☆

Classification predicts a category (discrete output). Regression predicts a continuous numerical value.

Deep Dive: In classification the output is one of a fixed set of categories: spam/not spam cat/dog/bird disease/healthy. Binary classification has two classes multiclass has more. The model output is typically a probability for each class and a threshold or argmax converts it to a final prediction. In regression the output is a continuous number: predicting tomorrow's temperature estimating a house price forecasting sales volume. The same algorithms often have both variants — linear regression vs logistic regression (despite the name logistic regression is a classifier) decision tree regressor vs classifier. Evaluation metrics differ: accuracy/F1 for classification RMSE/MAE/R2 for regression.

Real-World: A real estate platform uses regression to estimate property values (continuous output: $425000) and classification to predict whether a property will sell within 30 days (binary output: yes/no). Both models are trained on the same property feature data but with different target variables and evaluation strategies.

⚠ Common Mistakes: Using regression metrics (RMSE) to evaluate a classifier or vice versa. Treating a regression problem as classification by binning the output (losing information). Not recognizing that logistic regression IS a classifier despite the word 'regression' in its name.

🏭 Production Scenario: A demand forecasting system incorrectly used a classifier to predict inventory needs by bucketing demand into Low/Medium/High. The loss of continuous information caused systematic over-ordering. Switching to a regression model that predicted exact units improved inventory efficiency by 23%.

Follow-up questions: What is ordinal regression? How does multi-label classification differ from multiclass? What is the ROC curve and when is it used?

// ID: ML-BEG-003  ·  DIFFICULTY: 2/10  ·  ★★☆☆☆☆☆☆☆☆

Overfitting is when a model learns the training data too well — including its noise — and performs poorly on new data. Detect it by comparing training and validation accuracy. Prevent it with regularization dropout more data or simpler models.

Deep Dive: A model overfits when it memorizes training examples rather than learning generalizable patterns. The tell-tale sign is high training accuracy but significantly lower validation/test accuracy — the gap between them is your overfitting signal. Prevention techniques: regularization (L1/L2 add penalty terms for large weights) dropout (randomly deactivating neurons during training) early stopping (halt training when validation loss stops improving) data augmentation (artificially expand training data) cross-validation (use all data for both training and validation) and reducing model complexity. The bias-variance tradeoff is the theoretical framework: overfitting is high variance underfitting is high bias.

Real-World: An image classification model for medical diagnostics achieved 99% training accuracy but only 71% on the validation set. Analysis showed it was memorizing specific image artifacts from the training hospital's scanner. Fixing required data augmentation (random crops flips brightness changes) and L2 regularization bringing validation accuracy to 89%.

⚠ Common Mistakes: Evaluating model performance only on training data and reporting those numbers. Not setting aside a test set that is never touched during development. Using the validation set for hyperparameter tuning and then reporting validation accuracy as if it were test accuracy (data leakage).

🏭 Production Scenario: A production churn prediction model was deployed with 94% training accuracy. In production it performed at 61% barely better than always predicting 'no churn'. Investigation revealed no validation split was used and the model had memorized customer IDs that leaked into the feature set.

Follow-up questions: What is the bias-variance tradeoff? How does cross-validation work? What is regularization and what is the difference between L1 and L2?

// ID: ML-BEG-002  ·  DIFFICULTY: 3/10  ·  ★★★☆☆☆☆☆☆☆

Training set is used to fit the model. Validation set is used to tune hyperparameters and select the best model. Test set is held out completely and used only once to report final performance. Using only train/test leads to overfitting on the test set through repeated evaluation.

Deep Dive: Without a separate validation set developers tune hyperparameters (learning rate tree depth regularization strength) by evaluating on the test set. Each evaluation leaks information about the test set into the model selection process — the final reported test accuracy is optimistically biased. A proper split: 70% training (model learns from this) 15% validation (used during development for hyperparameter tuning and model selection) 15% test (locked away evaluated exactly once to report final performance). For small datasets k-fold cross-validation replaces the validation set by rotating which portion of training data is held out. The test set must never be touched during any development decision.

Real-World: An ML competition showed that teams who repeatedly submitted to the public leaderboard (which used the test set) were effectively overfitting to the test set through hundreds of submission cycles. Teams who maintained a strict held-out final test set reported the more realistic performance numbers.

⚠ Common Mistakes: Using the test set during hyperparameter tuning then reporting test set performance as if it were unbiased. Not stratifying the split for classification — random splits of imbalanced data can put almost no positive examples in the validation set. Time-series data: splitting randomly instead of chronologically leaks future information into training.

🏭 Production Scenario: A production recommendation system was developed with 50 rounds of hyperparameter tuning each evaluated on the same test set. Deployed performance was 15% lower than the reported test AUC. Post-mortem confirmed the test set had been evaluated 50 times during development causing effective test set overfitting.

Follow-up questions: What is k-fold cross-validation and when do you use it? How do you handle time-series data splitting? What is nested cross-validation?

// ID: ML-BEG-004  ·  DIFFICULTY: 3/10  ·  ★★★☆☆☆☆☆☆☆

A neural network is a series of connected layers of mathematical functions (neurons) that transform inputs into outputs. It learns by adjusting the connection weights using backpropagation — computing how much each weight contributed to the error and updating it to reduce the error.

Deep Dive: A neural network has an input layer (receives features) hidden layers (learn representations) and an output layer (produces predictions). Each neuron computes a weighted sum of its inputs adds a bias and applies an activation function (ReLU sigmoid tanh) to introduce non-linearity. Learning happens through: forward pass (compute prediction) loss computation (measure how wrong the prediction was using a loss function like cross-entropy or MSE) backpropagation (use chain rule to compute gradient of loss with respect to each weight) and gradient descent (update weights in the direction that reduces loss). This cycle repeats for many iterations (epochs) over the training data. The learning rate controls how large each weight update is.

Real-World: Image classification: the input layer receives pixel values early hidden layers learn to detect edges and colors middle layers detect shapes and textures later layers detect object parts and the output layer assigns class probabilities. This hierarchical feature learning happens automatically through training — no hand-engineering required.

⚠ Common Mistakes: Using too high a learning rate causing the loss to oscillate or diverge. Not normalizing inputs (neural networks are sensitive to input scale). Not enough data — neural networks need more data than traditional ML algorithms to generalize. Using too many layers for a simple problem when a shallower network would suffice.

🏭 Production Scenario: A production image recognition model for quality control on a manufacturing line was failing to converge during training. Investigation showed input images were not normalized — pixel values ranged 0-255 instead of 0-1. Adding a normalization layer as the first layer stabilized training and the model converged in 50 epochs.

Follow-up questions: What is the vanishing gradient problem? What is the difference between SGD Adam and RMSprop optimizers? What is batch size and how does it affect training?

// ID: ML-BEG-005  ·  DIFFICULTY: 3/10  ·  ★★★☆☆☆☆☆☆☆

Cross-validation trains and evaluates a model multiple times on different subsets of data giving a more reliable estimate of generalization performance especially for small datasets. The most common form is k-fold cross-validation.

Deep Dive: In k-fold cross-validation the dataset is split into k equal parts (folds). The model is trained k times each time using k-1 folds for training and 1 fold for validation. The final performance metric is the average across all k evaluations and you also get a standard deviation showing how stable the model is. Common choices: k=5 (20% validation each time) or k=10 (10% validation). Benefits over single split: uses all data for both training and validation (important for small datasets) provides confidence intervals on performance (single split gives one number — is it lucky or representative?) and reveals if the model is sensitive to which data is in training vs validation (high variance = potential overfitting). Stratified k-fold maintains class proportions in each fold — essential for imbalanced classification.

Real-World: A medical ML model for rare disease diagnosis had only 800 labeled examples. A single 80/20 split would train on 640 examples and validate on 160 — too few for either. 10-fold cross-validation trained 10 models each on 720 examples and validated on 80 giving a reliable performance estimate with confidence intervals and using all data for both training and evaluation.

⚠ Common Mistakes: Using k-fold cross-validation for hyperparameter tuning and reporting those scores as test performance (data leakage — use nested cross-validation instead). Not using stratified folds for imbalanced classification. Ignoring the standard deviation across folds — high variance means the model is sensitive to data splits which is itself a problem. Applying cross-validation to time-series data without using TimeSeriesSplit.

🏭 Production Scenario: A production model selection process used 5-fold cross-validation to compare 20 candidate models. The winning model had a mean AUC of 0.87 with standard deviation 0.02 — indicating stable performance across folds. The runner-up had mean AUC 0.86 with standard deviation 0.09 — highly variable and less trustworthy. The stable model was selected and performed as expected in production.

Follow-up questions: What is nested cross-validation and when do you need it? What is TimeSeriesSplit and why can't you use standard k-fold for time-series? What is sklearn Pipeline and why does it matter for cross-validation?

// ID: ML-BEG-006  ·  DIFFICULTY: 4/10  ·  ★★★★☆☆☆☆☆☆