Run a mock round

Class imbalance means one class is far rarer, so accuracy becomes misleading and models favor the majority.

Strategies:

• Resampling: oversample the minority (SMOTE synthesizes examples) or undersample the majority. Do this only on training folds, never before splitting.
• Class weights: set class_weight="balanced" or scale the loss so minority errors cost more.
• Threshold tuning: move the decision threshold to optimize the metric you care about, such as recall or F1.
• Metric choice: use precision-recall AUC, F1 or cost-based measures instead of accuracy or ROC-AUC.
• Anomaly framing: treat the rare class with one-class or isolation methods.
• Collect more minority data if feasible.

model = LogisticRegression(class_weight="balanced")

Always evaluate on a stratified holdout or with stratified cross-validation so each fold keeps the class ratio.

Regularization adds a penalty on model complexity to the loss, discouraging large weights and reducing overfitting.

L2 (ridge) adds the sum of squared weights. It shrinks weights smoothly toward zero but rarely makes them exactly zero, and it handles correlated features well.

L1 (lasso) adds the sum of absolute weights. It drives some coefficients exactly to zero, performing feature selection automatically, which helps with sparse or high-dimensional data.

Elastic net combines both, with alpha controlling overall strength and l1_ratio the mix.

Ridge(alpha=1.0)
Lasso(alpha=0.01)
ElasticNet(alpha=0.1, l1_ratio=0.5)

The strength parameter must be tuned by cross-validation. In neural networks L2 appears as weight decay, and dropout is another regularization technique. Regularization always trades a little training fit for better generalization.

Both are tree ensembles, differing in how trees are combined.

Random forest builds many deep, independent trees on bootstrap samples with random feature subsets, then averages or votes. Trees train in parallel and reduce variance. It is robust, hard to overfit, and needs little tuning.

Gradient boosting builds trees sequentially, each fitting the residual errors of the current ensemble. It reduces bias and often achieves higher accuracy, but is sensitive to hyperparameters and can overfit without regularization. Modern implementations add shrinkage, subsampling and column sampling.

RandomForestRegressor(n_estimators=500, max_features="sqrt")
GradientBoostingRegressor(n_estimators=500, learning_rate=0.05, max_depth=3)

Use a forest as a strong baseline with minimal tuning; use boosting when you need the best tabular performance. HistGradientBoosting, XGBoost and LightGBM are fast defaults.

MLOps applies DevOps practices to machine learning: versioning data, code and models; automating training and deployment; and monitoring models in production.

Key components:

• Experiment tracking and reproducibility, for example MLflow or Weights and Biases.
• Feature stores for consistent training and serving features.
• CI/CD for retraining, testing and rollout with canary or shadow deployment.
• A model registry with lineage and approval.
• Monitoring for performance, latency and cost.

Monitoring matters because models decay. Data drift is when input distributions shift away from training data; concept drift is when the relationship between inputs and target changes. Both degrade accuracy even when the code is unchanged.

Alert on drift, prediction distributions and business KPIs, and define a retraining trigger. Labels may arrive late, so use proxies until ground truth is available.

Cross-validation repeatedly splits data into training and validation folds to estimate how well a model generalizes. In k-fold, the data is split into k parts; the model trains on k-1 and validates on the remaining one, rotating so every point is validated once. The average score is the estimate.

from sklearn.model_selection import cross_val_score
scores = cross_val_score(model, X, y, cv=5, scoring="roc_auc")

Variants: stratified k-fold preserves class ratios for imbalanced data; grouped k-fold keeps all rows of one entity together to avoid leakage; time series split respects temporal order.

Use it for model selection, hyperparameter tuning and comparing algorithms. Pitfalls: preprocessing such as scaling or SMOTE must be fit inside each fold via a pipeline, otherwise information leaks and the score is optimistic. Keep a separate untouched test set for the final estimate.

Gradient descent minimizes a loss function by repeatedly moving parameters in the direction that decreases the loss. The gradient is the vector of partial derivatives, and you step opposite to it.

for epoch in range(epochs):
    preds = X @ w + b
    error = preds - y
    grad_w = (2/n) * X.T @ error
    grad_b = (2/n) * error.sum()
    w -= lr * grad_w
    b -= lr * grad_b

The learning rate is critical: too large and the loss diverges or oscillates, too small and training is slow. Variants: batch gradient descent uses all data per step (stable, slow), stochastic uses one example (noisy, fast), and mini-batch uses a small batch and is the standard. Momentum and Adam adapt the step per parameter. Always scale features, monitor the loss curve, and consider schedules or early stopping.

Bias is error from overly simple assumptions; a high-bias model underfits and misses patterns in both training and test data. Variance is error from sensitivity to the training sample; a high-variance model overfits, fitting noise so training error is low but test error is high.

Expected error decomposes into bias squared, variance and irreducible noise. As model complexity increases, bias falls and variance rises, so the sweet spot minimizes total error.

Diagnose with cross-validation: high bias means both training and validation error are high, so add features or complexity or reduce regularization. High variance means a large gap between low training error and high validation error, so get more data, simplify the model, add regularization, or use bagging. Regularization, early stopping, pruning and ensembling are the main levers.

Choose metrics that match the problem.

• MAE: mean absolute error, in the target's units, robust to outliers.
• MSE and RMSE: penalize large errors more; RMSE is in target units and sensitive to outliers.
• R-squared: fraction of variance explained, but it always rises with more features, so use adjusted R-squared.
• MAPE: percentage error, useful across scales but undefined when the actual is zero.

from sklearn.metrics import mean_absolute_error, root_mean_squared_error, r2_score

Always compare against a baseline, such as predicting the mean or the previous value. Inspect residual plots for patterns, heteroscedasticity and non-linearity, and check errors across segments, not just overall. Use cross-validation for stable estimates and a held-out test set for the final report.

Start with the objective and data: implicit signals such as views, clicks and purchases, plus explicit ratings and item or user features.

A common two-stage design:

• Candidate generation: retrieve a few hundred relevant items quickly. Use collaborative filtering such as matrix factorization or ALS, content-based similarity, or approximate nearest neighbour search over embeddings. This scales to millions of items.
• Ranking: score candidates with a richer model, gradient boosting or a neural network, using user, item and context features to optimize the target event.

user_vec = als_model.user_factors[user_id]
scores = item_factors @ user_vec
candidates = np.argpartition(scores, -200)[-200:]

Add business rules for diversity, freshness and filtering. Evaluate offline with recall@k and NDCG, then run online A/B tests on engagement and revenue. Address the cold start problem with popularity or content features, and monitor feedback loops and drift.