40 Essential Machine Learning Interview Questions and Answers for Fall 2025

40 Machine Learning Interview Questions and Answers

Q1. Why do we take the harmonic mean of precision and recall when computing the F1 score instead of a simple average?

The F1 score (the harmonic mean of precision and recall) balances the trade‑off between the two metrics. The harmonic mean penalises extreme values more than the arithmetic mean, ensuring that when one metric is much lower than the other, the overall score reflects that imbalance. This yields a more balanced evaluation for classification tasks where precision and recall may be inversely related.

Q2. Why does logistic regression, which is used for classification, contain the word “regression” in its name?

Logistic regression does not directly output class labels; it uses a linear model to estimate the probability of an event (0‑1). A threshold (e.g., 50 %) is then applied to convert the probability into a class such as “yes” or “no”. Hence, despite the term “regression”, the model ultimately predicts a class.

Q3. What is the purpose of activation functions in neural networks?

Activation functions introduce non‑linearity, enabling neural networks to learn complex patterns and relationships. Without them, a network would collapse to a linear model, limiting its ability to capture intricate features. Common activation functions include the sigmoid, tanh, and ReLU, each providing non‑linear transformations that allow networks to approximate complex functions for tasks like image recognition and natural‑language processing.

Q4. If you do not know whether the data have been scaled and must handle a classification problem without inspecting the data, which technique would you choose for Random Forest and Logistic Regression, and why?

Random Forest is the more appropriate choice. Logistic Regression is sensitive to feature scale, and unscaled features can degrade its performance. Random Forest, by contrast, is largely invariant to scaling because each tree splits on feature values independently of their magnitude. Therefore, when feature scaling is unknown, Random Forest tends to produce more reliable results.

Q5. In a binary classification task for cancer detection, which metric would you sacrifice—precision or recall—and why?

Recall (sensitivity) should be prioritised. Maximising recall ensures that as many true cancer cases as possible are identified, reducing false‑negative (missed) diagnoses, which can have severe consequences. While precision helps minimise false positives, the cost of missing a cancer case outweighs the cost of extra false positives in medical settings.

Q6. What is the significance of a p‑value when building machine‑learning models?

In classical statistics, a p‑value measures the significance of a specific effect or parameter. It can be used to identify features that are more strongly correlated with the target; the closer the p‑value is to 0, the higher the feature’s relevance.

Q7. How does dataset skew affect machine‑learning model performance or behaviour?

Skewed data introduces bias during training, especially for algorithms sensitive to class distribution. Models may favour the majority class, leading to poor predictions for minority classes. Skew also shifts decision boundaries (e.g., in logistic regression or SVM) toward the dominant class and can inflate accuracy metrics while masking poor performance on minority classes.

Q8. When might ensemble methods be useful?

Ensembles are valuable for complex, heterogeneous datasets or when improving model robustness and generalisation is desired. For example, in medical diagnosis where multiple tests provide complementary information, combining models such as Random Forest or Gradient Boosting mitigates individual model bias and uncertainty, yielding more reliable predictions.

Q9. How can outliers be detected in a dataset?

Common methods include:

Z‑score : flag points whose Z‑score exceeds a chosen threshold.

IQR (interquartile range) : mark points beyond 1.5 × IQR from the quartiles.

Visualization : box plots, histograms, or scatter plots reveal obvious deviations.

Machine‑learning models : one‑class SVM or Isolation Forest can be trained to identify anomalous instances.

Q10. Explain the bias‑variance trade‑off in machine learning and its impact on model performance.

The bias‑variance trade‑off describes the balance between error due to overly simplistic assumptions (high bias) and error due to sensitivity to training data noise (high variance). High bias leads to underfitting, while high variance leads to overfitting. Reducing bias usually increases variance and vice‑versa; the optimal model finds a sweet spot that minimises total error on both training and unseen data.

Q11. Describe how Support Vector Machines (SVM) work and their kernel trick. When would you choose SVM over other algorithms?

SVM seeks the optimal hyperplane that maximises the margin between classes. The kernel trick maps data into a higher‑dimensional space, turning non‑linearly separable data into linearly separable data.

SVM is preferred when:

Dealing with high‑dimensional data.

Clear class boundaries are desired.

Non‑linear relationships need to be captured via kernels.

Interpretability is less critical than predictive performance.

Q12. Explain the difference between Lasso (L1) and Ridge (L2) regularisation.

Both add penalty terms to the loss function to prevent overfitting. Lasso (L1) adds the absolute value of coefficients, encouraging sparsity by driving some coefficients to zero. Ridge (L2) adds the squared value of coefficients, discouraging large weights but rarely producing sparsity.

Choose Lasso when feature selection is important; choose Ridge when all features contribute meaningfully.

Q13. What is self‑supervised learning?

Self‑supervised learning generates labels from the data itself, using inherent structure or relationships as supervision. Typical tasks include predicting missing image patches, masking words in sentences, or forecasting video frames, allowing models to learn without manually annotated data.

Q14. Explain Bayesian optimisation for hyper‑parameter tuning and how it differs from grid search or random search.

Bayesian optimisation builds a probabilistic model of the objective function and uses it to select promising hyper‑parameter configurations, leveraging information from previous evaluations. Unlike exhaustive grid search or uninformed random search, it converges more efficiently, requiring fewer evaluations for complex, computationally expensive models.

Q15. Contrast semi‑supervised learning with self‑supervised learning.

Semi‑supervised learning : Uses both labelled and unlabelled data; the model learns from labelled samples while exploiting structure in unlabelled data to improve generalisation.

Self‑supervised learning : Generates its own supervisory signal from raw data, eliminating the need for external labels.

Q16. What is the significance of out‑of‑bag (OOB) error in machine‑learning algorithms?

OOB error is an unbiased estimate of a model’s generalisation error obtained from bootstrap samples that were not included in the training of each individual tree within an ensemble (e.g., bagging). It provides a validation metric without requiring a separate hold‑out set and can guide hyper‑parameter tuning.

Q17. Explain the concepts of Bagging and Boosting.

Bagging (Bootstrap Aggregating) : Generates multiple training subsets via random sampling with replacement, trains independent base models on each subset, and aggregates their predictions to reduce overfitting and improve generalisation.

Boosting : Trains a sequence of weak learners, each focusing on the errors of its predecessor by assigning higher weights to mis‑classified instances, thereby progressively improving overall performance.

Q18. What advantages does Random Forest have over a single decision tree?

Reduces overfitting by averaging predictions from many trees trained on different subsets.

Typically yields higher accuracy on complex datasets.

Provides feature‑importance scores to identify influential variables.

More robust to outliers due to the averaging effect.

Q19. How does Bagging reduce model variance?

Bagging trains multiple base models on different bootstrap samples of the training data. Averaging or aggregating their predictions smooths out the variance caused by any single noisy or atypical sample, leading to a more stable and less over‑fitted ensemble.

Q20. In the bootstrap‑aggregating process, can a single sample contain the same original record multiple times?

Yes. Because bootstrap sampling is performed with replacement, the same original row may appear multiple times within a single bootstrap sample, increasing diversity among the base models.

Q21. Relate Bagging to the “No Free Lunch” theorem.

The “No Free Lunch” theorem states that no single algorithm excels on all possible problems. Bagging embodies this principle by creating diverse models on different data subsets, acknowledging that different subsets may require different hypotheses for optimal performance.

Q22. Differentiate hard voting and soft voting in Boosting ensembles.

Hard voting : Each model casts a categorical vote; the majority class wins.

Soft voting : Models output class probabilities; the final prediction is based on the averaged (or weighted) probabilities, incorporating confidence levels.

Q23. How does voting‑boost differ from simple majority voting and Bagging?

Voting‑boost : Trains weak learners sequentially, re‑weighting mis‑classified instances to improve overall performance.

Simple majority voting (e.g., Bagging) : Gives each model equal vote weight; no sequential error correction.

Bagging : Independently trains multiple models on different subsets and aggregates predictions to reduce variance.

Q24. How does the choice of weak learner (e.g., decision stump vs. deeper tree) affect Boosting performance?

Weak learners that are too simple (e.g., decision stumps) have low computational cost and are less prone to overfitting, making them suitable for boosting. More complex learners (deeper trees) can capture richer patterns but risk overfitting, potentially degrading the ensemble’s generalisation.

Q25. What are forward fill and backward fill in data preprocessing?

Forward fill propagates the last observed non‑missing value forward to fill subsequent missing entries, useful for time‑series gaps.

Backward fill propagates the next observed non‑missing value backward to fill preceding missing entries, appropriate when future values are expected to resemble past ones.

Q26. Distinguish feature selection from feature extraction.

Feature selection : Chooses a subset of original features based on relevance, reducing dimensionality while preserving interpretability.

Feature extraction : Transforms the original data into a new set of features (e.g., PCA, t‑SNE) that capture most variance, often reducing dimensionality but sacrificing direct interpretability.

Q27. How does cross‑validation help improve model performance?

Cross‑validation repeatedly splits the dataset into training and validation folds, providing a robust estimate of a model’s generalisation ability. It helps detect overfitting, guides hyper‑parameter tuning, and yields a more reliable performance metric.

Q28. Compare feature scaling and feature normalisation.

Feature scaling : Rescales features to a common range (e.g., min‑max, Z‑score) to prevent large‑scale features from dominating learning algorithms.

Feature normalisation : Transforms features to have zero mean and unit variance (standardisation), a specific form of scaling aimed at achieving a standard normal distribution.

Q29. How should you choose an appropriate scaling/normalisation method for a specific ML task?

Consider the algorithm and data characteristics:

Min‑max scaling : Works well for algorithms sensitive to absolute scale (e.g., neural networks) when data are uniformly distributed.

Z‑score normalisation : Suited for algorithms assuming normally distributed features and provides robustness to outliers.

Robust scaling : Uses inter‑quartile range; ideal when the dataset contains outliers.

Q30. Compare Z‑score normalisation with Min‑Max scaling.

Z‑score : Centers data to mean 0 and standard deviation 1; robust to outliers and appropriate for normally distributed data.

Min‑Max scaling : Maps data to a fixed range (commonly [0, 1]); preserves the original distribution but is sensitive to outliers.

Q31. What is an “IVF score” and what does it mean for building ML models?

The term “IVF score” is not a standard metric in machine learning or feature engineering; additional context would be required to explain it.

Q32. How do you compute Z‑scores for a dataset that contains outliers, and what other factors should be considered?

Outliers can distort the mean and standard deviation, making Z‑scores unreliable. A robust alternative is to use the median absolute deviation (MAD) instead of the mean and standard deviation, providing a more stable estimate of central tendency and dispersion for outlier‑rich data.

Q33. Explain pre‑pruning and post‑pruning of decision trees, including their pros and cons.

Pre‑pruning : Stops tree growth early based on criteria such as information gain, reducing overfitting and improving training efficiency, but may lead to underfitting if the stopping condition is too strict.

Post‑pruning : Allows the tree to grow fully, then removes branches that contribute little to predictive performance, potentially improving accuracy but risking overfitting if pruning decisions are not well‑informed.

Q34. What are the core principles behind model quantisation and pruning, and how do they differ?

Model quantisation : Reduces the precision of weights and activations (e.g., from 32‑bit float to 8‑bit integer) to lower memory footprint and computational demand, facilitating deployment on resource‑constrained devices.

Model pruning : Removes unnecessary connections or entire neurons from a neural network, decreasing parameter count and inference time while preserving accuracy.

Q35. How would you approach an image‑segmentation problem?

Typical steps include:

Data preparation: collect a labelled dataset with images and segmentation masks.

Robust statistics: consider using median and IQR to mitigate outlier effects.

Outlier handling: decide whether to remove or transform anomalous pixels before computing statistics.

Model selection: choose a suitable architecture such as U‑Net, Mask R‑CNN, or DeepLab.

Data augmentation: apply rotations, flips, scaling, etc., to increase variability.

Model training: train on the labelled set, optionally leveraging transfer learning.

Hyper‑parameter tuning: adjust learning rate, batch size, regularisation, etc.

Evaluation: use metrics like IoU or Dice coefficient on a validation set.

Post‑processing: refine masks and remove artefacts.

Q36. What is GridSearchCV?

GridSearchCV (grid‑search cross‑validation) systematically explores a predefined hyper‑parameter grid, evaluating each combination via cross‑validation to identify the configuration that yields the best model performance.

Q37. Define false positives and false negatives and their importance.

False positive (FP) : The model incorrectly predicts the positive class for a negative instance.

False negative (FN) : The model incorrectly predicts the negative class for a positive instance.

In medical diagnosis, false positives can lead to unnecessary treatment, while false negatives can miss critical conditions, potentially causing harm.

Q38. What is PCA in machine learning, and can it be used for feature selection?

PCA (Principal Component Analysis) : A dimensionality‑reduction technique that transforms high‑dimensional data into a lower‑dimensional space while preserving as much variance as possible.

Using PCA for feature selection : PCA primarily reduces dimensionality rather than selecting original features, but by retaining the most informative components it indirectly performs a form of feature selection. When interpretability of individual features is crucial, explicit feature‑selection methods may be preferable.

Q39. Your model exhibits high bias and low variance. How would you address this?

Increase model complexity (e.g., switch from linear to non‑linear models).

Engineer additional relevant features.

Reduce regularisation strength.

Employ ensemble methods to combine diverse models.

Perform hyper‑parameter optimisation to find a better configuration.

Q40. How is the area under the ROC curve (AUC) interpreted?

The ROC curve plots true‑positive rate versus false‑positive rate at various thresholds. AUC quantifies overall model discrimination:

AUC = 1: perfect classifier (no false positives or false negatives).

AUC = 0.5: no better than random guessing.

AUC > 0.5: better than random; higher values indicate stronger discriminative ability, especially useful for imbalanced datasets.

Higher AUC values correspond to better model performance.