When Machine Learning Adds Value — and When It Does Not

Machine learning is a tool, not a research objective

Beginning with “we should use machine learning” reverses the analytical sequence. It makes a technique the goal before the problem, available evidence or intended decision has been defined.

Problem → Data → Objective → Method → Validation → Interpretation or decision

This order leaves room for machine learning when it earns its place and for a simpler statistical approach when that is more suitable. The best method is the one that appropriately addresses the question, data and decision context—not necessarily the most complex model available.

What machine learning is particularly good at

Machine-learning methods can be useful for prediction and classification, especially where relationships are nonlinear, interactions are complex or the number and form of inputs make manual specification difficult. They also support pattern recognition in images, text and other complex data, and can enable scalable analytical workflows when the task is clearly defined.

These strengths do not mean a model automatically discovers truth or identifies a causal mechanism. A pattern can improve prediction while remaining a proxy for an unmeasured process, a feature of the sample or an artefact of data collection.

Prediction and explanation are different goals

A predictive question

“Can we accurately estimate or classify an outcome?” prioritises performance on relevant, unseen observations. The model may use many features and complex relationships if those choices improve validated performance and remain suitable for the intended setting.

An explanatory or inferential question

“What is the relationship between particular variables and an outcome, with quantified uncertainty?” puts the meaning of specified parameters, assumptions and uncertainty at the centre. A model can predict accurately without explaining why an outcome occurs; equally, an interpretable estimated relationship need not be an optimal predictor.

Neither objective is inherently superior. The analytical strategy and reporting should state which one is being pursued. Related methodological domains are outlined in research areas and the site's analytics contributions.

Sometimes a regression model is the better model

Conventional regression may be preferable when interpretability is central, the question concerns specified relationships, uncertainty estimates matter or the sample is relatively small. It can also be a better fit for straightforward data structures or stakeholders who need transparent coefficients or effect estimates.

Regression and machine learning are not opposing camps. Regression can form a strong benchmark, and regularisation or flexible terms can bridge parts of the distinction. Comparing credible options is more informative than attaching value to a method's label.

Data quantity does not automatically mean data quality

A large dataset may still contain systematic missingness, measurement error, unreliable labels, class imbalance or selection bias. Leakage can expose the model to information unavailable at the point of use, producing deceptively strong evaluation results. Poor representativeness, temporal shifts and inconsistent definitions can further weaken performance after deployment.

Model sophistication cannot compensate for a poor data-generating process. Understanding how observations and outcomes were produced is therefore part of modelling, not preparation to be delegated and forgotten.

Validation is not optional

Evaluation must preserve genuinely unseen information. Depending on the problem, that may involve training and test separation, cross-validation that respects groups or time, and external validation in another relevant setting where possible. These strategies help detect overfitting and estimate out-of-sample performance.

Calibration may matter when predicted probabilities will be interpreted directly. The validation design and evaluation metrics should reflect the intended use, data structure and likely sources of shift; no single split or threshold is universally correct. Data science and analytics support should treat this design as a core part of the work.

Accuracy alone may be misleading

Overall accuracy can conceal poor performance for an uncommon but important class. Depending on the task, sensitivity or recall, specificity, precision, F1 score, ROC-AUC, calibration or a suitable prediction-error measure may reveal different strengths and weaknesses.

No metric is universally best. Its relevance depends on class balance, whether probability estimates or classifications are needed, and the consequences of different errors. Metrics should be interpreted together with uncertainty and performance across relevant groups and settings.

Interpretability and decision context matter

A technically strong model can still be unsuitable if users do not understand its intended use, outputs cannot be operationalised or interpretability requirements are high. Errors may have asymmetric consequences: a false positive and false negative rarely carry identical implications in every context.

In health applications, predictive outputs should be evaluated within an explicitly governed decision process rather than treated as autonomous clinical decisions. The machine-learning health decision-support case study provides project context; it is not a claim that machine learning is appropriate for every health question.

When machine learning may not add enough value

Machine learning may offer limited benefit with very small datasets, insufficient outcome events, simple relationships or weak data quality. It may also be a poor fit when appropriate validation is unavailable or the primary question requires highly interpretable inference about specified relationships.

If a simpler model performs comparably under relevant validation and is easier to explain, maintain and use, added complexity has not earned its place. That conclusion is not anti-technology; it is evidence-based model selection.