Multi-omics experiments now routinely produce datasets where the number of features exceeds the number of samples by three or four orders of magnitude: 20,000 genes, 500 metabolites, 10,000 methylation probes, and a few hundred patients. This is the regime where machine learning either earns its keep or quietly memorizes noise. The difference between a publishable biomarker panel and an overfit curiosity is almost never the choice of algorithm — it is feature selection, validation discipline, and honest interpretation.
The p >> n problem
Classical statistics assumes p < n; omics violates it almost always. With p = 20,000 and n = 200, ordinary least squares has infinitely many perfect solutions, and even a naive random split will find features that separate the classes by chance. The entire analytical pipeline must be designed around this constraint. Two things matter: shrinking the feature space with a method that is itself regularized, and making sure any supervised step (including feature selection) happens inside the cross-validation loop.
Feature selection: LASSO and relatives
The LASSO adds an L1 penalty to the regression, min ‖y − Xβ‖² + λ‖β‖₁, which drives most coefficients exactly to zero and yields a sparse, interpretable panel (Tibshirani 1996). Its variants dominate omics: elastic net mixes L1 and L2 to handle correlated gene modules, group LASSO selects whole pathways, and adaptive LASSO improves asymptotic properties. A caution that reviewers know well: LASSO selection is unstable under collinearity — rerun with a different bootstrap and the selected genes change. Report a stability frequency (how often each feature is picked across resamples) rather than a single list.
Feature selection: random forests and importance
Random forests take a different route: build many decorrelated decision trees on bootstrap samples and average them (Breiman 2001). For omics, they handle non-linear interactions and correlated features gracefully, and the Gini or permutation importance ranking gives an immediately usable shortlist. But impurity-based importance is biased toward high-cardinality features, and permutation importance is misleading under collinearity. Boruta, which compares real features against shadow copies, and the conditional permutation scheme of Strobl et al. are better choices when correlated pathways are the object of interest.
Supervised learning beyond linear models
Once the feature count is manageable, gradient boosting and support vector machines are strong defaults, and elastic-net logistic regression remains an excellent, well-calibrated baseline that should always be reported. Deep learning rarely wins on tabular omics at typical sample sizes; it wins when integrating raw sequence or image modalities. Whatever the model, class imbalance (rare disease subtypes, few responders) must be handled with stratified sampling, class weights, or proper resampling — never by resampling before splitting.
Unsupervised learning: clustering and dimensionality reduction
Multi-omics integration is often about structure, not labels. PCA and its sparse variant select variance components; t-SNE and UMAP give visualization-only embeddings that must never be clustered for downstream statistics. For true multi-view integration, methods like MOFA and similarity network fusion decompose shared and modality-specific factors, while consensus clustering (Monti et al. 2003) provides stability assessment for subtype discovery. Silhouette and gap statistics are weak arbiters of cluster count; resampling-based stability is stronger.
Overfitting and cross-validation done right
This is where most analyses fail. The rules are simple and non-negotiable: (1) split train/test before any supervised preprocessing; (2) nest feature selection and hyperparameter tuning inside the training folds; (3) use stratified grouped CV when samples cluster by patient, batch, or site; (4) do not tune on the test set, ever. Report the whole pipeline as a model, not just the final numbers. A tempting but fatal shortcut is selecting features on the full dataset and then cross-validating only the classifier — this leaks and inflates accuracy by 10–30 points. When n is small, repeated nested CV with an outer loop for estimation and an inner loop for tuning is the honest choice (Varma & Simon 2006).
Interpretation with SHAP and friends
Accurate black boxes do not advance biology. SHAP assigns each feature a contribution to each prediction via Shapley values from cooperative game theory, with theoretical guarantees of local accuracy and consistency (Lundberg & Lee 2017). TreeSHAP makes it exact and fast for ensembles. Practically, SHAP gives per-patient explanations, global importance rankings that respect interactions, and dependence plots that reveal non-monotonic effects hidden by linear models. It is not magic: SHAP explains the model, not the biology, and correlated features share credit arbitrarily. Pair it with a mechanistic read on the selected genes and pathways.
A practical checklist
- Fix the split first; write it down and never change it.
- Put selection inside CV; use nested CV for small n.
- Report stability, not a single feature list.
- Always include an elastic-net baseline and a random forest.
- Validate on an external cohort whenever one exists.
- Interpret with SHAP plus pathway context, not one or the other.
Multi-omics machine learning is not hard because the algorithms are exotic. It is hard because discipline is required at every step from split to interpretation. Get the validation right and the biology follows; get it wrong and nothing else matters.
References
- Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Series B. 1996;58(1):267–288.
- Breiman L. Random forests. Mach Learn. 2001;45(1):5–32.
- Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Adv Neural Inf Process Syst. 2017;30:4765–4774.
- Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Series B. 2005;67(2):301–320.
- Strobl C, Boulesteix AL, Zeileis A, Hothorn T. Bias in random forest variable importance measures. BMC Bioinformatics. 2007;8:25.
- Kursa MB, Rudnicki WR. Feature selection with the Boruta package. J Stat Softw. 2010;36(11):1–13.
- Monti S, Tamayo P, Mesirov J, Golub T. Consensus clustering: a resampling-based method for class discovery and visualization of gene expression microarray data. Mach Learn. 2003;52(1):91–118.
- Argelaguet R, Velten B, Arnol D, et al. Multi-omics factor analysis — a framework for unsupervised integration of multi-omics data sets. Mol Syst Biol. 2018;14(6):e8124.
- Varma S, Simon R. Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics. 2006;7:91.
- Chen T, Guestrin C. XGBoost: a scalable tree boosting system. Proc 22nd ACM SIGKDD. 2016:785–794.