Most reanalyses fail not because the statistics are wrong but because the data were never genuinely comparable. Public repositories are a goldmine, yet the gap between "downloaded a matrix" and "a defensible integrated dataset" is where projects die. This post lays out a disciplined workflow for mining GEO, TCGA, GTEx, ENCODE, and ArrayExpress, then integrating the result across omics layers without fooling yourself.
Know what each repository actually contains
Treat repositories as different instruments, not interchangeable pipes. GEO is heterogeneous: curated series (GSE) with mixed platforms, probe annotations, and inconsistent sample metadata. ArrayExpress mirrors this role and is now folded into BioStudies, but older accessions remain. TCGA and GTEx are harmonised by design but differ in tissue handling, consent, and processing — TCGA is tumour-focused with matched normals, GTEx is post-mortem normal tissue. ENCODE provides assays (ChIP-seq, ATAC-seq, RNA-seq) with explicit biosample and pipeline provenance. The first rule: never merge across repositories until you have reconciled platform, genome build, and quantification unit.
Retrieval that survives contact with reality
- Prefer processed matrices for exploration but re-derive raw counts when the study is central to your claim.
- Record the exact accession, platform ID, genome build, and download date for every series — provenance is part of the result.
- Pull sample metadata programmatically (GEOquery, TCGAbiolinks, recount3) rather than transcribing phenotype tables by hand.
- For ENCODE, capture the pipeline version in the file manifest, because peak calls change between releases.
A reproducible retrieval script is worth more than any single dataset. If you cannot re-run the download and land on byte-identical matrices, your integration is unreproducible.
Cross-database batch correction
Batch effects across repositories are usually larger than the biology of interest. For microarray-style data, ComBat and its covariate-preserving variant remain the workhorses; for RNA-seq counts, model the batch as a fixed or random term inside DESeq2 or limma-voom rather than pre-transforming counts. When you must combine externally processed data with your own, quantify the problem first: PCA and hierarchical clustering of samples coloured by batch reveal whether correction is even viable. If batch and biology are fully confounded — one study is all cases, another all controls — no algorithm rescues you, and attempting correction manufactures false confidence.
Integration strategies: early, late, and intermediate
There is no universal fusion method, only fusions matched to a question.
| Strategy | Use when | Tooling |
|---|---|---|
| Concatenation (early) | Layers already normalised, same samples | PCA, PLS |
| Latent factors (intermediate) | Blocks with different noise/scale | MOFA, iCluster |
| Supervised fusion (late) | Outcome-driven, small n | DIABLO, sPLS-DA |
MOFA and DIABLO in practice
MOFA is an unsupervised factor analysis that decomposes variance per omics block, giving interpretable factors with weights and a variance-explained table. It shines when you want structure without a label — but factor count selection is a real decision, not a default, and factors must be validated against independent metadata. DIABLO is supervised and multi-block: it seeks correlated features across blocks that jointly discriminate classes, with the caveat that it will happily overfit small cohorts unless cross-validation is nested and honest. For both, harmonise samples by subject identifier, never by row order, and hold a truly untouched test set.
Metadata is the hidden dataset
Integration quality tracks metadata quality almost perfectly. Harmonise controlled vocabulary (disease stage, sex, age, tissue), resolve conflicting units, and explicitly flag inferred values. The same sample appearing as "normal" in one series and "adjacent" in another is a semantic batch effect that no correction tool sees. Build a sample dictionary before touching any matrix.
Validation and honest reporting
Never validate a signature on data that participated in its construction. Use held-out cohorts, external repositories, or orthogonal assays. Report negative results and confounded batches in the limitations, because reviewers — and later, yourself — will ask why the TCGA and GTEx merge behaved oddly. Multi-omics integration is powerful but unforgiving: the layers amplify both signal and every upstream flaw.
References
- Barrett T, Wilhite SE, Ledoux P, et al. NCBI GEO: archive for functional genomics data sets—update. Nucleic Acids Res. 2013;41(D1):D991–D995.
- Edgar R, Domrachev M, Lash AE. Gene Expression Omnibus: NCBI gene expression and hybridization array data repository. Nucleic Acids Res. 2002;30(1):207–210.
- Weinstein JN, Collisson EA, Mills GB, et al. The Cancer Genome Atlas Pan-Cancer analysis project. Nat Genet. 2013;45(10):1113–1120.
- GTEx Consortium. The Genotype-Tissue Expression (GTEx) project. Nat Genet. 2013;45(6):580–585.
- ENCODE Project Consortium. An integrated encyclopedia of DNA elements in the human genome. Nature. 2012;489(7414):57–74.
- Johnson WE, Li C, Rabinovic A. Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics. 2007;8(1):118–127.
- Argelaguet R, Velten B, Arnol D, et al. Multi-Omics Factor Analysis—a framework for unsupervised integration of multi-omics data sets. Mol Syst Biol. 2018;14(6):e8124.
- Singh A, Shannon CP, Gautier B, et al. DIABLO: an integrative approach for identifying key molecular drivers from multi-omics assays. Bioinformatics. 2019;35(17):3055–3062.
- Ritchie ME, Phipson B, Wu D, et al. limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Res. 2015;43(7):e47.