MethylGPT Unlocks DNA Secrets – Age & Disease Prediction

In this study, researchers created MethylGPT, a transformer-based foundation model for the DNA methylome. The EWAS Data Hub and Clockbase were used to collect data on 226,555 human DNA methylation profiles across diverse tissue types. Following deduplication and quality checking, 154,063 samples were saved for pretraining. The model focused on 49,156 CpG sites that had been previously associated with specific qualities in order to maximize their biological significance.

The model was pre-trained with two complimentary loss functions, masked language modeling (MLM) loss and profile reconstruction loss, which allowed it to reliably predict methylation at masked CpG sites. The model has a mean squared error (MSE) of 0.014 and a Pearson correlation coefficient of 0.929 between predicted and actual methylation levels, showing excellent predictive accuracy. Researchers also tested the model’s ability to capture biologically relevant characteristics of DNA methylation. As a result, they examined the learnt representations of CpG sites in the embedding space.

They discovered that CpG sites clustered according to their genomic contexts, indicating that the model learnt the methylome’s regulatory properties. Furthermore, there was a significant distinction between autosomes and sex chromosomes, showing that MethylGPT detected higher-order chromosomal characteristics. Next, the researchers looked at zero-shot embedding spaces. This revealed a clear biological organization, with grouping by sex, tissue type, and genetic context.

The major tissue types formed well-defined clusters, demonstrating that the model acquired tissue-specific methylation patterns without manual supervision. Notably, MethylGPT prevented batch effects, which frequently skew results in complicated datasets. Furthermore, the female and male samples showed continuous separation, indicating sex-specific differences. Next, the researchers evaluated MethylGPT’s ability to estimate chronological age based on methylation patterns. To accomplish this, they employed a dataset containing approximately 11,400 samples from various tissue types.

Fine-tuning for age prediction resulted in significant age-dependent clustering. Notably, intrinsic age-related organization was present even before fine-tuning. Furthermore, MethylGPT beat other age prediction algorithms (such as Horvath’s clock and ElasticNet), attaining higher accuracy. Its median absolute error for age prediction was 4.45 years, indicating its robustness. MethylGPT was also extremely tolerant to missing data. It performed consistently with up to 70% missing data, beating multi-layer perceptron and ElasticNet techniques.

Analysis of methylation patterns during induced pluripotent stem cell (iPSC) reprogramming revealed a distinct rejuvenation trajectory, with samples gradually transitioning to a younger methylation state during the course of reprogramming. The model was also able to pinpoint the moment during reprogramming (day 20) when cells began to exhibit clear signals of epigenetic age reversal. Finally, the model’s capacity to forecast disease risk was evaluated. The pre-trained algorithm was refined to estimate the probability of 60 diseases and deaths. The model had an area under the curve of 0.74 and 0.72 on the validation and test sets, respectively.

Furthermore, they employed this illness risk prediction methodology to assess the effects of eight treatments on anticipated disease incidence. Interventions included smoking cessation, high-intensity exercise, and the Mediterranean diet, among others, with different degrees of efficacy across illness types. This demonstrated unique intervention-specific effects across disease categories, underscoring MethylGPT’s potential for predicting intervention outcomes and enhancing personalized intervention tactics.

Conclusions

The results show that transformer designs may accurately mimic DNA methylation patterns while maintaining biological relevance. The grouping of CpG sites according to regulatory factors and genomic context indicates that the model grasped fundamental aspects without explicit supervision. MethylGPT also performed better in age prediction across different tissues. Furthermore, its strong ability to handle missing data (≤ 70%) highlights its potential use in clinical and scientific settings.

For more information:  MethylGPT: a foundation model for the DNA methylome, Kejun Ying, Jinyeop Song, Haotian Cui, Yikun Zhang, Siyuan Li, Xingyu Chen, Hanna Liu, Alec Eames, Daniel L McCartney, Riccardo E. Marioni, Jesse R. Poganik, Mahdi Moqri, Bo Wang, Vadim N. Gladyshev bioRxiv 2024.10.30.621013; doi: 10.1101/2024.10.30.621013, https://www.biorxiv.org/content/10.1101/2024.10.30.621013v2