Researchers at the Department of Energy’s (DOE) Argonne National Laboratory rewrote part of the code for a genetic analysis to run on graphics processing units (GPUs). In its original version, this calculation would have required approximately 481,000 CPU core-hours, and it became feasible after this modification. The work focuses on genetic data collected from American veterans.
- Argonne researchers adapted a genetic analysis code to run on GPUs.
- This change made a calculation requiring about 481,000 CPU core-hours feasible.
- The study identified 13,672 genome regions linked to conditions across 635,969 veterans.
- The findings remain preclinical, but analysis time dropped from 14 days to under two hours.
More than a million veterans have contributed their DNA as part of the ‘Million Veteran Program,’ established by the Department of Veterans Affairs. Ravi Madduri, a senior scientist in Argonne’s Data Science and Learning division, who also holds a joint appointment in the Department of Human Genetics at the University of Chicago, describes this commitment solemnly: ‘Even after giving everything to their country, they found it in their hearts to give again, expecting nothing in return.’ For him, this generosity creates a debt: ‘When I look at this data, I have an obligation to keep the promise made to the veterans.’
A Code Problem Before a Machine Problem
A genome-wide association study (GWAS) seeks to determine if a genetic variant is associated with a given disease. The question is simple, but it multiplies by each variant tested and each pathology recorded in patient records, making it a huge mathematical problem. Madduri’s team first ran this calculation on the Summit system at the Oak Ridge Leadership Computing Facility, then on the Aurora system at the Argonne Leadership Computing Facility. On Summit, the task failed to complete on the CPUs: as the cohort grew, the matrix needed for each group no longer fit in memory, and the calculation stalled.
‘We decided to open the box and look at the code,’ explains Madduri. The problem turned out to be a programming flaw rather than a simple speed issue. The code, written for conventional CPUs like those used by the genetics community, spent almost all its time multiplying very large grids of numbers, an operation for which graphics processors are designed. Under the direction of Argonne’s Alex Rodriguez, the team rewrote this part for GPUs. The original tool, SAIGE, thus gave rise to a version named SAIGE-GPU, described in a separate paper published in the journal Bioinformatics.
Data That Cannot Travel
The results were published in the journal Science by the Million Veteran Program consortium, of which Madduri’s group is among the authors. The study covers 2,068 traits in 635,969 veterans and identified 13,672 regions of the genome linked to one or more conditions. About 1,600 of these regions appear only because nearly one-third of participants do not have European ancestry, whereas, according to the Science paper, most genetic studies conducted to date have relied primarily on people of European descent.
Madduri emphasizes the limits of these results: ‘All of this is preclinical. We are in the discovery phase.’ The analysis produces a statistical map indicating where to look, but turning one of these regions into a treatment requires years of additional work. He also highlights the time savings: operations that took 14 days at the time of the Science paper now require ‘less than one or two hours.’
For medical data, sharing remains an obstacle. Madduri notes that genetic data cannot be shared as freely as, for example, cosmology data. Federated learning proposes moving the model rather than the data: each institution trains the model on its own data, then sends back only the learned adjustments, which are combined into a common model. Madduri is one of the developers of APPFL, an Argonne Privacy-Preserving Federated Learning software framework. He adds, however, that the main difficulty remains authorization: each collaborator must work with their own institutional credentials, on a machine they are authorized to use. In his view, ‘a site that holds unique data makes the shared model more robust for everyone.’
