Retrospective Magnetic Resonance Image Harmonization for Robust Federated Learning

Project Summary Deep learning algorithms have revolutionized the field of medical image analysis with their ability to automatically extract information without the need for handcrafted features. These algorithms have been shown to be particularly effective for image segmentation (identifying regions of interest) in well-curated datasets. However, deep learning can perform suboptimally when applied to input data acquired differently from the training data. This issue of “out of distribution data” or “domain shift” hinders accurate and robust performance when applied to magnetic resonance (MR) images given the diversity of both subject populations and MR acquisition protocols. The appearance of MR images is notoriously sensitive to both hardware differences and software differences and as a result, routine use of deep learning has proven to be extremely challenging in clinical neuroimaging. In this project, we propose to surmount the challenges of domain shift in medical images by combining two approaches: federated learning (FL) and harmonization. To focus our work on this problem, we will specifically work with MR data from people with multiple sclerosis (MS), an inflammatory disease of the central nervous system and a leading cause of neurological disability in younger adults. Because reduction or stabilization of white matter lesion burden is one of the goals of MS therapy, there is strong interest in establishing accurate and reliable white matter lesion quantification from MR images. Machine learning would seem to offer a viable solution, but despite decades of research, there is no method for automated lesion segmentation that can be robustly applied across data from different research and clinical sites. One approach is to expand the amount of training using FL, which builds a deep learning model across multiple institutions without requiring sharing of patient data. This bypasses the legal and regulatory restrictions associated with sharing patient data, while enabling training with a much richer and more diversified dataset. The advantage of working with larger training data sets, however, brings the challenge of dealing not only with heterogeneous MR acquisitions but also inconsistent labeling protocols from different sites. Our team has recently developed state-of-the-art retrospective image harmonization techniques that mitigate variations in structural MRI contrast, spatial resolution, and even artifacts. We will perform three aims: 1) Augment and optimize image harmonization algorithms using FL; 2) Develop and validate an integrated approach to FL for harmonized lesion segmentation; 3) Implement and distribute an FL platform for MS lesion segmentation. Harmonization provides a task agnostic solution to addressing domain shift in MR imaging and will maximize the utility of training data that may be discrepant with respect to both acquisition properties and labeling protocols. The proposed FL framework will facilitate collaboration across institutions to provide a novel, vastly improved framework for shared image analysis algorithm development, leading to more robust measurements and more reproducible science.