| Title: | Distributed quantile regression for big data with missing covariates |
| Journal: | Statistics in Biosciences |
| Published: | 9 Jul 2026 |
| DOI: | https://doi.org/10.1007/s12561-026-09528-6 |
| URL: | https://link.springer.com/content/pdf/10.1007/s12561-026-09528-6.pdf |
Publication 20337
WARNING: the interactive features of this website use CSS3, which your browser does not support. To use the full features of this website, please update your browser.
Abstract
Data missing caused by non-response or drop-out appears routinely in modern medical studies. With the rapid development of information technology, medical data are becoming more and more massive in volume, and thus usually require distributed analysis in real-world applications, especially for cases where the data are collected from multiple centers or contain patient privacy. In this paper, we focus on developing efficient distributed algorithms to support quantile regression analysis in missing big data, which provides a powerful tool to model the skew and heterogeneous medical samples in reality, but currently remains a challenging issue. We employ the weighted quantile regression (WQR) technique to incorporate the missing information into the model and propose two two-stage distributed algorithms, IPW-ADMM and IPW-renewable, for efficient and privacy-preserving estimation of WQR. Both methods first estimate the missingness mechanism using a logistic model and then solve the WQR problem in a communication-efficient manner without sharing raw individual-level data. The IPW-ADMM algorithm parallelizes estimation using a multi-block alternating direction method of multipliers (ADMM), reformulating the nonsmooth WQR objective into a set of local subproblems. The IPW-renewable algorithm adopts a sequential renewable estimation framework with the smoothing technique, making it suitable for streaming or incremental data settings. Simulation studies demonstrate that both proposed methods achieve estimation accuracy comparable to the classical centralized interior point (IP) method, while offering substantial computational speedups in distributed environments. An application to the UK Biobank dataset further illustrates their practical utility in detecting heterogeneous genetic associations across quantiles under covariate missingness.</p>
5 Authors
- Ye Fan
- Hua Zhou
- Jin Zhou
- Tiejun Tong
- Nan Lin
1 Application
| Application ID | Title |
|---|---|
| 48152 | Developing statistical methods and computational algorithms for identifying biomarkers at Biobank-data scale for cardio-metabolic traits |
Enabling scientific discoveries that improve human health