Distributed Interpretable Machine Learning on GPUs

Description

The research introduces the Bi-linear consensus Alternating Direction Method of Multipliers (Bi-cADMM) to address large-scale regularized Sparse Machine Learning (SML) problems over a network of computational nodes. These problems involve minimizing local convex loss functions over a global decision vector with an explicit $\ell_0$ norm constraint to ensure sparsity. This approach generalizes various sparse regression and classification models, including sparse linear and logistic regression, sparse softmax regression, and sparse support vector machines. Bi-cADMM reformulates the original non-convex SML problem using bi-linear consensus and employs a hierarchical decomposition strategy. This strategy splits the problem into smaller, parallel-computable sub-problems through a two-phase approach: initial sample decomposition and distribution of local datasets across nodes, followed by a delayed feature decomposition on GPUs available to each node. GPUs handle data-intensive computations, while CPUs manage less demanding tasks. The algorithm is implemented in an open-source Python package called Parallel Sparse Fitting Toolbox (PsFiT). Computational experiments validate the efficiency and scalability of Bi-cADMM through numerical benchmarks on various SML problems with distributed datasets.

Period	30 Jun 2024 → 4 Jul 2024
Event title	33rd European Conference on Operational Research
Event type	Conference
Location	Copenhagen, DenmarkShow on map
Degree of Recognition	International