Introducing a new distributed computing framework that is resilient to slow compute nodes and capable of both approximate and exact linear operations. This innovative approach combines randomized sketching and polar codes within the context of coded computation. We have developed a sequential decoding algorithm that efficiently handles real-valued data for recovery, while maintaining low computational complexity. Furthermore, we have created an anytime estimator that produces accurate estimates even when the available node outputs cannot be decoded. We showcase the diverse applications of this framework in areas such as large-scale matrix multiplication and black-box optimization. To validate its practical scalability, we have implemented these methods on a serverless cloud computing system and present numerical results, including computations at the scale of ImageNet.
Randomized Polar Codes for Distributed Machine Learning with Anytime Capability
