Gradient Delay

SYNTHESIS: A semi-asynchronous path-integrated stochastic gradient method for distributed learning in computing clusters
To increase the training speed of distributed learning, recent years have witnessed a significant amount of interest in developing both …
Zhuqing Liu, Xin Zhang, Jia Liu
PDF Cite
Taming Convergence for Asynchronous Stochastic Gradient Descent with Unbounded Delay in Non-Convex Learning
Understanding the convergence performance of asynchronous stochastic gradient descent method (Async-SGD) has received increasing …
Xin Zhang, Jia Liu, Zhengyuan Zhu
PDF Cite