Excited to share our
@NeurIPSConf paper "Sketching for Distributed Deep Learning: A Sharper Analysis":
openreview.net/pdf?id=0G0VpMโฆ
We provide a significantly improved convergence analysis for sketching-based distributed learning frameworks by exploiting the properties of the deep learning losses, such as restricted strong smoothness. With that, we break the dimension dependence in the convergence error and communication cost -- showing the promise of sketching for larger models.
Unfortunately, I am not attending NeurIPS this year but visit our poster on Thursday 11-2 PST and ask Mayank any questions you may have.
w/ Mayank Shrivastava, Qiaobo Li,
@sanmikoyejo, Arindam Banerjee