Allreduce (or MPI) vs. Parameter server approaches

Allreduce (or MPI) vs. Parameter server approaches In the last 7 years or so there has been quite a bit of work on parallel machine learning approaches, enough that I felt like a summary might be helpful both for myself and others. In each case, I put in the earliest known citation. If I missed something please comment. One basic…
Original post: Allreduce (or MPI) vs. Parameter server approaches
Source: Machine Learning (Theory)