One weird trick for parallelizing convolutional neural networks

Code

• github.com/albarji/neurowriter
  tf★ 95
• github.com/gifale95/algonauts_2023
  pytorch★ 21
• github.com/rafaelmata357/Flower-image-classifier
  pytorch★ 0
• github.com/bekirufuk/CNN-pretrained-from-scratch-comparison
  none★ 0
• github.com/MisaOgura/flashtorch
  pytorch★ 0

Abstract

I present a new way to parallelize the training of convolutional neural networks across multiple GPUs. The method scales significantly better than all alternatives when applied to modern convolutional neural networks.

Reproductions