Will Deep Learning Scale to Supercomputers?

Mark O’Connor demonstrates Allinea Software Are supercomputers practical for Deep Learning applications? Over at the Allinea Blog, Mark O’Connor writes that a recent experiment with machine learning optimization on the Archer supercomputer shows that relatively simple models run at sufficiently large scale can readily outperform more complex but less scalable models. I needed high performance computing expertise to see that the performance was limited by workload imbalance between the nodes triggered by late senders. I combined that with domain expertise in playing Pong to reach the insight that source of the imbalance could be eliminated without harming the model’s performance (it’s better to have a few learners lose a reward signal than a lot of learners sitting idle). This dynamic between HPC expertise and domain expertise is central to both…

Link to Full Article: Will Deep Learning Scale to Supercomputers?

Pin It on Pinterest

Share This

Join Our Newsletter

Sign up to our mailing list to receive the latest news and updates about homeAI.info and the Informed.AI Network of AI related websites which includes Events.AI, Neurons.AI, Awards.AI, and Vocation.AI

You have Successfully Subscribed!