ACM, the Affiliation for Computing Equipment, at present named Torsten Hoefler, a professor at ETH Zurich, the recipient of the ACM Prize in Computing for elementary contributions to high-performance computing and the continued AI revolution.
Hoefler, who was a visitor on the @HPCpodcast on the positioning of this publication’s sister outlet, insideHPC, final November, developed most of the core capabilities of contemporary supercomputers and outlined key features of the algorithms for distributing AI fashions on them, the ACM stated.
The ACM Prize in Computing acknowledges early-to-mid-career laptop scientists whose analysis contributions have elementary influence and broad implications. The award carries a prize of $250,000, from an endowment supplied by Infosys Ltd., a digital companies and consulting firm.
ACM cited the work of Hoefler and his colleagues to scale community design and programming in supercomputers, which has revolutionized the capabilities of those
massive programs. For instance, AI algorithms can now be processed on lots of of 1000’s of nodes. Hoefler’s advances in interconnection networks, programming, and parallel algorithms broke new floor in facilitating the usage of large-scale massively parallel clusters.
“His quite a few improvements throughout the entire supercomputer stack—together with key contributions reminiscent of MPI-3 nonblocking collective operations, foundational parallelism methods for AI fashions, and high-performance networking programs—have pushed the boundaries of parallel programs design and translated into dramatic enhancements in supercomputer efficiency and scalability,” the ACM stated. “A lot of these improvements are included into the most important and strongest machines at present.”
The ACM cited these examples of Hoefler’s HPC-AI R&D:
Message Passing Interface 3: Hoefler performed a serious function within the evolution of the Message Passing Interface (MPI), a casual trade commonplace for exchanging messages between quite a few particular person nodes all through an HPC community. A messaging commonplace permits synchronization of the actions of every particular person laptop, sharing knowledge between nodes, and route and management of your complete parallel community.
The MPI-3 commonplace was adopted in 2012 and made attainable advances in HPC for simulations and AI functions over the previous a number of years. Hoefler chaired each the “Course of Topologies” and “Collective Operations” working teams for MPI-3. His nonblocking collective operations, reminiscent of Allreduce, Allgather, Bcast, and their respective blocking variations are included in numerous collective communication libraries—even past MPI-3. These operations energy the core of distributed deep studying at present.
3D Parallelism: Hoefler was among the many first to develop and uncover the now well-known notion of “3D parallelism,” which drives infrastructure design for the AI trade. Subsequently, he and his collaborators continued to develop methods for environment friendly pipelining, sparse communication, mannequin sparsity and quantization. This work has
enabled a cumulative 10-1000x acceleration of AI workloads in trendy computer systems.
Routing Protocols and Community Topologies: The low-level community routing protocols and community topologies that Hoefler and his colleagues developed for such networks as Myrinet and InfiniBand energy 1000’s of AI and HPC supercomputers. These contributions are central to high-performance AI programs that practice large-language fashions, reminiscent of ChatGPT.
“The capability of high-performance computer systems has develop into mind-boggling,” stated ACM President Yannis Ioannidis. “In simply the final two years, we now have ushered within the period of
exascale computer systems, which might carry out a billion billion calculations per second. However high-performance computer systems may do little with out the underlying algorithms and requirements that enable them to course of large influxes of knowledge.
“Whereas Hoefler launched many of those improvements whereas he was nonetheless a pupil, they continue to be the definitive method to program massively parallel programs at present,” Ioannidis continued. “The ensuing capabilities of those programs have allowed important advances in AI, the pure sciences, and lots of different areas.”
Salil Parekh, CEO, Infosys, stated, “We’re in the course of an thrilling period of AI, with nice promise for the longer term. Torsten Hoefler performed an necessary function in advancing high-performance computing, which in flip, fuelled the present AI revolution. The ACM Prize in Computing is meant to acknowledge to ‘early to mid-career professionals,’ whose work has depth and influence. With a lot of his necessary work being carried out in his 20s, Hoefler is an instance for younger people who age isn’t an impediment to achievement in computing. Infosys is proud to be the sponsor of this award because it was based in 2007.”
Hoefler is a Professor of Pc Science at ETH Zurich (the Swiss Federal Institute of Know-how), the place he serves as Director of the Scalable Parallel Computing
Laboratory. He’s additionally the Chief Architect for AI and Machine Studying on the Swiss Nationwide Supercomputing Centre (CSCS). Hoefler obtained a Diplom Informatik (Grasp of Pc Science) from Chemnitz College of Know-how and a PhD in Pc Science from Indiana College.
Hoefler’s honors embody the Max Planck-Humboldt Medal, an award for excellent mid- profession scientists; the IEEE CS Sidney Fernbach Award, which acknowledges excellent contributions within the software of high-performance computer systems; and the ACM Gordon Bell Prize, which acknowledges excellent achievement in high-performance computing. He’s a member of the European Academy of Sciences (Academia Europaea), a Fellow of IEEE, and a Fellow of ACM.
Hoefler will likely be formally offered with the ACM Prize in Computing at ACM’s annual awards banquet on Saturday, June 14 at The Palace Resort in San Francisco.
