No dia 16 de novembro, às 14:00, o DEI recebe Bruno Magalhães, Principal Researcher na Huawei Research Zurich, como convidado das DEI Talks. Na sala I-105, numa sessão aberta a todos, apresentará “Towards efficient AI models: MegaKernels for distributed training and efficient inference of large AI models”.
Sobre a DEI Talk:
“The scale of today’s AI models, with billions of parameters, turns their training and inference into distributed computing problems, bound mainly by data movement between memory, compute units and the network. Training needs compute large storage to hold billions of training parameters, and large compute power (FLOPs) to train the model on massive datasets in a feasible time. Inference requires low runtime for a positive user experience, and low running costs (“dollar per FLOP”). This talk will cover it.
For distributed training, the talk presents the main parallelization methods and the trade-off each one makes between memory per GPU and communication volume, given the network topology: data parallelism with sharding of the model state (ZeRO, FSDP), pipeline parallelism (1F1B, Zero Bubble, DualPipe), tensor parallelism (Megatron-LM), sequence and context parallelism (Ulysses, Ring Attention) and expert parallelism.
For efficient inference, we’ll discuss the roofline model that separates the compute-bound prefill from the memory-bandwidth-bound decode. Then techniques such as KV caching, PagedAttention, continuous batching, Grouped-Query Attention, Multi-head Latent Attention.
We will then introduce the concept of MegaKernels, adopted by systems such as DeepSeek to maximize performance. These kernels fuse computation and inter-GPU communication into a single kernel, reduce memory bandwidth, remove kernel launch runtime, and allows for computation/communication overlap. We will detail real implementations on the use cases of Tensor Parallelism, Expert Sharding and Expert parallelism.
Finally we will show that network latency (bandwidth) is critical to large model inference (training), and demonstrate how distributed model performance can be further optimized by programable switches that perform in-network computing of AI-related operations. We briefly introduce a performance modeling technique that extrapolates model performance onto new (unseen) network environment.
The methods are illustrated on Transformers, Mixture-of-Experts (MoE) models, diffusion and agentic models.”
Sobre o Orador:
“Bruno Magalhães is a principal AI Systems and Networks at Huawei Research, focusing on the efficiency for AI MegaKernels. Before he was a researcher at Microsoft Research where he worked on ML models for storage and reading of data in glass (Project Silica). Bruno holds a PhD from EPFL in Computational Neuroscience where he focused on (distributed) variable timestep interpolation of the electrical activity of the brain.”
