Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP

Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP Published June 11, 2026 Update on GitHub Upvote 63 Aritra Roy Gosthipaty ariG23498 Rémi Ouazan Reboul ror Sergio Paniego sergiopaniego Pedro Cuenca pcuenq Sayak Paul sayakpaul This is the second post of Profiling in PyTorch, a series where we slowly build the skill of reading profiler traces and use it to drive optimization: Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP (current) Profiling in PyTorch (Part 3): Attention is all you profile In the first part of this series "Profiling in PyTorch" , we used torch.add(torch.matmul(x, w), b) to learn how to read PyTorch profiler traces. We also discussed several other topics that came our way - the CPU dispatch chain, launch overhead, the difference between an overhead-bound and a compute-bound regime, and some internals of torch.compile .

This ProductLaunch is relevant to the technology intelligence record because it involves GitHub, NVIDIA, Hugging Face, Meta. The source article should remain the factual reference for follow-up coverage.

• Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP Published June 11, 2026 Update on GitHub Upvote 63 Aritra Roy Gosthipaty ariG23498 Rémi Ouazan Reboul ror Sergio Paniego sergiopaniego Pedro Cuenca pcuenq Sayak Paul sayakpaul This is the second post of Profiling in PyTorch, a series where we slowly build the skill of reading profiler traces and use it to drive optimization: Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP (current) Profiling in PyTorch (Part 3): Attention is all you profile In the first part of this series "Profiling in PyTorch" , we used torch.add(torch.matmul(x, w), b) to learn how to read PyTorch profiler traces.
• We also discussed several other topics that came our way - the CPU dispatch chain, launch overhead, the difference between an overhead-bound and a compute-bound regime, and some internals of torch.compile .
• In the second iteration (this blog post), we climb one rung up the ladder.
• We replace the hand-written matmul-add pair with an nn.Linear (with bias=True ).
• This is the building block every deep learning model uses.
• We then stack three of them (specific to our example), with an activation in between, to form a Multilayer Perceptron (MLP) block.