Assembly language programming, especially using SIMD instructions, can outperform C code by orders of magnitude, offering significant performance improvements in video decoding and other real-time applications. Despite modern compiler optimizations, handwritten assembly remains crucial for maximizing hardware efficiency, supporting legacy systems, and enabling software to run on billions of devices with limited hardware resources.
In the world of software development, C is often considered the gold standard for performance. However, recent insights from experts involved in projects like FFmpeg and VLC reveal that assembly code can deliver performance boosts nearly 100 times faster than C code. This revelation challenges common assumptions about compiler optimizations and highlights the enduring value of low-level programming.
The creators behind FFmpeg and VLC have built companies and projects grounded in low-level, real-time programming. While most companies accept C as sufficiently fast, these experts demonstrate that C is not always the fastest option. By writing assembly code directly, they achieve speed improvements that are simply unattainable through C alone.
One of the challenges faced by these projects is supporting a vast array of operating systems, from Windows XP to Windows 11, Mac OS versions from 10.7 to the latest, iOS 9 through iOS 26, various Linux distributions, BSD, Solaris, and even OS/2. This broad compatibility is maintained by a small team leveraging the ethos of open-source low-level programming.
Supporting older systems like iOS 9 requires clever techniques, such as mixing multiple versions of Apple's Xcode IDE and SDK to create a Frankenstein build that still runs on 32-bit iOS 9 devices. This dedication allows users to continue using older hardware, reducing the need for constant upgrades.
Assembly language programming involves writing instructions that the processor executes directly. Unlike C, where a compiler translates code into machine instructions, assembly allows precise control over CPU registers and instructions.
A key technique used in FFmpeg is SIMD (Single Instruction Multiple Data). SIMD enables a single instruction to operate on multiple data points simultaneously. For example, instead of adding 5 to a single number, SIMD can add 5 to 16 different numbers in one instruction. This is particularly effective for video processing, where operations on pixel grids can be parallelized.
While some developers use intrinsics—C functions that map closely to assembly instructions—the FFmpeg team writes pure handwritten assembly without abstractions. This approach yields speed improvements of 10x to 50x, with some functions achieving 62x faster performance than their C counterparts.
David is a software decoder for the AV1 video format, developed by the Alliance for Open Media (including Google, Netflix, Amazon, and Mozilla). AV1 is a complex codec that many believed required hardware decoding for performance.
The David project defied this assumption by creating an extremely optimized software decoder:
This massive assembly codebase is likely one of the largest in existence and is used in VLC and other AV1 playback stacks. It enables decoding of 720p video on devices with just one or two CPU cores, making it possible to run AV1 video on billions of devices worldwide.
Despite advances in compiler technology, including auto vectorization, handwritten assembly still outperforms compiler-generated code by large margins. The FFmpeg team has demonstrated hundreds of examples where compilers fall short.
This challenges the common belief that modern compilers can generate optimal code. Instead, the team argues that to fully exploit hardware capabilities, especially as Moore's Law slows, developers must write and optimize assembly code manually.
With hardware speed improvements plateauing, software optimization becomes critical. The demand for CPU and GPU power is exploding, but hardware speed gains are limited. Adding more cores helps but is not a complete solution.
Assembly programming allows developers to use the machine in unexpected ways, sometimes repurposing instructions designed for other tasks (e.g., cryptography instructions used in video processing) to gain speed.
In the David project, the team even bypasses standard operating system calling conventions to optimize function calls within their library. This involves creating custom calling conventions to share data efficiently between functions, reducing overhead from saving and restoring CPU registers.
Handling multiple calling conventions across different operating systems (Linux 32-bit, Windows 32-bit, Windows 64-bit, Linux 64-bit) is complex. The team developed lightweight abstraction layers to manage this complexity, enabling assembly code to run efficiently across platforms.
Assembly programming is described as an art form, requiring deep understanding of CPU and computer architecture. While niche industries like high-frequency trading have long used such techniques, the FFmpeg and David projects bring this level of optimization to mass-market software running on billions of devices.
In an era dominated by high-level languages like Python and JavaScript, and even AI-assisted coding, why invest in assembly?
This philosophy is also reflected in emerging efforts like Kyber, which focuses on ultra-low latency applications where every millisecond counts.
The work of the FFmpeg and David teams reveals that assembly programming remains a vital skill for achieving extreme performance optimizations. As hardware improvements slow and demands for computational power grow, low-level programming techniques will be essential for software to run efficiently on a wide range of devices, from legacy systems to cutting-edge platforms.
Understanding and embracing assembly language can unlock significant performance gains, reduce hardware upgrade cycles, and enable software to meet the demands of modern applications such as video decoding and AI inference.
