Niklas and another developer have created a fully OpenGL ES 3.0 compliant GPU driver for the M4 Mac Mini and MacBook Neo within approximately one month. This development process, which typically takes years, involved reverse engineering the AGX GPU's firmware ABI and user-space components. The driver has demonstrated sufficient performance to run Minecraft at 200 frames per second. The team utilized a clean room approach, avoiding any direct examination of Apple binaries, and relied on hardware traces and self-built shaders. They aim to make the driver available to end users soon, although it is not yet ready for public release.
The development process was divided into user and kernel space, reflecting the structure of modern GPU drivers. The kernel driver communicates with the GPU firmware, which operates on a custom RTOS called RTKit. The initial challenge was understanding the firmware ABI, which was complicated by Apple's design choices that split the kernel driver functionality between the AGX and the host system.
The developers faced several technical challenges, including capturing render work submitted after the GPU firmware started and issues related to compute workloads. They utilized a machine learning model, Codex, to assist in the reverse engineering process, which involved capturing GPU memory states and reconstructing objects in code. The team learned from their experiences, adjusting their strategies to improve efficiency in capturing and processing GPU workloads.
The user-space reverse engineering phase involved analyzing Metal programs to understand the GPU's operation. The team employed different strategies, with one developer focusing on hardware reverse engineering while the other prioritized building out the Mesa graphics library. The latter approach proved to be more efficient, leading to faster progress in the development of the driver.