Research

Inside neural accelerators

Each series studies one accelerator: its architecture, software interfaces and performance in practice. We explore new training and inference workloads and document our methods so others can check the results.

Apple M4 · Neural Engine

Inside the M4 Apple Neural Engine

  1. Part 1Architecture and program formatsA study of the Neural Engine's compilation and execution interfaces, and the binary format of a compiled program.
  2. Part 2BenchmarksThe chip peaks at 19 TFLOPS at 16-bit precision (Apple's 38 TOPS figure counts 8-bit operations). It slows down once data outgrows its 32 MB of on-chip memory, and CoreML makes small operations 2 to 4× slower.
  3. Part 3TrainingA 109M-parameter transformer trained from scratch, with the forward and backward passes on the chip at 2.8 W and the optimizer on the CPU. We later scaled it to Qwen3-0.6B.
  4. Part 4The complete machineWe mapped the hardware end to end, including the program format, task descriptors and lookup tables, and built programs from scratch that match Apple's compiler byte for byte.
  5. Part 4bInside the compilerWe traced Apple's compiler from graph to hardware binary. It runs 34 stages, covering graph recognition, lowering and scheduling.

Cited as the first public training on the Neural Engine by NYU Shanghai RITS and in the Orion paper. Code: github.com/maderix/ANE.

Next

Other AI accelerators

Coming soon