Research · neural processors
Systems Research
We study neural processors and share what we learn. Our research covers accelerator architecture, software interfaces and compiler behavior. We measure performance and explore new training and inference capabilities on everyday hardware.
- Hardware
- Apple Neural Engine (M4) today, with more neural processors to follow
- Depth
- Software interfaces, program formats and compiler design
- Results
- The first public training run on Apple's Neural Engine, and the first public decoding of its program format
- Cited by
- NYU Shanghai RITS and the Orion paper
Tool · rewrites model graphs
Rune
Speed up a model with a few short rules. Name a pattern and what replaces it. Rune rewrites every match in the model you exported and checks each edit before it lands.
- 1.54× faster on MediaPipe's BERT classifier (LiteRT), from one rule
- Works on ONNX, TFLite and torch.export
- Rejects any edit that fails its checks
Tool · compiles kernels
Blaze
Write the loops once. Compile for each processor. Blaze compiles kernels, the small programs that do a model's heavy math. You write the math as plain typed loops. The compiler plans the work once, and a backend for each processor turns that plan into native code.
- Backends for NVIDIA, Apple and AMD GPUs, and x86 and ARM CPUs
- An optional schedule sets tiling by hand; otherwise the compiler plans it