News
NVIDIA outlined three cuTile kernels for Mamba-2 and said optimized forward-pass code ran 1.14x to 1.82x faster than a Triton reference.
NVIDIA said it has used its cuTile Python DSL to implement and optimize the forward pass of Mamba-2, a sequence model used by Jane Street engineers in an early adoption effort. The Oct. 19, 2026 post describes three custom kernels and reports that the optimized cuTile version ran 1.14x to 1.82x faster than an open source Triton implementation in the benchmark setup NVIDIA disclosed.
Evidence and context
According to NVIDIA’s Oct. 19, 2026 developer blog, cuTile is a Python-based DSL and programming model for NVIDIA GPUs that abstracts thread- and warp-level scheduling behind tile-centric primitives. The company said the Mamba-2 work was done with Jane Street, which the post described as a deeply technical global trading firm using deep learning in quantitative research.
The article frames Mamba-2 as an architecture intended to balance the training efficiency of attention models with the fixed-memory inference characteristics of recurrent networks. NVIDIA said the forward pass was split into three kernels: chunk_cumsum to compute intra-chunk cumulative sums, fwd_pass_state to carry hidden state across chunks, and compute_out to fuse output generation while keeping matrix operations Tensor Core-friendly.
In the post’s explanation, the kernels rely on cuTile primitives such as ct.bid, ct.load, ct.store, ct.cumsum, and ct.mma. NVIDIA also said the implementation uses log-space reformulations, causal masking, and exp2-based scaling to preserve numerical stability while keeping computation in tiled matrix form rather than a memory-heavy scan.
The company said additional performance tuning included latency hints for loads and stores, operand swapping for Blackwell fifth-generation Tensor Cores, a memory-layout change to improve access patterns, and an autotuning search over cuTile parameters. NVIDIA said these optimizations were integrated with TileGym and benchmarked against an open source Mamba-2 kernel written in Triton.
For the benchmark, NVIDIA listed a single NVIDIA B300 SXM6 system with Blackwell Ultra compute capability 10.3, a 1920 MHz max SM clock and a 1100 W power limit. The software stack named in the post included mamba_ssm 2.3.2.post1, Triton 3.7.1, cuTile 1.5.0, TileIR assembler 13.4, PyTorch 2.13.0, CUDA 13.3, Python 3.12.3 and Ubuntu 24.04.4 LTS. NVIDIA said the measured speedups were kernel-level only and that end-to-end training performance would also depend on the backward pass.
The Oct. 19 post follows recent cuTile Python releases that NVIDIA said added Windows ARM64 support in version 1.7.0, dependent launch and TileIR bytecode export in 1.6.0, and additional specialization and autotuning features in 1.5.0. Background documentation supplied with the material says cuTile is part of CUDA Tile, NVIDIA’s tile-oriented programming model, and that its compiler abstracts hardware details such as Tensor Cores and the Tensor Memory Accelerator.
NVIDIA said a follow-up post would cover the backward pass, but it did not give a date for that publication.
Be the first to comment