Tensordyne is making AI compute more efficient with new math.

zach's tech blog

SubscribeSign in

Tensordyne is making AI compute more efficient using logarithms.<br>But is compute efficiency the thing that matters?

zach<br>Jun 17, 2026

Share

Most AI chip companies are focusing on speeding up generative AI workloads by changing the memory hierarchy. Groq achieved ultra-low-latency at a high cost by using super-fast but expensive SRAM memory, rather than the slower but more cost-effective high-bandwidth memory (HBM) that Nvidia relies on. Cerebras is using wafer-scale processing to take that idea even further, with each wafer providing 44GB of SRAM. d-Matrix is leveraging processing-in-memory to improve performance by reducing data movement to and from memory. Taalas is using read-only memory, which is even faster and cheaper than SRAM, but can’t be changed -- so each chip can only run one model.<br>Tensordyne is approaching AI chips differently. They’re using a different kind of number system based on logarithms that allows them to multiply numbers more efficiently than chips using conventional floating point math. But translating more efficient multipliers into better performance on real AI workloads is hard. Today on the blog, we’re looking at Tensordyne’s unique technology, how it positions their chips relative to others in the market, and trying to answer the question: do more efficient compute engines actually matter for AI chips?<br>Thanks for reading zach's tech blog! Subscribe for free to receive new posts and support my work.

Subscribe

Logarithmic multiplication

The core technical advantage of Tensordyne’s architecture comes from its logarithmic number format. Instead of storing a value A as just A , they actually store log2(A). Then, if they want to multiply A by another value, B , they can use the logarithm product rule to perform multiplication using addition:<br>log2(A x B) = log2(A) + log2(B)<br>This idea isn’t new, but nobody has successfully used it in a commercial AI accelerator before. Running large matrix multiplications requires a lot of multipliers, but it also requires a lot of adders. The log product rule makes multiplication easier, but it makes addition much harder. Tensordyne gets around this using an approximation:<br>log2(1+x) = x for 0<br>But this approximation isn’t enough by itself. Originally, Tensordyne used quantization-aware training to get models running despite this approximation, but convincing customers to re-train models just for Tensordyne chips would be a huge challenge. So they figured out a set of additional proprietary tricks to make their approximation accurate enough for large models.<br>Tensordyne’s software stack converts PyTorch models to run with their specific number format. I’m a bit skeptical about whether they can do this without meaningful accuracy degradation, but even if they can, this conversion process raises another issue. Modern optimized inference pipelines aren’t just written in PyTorch; they have custom kernels optimized for GPUs and their specific numeric quirks. If a customer wants to port that to a chip with a whole new number format, it’ll take a lot of work to match the level of performance of optimized GPU kernels. Tensordyne claims that they’ll have AI agents help customers convert their code to Tensordyne’s software and numerical format… which seems like a bit of a cop-out answer. Fundamentally, writing kernels is hard, and to reap the (very real) benefits of Tensordyne’s chips, customers may need to bite the bullet and port kernels to Tensordyne’s logarithmic number format.<br>This specialized number format also makes it much, much harder for Tensordyne to support training workloads. Inference workloads are relatively forgiving to numerical changes like quantization, but training workloads have major numerical stability concerns. Training pipelines are specifically designed around hardware quirks, and can cost hundreds of millions of dollars to run. It just doesn’t make sense to build a new training pipeline for Tensordyne’s chips for a workload like training, which is primarily dominated by networking and memory bandwidth anyways.<br>Speaking of memory bandwidth… while Tensordyne is touting their new number system, most AI chips are focused on achieving breakthrough performance using new memory hierarchies. Is Tensordyne focusing on the wrong bottleneck for generative AI workloads?<br>Thanks for reading zach's tech blog! This post is public so feel free to share it.

Share

Does more efficient compute matter?

There’s a reason why most AI chips have focused on innovating on the memory hierarchy, rather than fundamentally changing how the compute works: a lot of generative AI workloads are memory bound. If you want to use a single GPU to process a single LLM query, that GPU needs to load and unload all of the weights of every layer in the LLM to process that query. Even if you make the GPU much more efficient at matrix multiplication, you’re bottlenecked by the...