Something big just dropped in the AI world—and it’s not another Transformer variant. It’s something entirely different. Meet Hyena Edge, a new AI model architecture from Boston-based startup Liquid AI, spun out of MIT. Unveiled on April 25th just before ICLR 2025 in Singapore, Hyena Edge might just be the first real crack in the seemingly unshakable Transformer era.
An AI model designed to run efficiently on devices like smartphones. Unlike Transformer-based models, Hyena Edge replaces most attention mechanisms with gated convolutions (Hyena Y), offering faster performance, lower memory usage, and comparable or better accuracy.
Key Highlights:
- Faster & Leaner: Up to 30% faster on long sequences, runs natively on devices like the Samsung Galaxy S24 Ultra.
- Better Performance: Outperforms equivalent Transformer+ models in benchmarks (e.g., WikiText, HellaSwag, ARC).
- STAR Framework: Evolved using a Darwinian-inspired process that optimized architectures for speed and efficiency.
- Scalability: Convolution-based design scales linearly with input length, unlike attention’s quadratic scaling.
Why It Matters:
- On-Device AI: Enables powerful AI directly on phones without cloud reliance, preserving privacy and reducing latency.
- Open Source: Liquid AI plans to release Hyena Edge and other “Liquid Foundation” models, fostering innovation for mobiles, wearables, and embedded systems.
This could mark the beginning of a shift toward poly-architecture AI, where Transformers rule servers, and hybrids like Hyena Edge power local, efficient edge computing.
Real-World Performance
On benchmark tests, Hyena Edge didn’t just hold its own—it outperformed a parameter-matched GQA Transformer+ model:
| Benchmark | Transformer+ | Hyena Edge |
|---|---|---|
| WikiText Perplexity | 17.3 | 16.2 |
| LAMBADA | 10.8 | 9.4 |
| PIQA | 31.7 | 31.7 (tie) |
| Pile Accuracy | 71.1 | 72.3 |
| HellaSwag | 49.3 | 52.8 |
| WinoGrande | 51.4 | 54.8 |
| ARC Easy | 63.2 | 64.4 |
| ARC Challenge | 53.34 | 55.2 |