Looking to bring AI and machine learning workloads to Arm-based hardware, Arm is integrating its Arm Kleidi AI acceleration technology with PyTorch and ExecuTorch, the new on-device inference runtime from PyTorch.
The arrangement, announced September 16, is intended to extend AI performance benefits from the edge to the cloud. With support for PyTorch and ExecuTorch, the next generation of apps can run large language models (LLMs) on Arm CPUs. Arm has partnered with PyTorch and TensorFlow to integrate Arm Kleidi libraries directly into these frameworks. KleidiAI, a library for AI/ML framework developers, is also slated to be integrated into ExecuTorch in October 2024.
Since its launch four months ago, Kleidi has accelerated development and unlocked major AI performance uplifts on Arm CPUs, said Alex Spinelli, vice president of developer technology at Arm. In the cloud, Kleidi builds on existing work enhancing PyTorch with the Arm Compute Libraries (ACL) and offering a blueprint for optimizing AI on Arm. With Kleidi, applications can benefit from performance improvements when new framework versions are released, but without taking extra steps to build on Arm today.