Generative AI on Mobile:

How Arm CPUs Are Powering the Next Breakthrough

 

 

Explore how Arm CPUs are powering generative AI directly on mobile devices, enabling faster, private, and scalable AI experiences at the edge.

 

 

Generative AI Is Now on Mobile

Generative AI is no longer confined to cloud computing. Thanks to recent advancements, it now runs directly on mobile devices. This shift is powered by Arm CPUs, making AI experiences faster, more efficient, and more private.

Today’s smartphones can generate text, interpret language, and even create images and videos all locally without relying on the cloud. This is a game-changer for mobile AI.

Arm-Powered Smartphones Are Leading the Way

The latest flagship smartphones like the Samsung Galaxy S24, Google Pixel 8, and Vivo X100 Pro are equipped with Arm’s v9 CPU and GPU architecture. These high-performance devices are paving the way for real-time AI innovation at the edge.

Arm’s processor performance has doubled AI capacity roughly every two years, making mobile devices more capable than ever. With Arm’s future roadmap packed with new features and AI enhancements, this growth will only continue.

Why the CPU Matters for AI Workloads

AI operations often begin with the CPU. Tasks such as face recognition, body tracking, and camera filters are primarily handled by the CPU before involving accelerators like GPUs or NPUs.

Since most system-on-chips (SoCs) in today’s smartphones use Arm CPUs, this technology plays a central role in making AI possible for billions of users worldwide.

In fact, around 70% of AI workloads on mobile apps run on Arm CPUs. The combination of widespread adoption and unmatched flexibility makes Arm CPUs the go-to platform for mobile AI development.

Empowering Developers with Scalable AI Support

Arm CPUs support a variety of neural networks and data formats. The upcoming Armv9-A architecture, featuring the Scalable Matrix Extension (SME), will add even more AI functionality at the instruction set level.

To help developers harness this power, Arm is introducing Kleidi Libraries, which integrate directly with AI frameworks. These libraries will allow developers to build and deploy AI apps more easily while unlocking the full capabilities of the Arm CPU.

Running LLMs Like Llama and Phi on Mobile

At Mobile World Congress (MWC) 2024, Arm showcased a demo running Meta’s Llama2-7B language model on mobile devices using its CPU. Since then, engineers have added support for Meta’s Llama3 and Microsoft’s Phi-3 3.8B model.

These models are more powerful, efficient, and responsive. Arm’s AI-optimized software stack enables generative text at nearly 15 tokens per second, faster than average reading speed, all without accelerators.

The latest Phi-3 demo, using a chatbot named “Ada,” runs entirely on Arm CPUs. Ada serves as a virtual assistant for teaching science and coding an example of true edge AI.

Benefits of Edge AI on Arm CPUs

Running generative AI at the edge offers significant advantages:

• Speed: Lower latency with immediate responses

• Privacy: User data stays on the device

• Cost-efficiency: Less dependency on cloud resources

• Scalability: Easier deployment across billions of smartphones

The Arm AI software libraries further optimize performance and make it easier for developers to experiment with smaller, faster models. These libraries support fine-tuning, compression, and quantization for lightweight LLMs.

Open-Source Community: Accelerating Innovation

The open-source developer community is playing a major role in expanding AI on Arm. Within just 48 hours, developers got Llama3 and Phi-3 up and running on Arm CPUs. This shows the power of open collaboration in pushing the boundaries of edge AI.

Arm is actively encouraging and supporting this innovation, believing that open ecosystems fuel growth.

Looking Ahead: AI Everywhere on Arm

Arm’s mission is clear: to enable the most efficient generative AI experiences at the edge, with CPUs at the core of innovation. As models become smaller and more efficient, and as mobile devices grow more powerful, generative AI will become truly ubiquitous.

Through software innovation, AI-first hardware, and a robust developer ecosystem, Arm is building a future where AI is not only cloud-powered but also personal, private, and real-time right on your phone.

Explore how Arm CPUs are powering generative AI directly on mobile devices, enabling faster, private, and scalable AI experiences at the edge.

#ArmAI, #GenerativeAI, #MobileAI, #EdgeAI, #Phi3, #Llama3, #AIonArm, #SmartphoneAI, #MobileInnovation, #LLMonMobile,