In recent years, the field of artificial intelligence has made massive strides, especially in the realm of large language models (LLMs) and vision-language models (VLMs). However, running these advanced models locally on a mobile phone in an offline manner is still a challenging task — one that many developers, researchers, and tech enthusiasts are actively trying to solve.
In this post, I’ll walk you through Google’s AI Edge Gallery, a promising tool that aims to bring powerful AI capabilities directly to your Android device without requiring an internet connection. But before we dive into the details, let me set the stage with a quick reality check.
The Current State of LLMs on Mobile Devices
Despite the hype around local LLMs on phones, the truth is that even the smallest models can be temperamental when run offline on mobile devices. Many crash unexpectedly or deliver subpar results due to hardware limitations like memory constraints and lack of GPU acceleration.
So, while it’s possible to run models locally on your phone, don’t expect flawless performance — especially not yet. This area is wide open for innovation, and if you’re thinking about starting a project or a startup in this space, there’s plenty of room to make a real impact.
Introducing Google’s AI Edge Gallery
Google recently released an experimental app called AI Edge Gallery, designed specifically for running machine learning models directly on Android devices. While it’s currently Android-only, Google has hinted at plans to release an iOS version in the future.
The app allows users to:
- Run Gemini and other Google models offline
- Engage in multi-turn chat conversations
- Analyze and question images
- Experiment with prompts
- Generate code
- Benchmark model performance
- Switch between Hugging Face models
- Upload custom lightweight TensorFlow Lite (TFLite) models
All of this is done entirely on-device, ensuring privacy and eliminating dependency on cloud services.
How to Install the APK: Quick Guide
Method 1: Direct Installation (Easiest)
- Open your file manager and locate the downloaded
.apkfile (e.g., in the Downloads folder). - Tap the file to start installation.
- If prompted, go to Settings > Security > Unknown Sources and enable it.
- Confirm and complete the installation.
Method 2: ADB Installation (For Developers)
- On your Android device, go to Settings > About phone and tap Build number 7 times to enable Developer Options.
- Go to Developer options and enable USB debugging.
- Connect your device to your computer via USB.
- On your computer, open a terminal or command prompt.
- Navigate to the folder containing the APK.
- Run:
adb install -t ai-edge-gallery.apk
The -t flag allows installation of APKs intended for newer Android versions.
Done! If you want to know more on how the app work, you can go to the GitHub file.
Hands-On Experience with AI Edge Gallery
Tested the app on a low-end Android device with limited RAM and no GPU. Here’s the observation:
- The app supports models like Gemma 3 and Qwen, but only in quantized versions (e.g., Q4), which affects performance.
- Model download sizes are relatively small, but loading times can be slow on older hardware — up to a minute in some cases.
- Response quality varies; Noticed significant hallucinations and nonsensical outputs during testing.
While the app is clearly not production-ready, it’s an exciting step forward from Google, offering a glimpse into the future of on-device AI.
Pros and Cons of AI Edge Gallery
| Pros | Cons |
|---|---|
| Official Google app (trustworthy) | Still in early development |
| Runs models completely offline | Frequent crashes on lower-end devices |
| Supports multiple models and custom TFLite models | Limited model options currently |
| Privacy-preserving and no internet required | Performance issues on older hardware |
Final Thoughts
Running large language models locally on mobile devices is a compelling idea — and Google’s AI Edge Gallery is one of the most promising tools available today. It may not be perfect, but it’s a great starting point for developers and enthusiasts looking to explore the potential of on-device AI.
As more models become optimized for edge devices and hardware continues to improve, we’re likely to see dramatic improvements in performance and usability.
Until then, patience is key — and experimentation is highly encouraged.