Imagine asking your phone:
“Call ”
“Turn on the flashlight”
…and it obeys instantly — without sending your voice to a server across the world.
No internet.
No cloud GPU.
No latency.
Just pure, on-device intelligence.
That’s exactly what I built using Google’s FunctionGemma and a modified version of the Mobile Edge Gallery app. In this article, I’ll show how a regular Android phone can become an autonomous, offline AI agent using Edge AI.
The Problem: AI Is Usually “Heavy”
Most AI assistants today live in the cloud.
When you ask them to do something:
Your data leaves the deviceIt’s processed on massive server farmsThe response comes back
This introduces three fundamental problems:
Latency — Cloud round trips are slowPrivacy — Your voice and intent leave your deviceDependency — No internet = no intelligence
That’s not intelligence — that’s outsourcing thinking.
The Solution: Tiny, Mighty, and Fully Local
Instead of moving data to the brain, I moved the brain to the phone.
Here’s the exact recipe.
1. The Brain: FunctionGemma 270M (Fine-Tuned by Me)
I started with FunctionGemma, a specialized variant of Google’s Gemma models designed not just to talk, but to call functions.
Why FunctionGemma?
Because I didn’t want poetic responses — I wanted actions.
When a user says:
“I need to take a picture”
The model shouldn’t explain photography — it should output:
open_camera()
My Fine-Tuning Process
I fine-tuned the 270M parameter version (yes, tiny)Training data focused entirely on Mobile ActionsUsed Google’s official Colab notebook for function tuning
👉 Fine-tuning notebook
The Result
A lightweight LLM that understands intent → action, not intent → text.
📦 Download the fine-tuned model
👉 FunctionGemma 270M Mobile Actions (LiteRT)
2. The Translator: LiteRT (TensorFlow Lite Runtime)
Raw models are too slow and too heavy for mobile devices.
So I converted the fine-tuned model into LiteRT (.litertlm) format.
Why LiteRT?
Optimized for mobile CPUsNo GPU or NPU requiredRuns smoothly on most modern Android phonesNo overheating, no battery drain panic
This makes true offline AI practical, not theoretical.
3. The Body: Modified Mobile Edge Gallery App
Intelligence without action is useless.
So I took Google’s Mobile Edge Gallery app and slightly modified it to support custom mobile actions.
Accessibility Service (The Secret Sauce)
I added a custom Android Accessibility Service — a privileged background service that can:
Observe UI stateSimulate gesturesTrigger system APIs
The Execution Loop
Here’s what happens in real time:
User taps the mic and says
“Turn on the flashlight”Edge AI processes the command locallyModel outputsturnOnFlashlight()App parses the function callAccessibility Service triggers the Torch APIFlashlight turns ON
All of this happens in milliseconds — completely offline.
How to Try It Yourself
Want to experience real Edge AI?
Step 1: Download the Model
👉 FunctionGemma 270M LiteRT Model
Step 2: Install the Modified App
👉 Download Modified Mobile Edge Gallery APK
Step 3: Setup
Open the app and load the downloaded modelGo to Settings → AccessibilityEnable Mobile Actions ServiceGrant required permissions:OverlayRead ContactsPhone access
Step 4: Magic ✨
Tap the floating red mic and command your phone.
Why This Matters (Beyond a Demo)
This isn’t just a fun experiment — it’s a preview of the future.
Privacy-First Computing
Your voice, intent, and actions never leave your device.
Zero-Dependency Intelligence
Works:
In tunnelsOn flightsIn remote locationsWithout SIM or Wi-Fi
♿Accessibility Superpowers
Voice-controlled, intent-aware UI can radically improve device access for users with motor impairments — far beyond rigid command systems.
Final Thoughts
Edge AI isn’t coming.
It’s already here.
It’s fast.
It’s private.
And it fits in your pocket.
The future won’t be cloud-only — it’ll be local, intelligent, and autonomous.
And this is just the beginning.
🚀 How I Built an Offline AI Assistant That Controls Android Phone. was originally published in Coinmonks on Medium, where people are continuing the conversation by highlighting and responding to this story.