Why I Picked Chaquopy Over Linux Rootfs for On-Device Agents

An on-device agent should operate the phone, not just chat. Yet many runtime designs import desktop habits by shoving an entire Linux root filesystem onto Android.

In Zafiro, I took a path tailored to mobile hardware constraints: an embedded Python runtime via Chaquopy for routine decisions, while offloading deep terminal workloads to Termux.

Skipping gigabyte-scale rootfs downloads delivers instant execution. More importantly, this lightweight runtime evolved a core capability for on-device agents: synthesizing their own tools on the fly.

Mobile devices are not pocket desktops

Many on-device agents attempt to turn phones into portable workstations. To provide apt package management, compilers, and a full shell like a desktop, they package Alpine, Debian, or Ubuntu images.

The initial installer looks light at tens of megabytes. But launching the terminal triggers a blocking network download for a full Linux root filesystem. The archive easily exceeds a gigabyte, extracting into 2 to 3 GB of storage while users stare at a progress bar, worried the system will kill the app if they switch away.

On non-rooted devices, these runtimes rely on PRoot to intercept system calls via ptrace and emulate a Linux filesystem in user space. While this sandbox keeps the host system safe, intercepting every syscall creates constant CPU overhead and inflates resident memory. Even with chroot on rooted devices, running background system daemons, package managers, and shared libraries consumes hundreds of megabytes of idle RAM.

Android's memory management creates a bigger problem. A phone is not a wall-powered desktop with tens of gigabytes of RAM and unlimited swap. Android's Low Memory Killer watches physical memory levels constantly. When memory runs tight, high-memory background processes get killed without warning. A heavy environment running a full rootfs or VM struggles to stay alive in the background: switch apps or turn off the screen, and the system terminates the process, forcing another slow cold start next time.

On-device agents should finish the job and get out, not keep a heavy VM running in the background competing for RAM. For deep development tasks like compiling native C code locally, Android already has Termux. You can connect to it over a local SSH session, so the agent app never needs to carry a full Linux distribution inside its own process just to support an edge case.

Why Python for tool execution

After ruling out heavy Linux images, choosing an embedded scripting language came down to three pragmatic factors.

First, LLMs generate Python with significantly higher zero-shot accuracy than alternative scripting languages. Massive pretraining code corpora give models strong priors for standard data structures, regex patterns, and API calls, yielding higher Pass@1 rates on the first turn.

Second, the Python standard library covers the gaps in Android's stripped-down command line. Stock Android shell lacks even curl. Without extra binaries, an agent running shell commands cannot even dispatch a standard HTTP request. In Python, network requests, text parsing, and file I/O work out of the box.

Third, in-memory data pruning for screen control. Android Accessibility dumps massive view hierarchies full of empty containers, non-interactive nodes, and redundant nesting. Feeding raw dumps directly into prompts exhausts the context window within a few turns. Running Chaquopy inside the host process lets the agent inspect UI trees in place, stripping phantom nodes with list comprehensions and regex before returning a clean snapshot to the model. In-memory pruning is far more reliable and expressive than piping awk and sed in a raw shell.

Chaquopy brings Zafiro's APK size to roughly 29 MB. For a native Android application, 29 MB is not tiny. But paying that fixed binary footprint guarantees instant startup and reliable code generation, avoiding multi-gigabyte rootfs downloads entirely.

From rigid scripts to stepping stones

Running self-synthesized tools in Python seems obvious in hindsight, but the design did not start that way.

Early in Zafiro, custom tools were primitive: a simple module that ran user- or agent-defined shell scripts.

That early shell tool had one fatal limitation: it could not take arguments. Users or agents could only run hardcoded commands, like an intent to launch a specific Activity. The moment a task required dynamic input, raw shell syntax became clumsy and unsafe. Stock Android also lacks standard command-line utilities, offering no clean way to validate arguments, catch errors, or return structured output.

That argument limitation had to go. Unlike raw shell scripts, Python comes with typed function signatures. I experimented with letting the agent define Python functions on the fly, with the app extracting parameters automatically using reflection. Once released, users started creating their own custom Python tools right on their phones.

This reminded me of Kenneth Stanley's stepping stone theory in Why Greatness Cannot Be Planned: most valuable breakthroughs never follow a straight, planned path. Without that crude shell executor as a stepping stone, I would not have felt the real pain points of on-device tool calls, and I never would have built the reflective Python runtime.

Reflective meta-tools

Standard agent frameworks hardcode tools at build time: checking weather, reading calendars, setting alarms. But real-world needs span infinite long-tail workflows: scraping a specific webpage, downloading files in niche formats, or cleaning raw exports from a local photo gallery. Developers cannot anticipate every scenario up front.

Meta-tools solve this by letting the agent synthesize new tools on the fly. When facing a workflow without a built-in tool, the agent drafts a Python script, tests and validates it locally, and hot-registers it into the active session schema so subsequent turns can invoke it directly.

The main obstacle was parameter schema declaration. Forcing a model to output both Python code and a matching JSON schema wastes tokens and invites parsing mismatches whenever brackets or types misalign.

Instead, the runtime uses signature reflection. Every tool script must declare a typed main entrypoint. In CustomPyToolHarness.kt, the introspector loads the script in Chaquopy, extracts the docstring as the tool description, and maps parameter annotations directly into JSON schema primitives using inspect.signature.

This creates a closed-loop self-correction pipeline. If the generated script has syntax errors or untyped parameters, the introspector traps the error and returns a structured message with exact line numbers, letting the agent fix the script on the next turn without human intervention.

Living code and script self-repair

Meta-tools extend beyond one-off scripts into code that heals itself over time. Local web search illustrates this behavior in production.

Web search is essential for mobile assistants, yet commercial search APIs charge per request and enforce rate limits. As a local-first open source project, Zafiro avoids proxy servers. Instead, a seed script, py_web_search, bundles requests and BeautifulSoup to query public search engines directly from the phone at zero cost.

Search engine HTML layouts churn frequently, breaking static CSS selectors. In traditional client apps, broken parsers require releasing an application update. Because py_web_search is stored as an editable script managed by the meta-tool, the agent can inspect the broken source, fetch the new HTML layout, update its selectors, verify the extracted nodes, and overwrite the script on disk.

Moving from static compiled assets to living on-device logic gives autonomous agents a practical way to heal their own dependencies.

What 29 MB buys

Packaging a full Linux distro inside Android has its place for sandboxed tinkering on phones. But as models improve, the assumptions behind mobile agent runtimes are shifting.

LLMs do not need a virtual machine pretending to be a desktop. Given a high-level scripting runtime and lightweight scaffolding to access system APIs, a model can handle most tasks on its own: generating tools on the fly, cleaning data in memory, and fixing its own code when execution fails.

Adding 29 MB for Chaquopy was a deliberate engineering trade-off: zero post-install downloads with an instant Python environment, high Pass@1 code generation from strong LLM pretraining priors, a meta-tool system that lets the agent and user add typed tools dynamically, and zero-cost local search scripts that can heal their own selectors.

On resource-constrained mobile hardware, paying a small binary footprint for reliable execution is worth it.