Personal infrastructure project

Fleet Runner — architecture & build plan

One set of rails that turns a shelf of iOS and Android devices into a programmable fleet: on-device ML benchmarks, Maestro UI-test runs of GreenFolio across real devices, nightly Jerv battery-drain tests, and batch inference — all dispatched from one collector, all landing in one results database.

01Goals and non-goals

Goals. Two families of workloads on shared infrastructure:

• On-device ML — a runner app per platform that loads models through pluggable backends, executes benchmark/batch/pipeline workloads, and measures performance honestly.
• Real-device app testing — install a build of GreenFolio, Aliquant, or Jerv on N physical devices and run UI-test suites, battery-drain runs, and overnight background-survival soaks against it.

Both families share the collector, the job/result protocol, the device registry, the scheduler, and the dashboard. The fleet is programmable: a new capability is a new workload type, not a new system.

Non-goals (v1). No training on device, no model conversion on device, no public multi-tenant anything, no UI beyond a status screen on the phone and a plain dashboard on the collector. Old devices that can't run LLMs still earn their place — they're the UI-test matrix, the OEM-task-killer soak targets, and the vision/audio workload pool.

02System overview

Four pieces: the runner app (on every device), the host executor (a worker on the Mac that drives devices from outside), the collector (one small service: queue, registry, scheduler, results), and the artifact store (models and app builds, served by hash). Everything rides your Tailscale network, so the fleet works whether devices are on your desk or plugged in around the house — with the one caveat that host-driven jobs need the device reachable by the executor (USB hub or Wi-Fi adb).

03Two execution paths — the key architectural split

Not every workload can run inside an app on the device. iOS will not let an app install another app or drive its UI, and a battery test of Jerv must run Jerv itself, not a proxy. So every job declares its executor:

• executor: "device" — the on-device runner app claims the job by polling and executes it internally. ML benchmarks, batch inference, pipeline nodes.
• executor: "host" — the host executor claims the job and drives target devices from outside: adb on Android, xcrun devicectl + XCUITest on iOS, Maestro for cross-platform flows. UI tests, app installs, battery-drain and soak tests.

04The runner app

Two thin native apps — Kotlin on Android, Swift on iOS — sharing a protocol, not code. The contract is the JSON job/result schema plus an internal interface both implementations mirror:

ModelBackend            // one per inference engine
  capabilities() -> [formats, accelerators, maxModelSize]
  load(artifact)  -> LoadedModel   (timed: load_ms)
  run(input)      -> Output        (timed per call)
  unload()

WorkloadEngine          // interprets device-executor job specs
  benchmark   : warmup N, measure M iterations, sustained-load option
  batch       : pull items from collector, run, push outputs
  pipeline    : subscribe to trigger topic, run on event, publish result

TelemetryBeacon         // always-on, also during host-driven jobs
  battery, thermal, memory pressure, process-alive checks

Reporter                // ships results + beacon stream, retries offline

• Android: a foreground service keeps the runner alive; backends are LiteRT (vision/audio, NNAPI or GPU delegate) and llama.cpp via JNI (LLMs, GGUF). Min SDK low enough to cover the old devices — the LiteRT backend reaches much further back than llama.cpp needs to.
• iOS: isIdleTimerDisabled while plugged in and a status screen; backends are Core ML (vision/audio, .mlmodelc) and llama.cpp via SPM (LLMs, GGUF). MLX is a possible third backend later for newer chips.
• Model artifacts are downloaded from the collector, verified by hash, and cached on device.

05Workload catalog

ML backends per platform: LLMs via llama.cpp (GGUF, Metal on iOS); vision/audio via Core ML (.mlmodelc) and LiteRT (.tflite); Whisper via whisper.cpp. Conversion happens on the Mac, never on device. The artifact store holds models and app builds alike: name, kind, format, sha256, min-RAM hint.

06Job and result protocol

The whole system hinges on two JSON documents. An ML benchmark job:

{
  "job_id": "bench-qwen3-0.6b-q4-2026-08-13",
  "workload": "benchmark", "executor": "device",
  "model": { "name": "qwen3-0.6b", "format": "gguf", "quant": "Q4_K_M",
             "sha256": "…" },
  "backend": "llama.cpp",
  "params": { "prompt_tokens": 512, "gen_tokens": 128,
              "warmup_iters": 2, "measure_iters": 5 },
  "targets": { "match": "ram_mb >= 4000" },
  "constraints": { "require_charging": true }
}

A UI-test job rides the same schema — different workload, different executor, an app build instead of a model:

{
  "job_id": "gf-maestro-smoke-pr412",
  "workload": "ui-test", "executor": "host",
  "app":  { "name": "greenfolio-android", "build": "1.4.0-pr412",
            "sha256": "…" },
  "suite": { "kind": "maestro", "flows": "flows/smoke/*.yaml" },
  "targets": { "pool": "android-ui", "exclusive": true },
  "report_to": { "github_status": "addisdev/greenfolio-android@abc123" }
}

Results are rows in one table: metrics for benchmarks (prefill/decode tok/s, ttft, peak memory + method, thermal samples, battery delta), pass/fail + screenshot/video artifact refs for UI tests, and timestamped beacon curves for drain and soak runs. Version the schema from day one ("schema": 1), and make result posting idempotent by (job_id, device_id, iter).

07Metric normalization — the honest-numbers problem

• Measure at the same boundaries. Model load is its own metric, never folded into latency. For LLMs, prefill and decode are reported separately — a single "tokens/sec" hides the number that matters.
• Memory is labeled, not laundered. iOS reports phys_footprint; Android reports PSS. They are not the same quantity, so every row carries mem_method and the dashboard never charts them on one axis.
• Thermals are a curve, not a state. Sample the platform thermal state every N seconds; map to a shared enum (nominal / fair / serious / critical). The sustained-load benchmark is the honest number for real workloads.
• Battery runs are opt-in and controlled. Benchmarks require charging by default; drain and soak runs start unplugged from a fixed percentage with screen state pinned, and the collector's power control makes that repeatable.
• Warmups are mandatory. First-run compilation (ANE/NNAPI) is reported as its own metric, then excluded.
• Location is replayed, not real. Drain tests feed the app a recorded GPX route — Android via a mock location provider, iOS via a debug-scheme ReplayLocationProvider baked into the app build — so every nightly run walks the same 5 km. The GPS radio stays on, so the power cost is real; only the coordinates are scripted.

08The collector

One small TypeScript service (Node + Fastify) with SQLite — no external dependencies, runs forever on the Mac mini under launchd. Endpoints:

POST /devices/register        device checks in with descriptor + pools/tags
GET  /devices/:id/next-job    long-poll for device-executor work
GET  /executor/next-job       long-poll for host-executor work
GET  /artifacts/:sha256       models and app builds (range requests)
POST /results                 result rows + beacon samples, idempotent
POST /jobs                    enqueue (curl, CI, or the scheduler)
GET  /dash                    server-rendered dashboard

• Scheduler: cron entries that enqueue job templates — 02:00 jerv-drain on pool:battery, 03:00 gf-maestro-full on pool:android-ui. Nightly runs are just scheduled enqueues; nothing special.
• Device locks and pools: devices carry tags (ml-capable, android-ui, battery, os-pinned-a11); exclusive jobs lock their targets so a UI test never lands mid-benchmark. Battery-pool devices are excluded from opportunistic ML work.
• Power control: the collector toggles smart plugs (or per-device charge-limit tooling) — unplug the battery pool before a drain run, restore charging to the 40–70% band after. This is also the day-to-day battery-health mechanism.
• CI integration: a GitHub Actions step in greenfolio/aliquant/jerv repos uploads the build artifact, POSTs a job over Tailscale, polls for the verdict, and the collector posts a commit status back. Real-device smoke on every PR to develop; the full matrix nightly.

The dashboard starts as one page: device grid (last seen, battery, thermal, current job), job history with pass/fail, per-model comparison table, and drain-curve charts. Resist building more until the data demands it.

09What each app runs on the fleet

09bEval delivered: GreenFolio on-device plant ID

The fleet's first product payload. GreenFolio identifies plants today by sending photos to the cloud Plant.id API; the question was whether species identification can run on-device — offline in a greenhouse, at zero per-call cost — and on what hardware. Full method and per-image reports live in fleet-collector/evals/greenfolio-plant-id.md.

Method. 120 held-out images (16 species) sampled from the PlantNet-300K test split, center-cropped and resized once on the Mac so every device classifies identical bytes; ImageNet normalization on-device. Candidates: the Apache-2.0 litert-community PlantNet-300K ResNet18 (1081 species), our own post-training int8 quantization of it (100 validation-split calibration images — disjoint from the eval — float I/O preserved), and a 47-class houseplants model as a latency comparator.

Android rows: ATD emulator, CPU, 4 threads. iOS rows: iPhone 16 simulator on CPU — the Simulator's emulated GPU/ANE silently returned all-zero logits for this model, so the iOS runner forces CPU on sims and labels it; real iPhones use the ANE. Emulator/simulator numbers validate the pipelines and the relative comparison; the SM-X930 rows are queued via fan-out and fill in when the tablet wakes. Host fp32 accuracy matched device fp32 exactly, and Core ML matches LiteRT within a point on identical images — both platforms' preprocessing is faithful to the same weights.

10Build plan

11Fleet care (do these early)

• Charge management: batteries held at 100% forever swell. Smart plugs cycling around 40–70% — which Phase 4 turns into a collector feature, so buy controllable plugs from the start.
• Pin old OS versions now: disable auto-update on every device you want as a compatibility target. An updated device never goes back. Tag them os-pinned-* in the registry.
• Label and register: physical label = device_id in the registry. Future-you will thank you.
• Tailscale on everything from day one; a powered USB hub on the Mac mini for the host-executor pool, Wi-Fi adb for devices living elsewhere.
• Developer mode + trust once, per device: enable Android developer options / iOS developer mode and pair with the Mac while the shelf is in front of you, not job by job.