tools/gpx_replay.py: replay a GPX through the location pipeline (#23) #105

Merged

robert merged 1 commit from tooling/gpx-replay into main 2026-09-05 10:52:29 +02:00

Conversation 0 Commits 1 Files changed 11 +1390 -18

robert commented 2026-09-05 10:51:10 +02:00

Owner

Copy link

Closes #23.

Scope decision — read this first

#23's own listed file path is tools/gpx_replay.py, implying Python. Its actual acceptance criteria (read from the API, not guessed) are:

• Reads a GPX and emits positions at a configurable speed multiplier
• Can drive the Android emulator via mock locations
• Can drive the companion directly through a debug hook
• Supports pausing, jumping to an offset and injecting a deliberate off-route excursion
• Documented in docs/DEV.md

These are not about JVM-side pipeline verification (that concern is already covered by SpeedPipelineGpxReplayTest.kt/RealWorldFixtureTest.kt, which I did not touch or refactor — there is no shared-harness duplication to pay down here, because I did not build a second JVM replay harness). They are about driving the real, unmodified Android location stack from the outside: adb emu geo fix and a debug broadcast. Neither needs SpeedPipeline/StopDetector's algorithm reimplemented in Python — the real pipeline always runs in-process on the real device/emulator, unmodified. So: built the literal tools/gpx_replay.py Python path, not a Kotlin harness, because that's what these specific criteria actually call for once you separate "emit positions" from "process positions."

What's here

tools/gpx_replay.py (stdlib-only, uv run, same convention as tools/gen_message_keys.py):

• Parses raw <trkpt> sequences (same "raw points, not RDP-simplified routes" basis as SpeedPipelineGpxReplayTest.kt's own parseRawTrackPoints).
• --speed-multiplier paces emission by each point's real recorded <time> gap (falls back to NFR-B2's 1 Hz cadence for untimed GPX).
• --start-offset-seconds/--start-offset-index jump into a ride.
• --excursion-at-index/--excursion-offset-meters/--excursion-points splices a synthetic perpendicular-drift-and-return excursion into the point sequence, for testing off-route detection without a second recorded ride.
• Interactive p/q (pause/resume, quit) from stdin when it's a terminal.
• Three sinks: --sink print (stdout, default), --sink emulator (adb emu geo fix <lon> <lat> — AndroidLocationSource is untouched and can't tell the difference from real GPS), --sink debug-broadcast (adb shell am broadcast to the new debug hook below).
• tools/test_gpx_replay.py: 36 stdlib-unittest tests (Robert's xUnit/NUnit), including a cross-check of this script's GPX parsing against the same real ~119 km komoot fixture and ~119,020 m ground truth SpeedPipelineGpxReplayTest.kt/RealWorldFixtureTest.kt already established — proof this tool reads the identical point sequence, explicitly not a claim about SpeedPipeline itself (see the test's own docstring on why a haversine cross-check is not the kind of algorithm-duplication #23's own task brief was worried about).

The debug hook (:companion:location, :companion:ride):

• DebugBridgeLocationSource — a plain-Kotlin (no android.*) LocationSource you can inject() a fix into. Unit-tested for real (:companion:location gained a Kotest source set, same pattern as :companion:pebble's).
• parseDebugFixExtras — pure function parsing the broadcast's String extras into a LocationFix, unit-tested (14 tests: valid/missing/malformed/out-of-range/boundary lat-lon).
• DebugFixReceiver — thin BroadcastReceiver, dynamically registered by RideService.onCreate() only when ApplicationInfo.FLAG_DEBUGGABLE is set (never true for a release build; no BuildConfig.DEBUG since no module enables that build feature). Wired additively: debug-injected fixes merge alongside real GPS into the same SpeedPipeline, not a mode switch.

CI: slow-lane.yml's replay-regression and tag-lane.yml's replay-and-emulator-gates were both pre-existing placeholder jobs literally named after #23, deliberately left hard-failing with "update this step in the same PR that adds #23's harness." Both now run the tool's own test suite plus a --sink print pass over every real (non-adversarial) GPX fixture in the repo. tag-lane.yml still correctly hard-fails — now because G6's cue-position/leg-divergence/link-rate clauses and G7 (the no-network ride) need a cue engine/message scheduler that doesn't exist yet (#38 and friends), not because of anything #23 was responsible for. docs/CI.md updated to match.

docs/DEV.md: new "GPX replay tool" section with usage examples and an explicit manual checklist for the two things I could not verify live in this session (see below).

Honestly unverified (no device/emulator attached in this session)

• --sink emulator and --sink debug-broadcast against a real live target.
• Whether a plain, unprivileged adb shell am broadcast actually reaches a dynamically-registered RECEIVER_NOT_EXPORTED receiver on this project's target API levels — I chose the safe default (RECEIVER_NOT_EXPORTED) rather than guess at a platform detail I couldn't confirm; DebugFixReceiver's KDoc documents the RECEIVER_EXPORTED fallback and its trade-off if this turns out not to work. Worth a look from bastion-security before this is relied on for anything beyond a developer's own machine.

Both are called out in docs/DEV.md's manual checklist and in code KDoc, not silently assumed.

Verified for real, this session

• uv run python -m unittest tools/test_gpx_replay.py -v — 36/36 pass.
• ./gradlew :companion:location:test — 14/14 pass (new Kotest source set).
• ./gradlew :companion:core:test and :companion:pebble:test — unaffected, still green.
• ./gradlew :companion:ride:compileDebugKotlin and :companion:assembleDebug — compile clean with the new wiring.
• Manually ran tools/gpx_replay.py end-to-end against the real Havelchaussee fixture and every other real GPX fixture in the repo (--sink print), including an --excursion-at-index run showing a visible perpendicular drift-and-return in the printed coordinates.

https://claude.ai/code/session_01DAoXbRmJUf2uxNYBfdAXPt

Closes #23. ## Scope decision — read this first #23's own listed file path is `tools/gpx_replay.py`, implying Python. Its **actual acceptance criteria** (read from the API, not guessed) are: - Reads a GPX and emits positions at a configurable speed multiplier - Can drive the Android emulator via mock locations - Can drive the companion directly through a debug hook - Supports pausing, jumping to an offset and injecting a deliberate off-route excursion - Documented in docs/DEV.md These are **not** about JVM-side pipeline verification (that concern is already covered by `SpeedPipelineGpxReplayTest.kt`/`RealWorldFixtureTest.kt`, which I did not touch or refactor — there is no shared-harness duplication to pay down here, because I did not build a second JVM replay harness). They are about driving the **real, unmodified** Android location stack from the *outside*: `adb emu geo fix` and a debug broadcast. Neither needs SpeedPipeline/StopDetector's algorithm reimplemented in Python — the real pipeline always runs in-process on the real device/emulator, unmodified. So: **built the literal `tools/gpx_replay.py` Python path**, not a Kotlin harness, because that's what these specific criteria actually call for once you separate "emit positions" from "process positions." ## What's here **`tools/gpx_replay.py`** (stdlib-only, `uv run`, same convention as `tools/gen_message_keys.py`): - Parses raw `<trkpt>` sequences (same "raw points, not RDP-simplified routes" basis as `SpeedPipelineGpxReplayTest.kt`'s own `parseRawTrackPoints`). - `--speed-multiplier` paces emission by each point's real recorded `<time>` gap (falls back to NFR-B2's 1 Hz cadence for untimed GPX). - `--start-offset-seconds`/`--start-offset-index` jump into a ride. - `--excursion-at-index`/`--excursion-offset-meters`/`--excursion-points` splices a synthetic perpendicular-drift-and-return excursion into the point sequence, for testing off-route detection without a second recorded ride. - Interactive `p`/`q` (pause/resume, quit) from stdin when it's a terminal. - Three sinks: `--sink print` (stdout, default), `--sink emulator` (`adb emu geo fix <lon> <lat>` — `AndroidLocationSource` is untouched and can't tell the difference from real GPS), `--sink debug-broadcast` (`adb shell am broadcast` to the new debug hook below). - `tools/test_gpx_replay.py`: 36 stdlib-`unittest` tests (Robert's xUnit/NUnit), including a cross-check of this script's GPX parsing against the same real ~119 km komoot fixture and ~119,020 m ground truth `SpeedPipelineGpxReplayTest.kt`/`RealWorldFixtureTest.kt` already established — proof this tool reads the identical point sequence, explicitly **not** a claim about SpeedPipeline itself (see the test's own docstring on why a haversine cross-check is not the kind of algorithm-duplication #23's own task brief was worried about). **The debug hook** (`:companion:location`, `:companion:ride`): - `DebugBridgeLocationSource` — a plain-Kotlin (no `android.*`) `LocationSource` you can `inject()` a fix into. Unit-tested for real (`:companion:location` gained a Kotest source set, same pattern as `:companion:pebble`'s). - `parseDebugFixExtras` — pure function parsing the broadcast's String extras into a `LocationFix`, unit-tested (14 tests: valid/missing/malformed/out-of-range/boundary lat-lon). - `DebugFixReceiver` — thin `BroadcastReceiver`, dynamically registered by `RideService.onCreate()` only when `ApplicationInfo.FLAG_DEBUGGABLE` is set (never true for a release build; no `BuildConfig.DEBUG` since no module enables that build feature). Wired additively: debug-injected fixes merge alongside real GPS into the same `SpeedPipeline`, not a mode switch. **CI**: `slow-lane.yml`'s `replay-regression` and `tag-lane.yml`'s `replay-and-emulator-gates` were both pre-existing placeholder jobs literally named after #23, deliberately left hard-failing with "update this step in the same PR that adds #23's harness." Both now run the tool's own test suite plus a `--sink print` pass over every real (non-adversarial) GPX fixture in the repo. `tag-lane.yml` still correctly hard-fails — now because G6's cue-position/leg-divergence/link-rate clauses and G7 (the no-network ride) need a cue engine/message scheduler that doesn't exist yet (#38 and friends), not because of anything #23 was responsible for. `docs/CI.md` updated to match. **docs/DEV.md**: new "GPX replay tool" section with usage examples and an explicit manual checklist for the two things I could not verify live in this session (see below). ## Honestly unverified (no device/emulator attached in this session) - `--sink emulator` and `--sink debug-broadcast` against a real live target. - Whether a plain, unprivileged `adb shell am broadcast` actually reaches a dynamically-registered `RECEIVER_NOT_EXPORTED` receiver on this project's target API levels — I chose the safe default (`RECEIVER_NOT_EXPORTED`) rather than guess at a platform detail I couldn't confirm; `DebugFixReceiver`'s KDoc documents the `RECEIVER_EXPORTED` fallback and its trade-off if this turns out not to work. Worth a look from bastion-security before this is relied on for anything beyond a developer's own machine. Both are called out in `docs/DEV.md`'s manual checklist and in code KDoc, not silently assumed. ## Verified for real, this session - `uv run python -m unittest tools/test_gpx_replay.py -v` — 36/36 pass. - `./gradlew :companion:location:test` — 14/14 pass (new Kotest source set). - `./gradlew :companion:core:test` and `:companion:pebble:test` — unaffected, still green. - `./gradlew :companion:ride:compileDebugKotlin` and `:companion:assembleDebug` — compile clean with the new wiring. - Manually ran `tools/gpx_replay.py` end-to-end against the real Havelchaussee fixture and every other real GPX fixture in the repo (`--sink print`), including an `--excursion-at-index` run showing a visible perpendicular drift-and-return in the printed coordinates. https://claude.ai/code/session_01DAoXbRmJUf2uxNYBfdAXPt

robert added 1 commit 2026-09-05 10:51:10 +02:00

Add tools/gpx_replay.py and the debug fix-injection hook (#23) ... 7a382496e8

Implements #23's actual acceptance criteria (mock-location/emulator sink,
a debug-hook sink, pause/jump/off-route-excursion, docs/DEV.md), which turn
out to be about driving the *real*, unmodified Android location stack from
the outside rather than about JVM-side pipeline verification. tools/gpx_replay.py
never reimplements SpeedPipeline/StopDetector: it only emits GPS-shaped positions
from a GPX file, paced in real time, into one of three sinks:

- --sink print (stdout, used by this script's own tests)
- --sink emulator (`adb emu geo fix`, feeding AndroidLocationSource unmodified)
- --sink debug-broadcast (a new debug-build-only DebugFixReceiver ->
  DebugBridgeLocationSource, wired into RideService, additive alongside real GPS)

Also wires up the two CI jobs that were already named after this issue and
left as deliberate hard-fail placeholders: slow-lane.yml's replay-regression
and tag-lane.yml's replay-and-emulator-gates now run gpx_replay.py's own test
suite plus a --sink print pass over every real GPX fixture. Both still
correctly flag that G6's cue-position/leg-divergence/link-rate clauses and
G7 (no-network ride) remain blocked on a cue engine/message scheduler that
doesn't exist yet (#38 and friends) — not a #23 gap.

Unverified in this session (no device/emulator attached): the --sink emulator
and --sink debug-broadcast paths against a live target, and whether an
unprivileged `adb shell am broadcast` reaches a RECEIVER_NOT_EXPORTED
dynamically-registered receiver on this project's API levels. Both flagged
explicitly in docs/DEV.md's manual checklist and DebugFixReceiver's KDoc.

Claude-Session: https://claude.ai/code/session_01DAoXbRmJUf2uxNYBfdAXPt

robert merged commit b372c2f57b into main 2026-09-05 10:52:29 +02:00

robert referenced this pull request from a commit 2026-09-05 10:52:30 +02:00

Merge pull request 'tools/gpx_replay.py: replay a GPX through the location pipeline (#23)' (#105) from tooling/gpx-replay into main

robert referenced this pull request 2026-09-05 10:52:47 +02:00

tools/gpx_replay.py: replay a GPX through the location pipeline #23

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area:companion

Android companion app

  

area:docs

Documentation

  

area:shared

Message contract shared by both sides

  

area:tooling

Build tooling, SDK, dev scripts

  

area:watchapp

Pebble C watchapp

  

blocker

Blocks a large part of the project until resolved

  

kind:chore

Setup, scaffolding, maintenance

  

kind:feature

New functionality

  

kind:spike

Time-boxed investigation to de-risk a decision

  

kind:test

Tests and fixtures

No labels

area:companion

area:docs

area:shared

area:tooling

area:watchapp

blocker

kind:chore

kind:feature

kind:spike

kind:test

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

robert/PedalPebble!105

No description provided.