Skip to content

feat: Q10 map layers, calibration, path & no-go zones (+ Q7 layer reuse)#848

Open
tubededentifrice wants to merge 34 commits into
Python-roborock:mainfrom
tubededentifrice:q10-map-layers
Open

feat: Q10 map layers, calibration, path & no-go zones (+ Q7 layer reuse)#848
tubededentifrice wants to merge 34 commits into
Python-roborock:mainfrom
tubededentifrice:q10-map-layers

Conversation

@tubededentifrice

@tubededentifrice tubededentifrice commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds structured Q10 map layers, calibration, path placement, and vector overlays on top of #847. It also reuses the shared B01 grid-layer code for Q7 maps.

This PR is stacked on #847. Until #847 merges, GitHub will show those commits here too; the incremental work starts with feat: decompose Q10 map into separable layers.

Changes

  • Split Q10 occupancy grids into separable background, wall, floor, and per-room layers. Each layer can be rendered as a transparent PNG for frontend compositing.
  • Add GridCalibration and solve_calibration() to fit Q10 world coordinates to grid pixels from an active cleaning path.
  • Populate MapData.path, vacuum_position, and charger position once calibration is available.
  • Decode Q10 restricted-zone blobs into no-go, no-mop, and virtual-wall overlays in MapData.
  • Decode and apply Q10 erase zones from the map packet tail so erased cells are removed from rendered maps and layers.
  • Reuse the shared layer/calibration abstractions for Q7 SCMap payloads.
  • Preserve existing overlays on partial status updates and keep path/position overlays after reparsing a calibrated map.

Notes

  • Virtual walls, zoned-clean areas, and carpets remain best-effort until non-empty captures are available for each data point.
  • Q10 calibration currently needs a sufficiently dense current-session path; no-clean calibration is left for a later PR.

Testing

  • uv run pytest — 543 passed on the rebased stack
  • uv run pre-commit run --all-files
  • Focused coverage for Q10 layers, calibration, overlays, erase zones, parser edge cases, and Q7 layer reuse
  • Live checked against Q10 captures for map layers, calibration fit, path placement, and no-go/no-mop zones

Brings Q10 maps toward parity with V1 devices. Verified end-to-end against two
physical Q10 (roborock.vacuum.ss07) robots.

Protocol (reverse-engineered from live captures):
- Requesting device state (dpRequestDps) makes the robot push its current map as
  a protocol-301 MAP_RESPONSE a few seconds later (firmware throttles to ~once
  per minute).
- The "01 01" map packet carries a u32be map id, u16le grid width, and an
  LZ4-block-compressed occupancy grid followed by 47-byte room records
  (id + ascii name); room cells use value room_id*4. The payload is unencrypted,
  unlike the Q7 SCMap protobuf format.

Changes:
- roborock/map/b01_q10_map_parser.py: clean LZ4 block decoder + packet parser +
  renderer producing a PNG and MapData with room names.
- roborock/devices/rpc/b01_q10_channel.py: request_map() triggers and awaits the
  MAP_RESPONSE push.
- roborock/devices/traits/b01/q10/map.py: MapContentTrait (refresh/parse/image/
  rooms), wired into Q10PropertiesApi.
- cli: `map-image` and `rooms` now work for Q10 devices.
- Tests + a synthetic (no-PII) map fixture.

Map packet format documentation credit: the roborock-qseries-map-bridge project
(GPL-3.0): https://github.com/v1b3c0d3x3r/roborock-qseries-map-bridge
Adds parsing for the Q10 "02 01" live position packet (delivered on the same
protocol-301 channel as the map, only while the robot is moving).

The packet format was reverse-engineered and validated against live ss07
captures (the 18-byte-header layout documented elsewhere did NOT match this
firmware):
- 10-byte header (sequence counter at byte 3, then a constant type/flag).
- big-endian int16 (x, y) point pairs; this firmware sends the current position
  as a single point per packet rather than an accumulated path.
- Confirmed live: as R1 traversed the corridor, the decoded x moved from -163 to
  +169 with y ~0.

The full saved map packet (01 01) was checked too and does NOT carry the live
path (identical across captures during a clean), so position comes from 02 01.

- b01_q10_map_parser: parse_trace_packet() + Q10TracePacket/Q10Point.
- b01_q10_channel: request_trace() (marker-filtered).
- MapContentTrait.refresh_trace() exposes path + robot_position.
- cli: `q10-position` (reports gracefully when the robot is idle).
- Tests use a real captured position packet + a synthetic multi-point packet.
Live capture (R1 corridor run) disproved the earlier 'single current point
per packet' assumption: the same session emitted packets of 1, then 3, then
15 points, each a strict superset. The robot accumulates the full session
path server-side and returns it whole, so a client connecting mid-session
still gets the complete trail (matching the app showing it after a cold
launch). The parser already read all points; this corrects the docs and
adds a real 15-point fixture + test, and clarifies that byte 3 is a session
counter (tracks the device clean count) not a per-packet sequence.
@tubededentifrice tubededentifrice force-pushed the q10-map-layers branch 2 times, most recently from 8af91fe to 0e3af65 Compare June 14, 2026 11:42
@allenporter

Copy link
Copy Markdown
Contributor

If there are low level map parsing parts, you could send them out in parallel without blocking on the other changes if that would be helpful to parallelize review. otherwise, we can mark this in draft until the other blocking parts are reviewed and merged.

The Q10 has no synchronous get-map command. The previous MapContentTrait
faked one: refresh()/refresh_trace() sent a dpRequestDps and blocked awaiting
the next MAP_RESPONSE push with a timeout. That has no request/response
correlation and fights the firmware's ~60-70s push throttle.

Mirror the existing Q10 StatusTrait model instead:
- MapContentTrait is now a push-only TraitUpdateListener. The Q10PropertiesApi
  subscribe loop routes protocol-301 MAP_RESPONSE packets to
  update_from_map_response(), which parses the payload, updates the cached
  fields and notifies listeners.
- Drop request_map()/request_trace() and the trait's refresh()/refresh_trace().
- CLI map-image/rooms/q10-position now nudge the device with refresh() and wait
  on a map-trait update listener for the pushed data.
Adds a device-agnostic grid->layers module (b01_grid_layers) that splits a
single-byte occupancy grid into background/wall/floor/per-room layers via a
caller-supplied classifier, each renderable to a transparent RGBA PNG for
frontend compositing. Wires a Q10 classifier (confirmed against real ss07
captures: 243=background, 249=wall, 240=unsegmented floor, value=room_id*4
for room floor) and exposes layers on MapContentTrait, plus a q10-map-layers
CLI command that lists layers and can export per-layer PNGs. The shared module
is built classifier-first so Q7's 0/127/128 grid can reuse it later.
The Q10 packet carries no calibration (header fields are map-growth metadata;
room records hold flags, not coords), so the world<->pixel transform is solved
from a cleaning path: GridCalibration + solve_calibration() slide the path's
pixel bbox to maximise on-floor overlap. Validated on a live R1 corridor run
(184-pt path, 183/184 on floor; path renders along the corridor with the robot
at its end). MapContentTrait gains calibration, solve_calibration(),
render_path_on_map() and populates MapData.path/vacuum_position in grid-pixel
coords (consistent with the identity img_transformation). Adds a
q10-map-with-path CLI command. Resolution is fit per-map so nothing is
hardcoded; note origin_x landed exactly on header @14, hinting the header may
encode origin.
Reverse-engineered the dpRestrictedZoneUp blob from a live ss07 (7 real zones):
[version][count] + fixed 38-byte records of [type][nverts] + int16-BE vertex
pairs, in world coords. New b01_q10_overlays.parse_zone_blob decodes it
(type 0 = no-go, 3 = no-mop). MapContentTrait.load_overlays() stores zones +
virtual walls and, with calibration, places them as MapData.no_go_areas /
no_mopping_areas / walls in pixel space; the charger is derived from the path
origin (the dock). The property API feeds the overlay DPs to the map trait from
the status stream, and render_path_on_map() draws zones + dock + position.
Validated live on R1: 6 no-go + 1 no-mop zones land squarely inside rooms.

Virtual walls / zoned / carpets are empty on the test device, so their decoders
are best-effort/scaffolded; obstacles were not located on the device channel.
Demonstrates the device-agnostic b01_grid_layers module serves both devices:
the Q7 SCMap parser gains classify_q7_cell / decompose_q7_layers (0=background,
127=wall, 128=floor) and q7_calibration, which reads the world<->pixel transform
straight from the SCMap mapHead (minX/minY/resolution) -- no path fitting needed
(unlike the Q10). Q7's MapContentTrait now exposes layers + calibration.

Q7's raster has no per-room segmentation and its map carries no path or zones,
so Q7 reaches background/wall/floor layers + calibration only; per-room masks,
path and vector overlays remain Q10-only. Validated against the existing Q7
SCMap fixture (no Q7 hardware available to test live).
The map (01 01) packet has a vector section after the compressed grid that the
parser previously ignored: [count][vertices_per] + count polygons of int16-BE
(x,y) pairs = carpet areas (user-defined + auto/self-identifying). Confirmed on
two ss07 devices (R1: 3 carpets, RDC: 2) and explains why the dpCarpetUp DP is
empty -- carpets ride in the map, not a DP. parse_map_packet now returns
packet.carpets; MapContentTrait exposes them and, with calibration, rasterises
them into MapData.carpet_map and draws them. The remaining tail (a run-length
raster + trailing signature) is left for later -- likely the carpet pixel mask
and/or obstacles.
load_overlays(restricted_zone_up=None) treated an absent DP as 'clear', so a
status push carrying only the virtual-wall DP wiped the loaded no-go zones
(caught live: zones loaded as 7, rendered as 0). None now means 'unchanged';
an explicit empty blob still clears. Regression test added.
Two corrections to the Q10 (ss07) map rendering, both verified on live
hardware:

Orientation: the ss07 grid is stored top-down (row 0 = top of the home),
unlike the V1/Q7 bottom-up convention, so the inherited vertical flip
rendered every Q10 map upside down. Make the flip a per-device property
of GridLayers (Q10 = no flip; Q7 keeps flipping, untouched) and drop it
from the Q10 renderer and the path/overlay math so all layers, the
combined map and overlays stay consistent.

Erase zones: a controlled with/without diff on a live device proved the
map-packet tail "carpet" vector section is actually the app's *Erase*
zone list -- removing the two zones in-app dropped its count 2->0 while
the grid and the trailing raster stayed byte-identical (so the earlier
"decode Q10 carpets" commit mislabeled them, and we were drawing them as
purple polygons). Rename Q10Carpet -> Q10EraseZone and, once a
calibration is available, blank the cells inside each erase rectangle to
background and re-render so phantom floor (e.g. lidar seen through
floor-to-ceiling windows) drops out of the map and every layer, matching
the app. Validated on R1: a 57-point corridor path solved the
calibration and the three erase zones removed the three phantom
projections.
Follows the q10-maps push-driven refactor: the map trait no longer has
refresh()/refresh_trace(), so adapt the layers/path CLI commands to nudge the
device (dpRequestDps) and wait on a map-trait update listener, and route the
overlay DPs through the refactored _handle_message dispatch.
@tubededentifrice

Copy link
Copy Markdown
Contributor Author

@allenporter ok, #850 is the standalone part

@andrewlyeats

Copy link
Copy Markdown
Contributor

Really nice to see the Q10 calibration and layer work coming together here.

One thing that might help the no-clean calibration you've flagged as a follow-up: on our Q10 S5+ (ss07, fw 03.11.24) captures, the calibration data appears to be transmitted in the 0101 grid-frame header, so a GridCalibration can be derived without a session path (i.e. without solve_calibration's dense-path requirement). The header fields we see (absolute byte offsets in the 0101 frame, sub-type at 0–1; verified across our captures, scoped to ss07):

  • bytes 11–12 x_min, 13–14 y_min (s16 BE) — the map origin, in 5 mm units (= 0.1 grid-px; ÷10 gives grid-pixel coordinates). This is the value solve_calibration is recovering by fitting.
  • bytes 15–16 resolution (u16 BE) — reads 5 universally for us → 0.05 m/px (the 50 mm/px you use).
  • bytes 17–22 charger x / y / phi — the dock position and orientation, as an alternative to inferring the charger from the path's first point.

On our decode, reading the origin straight from the header lands the path on-floor at the same rate as the auto-fit (they agree on ~29 of 31 captures; the two misses are null/keepalive frames where x_min == y_min == 0, which you'd just skip and fall back to a fit). So a header-derived calibration looks viable docked / pre-clean, which would drop the "needs a dense current-session path" requirement.

Happy to help — I can share scrubbed, annotated header captures, or sketch a PR against this branch if you'd like — and thanks for all the Q10 work. (Separately, the 0201 path header also carries a live SLAM heading field at bytes 10–11 if orientation is ever useful alongside vacuum_position — glad to detail that too.)

Reconciles this branch's Q10 map-layers / calibration / path / overlay
work with the Q10 map support that landed on main via Python-roborock#847. The two
diverged after the branch's 2026-06-15 work; Python-roborock#847 (merged 2026-06-21)
absorbed several @andrewlyeats-validated protocol corrections this
branch predated. Net result is a union, not a one-side win:

Took from main (newer, validated protocol corrections):
- map parser width/height: two consecutive u16be fields (offsets 7/9)
  + _split_with_dims, fixing the 222x261 (cross-256-band) mis-split that
  the older u16le@8 read produced; _infer_layout kept as fallback.
- trace _drop_stray_leading_point hygiene.
- overlay zone-type constants: 0 no-go, 1 virtual-wall, 2 no-mop,
  3 threshold (corrects this branch's earlier 3=no-mop reading).

Kept from this branch (net-new features main lacks):
- erase-zone decode from the packet tail, decompose_layers / classifier,
  GridCalibration usage, push-driven trait (update_from_map_response,
  load_overlays, render_path_on_map), and the q10_map_layers /
  q10_map_with_path CLI commands.
- top-down (no-flip) rendering: the branch's overlay/path/calibration
  placement is built on un-flipped grid-pixel coords, so the base raster
  is rendered un-flipped to keep overlays aligned.

CLI _await_q10_map_push merges both improvements: the early
already-satisfied short-circuit and main's allow_cached_on_timeout.

test_map zone-type test updated to the corrected no-mop constant (2).
Full suite green (576 passed).
Implements @andrewlyeats' suggestion (PR Python-roborock#848 review): the ss07 01 01
grid-frame header carries the calibration, so a GridCalibration origin
can be read straight from the packet instead of being recovered by
solve_calibration's dense-path slide.

- Decode the header calibration fields into Q10MapPacket:
  Q10HeaderCalibration{origin_x, origin_y (5 mm units), resolution,
  charger x/y/phi}. origin_pixels() returns the grid-pixel origin
  (header value / 10, since the grid is 50 mm/px); keepalive frames
  (x_min == y_min == 0) report is_keepalive and yield no origin.
- Add solve_calibration_with_origin(): fits only resolution + Y sign
  around a fixed pixel origin, validated against on-floor points. With
  the 2-D offset slide gone, a short path confirms the fit instead of a
  dense clean.
- MapContentTrait.solve_calibration() now prefers the header origin
  (>= 4 path points) and falls back to the full fit (>= 20) for
  keepalive frames or when the header origin doesn't validate.

The header origin (5 mm units -> /10 px) is the unambiguous, verifiable
part of the report. The GridCalibration resolution still lives in the
path's native units (the branch fits ~13-16/px), so it is confirmed
against a short path here rather than read from the header's 50 mm/px
field; a fully path-free resolution awaits the annotated ss07 captures
@andrewlyeats offered.

Tests: header field decode + keepalive, solve_calibration_with_origin
(short path / off-floor reject / no points), and trait-level header vs
fallback paths. Full suite green (584 passed).
@tubededentifrice

Copy link
Copy Markdown
Contributor Author

Thanks @andrewlyeats, this was really useful! I've implemented the header-origin part on this branch.

The 01 01 header now decodes into a Q10HeaderCalibration (x_min/y_min @11–14, resolution @15–16, charger @17–22), and origin_pixels() returns the grid-pixel origin (÷10, since the grid is 50 mm/px). Keep alive frames (x_min == y_min == 0) are detected and skipped > fall back to the path-fit, exactly as you described.

To use it, I added solve_calibration_with_origin(): with the origin fixed from the header, only the resolution + Y sign are fit, so it now confirms from a short path instead of needing a dense clean (the trait prefers the header origin at >=4 points, falls back to the full fit at ≥20).

One thing I couldn't pin down without your data: the header resolution reads 50 mm/px, but my path-fit lands at ~13-16 per pixel, i.e. the path coordinates don't look like millimeters, so I can't yet turn the header's 50 mm/px straight into the GridCalibration resolution. That's the only reason it isn't fully path-free yet (origin is exact; resolution still gets a quick on-floor validation against a short path).

So I'd love those scrubbed, annotated header captures, especially a few where you've noted the path/trace coordinate units alongside x_min/resolution. If I can confirm the path-unit <=> pixel relationship, I think the docked/pre-clean (zero-path) case falls out directly. And yes, please do detail the 0201 SLAM heading field too, orientation alongside vacuum_position would be nice to have.

Thanks again for the cross-capture validation here.

Virtual walls (dpVirtualWallUp 57) use a different on-wire frame from the
restricted-zone DPs: a bare [count] byte (no version, no per-record
type/pad) then 8-byte (y, x) int16-BE records. Feeding such a blob to
parse_zone_blob mis-frames it (leading 0x01 read as a version, the next
coordinate byte as a record count), so virtual_walls silently came back
empty and the wall overlay never rendered.

Add parse_virtual_wall_blob (axes un-swapped to (x, y) so walls share the
restricted-zone coordinate order), point load_overlays at it for DP 57,
and correct the overlay module's docs that wrongly claimed parse_zone_blob
handled DP 57. Tested against a real ss07 read-back from the PR Python-roborock#850 thread.
@tubededentifrice

Copy link
Copy Markdown
Contributor Author

Confirmed on our hardware, you were right. Captured a no-go zone (DP-55) and a wall (DP-57) on our devices and ground-truthed both against the app:

  • DP-55: a deliberately wide no-go zone reads back wide (x-range >> y-range), matching the horizontal band as drawn, so our zone parser keeps the first wire word on x, no transpose.
  • DP-57: a wall drawn horizontally below a room only decodes horizontal when we don't swap. Our parse_virtual_wall_blob was swapping to (y,x) (from an earlier reading of the feat: B01 grid-layer decomposition + Q10 vector overlay decoding #850 notes), placing every wall 90° off. Dropped the swap so DP-57 shares DP-55's order, exactly as you predicted. Fixed in cea9c74, with the real capture added as a regression fixture.

Also fixed a related rendering bug where 2-point wall segments were being skipped entirely. Thanks for the nudge on this one. Still owe you the ruler-measured drive to pin the absolute path-unit scale.

Real DP-57 read-back from the R1 with one horizontal and one (near-)vertical
wall drawn in the app. Confirms the un-swapped axis order holds for both
orientations in a single blob -- the second wall's 4-unit x drift even matches
it not being drawn perfectly vertical. Locks in the mixed-orientation case the
inferred two-walls fixture only approximated.
…th-units/px

A dense ss07 path fits resolution 20.0 around the header origin, but the
[10.0..18.0] candidate range couldn't reach it (the fit railed at 18, or at 10
for sparse paths), biasing every header-anchored calibration. Ground-truthed on
the R1: a corridor drive registered at 20 (matching the format author's
independent value), and the dock->robot path span (3619 units) lined up with
the ruler-measured 8.81 m corridor at that scale -- ~2.4 mm/path-unit and a
~49 mm/px grid, confirming the header resolution=5. Widen to [12.0..26.0] so the
fit can land on 20.
…nvention

Scale: with the header resolution=5 (50 mm/px grid) and the confirmed 20
path-units/px, one path-unit is exactly 2.5 mm -- so a path-unit is not a
millimetre (the reviewer's open scale question). Refine the resolution-range
comment accordingly.

Heading: a live R1 clean confirmed the convention including the y-sign -- on
straight segments the reported heading equalled the direction of travel
atan2(dy,dx): +x read 0, -x read +/-180, a slight -y drift read negative. So
the on-map heading arrow points the right way.
@tubededentifrice

Copy link
Copy Markdown
Contributor Author

Pinned the scale on my robot: a dense path fits 20 path-units/px around the header origin, matching your internal "20". With the header's 50 mm/px grid that's 2.5 mm per path-unit (so not a millimeter). I corroborated it accross a ~8.8 m drive. Also widened our calibration range, which had been railing below 20. And while there, ground-truthed the trace heading (bytes 10–11) against a live drive, matches direction of travel exactly.

@tubededentifrice

Copy link
Copy Markdown
Contributor Author

@allenporter Lots of fun experiments here, I cross checked everything with my hardware I believe, should be ready to review, thanks :)

# Conflicts:
#	roborock/data/b01_q10/b01_q10_containers.py
#	roborock/devices/traits/b01/q10/__init__.py

@allenporter allenporter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to incorporate changes made form the other PR? this looks like its reverting something we iterated on in the last PR. if that is intentional then please update the PR description to describe why but im assuming its just a little stale and needs to be updated. Want to clairfy then i can take a look?

Thank you!

@andrewlyeats

Copy link
Copy Markdown
Contributor

The header-calibration and erase-zone work here is great to see. Both match what we get on our ss07 — the erase section decodes identically (count, vertices-per, int16 rectangles), and reading the origin straight from the grid header is exactly the piece that places everything without a fit.

One thing that might be useful: on our ss07 the packet tail continues past the erase section with more structure before the trailing raster — it isn't opaque. After _parse_erase_zones consumes its bytes, the rest decodes byte-exact:

  • carpet mask — a second compressed grid using the same LZ4 framing as the main grid ([u32 uncompressed_len][u16 compressed_len][block]). Empty on our current map (0 cells), but that's where carpet zones live.
  • AI-obstacles (the "cones" the app shows)[count: u8] then count × int16-BE (x, y), each coord raw / 50. Offset from your header pixel-origin (origin / 10) with the same Y-flip, all of them land on the floor footprint here (53/53). This is the layer that's otherwise cloud-only on camera models — on the lidar Q10 the marker positions are right in the frame.
  • skip-clean zones — a short [count: u8] + int16 (x, y) list (/10), then on the full-map (03 01) frame a concatenated path package (a 17-byte header + int16 points) — probably the "+ signature" you noted.

Byte-exact on one 03 01 frame:

29 + 8963 (grid) + 82 (erase) + 253 (carpet) + 213 (obstacle) + 1 (skip) + 40925 (path) = 50466

The same erase/carpet/obstacle/skip layers also appear in the finalized 01 01 frames (same dims + origin), so they ride the live grid, not just the saved map.

Totally your call whether obstacles are in scope for this PR vs. a follow-up — happy to share fixtures (a frame with N obstacles, plus before/after) if it helps, same as the #850 ones. Thanks again for driving the Q10 map work forward.

…-message model

PR Python-roborock#847 (the PR this branch is stacked on) landed a typed-message decode/
dispatch model on main: stream_decoded_messages() decodes each MAP_RESPONSE
push into a typed Q10Message (Q10DpsUpdate | Q10MapPacket | Q10TracePacket)
via decode_message(), and Q10PropertiesApi._handle_message routes by
isinstance. This branch predated that and carried the older push-driven model
(channel yielding DPS-only dicts, _handle_message taking the raw RoborockMessage
and calling map.update_from_map_response()), so the merges reconciled the
parsing details but left the dispatch architecture reverted -- decode_message
ended up orphaned (defined + unit-tested but with no production caller), which
is the revert @allenporter flagged.

Re-reconcile onto main's typed model instead of replacing it:

- b01_q10_channel.py: restore stream_decoded_messages() + decode_message()
  (now byte-identical to main); drop the DPS-only stream_decoded_responses().
- traits/b01/q10/__init__.py: restore main's typed isinstance dispatch. The
  branch's one net-new behavior -- feeding the no-go / virtual-wall overlay DPs
  to the map trait -- now rides on the Q10DpsUpdate branch.
- traits/b01/q10/map.py: replace update_from_map_response(message) with main's
  typed update_from_map_packet(packet) / update_from_trace_packet(packet). The
  trait caches the parsed Q10MapPacket (self._packet) so erase zones / overlays
  re-render without re-parsing wire bytes; parse_map_content() re-renders the
  cached packet. Drop raw_api_response (no raw bytes reach the trait in the
  typed model, matching main). Calibration / layers / erase / overlay features
  are unchanged, just layered on the typed entry points.
- tests: drive the typed methods; the trait-level "ignores non-map" case is
  now covered by the decode_message protocol tests and the dispatch integration
  tests.

decode_message is back on the production path. Full suite green (611 passed);
mypy + ruff clean.
@tubededentifrice

Copy link
Copy Markdown
Contributor Author

@allenporter Good catch, you were right, this was a stale revert, not intentional. Fixed in 496474c

The packet tail past the erase section is not opaque: it continues with a
carpet mask, then (currently undecoded) obstacle and skip-clean sections.
Decode the carpet mask here.

Framing (reported by @andrewlyeats on PR Python-roborock#848, then confirmed byte-exact on
our own ss07 hardware -- live captures from both the R1 and RDC):

- The carpet mask follows the erase section and uses the same framing as the
  main grid block: [u32 uncompressed_len][u16 compressed_len][LZ4 block].
- The decompressed mask is a full width*height grid in the same top-down pixel
  space as the main grid; a non-zero cell is carpet (the value is the carpet
  kind). The uncompressed length equals width*height exactly on every capture,
  which is used as the guard: if it doesn't hold we mis-located the section and
  leave carpet undecoded rather than emit garbage.

Verified on hardware (the synthetic fixture has an empty tail, so this could
not be exercised before): walking erase -> carpet -> obstacles -> skip consumes
both frames to the last byte, and the carpet masks are non-empty -- R1 = 1047
carpet cells (kind 4), RDC = 2856 cells (kinds 3 and 4).

- parse_map_packet now returns Q10MapPacket.carpet_mask (the decompressed grid).
- B01Q10MapParser populates MapData.carpet_map (flat grid indices y*width+x),
  which lines up with the rendered top-down raster and composes with the layers.
- Tests build a synthetic carpet section with the shared LZ4 helper and cover
  the decode, the MapData.carpet_map population, the no-carpet case, and the
  dimension-mismatch guard.

The obstacle and skip-clean sections share this tail; their framing is also
byte-exact (count byte + N int16 pairs) but both read count 0 on our current
maps, so decoding their values is left for a follow-up once we have a non-empty
capture.
@tubededentifrice

Copy link
Copy Markdown
Contributor Author

@andrewlyeats This was spot-on, I captured live 01 01 frames on my robots and your tail layout decodes byte-exact on our hardware too. Walking erase > carpet-mask > obstacles > skip consumes both frames to the last byte.

A couple of confirmations beyond what you had:

Carpet mask: exactly your framing ([u32 uncompressed][u16 compressed][LZ4 block], same as the main grid), and the uncompressed length equals width×height on the nose (325×283, 269×320), so it really is a full same-dims second grid. Unlike your map, ours aren't empty: R1 = 1047 carpet cells (kind 4), RDC = 2856 (kinds 3 and 4) ; looks like the value encodes the carpet kind. I'm using uncompressed == w*h as the locate-guard so a mis-framed tail just yields no carpet rather than garbage. This explains the empty dpCarpetUp: carpets ride in the map, not the DP. I've landed carpet on the branch (parse_map_packet().carpet_mask → MapData.carpet_map), so that part's done.

Obstacles / skip-clean: framing confirmed byte-exact ([u8 count] + N×int16), but here's where I'm stuck: both read count 0 on every frame our robots produce right now, including the finalized post-dock map. I ran a spot clean (table + chairs within a meter), but with lineLaserObstacleAvoidance it just reacts physically and never persists any "cone" markers, nothing gets written to the obstacle section. So I can confirm the shape but not the values (the /50 scaling, the on-floor placement you saw 53/53 on).

If that offer of a frame with N obstacles (plus a before/after) still stands, I'd love to take you up on it, that's the missing piece to validate the obstacle decode and wire it into MapData.obstacles. A 03 01 saved-map sample would be great too while we're at it: we only ever get 01 01 pushes here, so the trailing path package you mentioned is still unsampled, and our parser doesn't recognize 03 01 yet.

Thanks again, the byte-level cross-checks have been incredibly useful.

@tubededentifrice

Copy link
Copy Markdown
Contributor Author

@allenporter i think we can also merge this one; for the obstacles, we'll simply make another PR

@allenporter allenporter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, took me a little bit of time to gather my thoughts since this is a pretty large PR.


# Feed the map's vector-overlay data points (no-go zones / virtual
# walls) to the map trait so they are decoded as they arrive.
if B01_Q10_DP.RESTRICTED_ZONE_UP in message.dps or B01_Q10_DP.VIRTUAL_WALL_UP in message.dps:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This indicates to me the DPS fields on Status should probably live on the map trait? It seems like the update_from_dps API could potentially handle these types of calls.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. The map trait now implements update_from_dps and joins the _updatable_traits fan-out like the other read-model traits, picking out the vector-overlay DPs (dpRestrictedZoneUp / dpVirtualWallUp) it owns and ignoring everything else. The orchestrator no longer special-cases those data points. Added a small DpsUpdatable Protocol to type the fan-out list since the map trait extends TraitUpdateListener directly rather than UpdatableTrait.

Comment thread roborock/devices/traits/b01/q10/map.py Outdated
The orchestrator special-cased the no-go / virtual-wall data points,
reaching into the map trait's load_overlays after the generic DPS fan-out.
Per review feedback on Python-roborock#848, the map trait now implements update_from_dps
and joins the _updatable_traits fan-out like every other read-model trait,
picking out the vector-overlay DPs it owns and ignoring the rest. The
orchestrator no longer needs to know which DPs the map cares about.

A DpsUpdatable Protocol in the Q10 common module types the (now
heterogeneous) fan-out list, since the map trait extends TraitUpdateListener
directly rather than UpdatableTrait.
@andrewlyeats

Copy link
Copy Markdown
Contributor

@tubededentifrice The offer stands — two real 0301 saved-map frames with obstacle markers, map-id zeroed, everything else as-captured:

Gist: https://gist.github.com/andrewlyeats/6c0c5892ab61e6d1c75d4c42c9202903

  • q10_0301_frame_2obstacles.b64 — compact ~9.4 KB, 2 obstacles (W=219 H=254 ox=1651 oy=434 res=5; 5 erases + carpet, skip=0)
  • q10_0301_frame_53obstacles.b64 — dense ~50 KB, 53 obstacles

Both are 03 01, and both carry the trailing path package you said was still unsampled (17-byte header + int16 (x,y) points, same geometry as 0201) — ~1 KB on the compact frame, ~40 KB on the dense one — so there's a sample to teach the parser the 03 01 shape too.

Obstacle decode for the compact frame (scale /50; col = ox/10 + x/50, row = oy/10 − y/50, row Y-flipped):

raw=(-4633, -1946)  ->  grid px (72.4, 82.3)
raw=(-5231, -3852)  ->  grid px (60.5, 120.4)

Both inside the footprint. Before/after is just the count: the section reads count=0 on obstacle-free frames vs 2/53 here, same tail walk otherwise — so it should drop into MapData.obstacles once the count is non-zero. The one gotcha is the per-section scale: obstacles /50 but erases/skip /10.

On our unit the saved-map obstacle set is accumulated/historic (the app's "cones" build up across many cleans rather than one), so a single spot clean may not populate it — though your finalized map reading zero too points at something model- or setting-specific on your side. Either way these give you non-zero counts to validate against. Happy to grab a non-empty skip sample if that's a useful next step. Thanks for the byte-level cross-checks — glad the tail held up on your hardware.

Comment thread roborock/devices/traits/b01/q10/common.py Outdated
# Conflicts:
#	roborock/devices/traits/b01/q10/__init__.py
Addresses review feedback that `_render_packet` mutated ~8 trait fields
through 3-4 layers of side-effecting methods, making the trait hard to
review and blurring the state-management / pixel-work boundary.

- Add `roborock/map/b01_q10_render.py`: `render_q10_map()` composes a map
  packet + path + overlays + calibration into one `Q10MapRender` result
  (image + MapData + layers), and owns the erase blanking, world->pixel
  overlay placement, path drawing and calibration policy that previously
  lived on the trait. The whole data flow reads top-to-bottom in one
  function instead of nested mutations.
- `MapContentTrait` is now just state management: it accumulates the pushed
  inputs and rebuilds the single `Q10MapRender` wholesale on each change,
  exposing read-only properties. No more per-field clearing. The derived
  rendering scaffolding (erase zones, header calibration) is off the public
  surface; layers/calibration stay (used by the CLI + frontend compositing).
- Move the geometry/pixel tests to tests/map/test_b01_q10_render.py; the
  trait tests now cover state management.
- `UpdatableTrait` explicitly declares the `DpsUpdatable` protocol so the
  heterogeneous fan-out list's shared shape is documented (the map trait
  satisfies it structurally without being a converter-backed read-model).
@tubededentifrice

Copy link
Copy Markdown
Contributor Author

Done in a3b095f. Pushed the machinery down into a new b01_q10_render.py: render_q10_map() returns a single Q10MapRender (the MapData-equivalent you called out), and the trait just accumulates the pushed inputs and rebuilds it wholesale, so there's no field-clearing/nested-mutation to trace anymore. Geometry tests moved to the map module; trait tests are now state-management only. And yes on DpsUpdatable: made UpdatableTrait declare it explicitly rather than dropping the protocol, since the map trait implements it without being a converter-backed read-model.

@tubededentifrice

tubededentifrice commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

@andrewlyeats Thanks for the data! I've put a very draft PR at tubededentifrice#1 but I'm waiting for this one to merge first before getting more into it

@lboue

lboue commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

@tubededentifrice
You should join the discussion thread on Discord. It would be easier to talk there.
https://discord.com/channels/1466803116392579084/1466803217168994314

@tubededentifrice

Copy link
Copy Markdown
Contributor Author

@lboue I think I need to be invited to the server? that links redirects me to https://discord.com/channels/@me (if needed, my handle is same as here)

@Lash-L Lash-L requested a review from allenporter July 4, 2026 13:58
Resolve conflicts from Python-roborock#859 (decouple B01 protocol/transport layer):
- q7/map_content.py: adopt the new Q7MapRpcChannel.send_map_command(method,
  params) call style; drop the now-unused Q7RequestMessage/B01_Q7_DPS imports
  while keeping the Q10 layers/calibration decomposition.
- q10/test_map.py: route the subscribe-loop integration tests through the new
  parsed-Q10Message stream (FakeB01Q10Channel) instead of raw RoborockMessage;
  drop the obsolete _map_message helper; keep the new layer/calibration/overlay
  test coverage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants