Memory-aware rendering, ESP32 reliability, state-field showIf, and new apps

[mariusandra]

Summary

Memory leak on HDMI/framebuffer frames (fixed)

• Root cause: every Linux build shipped with -d:malloc — a no-op define (Nim's real flag is -d:useMalloc) — so binaries ran Nim's page allocator and the mallopt/malloc_trim tuning in setupRenderMemory()/reclaimRenderMemory() never governed image buffers. RSS ratcheted ~3MB/render (frame 60: ~180MB after boot → ~300MB → OOM reboot on a 451MB Pi). HDMI + local JPEG frames churned the most; e-ink rendered too rarely to notice.
• Fixed to -d:useMalloc in the deployer (FRAMEOS_NIM_FLAGS), frameos/Makefile, and the driver/scene library build tools; deployer tests updated (assertions + a hermeticity fixture so a ../pixie checkout doesn't break them on dev machines).
• Verified on hardware: frame 60 RSS now cycles 49–84MB and returns to baseline after renders.

Memory-aware rendering pipeline

• Pixie fork (pinned via nimble to FrameOS/pixie): runtime per-decode budgets, streaming baseline-JPEG decode through a 32KB window, decodeJpegInfo dimension probe, aspect-correct ScaledDecodeFit (stretch/cover/contain) across all scaled decode paths.
• PNG decodes now unfilter in place (pixie 853485d): the decode plan for a canvas-sized RGBA PNG drops from pixels + 2× scanlines (~4.5MB at 480×800) to pixels + scanlines (~3MB), fitting the budget on ESP32-class frames. PNGs can't stream like JPEGs (zippy has no streaming inflate), so multi-megapixel photos remain JPEG territory.
• frameos: availableRenderBytes() (Linux MemAvailable / ESP32 PSRAM), per-render decode budget refresh, decode-into-canvas hints for full-frame render/image + data/localImage chains, file-read budgets, ensureRenderAllocation guardrails.
• ESP32 OOM containment: budgets first, then a patched allocator that releases a 1MB PSRAM emergency reserve, then a fatal-OOM longjmp back to the render entry point — render fails, device survives.

showIf visibility conditions for state fields

The app-config showIf system now applies to scene state fields and custom event fields, with one set of semantics (top-level conditions OR-ed, and blocks AND-ed, values coerced by field type) evaluated consistently on every surface:

• The condition matcher moved out of appNodeLogic into a shared frontend/src/utils/showIf.ts, used by the diagram editor, scene control forms (ExpandedScene), split-screen options, schedule state payloads and template previews. Hidden fields are excluded from submission.
• The field definition editor gained a visual condition builder (ShowIfEditor: field/operator/value rows, match-any/match-all, raw-JSON fallback for complex conditions) for both state fields and custom event fields.
• The on-frame /c control page evaluates conditions live in the browser and server-side for the initial render (frameos/utils/show_if.nim); compiled-scene codegen emits showIf + default values into StateField literals; interpreted scenes parse them from scenes.json.
• Along the way, interpreted scenes now respect access: private (previously every state field was externally visible and settable — the long-standing TODO), and json state field defaults parse like compiled scenes.

New apps

• render/chart — line/bar/area charts from JSON (accepts [1,2,3], label/value pairs, or {labels, series}), multi-series palette, auto/fixed y-ranges, grid + labels, friendly degenerate-data messages; bars bucket to pixel columns when data outnumbers the plot width.
• data/googlePhotos — shared-album slideshows via the share link (the Library API can no longer read library content); extracts photo URLs from the share page, requests display-sized variants, caches the album list, random/sequential order.
• data/immich — random/album/favorites/memories from a self-hosted Immich server (x-api-key), with a new immich settings namespace wired through the settings page, embedded settings pull and the on-device admin whitelist. Album mode picks server-side via POST /api/search/random so large albums never buffer on-frame; older servers fall back to /api/assets/random with client-side filtering.
• render/zoomPan — Ken Burns-style smooth zoom/pan: each render samples a slightly different crop bilinearly straight onto the canvas (zero per-render allocations), deterministic per-cycle focal drift, persistable phase.
• Native EXIF parser (frameos/utils/exif.nim) — dependency-free, bounds-checked, fuzz-tested; localImage and downloadImage merge camera/lens/exposure/ISO/GPS metadata plus an exifSummary one-liner into their metadata state. exiftool still wins where installed; ESP32 frames get EXIF for the first time.
• Four sample scene templates (Chart, Google Photos, Immich, Ken Burns slideshow) with rendered previews; the Immich sample uses the new state-field showIf (albumId only shows when mode=album).

ESP32 13.3" frame (Spectra-6, 32MB)

• Error frames now render the actual error message with the compiled-in typeface (wrapped, centered, thin border) instead of the black plus marker; the marker remains only as the fallback if typesetting itself runs out of memory.
• Blank/new SD cards are formatted as FAT on mount failure and used as the assets folder (IDF only formats on FR_NO_FILESYSTEM, so transient errors on a good card never wipe it). Verified: blank 64GB card formatted + mounted on hardware.
• Wi-Fi joins the strongest BSSID (WIFI_ALL_CHANNEL_SCAN + WIFI_CONNECT_AP_BY_SIGNAL): on mesh networks the default fast-scan latched onto distant nodes (-83 dBm) and died in reason=34 low-ack loops, falling back to the provisioning portal. Verified: connects at boot at -54 dBm.
• HTTP redirects now work on-device: the manual esp_http_client open/read flow bypasses esp_http_client_perform() — the only place ESP-IDF honors max_redirection_count — so any 30x (Google Photos share short links, http→https upgrades) errored on ESP32 while working on the Pi. The glue now follows up to 5 redirects.
• Repo-scene coverage harness (test_repo_scenes_esp32.nim): parses and builds all 21 bundled sample templates and renders the offline-safe ones through the interpreter under a 4MB render-memory override (testing-only), asserting no render-chain errors. The SD-card and Ken Burns scenes stream a bundled 10-megapixel JPEG fixture into the canvas — large JPGs from SD render within budget. FRAMEOS_TEST_NETWORK=1 also renders the network-dependent scenes. Only the Chromium/RTSP samples are excluded (child processes).
• Scene uploads: USB payload readiness handshakes, chunked reads/writes, size-scaled timeouts; 4MB USB uploads on the 32MB flash profile; scenes re-uploaded after browser flashing (which erases cached state); HTTP fast deploy preferred when reachable.
• Rendering fixes: Wikimedia cover aspect preserved via bounded decode, embedded scaler honoring cover/contain/stretch, embedded-safe approximate text stroke.
• No backend media proxy: image size/format constraints are handled device-side.

Build & tooling

• nimble.lock regenerated to pin the pixie fork revision (CI resolved pixie >= 6.1.0 to upstream and failed on pixie/decodebudget). Added a nimble relock task: nimble 0.20.1's lock-update path silently writes an empty lock for this package, so always regenerate from scratch.
• Browser flash / deploy drawer / logging improvements (USB API bridging, trace summarization, drawer close routing).

Validation

• backend: full pytest suite green (682 passed, 3 skipped), including new codegen tests for showIf/value emission and a fix for the pixie-path test that failed on dev machines with a ../pixie checkout
• frameos: full Nim test suite green (35 test files incl. new showIf, EXIF, interpreter state-field and repo-scene tests); nim check src/frameos.nim against the locked pixie fork
• ESP32: --os:freertos --cpu:esp -d:frameosEmbedded cross-compile of embedded_main.nim clean with all new apps registered
• pixie fork: PNG suite + 10k-iteration fuzz green with the in-place unfilter; new budget regression test (480×800 RGBA decodes at a 4MB budget, rejected with a catchable error at 2MB)
• frontend: tsc clean after kea typegen; control-page showIf JS exercised via node against the shared-evaluator semantics
• adversarial multi-agent review of the new code confirmed and fixed 15 findings before merge (32-bit EXIF RangeDefect, ESP32 redirect gap, Immich album buffering, zoomPan per-render allocations, showIf semantic divergences, control-page placeholder XSS, and more)
• on-hardware verification (frame 59): SD auto-format + mount, typeset error frame on panel, Wi-Fi strongest-BSSID connect at boot, 11-scene upload; (frame 60): RSS flat post-deploy

🤖 Generated with Claude Code