diff --git a/README.md b/README.md index 121ef9343..85dc4827f 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,41 @@ Language: EN | [简中](README.zh-CN.md) MP4 to GIF export (4) +--- + +## Why Use This AKA Build? + +This fork is tuned for creator-led screen recordings, not just raw screen +capture. The goal is to make Recordly feel more like a lightweight creator +studio: readable scripts, fast cuts, camera-first moments, tight timelines, and +exports that still look sharp. + +- **Hidden teleprompter:** read from a separate teleprompter window while + recording without burning the script into the final export. +- **Full-screen face-cam switch:** jump between screen-first mode and + camera-full mode while recording, then keep those camera-full moments editable + in the timeline and preserved in export. +- **Magnet timeline editing:** keep the magnet on when deleted clips should + close the gap automatically, or turn it off when you want the gap to remain as + intentional black time. +- **Fast `B` cuts:** split clips from the keyboard with `B`, matching common + editor muscle memory for quick cleanup. +- **Higher-quality phone camera capture:** webcam sidecar recording requests a + 4K 16:9 stream when available, with 1080p as the minimum target instead of + silently accepting low-resolution camera input. +- **Higher webcam recording bitrate:** camera footage is recorded at a + 4K-friendly bitrate, so detailed face-cam footage has more room before + compression artifacts show up. +- **No low-res preview trap:** the floating webcam preview no longer opens the + camera as a tiny square stream that can cause some camera sources to negotiate + low-quality output. +- **Cleaner original-quality exports:** the highest MP4 quality path keeps the + full source bitrate instead of reducing it below the source-quality budget. + +Use this build if you record creator education, product walkthroughs, +Loom-style videos, sales demos, or social clips where you need your face-cam, +script, cuts, and final export to work together. + --- ### Backed by the community diff --git a/docs/superpowers/plans/2026-06-10-camera-layout-switch.md b/docs/superpowers/plans/2026-06-10-camera-layout-switch.md new file mode 100644 index 000000000..d93f9134f --- /dev/null +++ b/docs/superpowers/plans/2026-06-10-camera-layout-switch.md @@ -0,0 +1,834 @@ +# In-Recording Camera/Screen Layout Switch Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A HUD button + global `Alt+F10` hotkey during recording stamps timestamped layout events; the editor converts them to camera-full segments that hard-cut the preview and export between "webcam letterboxed over background" and the normal screen+bubble layout. + +**Architecture:** The HUD renderer owns the toggle state and the pause-adjusted clock (`getRecordingDurationMs` in `useScreenRecorder`); each toggle sends `{timeMs, mode}` to the main process, which accumulates events and writes a sidecar JSON at finalize (cursor-telemetry pattern). The editor reads the sidecar into `WebcamLayoutRegion[]`, persists them in the project, and both the Pixi preview and the WebCodecs export renderer hide the screen layer + letterbox the webcam during active regions. The native static-layout export gains one skip reason. + +**Tech Stack:** Electron IPC + globalShortcut, React/TS, Pixi (preview), WebCodecs exporter, vitest, Biome (TABS). + +**Spec:** `docs/superpowers/specs/2026-06-10-camera-layout-switch-design.md` + +**Conventions:** npm from `/Users/justmaiko/PROJECTS/Mini Tools/RECORDLY`; git from `/Users/justmaiko/PROJECTS/Mini Tools`. Baseline: ONE pre-existing test failure (`electron/ipc/paths/binaries.test.ts`); tsc clean. New `t()` keys go to ALL 10 locales (launch.json for HUD strings, settings.json for the editor checkbox); `npm run i18n:check` must show no NEW failures. **Execute AFTER the presets+flicker plan** — Task 6 builds on the ref-based `applyWebcamBubbleLayout` from that plan. + +--- + +### Task 1: Pure regions + geometry module (TDD) + +**Files:** +- Create: `src/components/video-editor/webcamLayoutRegions.ts` +- Test: `src/components/video-editor/webcamLayoutRegions.test.ts` + +- [ ] **Step 1: Failing test** — `webcamLayoutRegions.test.ts`: + +```ts +import { describe, expect, it } from "vitest"; +import { + eventsToWebcamLayoutRegions, + getLetterboxRect, + isCameraFullAtMs, + type WebcamLayoutEvent, +} from "./webcamLayoutRegions"; + +describe("eventsToWebcamLayoutRegions", () => { + it("pairs camera-full/screen events into regions", () => { + const events: WebcamLayoutEvent[] = [ + { timeMs: 5000, mode: "camera-full" }, + { timeMs: 9000, mode: "screen" }, + { timeMs: 20000, mode: "camera-full" }, + { timeMs: 31000, mode: "screen" }, + ]; + const regions = eventsToWebcamLayoutRegions(events); + expect(regions.map((r) => [r.startMs, r.endMs])).toEqual([ + [5000, 9000], + [20000, 31000], + ]); + expect(regions[0].id).not.toBe(regions[1].id); + }); + + it("extends an unterminated camera-full segment to the end", () => { + const regions = eventsToWebcamLayoutRegions([{ timeMs: 5000, mode: "camera-full" }]); + expect(regions).toHaveLength(1); + expect(regions[0].endMs).toBe(Number.MAX_SAFE_INTEGER); + }); + + it("dedupes consecutive same-mode events and sorts by time", () => { + const regions = eventsToWebcamLayoutRegions([ + { timeMs: 9000, mode: "screen" }, + { timeMs: 5000, mode: "camera-full" }, + { timeMs: 6000, mode: "camera-full" }, + ]); + expect(regions.map((r) => [r.startMs, r.endMs])).toEqual([[5000, 9000]]); + }); + + it("drops zero-length segments and ignores leading screen events", () => { + expect( + eventsToWebcamLayoutRegions([ + { timeMs: 0, mode: "screen" }, + { timeMs: 5000, mode: "camera-full" }, + { timeMs: 5000, mode: "screen" }, + ]), + ).toEqual([]); + }); + + it("ignores invalid events", () => { + expect( + eventsToWebcamLayoutRegions([ + { timeMs: Number.NaN, mode: "camera-full" }, + { timeMs: -5, mode: "camera-full" }, + ]), + ).toEqual([]); + }); +}); + +describe("isCameraFullAtMs", () => { + const regions = eventsToWebcamLayoutRegions([ + { timeMs: 5000, mode: "camera-full" }, + { timeMs: 9000, mode: "screen" }, + ]); + it("is true inside and false outside (end exclusive)", () => { + expect(isCameraFullAtMs(regions, 4999)).toBe(false); + expect(isCameraFullAtMs(regions, 5000)).toBe(true); + expect(isCameraFullAtMs(regions, 8999)).toBe(true); + expect(isCameraFullAtMs(regions, 9000)).toBe(false); + }); +}); + +describe("getLetterboxRect", () => { + it("fits wide content into a taller frame, centered, with padding", () => { + const rect = getLetterboxRect({ width: 1600, height: 900 }, { width: 1000, height: 1000 }, 50); + // available 900x900; 16:9 fit -> 900x506.25 + expect(rect.width).toBeCloseTo(900); + expect(rect.height).toBeCloseTo(506.25); + expect(rect.x).toBeCloseTo(50); + expect(rect.y).toBeCloseTo((1000 - 506.25) / 2); + }); + + it("fits tall content into a wider frame", () => { + const rect = getLetterboxRect({ width: 900, height: 1600 }, { width: 1920, height: 1080 }, 0); + expect(rect.height).toBeCloseTo(1080); + expect(rect.width).toBeCloseTo(1080 * (900 / 1600)); + expect(rect.y).toBeCloseTo(0); + }); + + it("degrades safely on invalid input", () => { + const rect = getLetterboxRect({ width: 0, height: 0 }, { width: 1000, height: 500 }, 10); + expect(rect).toEqual({ x: 10, y: 10, width: 980, height: 480 }); + }); +}); +``` + +- [ ] **Step 2: Run to verify FAIL** (module not found). +- [ ] **Step 3: Implement** — `webcamLayoutRegions.ts`: + +```ts +export type WebcamLayoutMode = "screen" | "camera-full"; + +export interface WebcamLayoutEvent { + timeMs: number; + mode: WebcamLayoutMode; +} + +export interface WebcamLayoutRegion { + id: string; + startMs: number; + endMs: number; +} + +interface SizeLike { + width: number; + height: number; +} + +export interface LetterboxRect { + x: number; + y: number; + width: number; + height: number; +} + +function isValidEvent(event: WebcamLayoutEvent): boolean { + return ( + Number.isFinite(event.timeMs) && + event.timeMs >= 0 && + (event.mode === "screen" || event.mode === "camera-full") + ); +} + +/** + * Converts recording-time toggle events into camera-full regions. Recording + * starts in "screen" mode implicitly; an unterminated camera-full segment + * runs to MAX_SAFE_INTEGER (renderers clamp to the video duration). + */ +export function eventsToWebcamLayoutRegions(events: WebcamLayoutEvent[]): WebcamLayoutRegion[] { + const sorted = events.filter(isValidEvent).sort((a, b) => a.timeMs - b.timeMs); + const regions: WebcamLayoutRegion[] = []; + let openStartMs: number | null = null; + + for (const event of sorted) { + if (event.mode === "camera-full") { + if (openStartMs === null) { + openStartMs = event.timeMs; + } + } else if (openStartMs !== null) { + if (event.timeMs > openStartMs) { + regions.push({ + id: `webcam-layout-${openStartMs}-${event.timeMs}`, + startMs: openStartMs, + endMs: event.timeMs, + }); + } + openStartMs = null; + } + } + + if (openStartMs !== null) { + regions.push({ + id: `webcam-layout-${openStartMs}-end`, + startMs: openStartMs, + endMs: Number.MAX_SAFE_INTEGER, + }); + } + + return regions; +} + +export function isCameraFullAtMs(regions: WebcamLayoutRegion[], timeMs: number): boolean { + return regions.some((region) => timeMs >= region.startMs && timeMs < region.endMs); +} + +/** Largest content-aspect rect centered inside frame minus padding on all sides. */ +export function getLetterboxRect( + content: SizeLike, + frame: SizeLike, + paddingPx: number, +): LetterboxRect { + const availableWidth = Math.max(0, frame.width - paddingPx * 2); + const availableHeight = Math.max(0, frame.height - paddingPx * 2); + if ( + !Number.isFinite(content.width) || + !Number.isFinite(content.height) || + content.width <= 0 || + content.height <= 0 + ) { + return { x: paddingPx, y: paddingPx, width: availableWidth, height: availableHeight }; + } + + const scale = Math.min(availableWidth / content.width, availableHeight / content.height); + const width = content.width * scale; + const height = content.height * scale; + return { + x: paddingPx + (availableWidth - width) / 2, + y: paddingPx + (availableHeight - height) / 2, + width, + height, + }; +} +``` + +- [ ] **Step 4: Run to verify PASS**; lint. +- [ ] **Step 5: Commit** — `git commit -m "Add webcam layout regions and letterbox geometry primitives"` + +--- + +### Task 2: Main process — event log, sidecar write/read, Alt+F10 + +**Files:** +- Create: `electron/ipc/recording/webcamLayoutEvents.ts` +- Test: `electron/ipc/recording/webcamLayoutEvents.test.ts` +- Modify: `electron/main.ts` (the `onRecordingStateChange` callback registered ~lines 974-985) +- Modify: `electron/ipc/recording/mac.ts` (`finalizeStoredVideo`, after `persistPendingCursorTelemetry` at line ~226 — this function finalizes BOTH mac and windows recordings) +- Modify: `electron/ipc/register/recording.ts` (new `ipcMain` handlers, near `get-video-audio-fallback-paths` at line ~1387) +- Modify: `electron/teleprompterShortcuts.ts` OR a new sibling module for the Alt+F10 registration helper + +- [ ] **Step 1: Failing test** — `webcamLayoutEvents.test.ts` (use `vi.mock("node:fs/promises")` or a temp dir via `os.tmpdir()` — follow the style of `electron/ipc/recording/diagnostics.test.ts` which tests sidecar IO): + +```ts +import { describe, expect, it, beforeEach } from "vitest"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { + beginWebcamLayoutSession, + getWebcamLayoutEventsPath, + persistWebcamLayoutEvents, + readWebcamLayoutEvents, + recordWebcamLayoutEvent, +} from "./webcamLayoutEvents"; + +describe("webcam layout events session", () => { + let videoPath: string; + + beforeEach(async () => { + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "recordly-layout-")); + videoPath = path.join(dir, "recording.mp4"); + }); + + it("persists recorded events as a sidecar and reads them back", async () => { + beginWebcamLayoutSession(); + recordWebcamLayoutEvent({ timeMs: 5000, mode: "camera-full" }); + recordWebcamLayoutEvent({ timeMs: 9000, mode: "screen" }); + await persistWebcamLayoutEvents(videoPath); + + const raw = JSON.parse(await fs.readFile(getWebcamLayoutEventsPath(videoPath), "utf8")); + expect(raw.version).toBe(1); + expect(raw.events).toHaveLength(2); + + const read = await readWebcamLayoutEvents(videoPath); + expect(read).toEqual([ + { timeMs: 5000, mode: "camera-full" }, + { timeMs: 9000, mode: "screen" }, + ]); + }); + + it("writes nothing when no events were recorded", async () => { + beginWebcamLayoutSession(); + await persistWebcamLayoutEvents(videoPath); + await expect(fs.stat(getWebcamLayoutEventsPath(videoPath))).rejects.toThrow(); + }); + + it("ignores events outside a session and invalid payloads", async () => { + recordWebcamLayoutEvent({ timeMs: 5000, mode: "camera-full" }); // no session begun + beginWebcamLayoutSession(); + recordWebcamLayoutEvent({ timeMs: Number.NaN, mode: "camera-full" } as never); + recordWebcamLayoutEvent({ timeMs: 5, mode: "bogus" } as never); + await persistWebcamLayoutEvents(videoPath); + await expect(fs.stat(getWebcamLayoutEventsPath(videoPath))).rejects.toThrow(); + }); + + it("returns empty array for missing/corrupt sidecars", async () => { + expect(await readWebcamLayoutEvents(videoPath)).toEqual([]); + await fs.writeFile(getWebcamLayoutEventsPath(videoPath), "not json"); + expect(await readWebcamLayoutEvents(videoPath)).toEqual([]); + }); +}); +``` + +- [ ] **Step 2: Run to verify FAIL.** +- [ ] **Step 3: Implement** — `electron/ipc/recording/webcamLayoutEvents.ts`: + +```ts +import fs from "node:fs/promises"; + +export type WebcamLayoutMode = "screen" | "camera-full"; + +export interface WebcamLayoutEvent { + timeMs: number; + mode: WebcamLayoutMode; +} + +let sessionEvents: WebcamLayoutEvent[] | null = null; + +export function getWebcamLayoutEventsPath(videoPath: string): string { + return `${videoPath}.webcam-layout-events.json`; +} + +export function beginWebcamLayoutSession(): void { + sessionEvents = []; +} + +function isValidEvent(event: WebcamLayoutEvent): boolean { + return ( + Number.isFinite(event?.timeMs) && + event.timeMs >= 0 && + (event.mode === "screen" || event.mode === "camera-full") + ); +} + +export function recordWebcamLayoutEvent(event: WebcamLayoutEvent): void { + if (!sessionEvents || !isValidEvent(event)) { + return; + } + sessionEvents.push({ timeMs: Math.round(event.timeMs), mode: event.mode }); +} + +/** Writes the sidecar next to the final video and ends the session. No-op without events. */ +export async function persistWebcamLayoutEvents(videoPath: string): Promise { + const events = sessionEvents; + sessionEvents = null; + if (!events || events.length === 0) { + return; + } + try { + await fs.writeFile( + getWebcamLayoutEventsPath(videoPath), + JSON.stringify({ version: 1, events }), + "utf8", + ); + } catch (error) { + console.warn("[webcam-layout] Failed to persist layout events:", error); + } +} + +export async function readWebcamLayoutEvents(videoPath: string): Promise { + try { + const raw = await fs.readFile(getWebcamLayoutEventsPath(videoPath), "utf8"); + const parsed = JSON.parse(raw) as { events?: WebcamLayoutEvent[] }; + return Array.isArray(parsed.events) ? parsed.events.filter(isValidEvent) : []; + } catch { + return []; + } +} +``` + +- [ ] **Step 4: Run to verify PASS.** + +- [ ] **Step 5: Wire lifecycle + hotkey.** + +(a) In `electron/teleprompterShortcuts.ts` add (same warn-and-continue style as the existing functions): + +```ts +const CAMERA_LAYOUT_SHORTCUT = "Alt+F10"; + +export function registerCameraLayoutShortcut(onPressed: () => void): void { + try { + const registered = globalShortcut.register(CAMERA_LAYOUT_SHORTCUT, onPressed); + if (!registered) { + console.warn(`[camera-layout] Could not register global shortcut ${CAMERA_LAYOUT_SHORTCUT}`); + } + } catch (error) { + console.warn( + `[camera-layout] Could not register global shortcut ${CAMERA_LAYOUT_SHORTCUT}:`, + error, + ); + } +} + +export function unregisterCameraLayoutShortcut(): void { + try { + globalShortcut.unregister(CAMERA_LAYOUT_SHORTCUT); + } catch { + // Best effort. + } +} +``` + +(b) In `electron/main.ts`, locate the `onRecordingStateChange` callback (registered with `registerIpcHandlers`/recording registration around lines 974-985 — find it by grepping `onRecordingStateChange`). Verify it fires with `recording: true` on start and `false` on stop for the macOS native path (trace from `recording.ts` broadcast sites at ~845-849 and ~1845-1849; if start does NOT route through the callback, hook the same locations where `"recording-state-changed"` is broadcast). Add: + +```ts +// on recording start: +beginWebcamLayoutSession(); +registerCameraLayoutShortcut(() => { + getHudOverlayWindow()?.webContents.send("webcam-layout-hotkey"); +}); +// on recording stop: +unregisterCameraLayoutShortcut(); +``` + +with imports from `./ipc/recording/webcamLayoutEvents` and `./teleprompterShortcuts`; `getHudOverlayWindow` is already exported from `./windows` (check the existing import block). + +(c) In `electron/ipc/recording/mac.ts` `finalizeStoredVideo`, after `persistPendingCursorTelemetry(videoPath)` (line ~226, inside its try or as a sibling step): + +```ts + await persistWebcamLayoutEvents(videoPath); +``` + +import from `./webcamLayoutEvents`. + +(d) In `electron/ipc/register/recording.ts`, near `get-video-audio-fallback-paths` (~1387): + +```ts + ipcMain.on("webcam-layout-toggle", (_event, payload: { timeMs: number; mode: string }) => { + recordWebcamLayoutEvent({ + timeMs: payload?.timeMs, + mode: payload?.mode as WebcamLayoutMode, + }); + }); + + ipcMain.handle("get-webcam-layout-events", async (_event, videoPath: string) => { + if (!videoPath) { + return { success: true, events: [] }; + } + return { success: true, events: await readWebcamLayoutEvents(videoPath) }; + }); +``` + +- [ ] **Step 6: Verify** — `npx tsc --noEmit`, `npx vitest --run electron/` (only the 1 pre-existing failure), biome on touched files. +- [ ] **Step 7: Commit** — `git commit -m "Record webcam layout toggle events and persist sidecar at finalize"` + +--- + +### Task 3: Preload bridge + types + +**Files:** +- Modify: `electron/preload.ts` (next to the teleprompter methods, ~lines 185-205) +- Modify: `electron/electron-env.d.ts` (next to the teleprompter types) + +- [ ] **Step 1: Preload additions:** + +```ts + webcamLayoutToggle: (payload: { timeMs: number; mode: "screen" | "camera-full" }) => { + ipcRenderer.send("webcam-layout-toggle", payload); + }, + onWebcamLayoutHotkey: (callback: () => void) => { + const listener = () => { + callback(); + }; + ipcRenderer.on("webcam-layout-hotkey", listener); + return () => { + ipcRenderer.removeListener("webcam-layout-hotkey", listener); + }; + }, + getWebcamLayoutEvents: (videoPath: string) => { + return ipcRenderer.invoke("get-webcam-layout-events", videoPath) as Promise<{ + success: boolean; + events: Array<{ timeMs: number; mode: "screen" | "camera-full" }>; + }>; + }, +``` + +- [ ] **Step 2: Types** in `electron-env.d.ts`: + +```ts + webcamLayoutToggle: (payload: { timeMs: number; mode: "screen" | "camera-full" }) => void; + onWebcamLayoutHotkey: (callback: () => void) => () => void; + getWebcamLayoutEvents: (videoPath: string) => Promise<{ + success: boolean; + events: Array<{ timeMs: number; mode: "screen" | "camera-full" }>; + }>; +``` + +- [ ] **Step 3: Verify** tsc + biome. **Commit** — `git commit -m "Expose webcam layout IPC through preload bridge"` + +--- + +### Task 4: HUD — toggle state, button, hotkey listener + +**Files:** +- Modify: `src/hooks/useScreenRecorder.ts` (state near line 333; reset where recording starts; expose in the return at ~2123) +- Modify: `src/components/launch/LaunchWindow.tsx` (props at ~199-211 + hotkey effect) +- Modify: `src/components/launch/RecordingControls.tsx` (props 8-18, button after the mic button at ~61-76) +- Modify: all 10 `src/i18n/locales/*/launch.json` (`recording` section keys) + +- [ ] **Step 1: Hook state.** In `useScreenRecorder.ts`: + +```ts + const [cameraFullActive, setCameraFullActive] = useState(false); + const cameraFullActiveRef = useRef(false); +``` + +Find where recording actually begins (where `startTime.current` is set / recording state flips true) and reset there: + +```ts + cameraFullActiveRef.current = false; + setCameraFullActive(false); +``` + +Add the toggle (near `getRecordingDurationMs`, lines ~456-463; determine the hook's actual recording-active flag name by reading the hook — LaunchWindow consumes `paused`/`elapsed`, so a boolean like `recording`/`isRecording` exists; use it): + +```ts + const toggleCameraLayout = useCallback(() => { + if (!recordingRefOrState || !webcamEnabled) { + return; + } + const timeMs = Math.round(getRecordingDurationMs(Date.now())); + const next = !cameraFullActiveRef.current; + cameraFullActiveRef.current = next; + setCameraFullActive(next); + window.electronAPI?.webcamLayoutToggle?.({ + timeMs, + mode: next ? "camera-full" : "screen", + }); + }, [getRecordingDurationMs, webcamEnabled /* + the recording flag */]); +``` + +(`recordingRefOrState` is a placeholder NAME ONLY for the hook's existing recording-active flag — substitute the real one; everything else is literal.) Export `cameraFullActive` and `toggleCameraLayout` from the hook's return object (~line 2123) and its TS interface (~line 148). + +- [ ] **Step 2: LaunchWindow.** Destructure `cameraFullActive, toggleCameraLayout` from `useScreenRecorder()` (lines 59-81). Add the hotkey listener effect: + +```ts + useEffect(() => { + const unsubscribe = window.electronAPI?.onWebcamLayoutHotkey?.(() => { + toggleCameraLayout(); + }); + return unsubscribe; + }, [toggleCameraLayout]); +``` + +Pass to RecordingControls (lines 199-211): + +```tsx + webcamEnabled={webcamEnabled} + cameraFullActive={cameraFullActive} + onToggleCameraLayout={toggleCameraLayout} +``` + +- [ ] **Step 3: RecordingControls button.** Extend the props interface (lines 8-18): + +```ts + webcamEnabled: boolean; + cameraFullActive: boolean; + onToggleCameraLayout: () => void; +``` + +After the microphone button block (~line 76), add (icons: import `MonitorIcon` and `UserSquareIcon` from `@phosphor-icons/react` — verify they exist in `node_modules/@phosphor-icons/react/dist/index.d.ts`; fall back to `MonitorIcon`/`VideoCameraIcon` if `UserSquareIcon` is missing): + +```tsx + {webcamEnabled && ( + + )} +``` + +Match the exact Button size/className conventions used by the neighboring buttons in this file (read them; adjust `size`, icon size, and classes to match). + +- [ ] **Step 4: i18n** — add to the `recording` object in all 10 `launch.json` locales: + +en: `"cameraLayoutToCameraFull": "Camera fullscreen", "cameraLayoutToScreen": "Back to screen"` +es: `"cameraLayoutToCameraFull": "Cámara a pantalla completa", "cameraLayoutToScreen": "Volver a la pantalla"` +fr: `"cameraLayoutToCameraFull": "Caméra plein écran", "cameraLayoutToScreen": "Retour à l'écran"` +it: `"cameraLayoutToCameraFull": "Fotocamera a schermo intero", "cameraLayoutToScreen": "Torna allo schermo"` +ko: `"cameraLayoutToCameraFull": "카메라 전체 화면", "cameraLayoutToScreen": "화면으로 돌아가기"` +nl: `"cameraLayoutToCameraFull": "Camera volledig scherm", "cameraLayoutToScreen": "Terug naar scherm"` +pt-BR: `"cameraLayoutToCameraFull": "Câmera em tela cheia", "cameraLayoutToScreen": "Voltar para a tela"` +ru: `"cameraLayoutToCameraFull": "Камера на весь экран", "cameraLayoutToScreen": "Вернуться к экрану"` +zh-CN: `"cameraLayoutToCameraFull": "摄像头全屏", "cameraLayoutToScreen": "返回屏幕"` +zh-TW: `"cameraLayoutToCameraFull": "攝影機全螢幕", "cameraLayoutToScreen": "返回螢幕"` + +- [ ] **Step 5: Verify** — tsc, biome, `npm run i18n:check` (no NEW failures), full `npx vitest --run src/` quick pass. +- [ ] **Step 6: Commit** — `git commit -m "Add camera layout toggle to recording HUD with Alt+F10 relay"` + +--- + +### Task 5: Editor — types, sidecar load, persistence, checkbox + +**Files:** +- Modify: `src/components/video-editor/types.ts` (re-export `WebcamLayoutRegion` from `./webcamLayoutRegions` or import where needed) +- Modify: `src/components/video-editor/projectPersistence.ts` (add `webcamLayoutRegions` + `webcamLayoutRegionsEnabled` to `ProjectEditorState` + `normalizeProjectEditor` with defaults `[]` / `true`; follow exactly how `zoomRegions` or `annotationRegions` are normalized/persisted there) +- Modify: `src/components/video-editor/VideoEditor.tsx` (state; sidecar load near the cursor-telemetry load at ~3060-3127; project open at ~1988; include in project save; pass into export config and VideoPlayback) +- Modify: `src/components/video-editor/SettingsPanel.tsx` (checkbox in the webcam section) +- Modify: all 10 `src/i18n/locales/*/settings.json` (`effects` section key) +- Test: extend `src/components/video-editor/projectPersistence` tests if a test file exists for `normalizeProjectEditor` (check `projectPersistence` imports in existing `*.test.ts` — `editorPreferences.test.ts` exercises `normalizeProjectEditor` indirectly); at minimum add a round-trip normalization test. + +- [ ] **Step 1: State + persistence.** Add to `ProjectEditorState` (in `projectPersistence.ts` or `types.ts` — wherever the interface lives; grep `interface ProjectEditorState`): + +```ts + webcamLayoutRegions: WebcamLayoutRegion[]; + webcamLayoutRegionsEnabled: boolean; +``` + +In `normalizeProjectEditor`, normalize: array items require finite `startMs < endMs` and a string `id` (coerce/regenerate ids if missing); default `[]`; `webcamLayoutRegionsEnabled` boolean default `true`. Write a normalization test: + +```ts + it("normalizes webcam layout regions with defaults", () => { + const normalized = normalizeProjectEditor({}); + expect(normalized.webcamLayoutRegions).toEqual([]); + expect(normalized.webcamLayoutRegionsEnabled).toBe(true); + + const withRegions = normalizeProjectEditor({ + webcamLayoutRegions: [ + { id: "a", startMs: 1000, endMs: 2000 }, + { id: "bad", startMs: 5, endMs: 5 }, + ], + webcamLayoutRegionsEnabled: false, + }); + expect(withRegions.webcamLayoutRegions).toEqual([{ id: "a", startMs: 1000, endMs: 2000 }]); + expect(withRegions.webcamLayoutRegionsEnabled).toBe(false); + }); +``` + +(place it in the existing test file that covers `normalizeProjectEditor`; if none covers it directly, create `src/components/video-editor/projectPersistence.webcamLayout.test.ts` with the imports it needs.) + +- [ ] **Step 2: VideoEditor state + load.** Add state: + +```ts + const [webcamLayoutRegions, setWebcamLayoutRegions] = useState([]); + const [webcamLayoutRegionsEnabled, setWebcamLayoutRegionsEnabled] = useState(true); +``` + +Project open (~line 1988 where `setWebcam(normalizedEditor.webcam)` runs): also `setWebcamLayoutRegions(normalizedEditor.webcamLayoutRegions)` and `setWebcamLayoutRegionsEnabled(normalizedEditor.webcamLayoutRegionsEnabled)`. Project save: include both fields wherever the editor state object is assembled for persistence (grep the save payload that includes `zoomRegions`). + +Sidecar load for fresh recordings — next to the cursor telemetry loader (~3060-3127), same guard style: + +```ts + useEffect(() => { + let cancelled = false; + if (!videoSourcePath) { + return; + } + void (async () => { + try { + const result = await window.electronAPI.getWebcamLayoutEvents?.(videoSourcePath); + if (cancelled || !result?.success || result.events.length === 0) { + return; + } + setWebcamLayoutRegions((current) => + current.length > 0 ? current : eventsToWebcamLayoutRegions(result.events), + ); + } catch (error) { + console.warn("Failed to load webcam layout events:", error); + } + })(); + return () => { + cancelled = true; + }; + }, [videoSourcePath]); +``` + +(The `current.length > 0 ? current : ...` guard prevents clobbering regions already loaded from a saved project.) + +- [ ] **Step 3: Checkbox.** In `SettingsPanel.tsx`'s webcam section (near the custom-position toggle — grep `Custom position` / `customPosition` for the exact toggle-row pattern used there), add an identical toggle row, shown only when `webcamLayoutRegions.length > 0` (pass the count or a boolean prop down from VideoEditor along with value+setter): + +label: `tSettings("effects.webcamUseRecordedSwitches", "Use recorded camera switches")` + +Wire `webcamLayoutRegionsEnabled` + `setWebcamLayoutRegionsEnabled` through SettingsPanel props following how other webcam toggles flow (read the component's props interface and VideoEditor call site; add three props: `webcamLayoutRegionsAvailable: boolean`, `webcamLayoutRegionsEnabled: boolean`, `onWebcamLayoutRegionsEnabledChange: (enabled: boolean) => void`). + +i18n `settings.json` all 10 locales, inside the `effects` object: +en: `"webcamUseRecordedSwitches": "Use recorded camera switches"` +es: `"webcamUseRecordedSwitches": "Usar cambios de cámara grabados"` +fr: `"webcamUseRecordedSwitches": "Utiliser les changements de caméra enregistrés"` +it: `"webcamUseRecordedSwitches": "Usa i cambi di fotocamera registrati"` +ko: `"webcamUseRecordedSwitches": "녹화된 카메라 전환 사용"` +nl: `"webcamUseRecordedSwitches": "Opgenomen camerawissels gebruiken"` +pt-BR: `"webcamUseRecordedSwitches": "Usar trocas de câmera gravadas"` +ru: `"webcamUseRecordedSwitches": "Использовать записанные переключения камеры"` +zh-CN: `"webcamUseRecordedSwitches": "使用录制的摄像头切换"` +zh-TW: `"webcamUseRecordedSwitches": "使用錄製的攝影機切換"` + +- [ ] **Step 4: Effective regions plumbed onward.** In VideoEditor: + +```ts + const effectiveWebcamLayoutRegions = useMemo( + () => (webcamLayoutRegionsEnabled ? webcamLayoutRegions : []), + [webcamLayoutRegions, webcamLayoutRegionsEnabled], + ); +``` + +Pass `webcamLayoutRegions={effectiveWebcamLayoutRegions}` to `` (call site ~line 5100-5157) and add `webcamLayoutRegions: effectiveWebcamLayoutRegions` to the export config object (grep where `zoomRegions` enters the exporter config). VideoPlayback prop typing comes in Task 6 — to keep this task compiling, do Task 6's prop addition signature first if needed, or land Tasks 5+6 in one commit; prefer adding the prop as part of this task with a no-op default `[]` in VideoPlayback and the rendering logic in Task 6. + +- [ ] **Step 5: Verify** — tsc, biome, vitest video-editor suite, i18n:check (no NEW failures). +- [ ] **Step 6: Commit** — `git commit -m "Load, persist, and gate webcam layout regions in the editor"` + +--- + +### Task 6: Preview rendering (camera-full in VideoPlayback) + +**Files:** +- Modify: `src/components/video-editor/VideoPlayback.tsx` + +Builds on the presets+flicker plan's ref-based `applyWebcamBubbleLayout`. + +- [ ] **Step 1: Investigate before coding.** Confirm in code: `cameraContainerRef` (line ~490, created ~1950) holds ONLY the screen video content (not the wallpaper background); toggling `cameraContainerRef.current.visible = false` leaves the background visible. Identify where `currentTime` updates reach the component (`currentTimeRef` effect ~1659-1667). If background lives inside the same container, find the screen-content child (sprite/mask) to hide instead, and report the deviation in the commit message. + +- [ ] **Step 2: Implement.** + +(a) Add prop `webcamLayoutRegions?: WebcamLayoutRegion[]` (default `[]`) to the component's props (interface near line 331's `currentTime`). + +(b) Mirror into a ref (alongside the existing webcam input refs from the flicker fix): + +```ts + const webcamLayoutRegionsRef = useRef([]); + webcamLayoutRegionsRef.current = webcamLayoutRegions ?? []; + const cameraFullActiveRef = useRef(false); +``` + +(c) Add an updater that recomputes the active mode and applies visibility + bubble layout; call it whenever currentTime changes (in the existing `currentTimeRef` update effect ~1659) and whenever the regions prop changes (small effect): + +```ts + const updateWebcamLayoutMode = useCallback(() => { + const active = isCameraFullAtMs(webcamLayoutRegionsRef.current, currentTimeRef.current); + if (cameraFullActiveRef.current === active) { + return; + } + cameraFullActiveRef.current = active; + const cameraContainer = cameraContainerRef.current; + if (cameraContainer) { + cameraContainer.visible = !active; + } + applyWebcamBubbleLayout(animationStateRef.current.appliedScale || 1); + }, [applyWebcamBubbleLayout]); +``` + +(d) In `applyWebcamBubbleLayout`, branch on `cameraFullActiveRef.current`: when active, instead of `getWebcamOverlaySizePx`/`getWebcamOverlayPosition`, compute the letterbox rect from the CROPPED webcam aspect and the overlay size: + +```ts + if (cameraFullActiveRef.current) { + const crop = normalizeWebcamCropRegion(webcamCropRegion); // or however crop is accessed; the cropped aspect is (crop.width * videoW) / (crop.height * videoH) + const videoDims = webcamVideoDimensionsRef.current; // mirror webcamVideoDimensions into a ref like the other inputs + const contentAspectWidth = videoDims ? crop.width * videoDims.width : 16; + const contentAspectHeight = videoDims ? crop.height * videoDims.height : 9; + const rect = getLetterboxRect( + { width: contentAspectWidth, height: contentAspectHeight }, + { width: overlay.clientWidth, height: overlay.clientHeight }, + Math.min(overlay.clientWidth, overlay.clientHeight) * 0.04, + ); + bubble.style.display = "block"; + bubble.style.left = `${rect.x}px`; + bubble.style.top = `${rect.y}px`; + bubble.style.width = `${rect.width}px`; + bubble.style.height = `${rect.height}px`; + bubble.style.aspectRatio = "auto"; + // keep the existing squircle/shadow styling, recomputed for rect dims + ... + return; + } +``` + +The existing inner crop "cover" math fills the bubble with the cropped region; with the bubble at exactly the cropped aspect, cover === fit, so the whole (cropped) camera frame is visible. Keep the squircle clip + drop-shadow code paths but compute `getSquircleSvgPath` with `width: rect.width, height: rect.height` and a radius matching the screen frame's `borderRadius` value if accessible via ref, else reuse `webcamCornerRadius`. + +(e) The bubble is square only in bubble mode — make sure the non-camera-full branch restores `aspectRatio: "1 / 1"` (it already sets it each call). + +- [ ] **Step 3: Manual verification.** `npm run dev`, record a short clip with webcam + a couple of toggles (HUD button + Alt+F10), open in editor: scrubbing/playback shows hard cuts — camera letterboxed over background, screen hidden — exactly at press points; checkbox off restores normal layout everywhere; zoom regions still behave during screen segments. +- [ ] **Step 4: tsc/biome/vitest** (no new failures). **Commit** — `git commit -m "Render camera-full layout segments in editor preview"` + +--- + +### Task 7: Export rendering + native skip reason + +**Files:** +- Modify: `src/lib/exporter/modernFrameRenderer.ts` (config interface ~98-162; webcam compositing ~2420-2523; screen/zoom draw ~2102-2130) +- Modify: `src/lib/exporter/modernVideoExporter.ts` (config pass-through ~590-612; `getNativeStaticLayoutSkipReasons` ~1498-1582) +- Test: extend the existing exporter test that covers skip reasons (grep `unsupported-annotation-overlay` in `src/lib/exporter/*.test.ts` to find it) + +- [ ] **Step 1: Config plumbing.** Add `webcamLayoutRegions?: WebcamLayoutRegion[]` to `FrameRenderConfig` (modernFrameRenderer.ts) and the exporter config (modernVideoExporter.ts), passing it through where `zoomRegions`/`webcam` flow (~590-612). Import the type + `isCameraFullAtMs` + `getLetterboxRect` from `@/components/video-editor/webcamLayoutRegions`. + +- [ ] **Step 2: Skip reason (TDD).** Find the existing test exercising `getNativeStaticLayoutSkipReasons` outputs (grep in `src/lib/exporter/*.test.ts` for `unsupported-annotation-overlay` or `nativeStaticLayoutSkipReasons`); add a case asserting that a config with one webcam layout region yields `"unsupported-webcam-layout-regions"`. Then implement in `getNativeStaticLayoutSkipReasons` (next to the annotation check at ~1553): + +```ts + if ((this.config.webcamLayoutRegions ?? []).length > 0) { + reasons.push("unsupported-webcam-layout-regions"); + } +``` + +If no directly-callable test exists for skip reasons (the method is private), verify via the existing test pattern for that file — read how `modernVideoExporter.nativeStaticLayout.test.ts` exercises decisions and extend it; if it's genuinely untestable without large scaffolding, document that in the commit and rely on the route-decision log check in Step 4. + +- [ ] **Step 3: Per-frame camera-full rendering.** Investigate the per-frame render path in modernFrameRenderer.ts (the method that applies zoom transforms ~2102-2130 and positions the webcam container/sprite). Implement: for each frame at `frameTimeMs`, if `isCameraFullAtMs(this.config.webcamLayoutRegions ?? [], frameTimeMs)`: + - hide the screen content container for that frame (set `.visible = false` on the container that holds the screen video — identify the exact member: the renderer mirrors VideoPlayback's structure; grep `cameraContainer` / the container that gets the zoom transform); + - position the webcam container/sprite to the letterbox rect: content aspect = cropped webcam aspect (the crop source rect is computed in `refreshWebcamFrameCache`/`getWebcamCropSourceRect` ~2491-2523), frame = output `this.config.width/height`, padding = `Math.min(width, height) * 0.04` (same constant as preview — extract it as `CAMERA_FULL_PADDING_FRACTION = 0.04` exported from `webcamLayoutRegions.ts` and use it in BOTH preview and export); + - keep the webcam's corner-radius mask + shadow consistent with bubble mode (reuse the existing mask graphics with the new rect). + Else restore visibility + normal webcam placement. Make the visibility/placement assignment unconditional per frame (no stateful flicker between segments). + +- [ ] **Step 4: Verification.** + - `npx vitest --run src/lib/exporter` — no new failures, new skip-reason test passes. + - Manual: export the Task 6 test project to MP4 (default settings, macOS → confirm in the export logs/route decisions that the native static layout was rejected with `unsupported-webcam-layout-regions` and the WebCodecs/breeze path ran); play the MP4 — cuts match the preview, camera letterboxed over background, no screen content during camera-full segments, audio continuous. + - Export the same project with the checkbox OFF — normal layout throughout, and native static layout is eligible again (no skip reason). +- [ ] **Step 5: Commit** — `git commit -m "Composite camera-full layout segments in export with native-path skip reason"` + +--- + +### Task 8: Final verification + +- [ ] **Step 1:** `npm test` (only the 1 pre-existing failure), `npx tsc --noEmit` clean, `npm run i18n:check` (no NEW failures), biome on all touched files. +- [ ] **Step 2:** End-to-end manual: record with webcam, press the HUD button twice and Alt+F10 once (three toggles), pause/resume between toggles, stop → editor opens → segments land at press points (pause-adjusted); export → MP4 matches; checkbox off → disabled everywhere; project save/reopen → regions persist. +- [ ] **Step 3:** Commit any verification fixes. diff --git a/docs/superpowers/plans/2026-06-10-camera-track-and-style.md b/docs/superpowers/plans/2026-06-10-camera-track-and-style.md new file mode 100644 index 000000000..1ffc5f739 --- /dev/null +++ b/docs/superpowers/plans/2026-06-10-camera-track-and-style.md @@ -0,0 +1,317 @@ +# Timeline Camera Track + Facecam Style Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Show camera-full segments as an editable track on the timeline (drag edges, move, delete, double-click-to-add), and add a fit/fill facecam style chosen in the HUD before recording and changeable in the editor. + +**Architecture:** The camera track copies the zoom-row pattern (row constant → `buildTimelineItems` entry → dnd bindings kind → selection/delete wiring → VideoEditor props). The style is a `"fit" | "fill"` value carried in the layout-events sidecar (v2), normalized into project state, and branched on in both renderers via a new `getCoverRect` pure function. + +**Tech Stack:** React/TS, dnd-timeline, Pixi preview, WebCodecs exporter, vitest, Biome (TABS). + +**Spec:** `docs/superpowers/specs/2026-06-10-camera-track-and-style-design.md` + +**Conventions:** npm from `/Users/justmaiko/PROJECTS/Mini Tools/RECORDLY`; git from `/Users/justmaiko/PROJECTS/Mini Tools`. Baseline: ONE pre-existing test failure (`electron/ipc/paths/binaries.test.ts`); tsc CLEAN. i18n keys must go to ALL 10 locales; `npm run i18n:check` may not gain new failures. Execute BEFORE the magnet/gaps plan (its display mapping wraps this track too). + +--- + +### Task 1: Geometry + segment-editing pure functions (TDD) + +**Files:** +- Modify: `src/components/video-editor/webcamLayoutRegions.ts` +- Test: `src/components/video-editor/webcamLayoutRegions.test.ts` (extend) + +- [ ] **Step 1: Failing tests** — append to the existing test file: + +```ts +describe("getCoverRect", () => { + it("covers the frame, cropping the long axis, centered", () => { + const rect = getCoverRect({ width: 1600, height: 900 }, { width: 1000, height: 1000 }); + // scale = max(1000/1600, 1000/900) = 1.111... -> 1777.8 x 1000 + expect(rect.height).toBeCloseTo(1000); + expect(rect.width).toBeCloseTo(1000 * (1600 / 900)); + expect(rect.x).toBeCloseTo((1000 - 1000 * (1600 / 900)) / 2); + expect(rect.y).toBeCloseTo(0); + }); + + it("degrades safely on invalid content", () => { + expect(getCoverRect({ width: 0, height: 0 }, { width: 1000, height: 500 })).toEqual({ + x: 0, + y: 0, + width: 1000, + height: 500, + }); + }); +}); + +describe("clampWebcamLayoutSpan", () => { + const others = [ + { id: "a", startMs: 1000, endMs: 2000 }, + { id: "b", startMs: 5000, endMs: 6000 }, + ]; + + it("clamps to neighbors and duration", () => { + expect( + clampWebcamLayoutSpan({ startMs: 1500, endMs: 5500 }, others, "x", 10000), + ).toEqual({ startMs: 2000, endMs: 5000 }); + expect(clampWebcamLayoutSpan({ startMs: -50, endMs: 800 }, others, "x", 10000)).toEqual({ + startMs: 0, + endMs: 800, + }); + expect( + clampWebcamLayoutSpan({ startMs: 9500, endMs: 12000 }, others, "x", 10000), + ).toEqual({ startMs: 9500, endMs: 10000 }); + }); + + it("ignores the region's own id and enforces minimum length", () => { + expect( + clampWebcamLayoutSpan({ startMs: 1000, endMs: 1010 }, others, "a", 10000), + ).toEqual({ startMs: 1000, endMs: 1100 }); + expect(clampWebcamLayoutSpan({ startMs: 3000, endMs: 3010 }, others, "x", 10000)).toEqual( + { startMs: 3000, endMs: 3100 }, + ); + }); + + it("returns null when no valid placement exists", () => { + expect( + clampWebcamLayoutSpan({ startMs: 1200, endMs: 1300 }, others, "x", 10000), + ).toBeNull(); + }); +}); + +describe("normalizeWebcamLayoutStyle", () => { + it("accepts fit/fill and falls back to fit", () => { + expect(normalizeWebcamLayoutStyle("fill")).toBe("fill"); + expect(normalizeWebcamLayoutStyle("fit")).toBe("fit"); + expect(normalizeWebcamLayoutStyle("bogus")).toBe("fit"); + expect(normalizeWebcamLayoutStyle(undefined)).toBe("fit"); + }); +}); +``` + +- [ ] **Step 2: Run to verify FAIL** — `npx vitest --run src/components/video-editor/webcamLayoutRegions.test.ts`. +- [ ] **Step 3: Implement** — add to `webcamLayoutRegions.ts`: + +```ts +export type WebcamLayoutStyle = "fit" | "fill"; + +export const MIN_WEBCAM_LAYOUT_REGION_MS = 100; + +export function normalizeWebcamLayoutStyle(value: unknown): WebcamLayoutStyle { + return value === "fill" ? "fill" : "fit"; +} + +/** Largest content-aspect rect that fully covers the frame, centered (crops overflow). */ +export function getCoverRect(content: SizeLike, frame: SizeLike): LetterboxRect { + if ( + !Number.isFinite(content.width) || + !Number.isFinite(content.height) || + content.width <= 0 || + content.height <= 0 + ) { + return { x: 0, y: 0, width: frame.width, height: frame.height }; + } + const scale = Math.max(frame.width / content.width, frame.height / content.height); + const width = content.width * scale; + const height = content.height * scale; + return { + x: (frame.width - width) / 2, + y: (frame.height - height) / 2, + width, + height, + }; +} + +/** + * Clamps a dragged/resized camera segment span against its neighbors and the + * video duration. Returns null when the span cannot fit anywhere valid. + */ +export function clampWebcamLayoutSpan( + span: { startMs: number; endMs: number }, + regions: WebcamLayoutRegion[], + ownId: string, + durationMs: number, +): { startMs: number; endMs: number } | null { + const others = regions + .filter((region) => region.id !== ownId) + .sort((a, b) => a.startMs - b.startMs); + + let startMs = Math.max(0, Math.round(span.startMs)); + let endMs = Math.min(durationMs, Math.round(span.endMs)); + + for (const other of others) { + // Clamp the span out of each overlapping neighbor, preferring the side + // the span already leans toward. + const overlaps = startMs < other.endMs && endMs > other.startMs; + if (!overlaps) continue; + if (startMs >= other.startMs) { + startMs = Math.max(startMs, other.endMs); + } else { + endMs = Math.min(endMs, other.startMs); + } + } + + if (endMs - startMs < MIN_WEBCAM_LAYOUT_REGION_MS) { + endMs = startMs + MIN_WEBCAM_LAYOUT_REGION_MS; + if (endMs > durationMs) return null; + // Re-check the stretched span against neighbors. + for (const other of others) { + if (startMs < other.endMs && endMs > other.startMs) return null; + } + } + + return startMs < endMs ? { startMs, endMs } : null; +} +``` + +- [ ] **Step 4: Run to verify PASS**; lint. Adjust the clamp implementation if any listed expectation fails — the TESTS are the contract. +- [ ] **Step 5: Commit** — `git commit -m "Add cover-rect geometry and camera segment clamping primitives"` + +--- + +### Task 2: Style end-to-end plumbing (sidecar v2 + HUD popover + editor state) + +**Files:** +- Modify: `electron/ipc/recording/webcamLayoutEvents.ts` + its test (style in sidecar) +- Modify: `electron/windows.ts` (cache style next to `selectedWebcamDeviceId`), `electron/preload.ts`, `electron/electron-env.d.ts` +- Modify: `src/components/launch/popovers/WebcamPopover.tsx`, `src/components/launch/LaunchWindow.tsx` +- Modify: `src/components/video-editor/projectPersistence.ts` (+ its webcamLayout test), `src/components/video-editor/VideoEditor.tsx`, `src/components/video-editor/SettingsPanel.tsx` +- Modify: all 10 `src/i18n/locales/*/launch.json` and `*/settings.json` + +- [ ] **Step 1: Sidecar v2 (TDD).** Failing tests first in `electron/ipc/recording/webcamLayoutEvents.test.ts`: + +```ts + it("persists the layout style and reads it back, defaulting v1 sidecars to fit", async () => { + beginWebcamLayoutSession(); + setWebcamLayoutSessionStyle("fill"); + recordWebcamLayoutEvent({ timeMs: 5000, mode: "camera-full" }); + await persistWebcamLayoutEvents(videoPath); + + const raw = JSON.parse(await fs.readFile(getWebcamLayoutEventsPath(videoPath), "utf8")); + expect(raw.version).toBe(2); + expect(raw.style).toBe("fill"); + + const read = await readWebcamLayoutSidecar(videoPath); + expect(read.style).toBe("fill"); + expect(read.events).toHaveLength(1); + + // v1 compatibility + await fs.writeFile( + getWebcamLayoutEventsPath(videoPath), + JSON.stringify({ version: 1, events: [{ timeMs: 1, mode: "camera-full" }] }), + ); + const v1 = await readWebcamLayoutSidecar(videoPath); + expect(v1.style).toBe("fit"); + expect(v1.events).toHaveLength(1); + }); +``` + +Implement in `webcamLayoutEvents.ts`: module state `sessionStyle: "fit" | "fill" = "fit"` reset in `beginWebcamLayoutSession()`; `export function setWebcamLayoutSessionStyle(style)` (validate, default fit); persist writes `{ version: 2, style: sessionStyle, events }`; add `readWebcamLayoutSidecar(videoPath): Promise<{ style: "fit" | "fill"; events: WebcamLayoutEvent[] }>` (keep the existing `readWebcamLayoutEvents` delegating to it for compatibility). Update the existing IPC handler `get-webcam-layout-events` in `electron/ipc/register/recording.ts` to return `{ success, style, events }` via the new reader. + +- [ ] **Step 2: HUD style choice.** In `electron/windows.ts` next to the `webcam-device-changed` handler: `let selectedWebcamLayoutStyle = "fit";` + `ipcMain.on("webcam-layout-style-changed", ...)` caching a validated value, and have the recording-start path apply it: in `electron/main.ts` where `beginWebcamLayoutSession()` is called, also call `setWebcamLayoutSessionStyle(getSelectedWebcamLayoutStyle())` (export a getter from windows.ts; import the setter from webcamLayoutEvents). Preload + env.d.ts: `webcamLayoutStyleChanged(style: "fit" | "fill"): void`. In `LaunchWindow.tsx`: persist the choice in localStorage (`recordly-webcam-layout-style`), push on mount + change (same `useEffect` pattern as `webcamDeviceChanged`). In `WebcamPopover.tsx`: a two-item style selector after the floating-preview item — two `DropdownItem`s (selected state per current style) labeled `t("recording.webcamStyleFit", "Camera fullscreen: fit with background")` and `t("recording.webcamStyleFill", "Camera fullscreen: fill screen")`, wired via two new props (`webcamLayoutStyle`, `onWebcamLayoutStyleChange`) passed from LaunchWindow. i18n keys in all 10 `launch.json` (translate naturally; en values as above). + +- [ ] **Step 3: Editor state.** `projectPersistence.ts`: add `webcamLayoutStyle: WebcamLayoutStyle` to `ProjectEditorState`, normalized via `normalizeWebcamLayoutStyle` (default fit) — extend `projectPersistence.webcamLayout.test.ts` first (failing): + +```ts + it("normalizes webcam layout style", () => { + expect(normalizeProjectEditor({}).webcamLayoutStyle).toBe("fit"); + expect(normalizeProjectEditor({ webcamLayoutStyle: "fill" }).webcamLayoutStyle).toBe("fill"); + expect(normalizeProjectEditor({ webcamLayoutStyle: "junk" }).webcamLayoutStyle).toBe("fit"); + }); +``` + +`VideoEditor.tsx`: state `webcamLayoutStyle` (default "fit"); set from `normalizedEditor` on project open; include in the persisted-state payload (same funnel as `webcamLayoutRegions` — `buildPersistedEditorState` + `currentPersistedEditorState` memo + deps); seed from the sidecar on fresh load (the existing sidecar-load effect now receives `style` — apply it with the same only-if-unset spirit: track whether the project provided one; simplest correct rule: apply sidecar style only when also seeding regions from the sidecar, i.e. inside the same `current.length > 0 ? current : ...` branch — restructure that setter into an explicit `if` so both states update together). Pass to `` and into the export config (`webcamLayoutStyle` field added to `FrameRenderConfig` + `VideoExporterConfig` + pass-through, like `webcamLayoutRegions` was). `SettingsPanel.tsx`: next to the "Use recorded camera switches" row, a small two-option segmented/select control bound to three new props (`webcamLayoutStyle`, `onWebcamLayoutStyleChange`, visible under the same `webcamLayoutRegionsAvailable` gate) with label `tSettings("effects.webcamLayoutStyle", "Camera fullscreen style")` and option labels `tSettings("effects.webcamLayoutStyleFit", "Fit with background")` / `tSettings("effects.webcamLayoutStyleFill", "Fill screen")` — copy whichever two-option control pattern already exists in SettingsPanel (grep for a segmented/two-button group; else two small Buttons with selected styling). i18n keys in all 10 `settings.json`. + +- [ ] **Step 4: Verify** — vitest (electron + video-editor suites), tsc, biome, i18n:check (no new failures). **Commit** — `git commit -m "Carry facecam fullscreen style from HUD through sidecar into project state"` + +--- + +### Task 3: Renderers honor fit/fill + +**Files:** +- Modify: `src/components/video-editor/VideoPlayback.tsx` (camera-full branch in `applyWebcamBubbleLayout`) +- Modify: `src/lib/exporter/modernFrameRenderer.ts` (camera-full branch in `updateWebcamOverlay`) +- Test: extend `src/lib/exporter/modernFrameRenderer.test.ts` (the existing camera-full layout test seam) + +- [ ] **Step 1: Preview.** `VideoPlayback.tsx` gains prop `webcamLayoutStyle?: "fit" | "fill"` (default "fit"), mirrored into `latestWebcamLayoutInputs`. In the camera-full branch: when style is `fill`, use `getCoverRect(croppedDims, { width: overlay.clientWidth, height: overlay.clientHeight })` (no padding), set the bubble rect to the FRAME (x:0, y:0, overlay size) with the inner video positioned by the cover rect — simplest faithful approach given the existing structure: keep the bubble at the full overlay rect with `clipPath` removed (no squircle), no drop-shadow, and let the existing inner "cover" crop math fill it (it already covers a bubble of any aspect — with the bubble at the frame's aspect, cover crops exactly as `getCoverRect` describes). So `fill` = bubble rect {0, 0, overlayW, overlayH}, no squircle clip (reset `clipPath` to `none`), `filter: none`. `fit` = current behavior. Ensure switching styles resets the styles it doesn't set (clipPath/filter must be explicitly restored in fit + bubble modes — they already are set every call; verify and set `clipPath`/`filter` explicitly in ALL branches). +- [ ] **Step 2: Export.** `modernFrameRenderer.ts`: config field `webcamLayoutStyle` (added in Task 2); in the camera-full path, when `fill`: layout rect = full output frame `{x:0, y:0, width: config.width, height: config.height}` with mask radius 0 and shadow disabled, relying on the existing sprite cover-fit within the layout rect to crop. Extend the existing camera-full unit test with a `fill` case asserting the webcam layout rect equals the full frame and the screen container is hidden. +- [ ] **Step 3: Verify** — `npx vitest --run src/lib/exporter src/components/video-editor` (no new failures), tsc, biome. **Commit** — `git commit -m "Render facecam fill style in preview and export"` + +--- + +### Task 4: Camera track on the timeline + +**Files:** +- Modify: `src/components/video-editor/timeline/core/constants.ts` (add `CAMERA_ROW_ID = "row-camera"`) +- Modify: `src/components/video-editor/timeline/model/timelineModel.ts:34-90` (`buildTimelineItems` — camera items, `variant: "camera"`) +- Modify: `src/components/video-editor/timeline/TimelineEditor.tsx:33-79` (props) and its row rendering (add the camera row, slim height, rendered above/below the zoom row matching existing row markup) +- Modify: `src/components/video-editor/timeline/hooks/useTimelineDndBindings.ts:33-178` (`resolveItemKind` + `handleItemSpanChange` camera case) +- Modify: `src/components/video-editor/timeline/Item.tsx` (camera variant styling — blue bar, reuse existing variant styling switch; find how `variant` maps to classes and add "camera" with blue fill e.g. `bg-blue-500/70 border-blue-400`) +- Modify: `src/components/video-editor/timeline/hooks/useTimelineSelection.ts` (selectedCameraId + deleteSelectedCamera), `timeline/hooks/utils/timelineSelectionUtils.ts` (target "camera"), `timeline/hooks/useTimelineKeyboardShortcuts.ts:93-120` (delete dispatch) +- Modify: `src/components/video-editor/TimelineCanvas.tsx` only if row hover/click plumbing requires it for double-click-add (see Step 3) +- Modify: `src/components/video-editor/VideoEditor.tsx` (handlers + props at the `` call ~6302-6349) +- Test: `src/components/video-editor/timeline/model/timelineModel.test.ts` if it exists (grep; extend with camera items), else rely on the pure clamp tests from Task 1 + manual. + +- [ ] **Step 1: Read the zoom-row pattern end to end first** (all files above) — the explorer-confirmed flow: VideoEditor props → `buildTimelineItems` → row render → `useItem` drag/resize → `handleItemSpanChange(id, span, rowId)` → parent callback. Copy it for camera with these specifics: + - Items: `{ id, rowId: CAMERA_ROW_ID, span: { start: region.startMs, end: region.endMs }, label: "Camera", variant: "camera" }`. + - The row renders only when `cameraRegions.length > 0 || cameraTrackVisible` (pass `cameraTrackVisible = webcam available` from VideoEditor; spec: track shown when webcam usable). + - When `webcamLayoutRegionsEnabled` is false, pass a `disabled`/dimmed visual (Item has a `disabled` prop per `useItem({ disabled })`; use a dimmed class instead so segments stay editable — spec says dimmed, still gating only rendering). +- [ ] **Step 2: VideoEditor handlers** (place near `handleZoomSpanChange` siblings): + +```ts + const handleCameraSpanChange = useCallback( + (id: string, span: { start: number; end: number }) => { + setWebcamLayoutRegions((prev) => { + const clamped = clampWebcamLayoutSpan( + { startMs: span.start, endMs: span.end }, + prev, + id, + Math.round(duration * 1000), + ); + if (!clamped) return prev; + return prev + .map((region) => + region.id === id ? { ...region, startMs: clamped.startMs, endMs: clamped.endMs } : region, + ) + .sort((a, b) => a.startMs - b.startMs); + }); + }, + [duration], + ); + + const handleCameraDelete = useCallback((id: string) => { + setWebcamLayoutRegions((prev) => prev.filter((region) => region.id !== id)); + }, []); + + const handleCameraAddAtMs = useCallback( + (timeMs: number) => { + setWebcamLayoutRegions((prev) => { + const span = clampWebcamLayoutSpan( + { startMs: timeMs, endMs: timeMs + 3000 }, + prev, + "", + Math.round(duration * 1000), + ); + if (!span) return prev; + return [ + ...prev, + { id: `webcam-layout-${Date.now()}-${Math.round(span.startMs)}`, ...span }, + ].sort((a, b) => a.startMs - b.startMs); + }); + }, + [duration], + ); +``` + +(`duration` is the source duration in seconds in VideoEditor — verify the actual variable name and use it.) Pass regions + handlers + `selectedCameraId` state through TimelineEditor props. +- [ ] **Step 3: Double-click to add.** Follow the zoom row's hover/click pattern (`TimelineCanvas.tsx:173-186` uses single-click with `zoomRowHoverMs`); implement the camera row with `onDoubleClick` on the row element using the same hover-ms tracking (or `pixelsToValue` of the event offset, matching however the zoom row computes hover ms). Empty-space only: ignore if the ms falls inside an existing region. +- [ ] **Step 4: Selection + delete.** Extend `resolveDeleteSelectionTarget` (+ its test if one exists — grep `timelineSelectionUtils.test`), `useTimelineSelection`, `useTimelineKeyboardShortcuts` per the zoom/clip pattern. Clicking a camera item selects it (Item click → `onSelectCamera(id)` — copy how zoom items select). +- [ ] **Step 5: Verify** — tsc, biome on touched files, `npx vitest --run src/components/video-editor` no new failures. Manual (dev): record/open a project with camera segments → blue bars on the track; drag edges/body (clamped, no overlap), double-click adds, select+Delete removes; preview cuts follow edits live; checkbox-off dims the bars. +- [ ] **Step 6: Commit** — `git commit -m "Add editable camera-full track to the timeline"` + +--- + +### Task 5: Final verification + +- [ ] **Step 1:** `npm test` (only the 1 pre-existing failure), `npx tsc --noEmit` clean, `npm run i18n:check` no new failures, biome on all touched files. +- [ ] **Step 2:** Manual end-to-end: HUD → pick "Fill screen" style → record with toggles → editor shows fill-style camera-full + segments on track → switch style in settings → preview + export flip style; export MP4 with edited segments matches the timeline. +- [ ] **Step 3:** Commit fixes if any. diff --git a/docs/superpowers/plans/2026-06-10-hidden-teleprompter.md b/docs/superpowers/plans/2026-06-10-hidden-teleprompter.md new file mode 100644 index 000000000..eb93b90ce --- /dev/null +++ b/docs/superpowers/plans/2026-06-10-hidden-teleprompter.md @@ -0,0 +1,1233 @@ +# Hidden Teleprompter + Preview Echo Fix Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Copy the Recordly screen recorder into `Mini Tools/RECORDLY`, add a capture-hidden teleprompter window with auto-scroll and global hotkeys, and fix the double-echo voice bug in the editor preview. + +**Architecture:** Recordly is an Electron + React/Vite/TS app. Each window is created by a `createXWindow()` function in `electron/windows.ts` and the renderer routes on a `?windowType=...` query param in `src/App.tsx`. The teleprompter is one more window type, hidden from capture via `BrowserWindow.setContentProtection(true)` (already used for the HUD overlay). Global hotkeys use Electron's `globalShortcut` (first use in this repo). The echo bug is a one-line logic fix in the preview audio routing engine plus a regression test. + +**Tech Stack:** Electron, React 18, TypeScript, Vite, Tailwind, Biome (tabs, lint), vitest. + +**Spec:** `docs/superpowers/specs/2026-06-10-hidden-teleprompter-design.md` + +**Conventions to follow:** +- Indent with TABS (Biome-enforced). Run `npx biome check --write ` on every file you touch. +- All commits go to the `Mini Tools` git repo (repo root is `/Users/justmaiko/PROJECTS/Mini Tools`, the project lives in the `RECORDLY/` subdirectory). +- All `npm` commands run from `/Users/justmaiko/PROJECTS/Mini Tools/RECORDLY`. + +--- + +### Task 1: Copy the repo and establish a baseline + +**Files:** +- Create: entire `RECORDLY/` tree (copied from the existing local clone) + +- [ ] **Step 1: Copy the existing clone (without git history)** + +The user already has a clone of `https://github.com/webadderallorg/Recordly` at `_external/Recordly`. Copy it without `.git` (note: trailing slashes matter; `docs/superpowers/` already exists in the destination and must be preserved — rsync without `--delete` merges, which is what we want): + +```bash +rsync -a --exclude=".git" --exclude=".DS_Store" \ + "/Users/justmaiko/PROJECTS/_external/Recordly/" \ + "/Users/justmaiko/PROJECTS/Mini Tools/RECORDLY/" +``` + +- [ ] **Step 2: Install dependencies** + +```bash +cd "/Users/justmaiko/PROJECTS/Mini Tools/RECORDLY" && npm install +``` + +Expected: completes successfully. The `postinstall` script rebuilds `uiohook-napi` and may build native helpers — this can take a few minutes. If a native helper build fails (e.g. missing Xcode component), note the error and continue: dev mode and tests do not require all packaged helpers. + +- [ ] **Step 3: Run the test suite to establish a baseline** + +```bash +cd "/Users/justmaiko/PROJECTS/Mini Tools/RECORDLY" && npm test +``` + +Expected: PASS. If any tests fail, record exactly which ones — they are pre-existing failures on this machine and must still fail/pass identically after our changes (no new failures). + +- [ ] **Step 4: Commit** + +```bash +cd "/Users/justmaiko/PROJECTS/Mini Tools" && git add RECORDLY && git commit -m "Vendor Recordly v1.3.3 from webadderallorg/Recordly (no upstream history)" +``` + +--- + +### Task 2: Fix the editor preview double-echo (mic played twice) + +**Root cause (verified):** On macOS, when recording with microphone but **no system audio**, the native helper (`electron/native/ScreenCaptureKitRecorder.swift:344`) writes mic audio **both** into the video's inline audio track and into a `.mic.m4a` sidecar. For that recording shape, `getCompanionAudioFallbackInfo` (`electron/ipc/recording/diagnostics.ts:528`) returns `paths = [micPath]` (video path NOT included). The routing engine (`src/lib/exporter/audioRoutingEngine.ts:124`) then computes `muteEmbeddedPreview: false`, so during preview the `