Skip to content

feat(hdr): shader transitions between HDR scenes + SDR rendering fixes#268

Open
vanceingalls wants to merge 2 commits intographite-base/268from
feat/hdr-layered-compositing
Open

feat(hdr): shader transitions between HDR scenes + SDR rendering fixes#268
vanceingalls wants to merge 2 commits intographite-base/268from
feat/hdr-layered-compositing

Conversation

@vanceingalls
Copy link
Copy Markdown
Collaborator

@vanceingalls vanceingalls commented Apr 14, 2026

Summary

Adds 15 GPU-style shader transitions (crossfade, swirl-vortex, glitch, domain-warp, light-leak, etc.) that blend between HDR scenes at full 16-bit precision. Also fixes SDR video rendering in HDR compositions and a critical streaming encoder bug that caused frame-to-frame flickering.

What it does

Shader transitions (Phase 5)

15 GLSL shaders ported to TypeScript, operating directly on rgb48le pixel buffers:

  • Simple: crossfade, flash-through-white
  • Warps: whip-pan, cinematic-zoom, cross-warp-morph, swirl-vortex
  • Distortion: glitch, ripple-waves, thermal-distortion, gravitational-lens, domain-warp
  • Effects: chromatic-split, sdf-iris, ridged-burn, light-leak

Each shader is a function: `(fromBuf, toBuf, outBuf, width, height, progress) → void` that reads from two scene buffers and writes blended pixels to the output.

Dual-scene compositing: For each transition frame, both the outgoing and incoming scenes are independently flattened into rgb48le buffers (HDR video blit + DOM screenshot composite), then blended by the shader function.

Scene detection: Compositions declare transitions via `window.__hf.transitions` (an array of `{ time, duration, shader, fromScene, toScene }`). The orchestrator queries this at render start and pre-computes frame ranges for each transition.

Critical bug fix: streaming encoder buffer race

`writeFrame()` passed the caller's buffer directly to `ffmpeg.stdin.write()`. Node.js streams hold a reference to the buffer and drain asynchronously. The transition loop reuses pre-allocated buffers and zero-fills them per frame — if the pipe hadn't fully drained, ffmpeg read partially-overwritten data, causing a horizontal banding artifact that flickered frame-to-frame. Fix: `Buffer.from(buffer)` copies the data before writing to the pipe.

SDR rendering fixes

Three stacked bugs made SDR video invisible in HDR compositions:

  1. `convertSdrToHdr` produced bt2020 pixels that Chrome misinterpreted as sRGB → near-zero values
  2. `syncVideoFrameVisibility` didn't restore active video `` elements after hiding inactive ones
  3. Injected `` overlays used `position: relative` and got clipped by `overflow: hidden` wrappers

Other fixes

  • `fileServer.ts` bridge script now uses `Object.assign` to preserve existing `window.__hf` properties (was overwriting the transitions array)
  • HDR frame directories cleaned up after each video ends (prevents disk exhaustion on long renders)
  • Lint rule updated to skip `data-start` validation on composition root elements

Files changed

File What changed
`packages/engine/src/utils/shaderTransitions.ts` NEW — 15 shader functions, `sampleRgb48le` (bilinear), noise functions (hash, vnoise, fbm), PQ/HLG LUT conversion. ~1000 lines.
`packages/engine/src/utils/shaderTransitions.test.ts` NEW — 89 tests: boundary conditions, all 15 shaders smoke-tested, HDR linearization roundtrips
`packages/engine/src/services/streamingEncoder.ts` `Buffer.from()` copy before pipe write (race condition fix)
`packages/engine/src/services/screenshotService.ts` `syncVideoFrameVisibility` restores active `` elements
`packages/engine/src/services/videoFrameExtractor.ts` `skipSdrConversion` parameter
`packages/engine/src/services/videoFrameInjector.ts` Border-radius, layout dimensions on `ElementStackingInfo`
`packages/producer/src/services/renderOrchestrator.ts` Scene detection, dual-scene transition compositing, `blitHdrVideoLayer` helper, frame dir cleanup
`packages/producer/src/services/fileServer.ts` `Object.assign` fix for `window.__hf` preservation
`packages/shader-transitions/src/hyper-shader.ts` Writes `window.__hf.transitions` after config parse
`compositions/hdr-transition-test/index.html` NEW — 2-scene integration test composition

How to test

  1. Render `compositions/hdr-transition-test` — 2 scenes with a swirl-vortex transition. Should be smooth, no flickering.
  2. Render `compositions/hdr-all-transitions` — 7 transitions demoing crossfade, swirl-vortex, glitch, domain-warp, sdf-iris, ridged-burn, light-leak. Works at both 1080p and 4K.
  3. Any composition mixing SDR + HDR video should render both correctly.

Stack position

6 of 6 — Top of the HDR stack. Stacked on #290 (GSAP transforms). This completes the HDR compositing pipeline.

Full stack: #258#265#288#289#290#268

🤖 Generated with Claude Code

@vanceingalls vanceingalls changed the title feat(engine): HDR two-pass compositing — DOM layer + native HLG video feat(hdr): layered HDR compositing with z-ordering, transforms, and CSS masks Apr 15, 2026
@vanceingalls vanceingalls force-pushed the fix/hdr-output-pipeline branch from 85de722 to 0b70603 Compare April 16, 2026 00:07
@vanceingalls vanceingalls force-pushed the feat/hdr-layered-compositing branch from eabca43 to a7db7f6 Compare April 16, 2026 00:07
Copy link
Copy Markdown
Collaborator Author

vanceingalls commented Apr 16, 2026

@vanceingalls vanceingalls changed the base branch from fix/hdr-output-pipeline to graphite-base/268 April 16, 2026 00:10
@vanceingalls vanceingalls force-pushed the feat/hdr-layered-compositing branch 2 times, most recently from 32cf08c to 6976409 Compare April 16, 2026 00:51
@vanceingalls vanceingalls force-pushed the feat/hdr-layered-compositing branch from 6976409 to c62a8f6 Compare April 16, 2026 00:54
@vanceingalls vanceingalls changed the title feat(hdr): layered HDR compositing with z-ordering, transforms, and CSS masks feat(hdr): shader transitions between HDR scenes + SDR rendering fixes Apr 17, 2026
…ering

Port 15 GLSL shaders to TypeScript rgb48le pixel math for HDR-native
scene transitions. Dual-scene compositing flattens each scene
independently then blends with per-pixel shader. Scene detection via
window.__hf.transitions protocol. Fix streaming encoder buffer race
condition (Node streams hold reference, zero-fill overwrites before
pipe drains). Fix SDR video rendering in HDR compositions (three
stacked bugs). 211 engine tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vanceingalls vanceingalls force-pushed the feat/hdr-layered-compositing branch from da57514 to a593e3f Compare April 17, 2026 06:00
…ions

When a composition mixes PQ and HLG HDR sources, the pipeline now
converts each video's native transfer to match the output transfer
function during blit. Uses a composite 65536-entry LUT (source EOTF
→ linear → target OETF) for O(1) per-sample conversion. Tracks each
video's individual transfer via ffprobe at detection time.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vanceingalls vanceingalls force-pushed the feat/hdr-layered-compositing branch from a0a2680 to 8830de9 Compare April 17, 2026 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant