feat: sidecar WebVTT subtitle passthrough into HLS output#12
Open
svensson00 wants to merge 8 commits into
Open
feat: sidecar WebVTT subtitle passthrough into HLS output#12svensson00 wants to merge 8 commits into
svensson00 wants to merge 8 commits into
Conversation
Add optional sidecar WebVTT subtitles to the HLS output. When SUBTITLE_URL is set, the encoder fetches the WebVTT source alongside the A/V input and publishes it as a segmented WebVTT rendition referenced from the master playlist as an #EXT-X-MEDIA:TYPE=SUBTITLES group. Unset leaves today's video+audio only output unchanged, so this is fully backward compatible. The config is shaped as a track array so more tracks can be added later, and the sidecar input keeps the MPEG-TS style muxing untouched so in-band subtitle carriage can be added later without reworking this path. ffmpeg's hls muxer only advertises the subtitle group on the single variant that carries the subtitle stream, and crashes if any other variant references the group directly, so the stream is attached to one variant and the SUBTITLES reference is broadened to every variant afterwards by rewriting the master playlist once it has been written. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Harden the seam that broadens the subtitle group across the ABR ladder: - Purge the hls output dir at the start of every ffmpeg run so a stale, already-rewritten master from a previous crashed run cannot be mistaken for this run's output, and only mark the master finalized after a write this process performed. - Turn finalizeSubtitleMaster into a readiness gate that returns success or failure. The running flip and the pull-push start now wait for it, so a one-shot downstream consumer can no longer cache the un-broadened master for the whole session. - Write the rewritten master via a temp file plus rename so readers never see a truncated playlist, and sanity-check the master is complete (the subtitle rendition plus every expected variant) before rewriting rather than trusting mere existence. - Bound the sidecar subtitle input with -rw_timeout so a stalled subtitle server cannot pin the encoder in starting with no output. - Surface finalize failures at warn/error level and align the readiness gate predicate with the rewrite predicate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move subtitle env parsing into a side-effect-free config module so it can be unit tested without starting the server. Wrap the URL parse so a typo yields an intentional error instead of a raw ERR_INVALID_URL, and warn when SUBTITLE_URL is set while HLS_ONLY is false, since only the HLS output carries subtitles. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…passthrough # Conflicts: # readme.md # src/encoder.test.ts # src/encoder.ts # src/server.ts
The workflow added on main is not prettier-clean (double-quoted strings), which fails the pretty check. The check only runs on pull requests, so it was not caught when the workflow was pushed. Format it so the check passes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #10
What
Adds optional sidecar WebVTT subtitle passthrough into the HLS output. When
SUBTITLE_URLis set, the encoder fetches a WebVTT source alongside the A/V input and publishes it as a segmented WebVTT rendition referenced from the HLS master playlist as an#EXT-X-MEDIA:TYPE=SUBTITLESgroup, available across the whole ABR ladder. Unset leaves today's video+audio only output unchanged, so this is fully backward compatible.Issue #10 lists two possible ingest forms and states a concrete near-term need for the sidecar form. This PR implements the sidecar form as the contributed choice: a WebVTT source URL fetched alongside the A/V input. The design keeps MPEG-TS style muxing untouched, so in-band subtitle carriage can be added later on the same path without reworking it. It composes with the configurable-input proposal (SRT caller, merged as #11) and works in both RTMP-listener and SRT-caller mode.
Config
SUBTITLE_URLhttp/https). If not set, output stays video+audio onlySUBTITLE_LANGUAGEenundSUBTITLE_NAMEEnglishSUBTITLE_LANGUAGESUBTITLE_DEFAULTtrue/1)falseThe config is shaped internally as a track array so more tracks can be added later; a single track is read from the environment today.
Design choices
-var_stream_map, and it segfaults, reproducibly, if any additional variant references that group directly (tested for bothmpegtsandfmp4). So the subtitle stream is attached to one variant (crash-free) and theSUBTITLESreference is then broadened to every variant by rewriting the master playlist once ffmpeg has written it. The rewrite (rewriteMasterPlaylist) also applies the configuredLANGUAGE,NAME,DEFAULTandAUTOSELECT.runningand the pull-push egress is not started until the master has been finalized, so a one-shot downstream consumer cannot cache the un-broadened, single-variant master for the whole session. A not-yet-complete master keeps the encoder instartingand is retried on the next monitor tick.rename()d into place so@fastify/staticand the pull-push fetcher never read a truncated master, and the master is sanity-checked as complete (subtitle rendition present plus every expected variant) before it is rewritten.-rw_timeoutso a stalled or unreachable subtitle server cannot pin the encoder instartingwith no output.LANGUAGE/NAME/DEFAULTthrough the master rewrite avoids ffmpeg's%vsegment-naming corruption when those attributes are set on a variant-carried subtitle in-var_stream_map.Contract verification
Grounded against the ffmpeg binary the repo's
Dockerfileproduces (node:18-alpine+apk add --no-cache ffmpeg), not against ffmpeg docs of another version.node:18-alpine, digestsha256:8d6421d663b4c28fd3ebc498332f249011d118945588d0a35cb9bc4b8ca09d9e3.21.3ffmpeg version 6.1.2, built with gcc 14.2.0 (Alpine 14.2.0), configuration includes--enable-libsrtand--enable-opensslffmpeg -h muxer=hls):Default subtitle codec: webvtt.,-hls_subtitle_path,-var_stream_mappresentffmpeg -protocols):file,http,https,tlspresent, sohttp(s)sidecar URLs are supported by this build-rw_timeout: documented asTimeout for IO operations (in microseconds); empirically a hunghttpserver (accepts, sends nothing) is bounded, ffmpeg exits withOperation timed outafter the configured interval rather than hangingEnd-to-end probe: a two-variant ABR encode with a sidecar WebVTT input, using the exact args this code generates (
-rw_timeout <us> -i <sidecar.vtt>,-map 1:0 -c:s webvtt,-var_stream_map "v:0,a:0 v:1,a:1,s:0,sgroup:subs"), exits 0 and writesmedia_1_vtt.m3u8plus.vttsegments. ffmpeg's master playlist (before the rewrite step):Only
media_1references the subtitle group;media_0does not. The multi-variant crash was reproduced against this same binary when trying to make ffmpeg reference the group from both variants directly. After running that real master throughrewriteMasterPlaylist, every variant references the group and the rendition carries the configured labels:Subtitle-exhaustion probe (A/V 12s, sidecar VTT with cues only for 0 to 4s): ffmpeg exits 0, the A/V variants continue for the full duration, and the WebVTT rendition is padded with empty segments (
media_10.vtt,media_11.vtt,media_12.vtt) rather than aborting. The vtt rendition segment naming in this build ismedia_<variant><index>.vtt.Tests
npm test,npm run typecheck,npm run lintpass locally (42 tests). Unit tests cover: subtitle input arg generation in both RTMP and SRT-caller mode (including the-rw_timeoutbound and the SRT + subtitles arg order), output arg /var_stream_mapgeneration, the no-subtitle byte-identical path, the master rewrite happy path (full-string golden), rewrite idempotency, single-variant masters, themediaIndexclamp, aMEDIAline withoutURI(left alone),masterIsCompletegating, the disk finalize seam in a temp dir (atomic write, idempotency, refusal on an incomplete master), a monitor-loop test that the finalize gate holds in SRT mode until the master is complete, and config parsing (protocol rejection, malformed-URL message, defaults, name fallback, bool parsing).Known limitations
Reconciliation with PR #11 (SRT caller, now merged)
PR #11 merged to main first, so this branch has been merged with main and the
generateInputcollision reconciled here (RESOLVED):generateInputis now(rtmpPort, streamKey, inputUrl?, subtitles = []). The sidecar-rw_timeout/-iargs are appended in BOTH the RTMP-listener and the SRT-caller branch, so the subtitle is always input index 1 and-map 1:0resolves to it in either mode. A unit test asserts the SRT + subtitles arg order (SRT-ifirst, sidecar second).subtitlesReadyfinalize gate is woven into feat: configurable input with SRT caller mode (INPUT_URL) #11's restructured monitor loop, gating the samerunningflip (andstartPullPush) in its new location.subtitleMasterFinalizedand the finalize attempt counter are reset instartFFmpeg, which runs after feat: configurable input with SRT caller mode (INPUT_URL) #11'scleanup()purge on both the initial dial and every re-dial. Without this, a re-dial after a briefly-connected attempt would leave the fresh master un-broadened. feat: configurable input with SRT caller mode (INPUT_URL) #11'scleanup()remains the single owner of the HLS-dir purge (its stale-index protection is preserved); this branch does not add a second purge.redactSecrets; the subtitle URL flows through the samespawnargsredaction, and no finalize log prints the URL.starting, and only goesrunning(broadening the master) once ffmpeg has written a complete master.The combined SRT-caller + sidecar case is verified at the argument level (unit test) and its output stage is the same pipeline proven end to end in Contract verification; a live SRT-listener container probe of the combined case was not run.
CI note
All four checks pass. The pre-existing
docs/api.yamlprettier issue is excluded by the.prettierignoreadded in #14. A separatechorecommit in this PR formats.github/workflows/sync-fork.yml(added on main with double-quoted strings): the pretty check only runs on pull requests, so that violation was never caught on the push to main and would otherwise fail the check here.