feat(replay): Capture SurfaceView content (experimental)#5333
Draft
feat(replay): Capture SurfaceView content (experimental)#5333
Conversation
Contributor
|
📲 Install BuildsAndroid
|
romtsn
force-pushed
the
rz/feat/replay-capture-surface-views
branch
from
6a58483 to
b287001
Compare
SurfaceView (used by Unity, video players, maps, and similar) renders to a separate Surface that is composited by SurfaceFlinger outside of the View hierarchy. PixelCopy.request(window, ...) only captures the Window surface, so SurfaceView regions appeared as transparent/black holes in Session Replay recordings. When the experimental option options.sessionReplay.isCaptureSurfaceViews is enabled, each visible SurfaceView is now captured separately via PixelCopy.request(surfaceView, ...) and composited onto the screenshot using PorterDuff.DST_OVER, so the SurfaceView content draws behind the Window content (which has transparent holes where the SurfaceViews are). Because SurfaceView redraws do not trigger ViewTreeObserver.OnDrawListener, the recorder bypasses the contentChanged guard when SurfaceViews are present, so subsequent frames are re-captured at the configured frame rate instead of reusing the last screenshot. The option defaults to false to preserve existing behavior. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
romtsn
force-pushed
the
rz/feat/replay-capture-surface-views
branch
from
b287001 to
df58de1
Compare
Contributor
Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| d217708 | 411.22 ms | 430.86 ms | 19.63 ms |
| 62b579c | 299.75 ms | 364.84 ms | 65.09 ms |
| f064536 | 349.86 ms | 417.66 ms | 67.80 ms |
| 5865051 | 333.08 ms | 355.34 ms | 22.26 ms |
| cf708bd | 408.35 ms | 458.98 ms | 50.63 ms |
| 9fbb112 | 401.87 ms | 515.87 ms | 114.00 ms |
| fcec2f2 | 311.35 ms | 384.94 ms | 73.59 ms |
| d15471f | 286.65 ms | 314.68 ms | 28.03 ms |
| b8bd880 | 314.56 ms | 336.50 ms | 21.94 ms |
| c8125f3 | 397.65 ms | 485.14 ms | 87.49 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| d217708 | 1.58 MiB | 2.10 MiB | 532.97 KiB |
| 62b579c | 0 B | 0 B | 0 B |
| f064536 | 1.58 MiB | 2.20 MiB | 633.90 KiB |
| 5865051 | 0 B | 0 B | 0 B |
| cf708bd | 1.58 MiB | 2.11 MiB | 539.71 KiB |
| 9fbb112 | 1.58 MiB | 2.11 MiB | 539.18 KiB |
| fcec2f2 | 1.58 MiB | 2.12 MiB | 551.51 KiB |
| d15471f | 1.58 MiB | 2.13 MiB | 559.54 KiB |
| b8bd880 | 1.58 MiB | 2.29 MiB | 722.92 KiB |
| c8125f3 | 1.58 MiB | 2.10 MiB | 532.32 KiB |
Previous results on branch: rz/feat/replay-capture-surface-views
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 7dc89d7 | 282.02 ms | 363.64 ms | 81.62 ms |
| 1acd813 | 298.43 ms | 351.19 ms | 52.76 ms |
| 9ef0a25 | 335.00 ms | 342.14 ms | 7.14 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 7dc89d7 | 0 B | 0 B | 0 B |
| 1acd813 | 0 B | 0 B | 0 B |
| 9ef0a25 | 0 B | 0 B | 0 B |
Add unit tests for the new SurfaceView capture support and extract a compositeSurfaceViewInto helper so the drawing contract can be verified with hand-built bitmaps (Robolectric's ShadowPixelCopy cannot produce meaningful SurfaceView pixels because there is no real GL producer). The tests cover: - ViewHierarchyNode.fromView returns SurfaceViewHierarchyNode vs. generic - View.traverse collects SurfaceView nodes when a list is supplied, not when it is null, and skips invisible SurfaceViews - PixelCopyStrategy leaves hasSurfaceViews false when the option is off - PixelCopyStrategy flags hasSurfaceViews true when the option is on - PixelCopyStrategy completes gracefully when a SurfaceView has no valid surface (the common Robolectric case) - compositeSurfaceViewInto fills transparent holes behind existing window content via DST_OVER, and respects both window offset and scale factors Also fixes a latent NPE in captureSurfaceViews when SurfaceHolder.surface is null (not just invalid) — happens before the surface is created. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
romtsn
force-pushed
the
rz/feat/replay-capture-surface-views
branch
from
261550d to
cb424d1
Compare
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📜 Description
Adds an experimental option
SentryReplayOptions.isCaptureSurfaceViews(defaultfalse). When enabled,PixelCopyStrategycaptures each visibleSurfaceViewon screen viaPixelCopy.request(surfaceView, ...)and composites the result onto the screenshot usingPorterDuff.DST_OVER, soSurfaceViewcontent no longer renders as a transparent hole in the replay.Since
SurfaceViewredraws do not triggerViewTreeObserver.OnDrawListener,ScreenshotRecorderalso bypasses thecontentChangedguard whenSurfaceViews are present, so frames are re-captured at the configured frame rate instead of reusing the previous screenshot.Example replay: https://sentry-sdks.sentry.io/explore/replays/0c28984d7c24464aa204495b0d95c53b/
💡 Motivation and Context
SurfaceView(used by Unity, ExoPlayer, maps, camera previews, etc.) renders to its ownSurfacethat is composited by SurfaceFlinger outside the View hierarchy.PixelCopy.request(window, ...)only captures the Window surface, so anything drawn on aSurfaceViewwas missing from Session Replay — a common complaint for apps embedding Unity or other 3D/video content.The feature is gated behind an experimental flag and defaults to
falseto preserve existing behavior and avoid the extra per-frame cost (one additional PixelCopy per visible SurfaceView + a compositing pass) for apps that don't need it.Closes #5257
💚 How did you test it?
Manually verified on a Unity-as-a-Library Android sample app: with the flag off, the Unity area is transparent/black in the replay (unchanged); with the flag on, the Unity frames appear in the replay video and animate at the configured frame rate. Also verified that non-SurfaceView screens behave identically to before. See example replay linked above.
📝 Checklist
sendDefaultPIIis enabled.