feat: SG-42200: 10-bit Metal presentation#1327
Draft
cedrik-fuoco-adsk wants to merge 15 commits into
Draft
Conversation
cedrik-fuoco-adsk
force-pushed
the
metal-10bit-macos
branch
from
6e39186 to
f4e483d
Compare
Decouple RvDocument from GLView by routing the active view through a generic QWidget* m_viewWidget and a viewWidget() accessor, instead of hardcoding m_glView at every shared call site (focus, geometry, stacked-layout, popup/menu coordinate mapping in Mu/Py UI commands). RvApplication now tolerates a null view()/GL context and uses the session's control video device for the primary display group, so an alternative presentation backend can be plugged in. No backend-specific code is introduced here; the GL path behaves exactly as before. This is the shared base for the Metal (macOS) and Vulkan (Linux) 10-bit presentation backends. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
Builds on the neutral view-abstraction base. Adds MetalView (a QWidget backed by a CAMetalLayer over an IOSurface) and QTMetalVideoDevice, wired into RvDocument behind '#if defined(PLATFORM_DARWIN) && defined(USE_METAL)'. USE_METAL is an Apple-only CMake option (default OFF). On the Metal path view() is null (no GL window): DiagnosticsView is not created (a QOpenGLWidget anywhere in the hierarchy forces macOS into a GL backing layer), GL-info queries are skipped, and the control video device drives presentation. Backend selection and shared plumbing live in the neutral base; this commit only adds the Metal-specific pieces. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
(alt-tab, uncover) and only restored the image once the mouse moved back into the view. MetalView is a plain QWidget with no paintEvent override, so on expose Qt's backing store painted a black background over the native NSView/CALayer, and nothing re-presented the layer's IOSurface contents until a later mouse event triggered a session redraw. Add WA_PaintOnScreen / WA_OpaquePaintEvent and override paintEngine() to return nullptr so Qt yields the native surface and stops drawing the black background over the layer. Override paintEvent to re-present the cached IOSurface (a cheap re-assign of layer.contents, no GL readback) so the last frame is restored immediately on expose, falling back to render() only when no frame has been produced yet. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
- Introduced a new method in MetalView to present an externally-owned IOSurface directly to the CALayer, improving performance by avoiding GL readback. - Added a mechanism to cache the last presented IOSurface for quick re-presentation on expose events. - Implemented a coalescing update request system to optimize redraw requests, ensuring only one update is posted per event loop cycle. - Updated QTMetalVideoDevice to manage a ring of IOSurfaces for zero-copy rendering, enhancing the efficiency of the Metal backend. These changes aim to streamline the rendering process and improve the responsiveness of the Metal display backend on macOS. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
- Introduced a new method in MetalView to check for 10-bit presentation capability, allowing the application to utilize Metal for rendering when supported. - Updated RvDocument to conditionally create a MetalView based on user preferences for 10-bit display formats, falling back to OpenGL if necessary. - Refactored the GL view creation logic into a separate method for better organization and clarity. These changes enhance the rendering capabilities on macOS, providing improved support for high dynamic range content. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
- Removed the conditional USE_METAL option from rv_options.cmake, simplifying the configuration for macOS builds. - Updated CMakeLists.txt files in RvCommon and IPCore to always include Metal-related source files and compile definitions, ensuring consistent Metal support across the application. These changes streamline the build process for macOS, enhancing the integration of the Metal rendering backend. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
- Added Qt::AA_ShareOpenGLContexts attribute in main.cpp to enable resource sharing across QOpenGLContexts, ensuring proper rendering of diagnostics tools on Metal/Vulkan backends. - Updated MetalView to request a fresh render on resize events, improving the display of the Live Review panel. - Refactored RvDocument to utilize viewWidget() for better compatibility with Metal/Vulkan backends, ensuring the active presentation widget is correctly referenced. - Enhanced QTMetalVideoDevice to share the OpenGL context with the global context, allowing diagnostics QOpenGLWidget to access textures from the offscreen render context. - Introduced activeViewFirstPaintCompleted() method in RvDocument to check if the active view has completed its first paint, accommodating both GLView and MetalView. These changes improve the integration and functionality of the Metal and Vulkan rendering backends, enhancing the overall user experience on macOS. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
- Updated MetalView to handle QEvent::UpdateRequest alongside QEvent::Paint, ensuring more responsive activity timer management. - Modified QTGLVideoDevice to conditionally redraw based on the visibility of the view, improving rendering performance on macOS. - Enhanced comments in the code to clarify the handling of OpenGL context and Metal rendering specifics. These changes aim to optimize the rendering process and improve the user experience during video playback and review sessions on macOS. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
…ontext management - Introduced renderImmediately() method in MetalView to allow synchronous rendering, bypassing the UpdateRequest coalescing for immediate updates. - Updated ensureGLContext() in QTMetalVideoDevice to return a boolean indicating the success of context creation and setup, enhancing error handling. - Modified makeCurrent() and defaultFBO() methods to handle GL context readiness checks, ensuring proper rendering behavior. - Improved redrawImmediately() to conditionally call renderImmediately() based on view visibility, optimizing rendering performance. These changes aim to enhance the rendering efficiency and reliability of the Metal backend on macOS, improving the overall user experience during video playback and review sessions. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
…ent and fallback handling - Added support for 10-bit Metal presentation capability checks in MetalView, including detailed logging for device and IOSurface availability. - Implemented a mechanism in QTMetalVideoDevice to latch interop failures and schedule retries instead of permanently disabling the zero-copy path. - Updated RvDocument to handle fallback from MetalView to OpenGL GLView upon context failures, ensuring a seamless user experience during rendering issues. - Enhanced error handling and logging throughout the Metal rendering process to provide better diagnostics and recovery options. These changes aim to improve the reliability and performance of the Metal backend on macOS, enhancing the overall user experience during video playback and review sessions. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
- Enhanced MetalView to include checks for successful IOSurface locking and null base address, preventing potential crashes and memory corruption. - Updated MuUICommands and PyUICommands to handle cases where viewWidget() may be null during event processing, ensuring robust behavior during UI interactions. - Added appropriate logging for error conditions to aid in diagnostics. These changes aim to enhance the stability and reliability of the Metal backend and UI interactions on macOS, improving the overall user experience during video playback and review sessions. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
…ndering and context management - Updated MetalView to handle null checks for viewWidget() during event processing, ensuring robust behavior in UI interactions. - Enhanced QTMetalVideoDevice to create a shared context worker device for threaded uploads, improving rendering performance and resource management. - Refactored RvDocument to better manage transitions between Metal and OpenGL views, including improved logging for context failures and fallback handling. - Improved error handling and diagnostics throughout the Metal rendering process, enhancing overall stability and user experience during video playback. These changes aim to optimize the Metal backend's performance and reliability on macOS, providing a smoother user experience during video review sessions. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
…agement and error handling - Updated QTMetalVideoDevice to ensure QOffscreenSurface is created on the GUI thread, enhancing thread safety during rendering. - Improved error handling in QTMetalVideoDevice to manage context creation failures and ensure valid OpenGL contexts. - Enhanced ImageRenderer to handle cases where the upload thread device may be unavailable, falling back to synchronous uploads when necessary. - Added explicit GL flushing in the upload thread to prevent synchronization issues, ensuring smoother rendering performance. These changes aim to optimize the Metal backend's reliability and performance on macOS, providing a better user experience during video playback and review sessions. Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
The CPU fallback triggered when IOSurface GL interop is unavailable used glReadPixels(GL_FLOAT) + a nested per-pixel float→10-bit pack loop and explicit Y-flip — approximately 25 M float ops per frame at 4K. Replace it with the same GPU-blit approach applied to the Vulkan path: - blit the RGBA16F render FBO into a persistent GL_RGB10_A2 flip FBO (GPU does the float→10-bit quantisation and Y-flip in one pass) - glFinish() synchronises the Metal command stream (required on macOS GL-on-Metal; glReadPixels alone does not flush the encoder) - glReadPixels(GL_BGRA, GL_UNSIGNED_INT_2_10_10_10_REV) reads the pre-packed ARGB2101010LE pixels directly — no CPU conversion loop GL_BGRA (not GL_RGBA as on the Vulkan side) is required because IOSurface expects kCVPixelFormatType_ARGB2101010LEPacked (A2|R10|G10|B10) while the Vulkan swapchain uses A2|B10|G10|R10. The alpha fix (glColorMask + glClear after the blit) mirrors the fast path: IPCore clears the render FBO with alpha=0 and the CA compositor honours per-pixel IOSurface alpha, so without the fix the background appears transparent over the gray window backdrop. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
Returns false from ensureIOSurfaceTextures() unconditionally when set, routing every frame through the GPU-blit CPU fallback path. Mirrors RV_VULKAN_FORCE_CPU_PRESENT on the Vulkan side. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Cédrik Fuoco <cedrik.fuoco@autodesk.com>
cedrik-fuoco-adsk
force-pushed
the
metal-10bit-macos
branch
from
f4e483d to
4c71ed7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: SG-42200: 10-bit Metal presentation
Linked issues
Summarize your change
Adds a macOS-only 10-bit Metal presentation path for OpenRV when the user requests 10+2 display format and the hardware supports it. IPCore still renders through an offscreen OpenGL 2.1 context (GL-on-Metal); the new backend delivers pixels to the screen via IOSurface → CALayer instead of a
QOpenGLWidgetswap chain.The work is built on a platform-neutral view abstraction that decouples
RvDocumentfromGLView: shared code routes throughviewWidget()and the session's control video device, so alternative backends (this Metal path, and the existing Vulkan 10-bit path on Linux) can plug in without touching every call site.When 10-bit Metal is unavailable or fails at runtime, the app falls back to the legacy OpenGL
GLViewwith logging.Describe the reason for the change
OpenRV's main window has historically been a
QOpenGLWidget. True 10-bit display on macOS is unreliable through that path: requesting a 10+2 GL surface format does not reliably produce correct scanout, and aQOpenGLWidgetanywhere in the window hierarchy can force Qt onto_NSOpenGLViewBackingLayer, breaking CALayer/IOSurface presentation (including 4× tiling artifacts with device pixel ratio scaling).This PR introduces a dedicated
MetalView(nativeNSView+CALayerbacked by IOSurface) andQTMetalVideoDevice, which wraps it as aTwkGLF::GLVideoDevicesoImageRendererand the rest of IPCore continue to render into offscreen FBOs unchanged. The zero-copy path blits RGBA16F → IOSurface-backed RGB10_A2; when GL–IOSurface interop is unavailable, a GPU-blit + packed readback fallback avoids the previous per-pixel CPU pack loop (~25M float ops/frame at 4K).Additional fixes address real usability bugs on the new path: black flash on window re-expose (alt-tab / uncover), context/thread safety for upload threads, interop failure retry instead of permanently disabling zero-copy, and diagnostics overlay integration via
Qt::AA_ShareOpenGLContexts.Describe what you have tested and on which operating system
USE_METAL=ON, 10+2 display prefs: main window presents correctly, no banding vs 8-bit GL pathsupports10BitPresentation()fails or GL context creation fails →GLViewwith log messageOS tested: macOS (version / hardware — please confirm)
Add a list of changes, and note any that might need special attention during the review
Core architecture (review carefully)
RvDocumentusesQWidget* m_viewWidget+viewWidget()instead of hardcodedm_glView;MuUICommands/PyUICommandsupdated for null-safe coordinate mappingRvApplicationview(); primary display group driven by session control video deviceMetalViewWA_PaintOnScreen/WA_OpaquePaintEvent,paintEngine() → nullptr, IOSurface cache + re-present on expose, coalescedUpdateRequest,renderImmediately()for synchronous redrawQTMetalVideoDeviceglReadPixels(CPU fallback); interop failure latch with periodic retryRvDocumentMetalViewvsGLViewwhen 10+2 prefs +MetalView::supports10BitPresentation();fallbackMetalToGLView()on context failureImageRenderer/SessionKnown limitations (not blockers, but reviewers should be aware)
DesktopVideoDevicepath, not ported)