pybind11: bind member functions via a forwarding lambda (fixes MSVC C2440 on overloaded auto-returning members of class templates)

[Fedr]

Problem

Member functions are bound by disambiguating overloads with a pointer-to-member cast:

static_cast<Ret (C::*)(Args) const>(&C::name)

MSVC (through 19.5x / VS 2026) rejects that cast when the member is overloaded, has a deduced (auto) return type, and the enclosing class is a template — all three are required:

error C2440: 'static_cast': cannot convert from 'overloaded-function' to '... (C::*)(...)'
note: None of the functions with this name in scope match the target type

GCC and Clang accept it. In MeshLib this hits e.g. MR::Buffer<T,I>::data():

[[nodiscard]] auto data()       { return data_.get(); }
[[nodiscard]] auto data() const { return data_.get(); }

Minimal repro (Clang 22 and GCC 15 compile; MSVC 19.5x fails), https://gcc.godbolt.org/z/nKnd5ffbE :

template <class T>
struct B {
    T* p = nullptr;
    auto data()       { return p; }
    auto data() const { return p; }
};
auto f() { return static_cast<int* (B<int>::*)()>( &B<int>::data ); }

Every pointer-forming spelling fails the same way on MSVC (static_cast, C-style cast, typed-init of a member pointer) — the failure is resolving the address of the overload set against a target type, not the cast itself.

Fix

Bind the member via a captureless forwarding lambda instead of a pointer-to-member:

+[](_pb11_C const_ &self, /*params*/) -> decltype(auto) { return self.name(/*params*/); }

The lambda calls the function by name on a concrete object, so the const/non-const overload is selected by the object's const-ness and the address of an overloaded member is never taken. Being captureless, it decays (via the leading +) to a function pointer, so it remains a valid non-type template argument to TryAddFunc, and TryAddFunc is unchanged.

This mirrors how friend functions are already bound (DETAIL_MB_PB11_FUNC_PTR_OR_LAMBDA_cl), reuses the existing DETAIL_MB_PB11_MAKE_PARAM_DECLS / _USES helpers, and shares the same documented "extra move per argument" cost. Static methods get the no-self variant calling _pb11_C::name(...).

Scope / notes

• Touches only DETAIL_MB_PB11_DISPATCH_MEMBER_method in targets/pybind11/core.h. The generated .cpp (from data_to_macros) is unchanged — the same macro invocations now expand to a lambda.
• Conversion operators (DETAIL_MB_PB11_DISPATCH_MEMBER_conv_op) keep their cast: their return type is explicit (operator T), not deduced, so they don't hit the bug.
• The lambda calls name_ unqualified to preserve virtual dispatch; this matches the common case where the method is defined in the class being bound. Worth a look for inherited members that are name-hidden in a derived class (the old cast named the base explicitly) — flagging for review.
• Upstream MSVC bug report: https://developercommunity.visualstudio.com/t/C2440:-static_cast-to-select-an-overload/11107969

Validated end-to-end against MSVC by building MeshLib's Python bindings with MSVC (MeshLib PR #6272).