Skip to content

Rearchitect vcpkg port for official submission#1466

Open
bmehta001 wants to merge 59 commits into
microsoft:mainfrom
bmehta001:bhamehta/vcpkg-port-clean
Open

Rearchitect vcpkg port for official submission#1466
bmehta001 wants to merge 59 commits into
microsoft:mainfrom
bmehta001:bhamehta/vcpkg-port-clean

Conversation

@bmehta001

@bmehta001 bmehta001 commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

This replaces #1414 with a clean history based directly on current main. The earlier PR had accumulated too much unrelated branch history, so this PR carries only the vcpkg/CMake port layer.

Goals

  • Build the SDK as a vcpkg overlay port using CMake, not legacy scripts/MSBuild.
  • Use vcpkg-provided dependencies when building under the vcpkg toolchain.
  • Install CMake package config/targets for downstream consumers.
  • Add consumer validation scripts for Windows, Linux, macOS, iOS, and Android.

Scope

  • CMake vcpkg dependency mode (MATSDK_USE_VCPKG_DEPS) and install/export support.
  • Modern tools/ports/mstelemetry overlay port files.
  • tests/vcpkg consumer project and platform scripts.
  • Example/docs/wrapper CMake updates needed for vcpkg consumption.
  • Remove the unused tools/vcpkg submodule; tests and CI bootstrap vcpkg from an external VCPKG_ROOT.

Core SDK only (no private modules)

  • The port builds the core MSTelemetry::mat target only.
  • The optional Microsoft-proprietary modules (Privacy Guard, Sanitizer, ECS/EXP, Compliant-by-Default, AFD, CDS) in the private lib/modules submodule are not available via the vcpkg port: official vcpkg source archives pin an immutable commit and do not fetch git submodules, and that submodule is an internal-only repository.
  • Module-only public headers (CompliantByDefaultFilterApi.hpp, IAFDClient.hpp, ICdsFactory.hpp, IECSClient.hpp) are excluded from the installed headers.
  • The Obj-C/Swift wrapper layers are not built by the port (BUILD_OBJC_WRAPPER/BUILD_SWIFT_WRAPPER=OFF); the core SDK — including the Apple HttpClient_Apple HTTP stack — is still built on macOS/iOS.
  • docs/building-with-vcpkg.md documents this scope and how to build modules from a full --recurse-submodules source checkout.

HTTP client per platform

  • Windows: WinInet. macOS/iOS: NSURLSession (Apple HTTP). Linux/Android: libcurl (curl[openssl]), declared only for linux | android in vcpkg.json.

Android API level

  • vcpkg's built-in Android triplets default to API 28. This repo's Android minSdk is 23, so the Android consumer test defaults to API 23 using custom overlay triplets in tests/vcpkg/triplets (VCPKG_CMAKE_SYSTEM_VERSION 23) and requires Ninja so the host IDE generator does not select a different NDK.
  • The whole dependency graph must be built with the same Android API triplet; do not mix API 28-built dependencies into an API 23 consumer binary.

Not in scope

Validation performed

  • Hand-authored vcpkg/CMake files pass git diff --check.
  • Windows vcpkg consumer test passed (x64-windows-static, C++11 consumer project, vcpkg_test including LogManager::Initialize() / FlushAndTeardown()).
  • Windows vcpkg install header check: module-only headers are not installed; core headers remain.
  • Linux vcpkg consumer test passed from WSL ext4 (x64-linux).
  • Android vcpkg cross-build passed for both API 23 (custom arm64-android-api23 overlay triplet) and API 28 (built-in arm64-android) using native libcurl HTTP.
  • Verified the upstream archive SHA512 used by tools/ports/mstelemetry/portfile.cmake.
  • Real send check: built ONNX Runtime for Android with --use_telemetry against this branch's SDK source and ran a telemetry harness on an emulator; telemetry events were emitted successfully.
  • Shell syntax checks passed for the Linux, macOS, iOS, and Android vcpkg scripts.

Follow-up after this merges

  • Submit the official port to microsoft/vcpkg using this repo's merged commit/tag as the source (and run vcpkg format-manifest + vcpkg x-add-version in the vcpkg repo).
  • Decide whether to keep or remove the in-repo overlay port once the official vcpkg port is available.
  • Cut a release/tag only if the official vcpkg port should target a stable release rather than a source commit.
  • Add optional vcpkg features for module support later if/when cpp_client_telemetry_modules can be fetched from a suitable source.
  • TODO (example CMake): Replace the stale #set(CMAKE_SYSTEM_PROCESSOR i386) hint in the example CMakeLists.txt files. Setting CMAKE_SYSTEM_PROCESSOR after project() does not cross-compile — it only changes the multiarch lib-dir lookup and the armv7l branch, so uncommenting it still yields a host-arch binary. A real i386 build needs -m32/multilib or a toolchain file set before project(). Document the correct approach or drop the misleading comment.

Backward compatibility / migration

The previous overlay port detected a local --recurse-submodules checkout and built the SDK from it (via build.sh/install.sh or MSBuild), which included the private lib/modules submodules. That path is removed; the port now always builds the core SDK from a pinned, immutable upstream archive.

  • vcpkg install --head --overlay-ports=tools/ports mstelemetry (built from local checkout, with private modules) -> vcpkg install mstelemetry builds the core SDK only, from a pinned commit.
  • response_file_linux.txt / response_file_mac.txt are removed; use a standard triplet.
  • Module users should build from a full --recurse-submodules source checkout instead.

Impact assessment: nothing visible depends on the old module-overlay path -- no repo CI uses it, no public GitHub repo references it, and modern consumers (e.g. ONNX Runtime) consume the SDK via CMake FetchContent rather than the overlay port. The only potentially affected group is internal pipelines that built the SDK with private modules via the overlay. See docs/building-with-vcpkg.md for the full migration note.

bmehta001 and others added 2 commits June 5, 2026 12:53
Add a clean vcpkg-oriented CMake and overlay-port path so the SDK can be consumed through vcpkg across the supported platform set. This keeps vendored dependency builds available by default while enabling vcpkg-provided sqlite, zlib, nlohmann-json, and curl when the vcpkg toolchain is active.

Files changed: CMake install/config support, overlay port metadata, vcpkg consumer tests, examples/docs, Obj-C/Swift wrapper source selection, and vcpkg test workflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Point the overlay mstelemetry port at the clean vcpkg source snapshot in this replacement PR and update the archive SHA512 used by vcpkg_from_github.

Files changed: tools/ports/mstelemetry/portfile.cmake

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bmehta001 bmehta001 requested a review from a team as a code owner June 5, 2026 17:57
@bmehta001 bmehta001 requested a review from Copilot June 5, 2026 18:56
@bmehta001 bmehta001 self-assigned this Jun 5, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 41 out of 43 changed files in this pull request and generated no new comments.

bmehta001 and others added 2 commits June 5, 2026 19:26
Use Android-specific PAL implementations when the top-level CMake path targets Android, and remove the missing bundled zlib simd_stub source from the Android legacy dependency path. Also make the vcpkg consumer test build as C++11 to match the SDK consumer compatibility posture.

Files changed: lib/CMakeLists.txt, tests/vcpkg/CMakeLists.txt

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update the overlay port REF and SHA512 to the source snapshot that includes the Android CMake source fixes and C++11 vcpkg consumer test.

Files changed: tools/ports/mstelemetry/portfile.cmake

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bmehta001 bmehta001 force-pushed the bhamehta/vcpkg-port-clean branch from 6900a6a to 79b1807 Compare June 6, 2026 00:26
bmehta001 and others added 2 commits June 5, 2026 23:13
GitHub source archives used by the public vcpkg port do not include the lib/modules submodule. Avoid installing public headers whose exported factories/functions are implemented only by absent optional modules, so core vcpkg consumers do not get linkable-looking APIs without implementations.

Files changed: lib/include/CMakeLists.txt

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update the overlay port REF and SHA512 to the source snapshot that excludes module-only public headers from the core vcpkg install.

Files changed: tools/ports/mstelemetry/portfile.cmake

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 42 out of 44 changed files in this pull request and generated 2 comments.

Comment thread lib/CMakeLists.txt
Comment thread lib/include/CMakeLists.txt
bmehta001 and others added 12 commits June 6, 2026 00:07
Build the Android vcpkg package with the native Curl HTTP client instead of the Java-backed Android HTTP client, so native consumers using find_package(MSTelemetry) can initialize the SDK without the Java bridge creating the Android HTTP singleton. Strengthen the vcpkg consumer test by calling LogManager::Initialize and FlushAndTeardown.

Files changed: CMakeLists.txt, lib/CMakeLists.txt, tools/ports/mstelemetry/vcpkg.json, tests/vcpkg/main.cpp

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update the overlay port REF and SHA512 to the source snapshot that uses Curl for Android vcpkg consumers.

Files changed: tools/ports/mstelemetry/portfile.cmake

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The SDK only needs HTTP/HTTPS. Disable curl default features and enable only ssl so Android vcpkg builds do not pull in unused non-HTTP protocol objects that fail to link on the Android NDK.

Files changed: tools/ports/mstelemetry/vcpkg.json

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update the overlay port REF and SHA512 to the source snapshot that disables unused curl non-HTTP protocols.

Files changed: tools/ports/mstelemetry/portfile.cmake

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document and use Android API 28 for the vcpkg Android consumer test, matching vcpkg's Android triplet default. Keep curl default features disabled and request OpenSSL explicitly for the native Android vcpkg HTTP path.

Files changed: tools/ports/mstelemetry/vcpkg.json, tests/vcpkg/test-vcpkg-android.sh, tests/vcpkg/README.md, docs/building-with-vcpkg.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update the overlay port REF and SHA512 to the source snapshot that aligns Android vcpkg validation with API 28 and requests OpenSSL explicitly for curl.

Files changed: tools/ports/mstelemetry/portfile.cmake

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Export the vcpkg-resolved dependency targets through MSTelemetry::mat so static-library consumers receive sqlite, zlib, nlohmann-json, and curl link requirements. Also fix the template header install exclusion and update the package config Curl comment for Android vcpkg builds.

Files changed: lib/CMakeLists.txt, lib/include/CMakeLists.txt, cmake/MSTelemetryConfig.cmake.in

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update the overlay port REF and SHA512 to the source snapshot that exports vcpkg dependency targets to downstream static-library consumers.

Files changed: tools/ports/mstelemetry/portfile.cmake

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use API 23 overlay triplets for Android vcpkg validation so dependencies match the repo's Android minSdk instead of requiring API 28 consumers.

Remove the unused bundled vcpkg submodule because tests and CI use an external VCPKG_ROOT.

Keep Apple sample and legacy tests aligned with the dependency model: core vcpkg builds use Apple HTTP without Obj-C/Swift wrappers, and Android legacy tests reuse bundled zlib.

Files changed:

- .github/workflows/test-vcpkg.yml

- .gitmodules

- docs/building-with-vcpkg.md

- examples/swift/SampleXcodeApp/SwiftWrapperApp.xcodeproj/project.pbxproj

- lib/CMakeLists.txt

- tests/functests/CMakeLists.txt

- tests/unittests/CMakeLists.txt

- tests/vcpkg/README.md

- tests/vcpkg/test-vcpkg-android.sh

- tests/vcpkg/triplets/*.cmake

- tools/ports/mstelemetry/vcpkg.json

- tools/vcpkg

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Point the overlay port at the source snapshot that removes the bundled vcpkg submodule, aligns Android validation with API 23, and keeps Apple/core dependency behavior consistent.

Files changed:

- tools/ports/mstelemetry/portfile.cmake

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Seed MATSDK_INSTALL_DIR from the cache/environment and fall back to /usr/local, removing the intermediate _MATSDK_DEFAULT_INSTALL_DIR temporary while preserving -DMATSDK_INSTALL_DIR override precedence.

Files changed:

- examples/cmake/MSTelemetrySample.cmake

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Point the overlay port REF/SHA512 at the source snapshot that simplifies the MATSDK_INSTALL_DIR default in the sample helper.

Files changed:

- tools/ports/mstelemetry/portfile.cmake

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bmehta001 and others added 5 commits June 8, 2026 18:44
Point the mstelemetry overlay port at the commit that adds MinGW curl dependency coverage and the MinGW-safe unused-parameter macro.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Revert the experimental MinGW curl/macro changes and make the port manifest match the documented Windows support boundary: MSVC Windows triplets are supported, MinGW is not.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
EventProperties initializes EventInfo.Level by default, so the vcpkg consumer test should verify the four custom typed keys are present rather than requiring the entire property map to contain exactly four entries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Verify string, double, and bool EventProperty values in addition to key presence and the existing int64 check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bmehta001 and others added 2 commits June 8, 2026 18:59
…1/cpp_client_telemetry into bhamehta/vcpkg-port-clean
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 48 out of 50 changed files in this pull request and generated 2 comments.

Comment thread tests/unittests/CMakeLists.txt
Comment thread tests/functests/CMakeLists.txt
bmehta001 and others added 8 commits June 8, 2026 22:03
…rning

zlib_bundled compiles zlib's pristine sources without its configure step, so
gz*.c lacked <unistd.h> and called POSIX read/write/lseek/close via implicit
(int-returning) declarations -- which we were silencing with
-Wno-implicit-function-declaration. On Android LP64 those functions return
ssize_t/off_t, so the implicit declarations are a latent prototype mismatch.

Define Z_HAVE_UNISTD_H (as zlib's own POSIX configure would) so zconf.h pulls
in <unistd.h> and the functions get their real declarations. All 15 bundled
zlib sources then compile clean under the project's -Werror with no warning
suppression (validated with the Android NDK r29 Clang).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two warning-suppression flags were broader than the issue they covered:

- lib/http/HttpClient_Android.cpp declares an unused `Tag` log
  constant.  Instead of suppressing -Wunused-const-variable for the
  whole file, annotate the constant with __attribute__((unused)) (a log
  tag is a standard convention and may be used later) and drop the
  per-file -Wno-unused-const-variable from lib/CMakeLists.txt.

- The global -Wno-reorder added to CXX_EXTRA_WARN_FLAGS in CMakeLists.txt
  was redundant.  The only member-init-order warning is in the private
  submodule's Sanitizer.cpp, which already has its own per-file
  -Wno-reorder scope in lib/CMakeLists.txt.  msft/main builds the full
  tree (with submodules) under -Wall -Werror -Wextra using only that
  per-file scope, so remove the global flag.

Files changed:
- CMakeLists.txt
- lib/CMakeLists.txt
- lib/http/HttpClient_Android.cpp

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Point the overlay portfile at the f98700d source snapshot so the
mstelemetry vcpkg port builds the warning-suppression scoping change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Tag log-component string was unused and is not part of the SDK
logging convention: the LOG_TRACE/INFO/WARN/ERROR macros resolve the
component automatically via getMATSDKLogComponent(), and the other
HttpClient backends (Curl, WinInet, WinRt) define no such constant.
HttpClient_Android.cpp performs no logging at all, so drop the dead
constant entirely instead of annotating it __attribute__((unused)).

Files changed:
- lib/http/HttpClient_Android.cpp

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Point the mstelemetry overlay portfile at the 6e15243 source snapshot.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
main.cpp includes LogManager.hpp, which on Windows transitively pulls in
<ObjBase.h> and the legacy empty
ear macro from the Windows SDK.  The
local static bool near(double, double) helper then expands to
static bool (double, double) and fails to compile (C2062), breaking the
test-vcpkg-windows.ps1 consumer build.  Rename it to nearlyEqual().

Files changed:
- tests/vcpkg/main.cpp

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tests/functests and tests/unittests linked the CURL::libcurl imported
target unconditionally.  On the project's minimum CMake (3.5), a legacy
(non-vcpkg) find_package(CURL) only sets CURL_LIBRARIES/CURL_INCLUDE_DIRS
and does not define CURL::libcurl until CMake 3.12, so configuration
would fail.  Mirror the root CMakeLists.txt guard: prefer the imported
target when present and fall back to \ otherwise.

Files changed:
- tests/functests/CMakeLists.txt
- tests/unittests/CMakeLists.txt

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 49 out of 51 changed files in this pull request and generated 1 comment.

Comment thread tools/ports/mstelemetry/vcpkg.json Outdated
Comment thread docs/building-with-vcpkg.md
Comment thread examples/c/SampleC-Guest/CMakeLists.txt Outdated
Comment thread tools/ports/mstelemetry/vcpkg.json Outdated
bmehta001 and others added 5 commits June 9, 2026 16:41
Per vcpkg convention (review feedback from @walbourn), port-version defaults to
0 and should be omitted when it is 0. Remove the explicit field.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Per review feedback from @walbourn: keep the 3.5 minimum so consumers on older
CMake (distro defaults, the Android NDK CMake, etc.) still build, while setting
the policy version to 3.29 for modern policy behavior and to avoid the CMake
3.31+ deprecation warning that a bare VERSION 3.5 emits.

CMake older than 3.12 sees the extra dots as version separators and ignores the
...3.29, treating it as VERSION 3.5 (documented behavior), so this is fully
backward compatible. Verified CMake 4.3.2 configures cleanly with no error or
deprecation and CMAKE_MINIMUM_REQUIRED_VERSION=3.5.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Point the mstelemetry overlay portfile at the 0604492 source snapshot.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The main merge brought lib/include/public/Version.hpp (3.10.159.1), a tarball
source, so point the overlay portfile at the 65a0c75 merge commit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants