Sync fork#1
Open
kislaykishore wants to merge 1400 commits into
Open
Conversation
…ics (#4348) * Update histogram buckets for GCSFuse distribution metrics
Changes to improve retry logic for kernel reader.
#4353) * fix_e2e_script * fix_e2e_script * fix_e2e_script * fix_e2e_script * fix_e2e_script * fix_e2e_script * fix_e2e_script * small fix
#4354) * fix(local-file-unit-test): fixing CreateHardLink test for newer kernel * fixing review comments
…ts/ml_tests/checkpoint/Jax/protobuf-5.29.6' into alleaditya_merge_dependency_updgrades
…ts/ls_metrics/protobuf-5.29.6' into alleaditya_merge_dependency_updgrades
…ts/presubmit_test/pr_perf_test/protobuf-6.33.5' into alleaditya_merge_dependency_updgrades
…ts/testing_on_gke/examples/protobuf-6.33.5' into alleaditya_merge_dependency_updgrades
…ts/testing_on_gke/examples/pyasn1-0.6.2' into alleaditya_merge_dependency_updgrades
…ts/micro_benchmarks/pyasn1-0.6.2' into alleaditya_merge_dependency_updgrades
…ts/presubmit_test/pr_perf_test/pyasn1-0.6.2' into alleaditya_merge_dependency_updgrades
* Update smeantics doc about buffered reader * Address comments
…pendency_updgrades chore(deps): Merge all dependabot PRs
…ters simultaneously (#4330) ### Description Add support to export traces to multiple exporters simultaneously ### Link to the issue in case of a bug fix. b/479670309 ### Testing details 1. Manual . Tested manually by providing configuration to export to all of stdout & gcptrace simultaneously. ### Any backward incompatible change? If so, please explain. NA
* build(deps): bump the go-dependencies group across 1 directory with 16 updates Bumps the go-dependencies group with 9 updates in the / directory: | Package | From | To | | --- | --- | --- | | [cloud.google.com/go/auth](https://github.com/googleapis/google-cloud-go) | `0.18.0` | `0.18.1` | | [github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go) | `0.54.0` | `0.55.0` | | [github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go) | `1.30.0` | `1.31.0` | | [github.com/fsouza/fake-gcs-server](https://github.com/fsouza/fake-gcs-server) | `1.52.2` | `1.53.0` | | [go.opentelemetry.io/contrib/detectors/gcp](https://github.com/open-telemetry/opentelemetry-go-contrib) | `1.39.0` | `1.40.0` | | [go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc](https://github.com/open-telemetry/opentelemetry-go-contrib) | `0.64.0` | `0.65.0` | | [go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace](https://github.com/open-telemetry/opentelemetry-go-contrib) | `0.64.0` | `0.65.0` | | [go.opentelemetry.io/otel/exporters/prometheus](https://github.com/open-telemetry/opentelemetry-go) | `0.61.0` | `0.62.0` | | [go.opentelemetry.io/otel/exporters/stdout/stdouttrace](https://github.com/open-telemetry/opentelemetry-go) | `1.39.0` | `1.40.0` | Updates `cloud.google.com/go/auth` from 0.18.0 to 0.18.1 - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](googleapis/google-cloud-go@v0.18.0...auth/v0.18.1) Updates `github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric` from 0.54.0 to 0.55.0 - [Release notes](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/releases) - [Commits](GoogleCloudPlatform/opentelemetry-operations-go@v0.54.0...v0.55.0) Updates `github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/releases) - [Commits](GoogleCloudPlatform/opentelemetry-operations-go@detectors/gcp/v1.30.0...detectors/gcp/v1.31.0) Updates `github.com/fsouza/fake-gcs-server` from 1.52.2 to 1.53.0 - [Release notes](https://github.com/fsouza/fake-gcs-server/releases) - [Commits](fsouza/fake-gcs-server@v1.52.2...v1.53.0) Updates `go.opentelemetry.io/contrib/detectors/gcp` from 1.39.0 to 1.40.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go-contrib@v1.39.0...v1.40.0) Updates `go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc` from 0.64.0 to 0.65.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.64.0...zpages/v0.65.0) Updates `go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace` from 0.64.0 to 0.65.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.64.0...zpages/v0.65.0) Updates `go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp` from 0.64.0 to 0.65.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.64.0...zpages/v0.65.0) Updates `go.opentelemetry.io/otel` from 1.39.0 to 1.40.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.39.0...v1.40.0) Updates `go.opentelemetry.io/otel/exporters/prometheus` from 0.61.0 to 0.62.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@exporters/prometheus/v0.61.0...exporters/prometheus/v0.62.0) Updates `go.opentelemetry.io/otel/exporters/stdout/stdouttrace` from 1.39.0 to 1.40.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.39.0...v1.40.0) Updates `go.opentelemetry.io/otel/metric` from 1.39.0 to 1.40.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.39.0...v1.40.0) Updates `go.opentelemetry.io/otel/sdk` from 1.39.0 to 1.40.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.39.0...v1.40.0) Updates `go.opentelemetry.io/otel/sdk/metric` from 1.39.0 to 1.40.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.39.0...v1.40.0) Updates `go.opentelemetry.io/otel/trace` from 1.39.0 to 1.40.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.39.0...v1.40.0) Updates `google.golang.org/api` from 0.259.0 to 0.264.0 - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](googleapis/google-api-go-client@v0.259.0...v0.264.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/auth dependency-version: 0.18.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: go-dependencies - dependency-name: github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric dependency-version: 0.55.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace dependency-version: 1.31.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: github.com/fsouza/fake-gcs-server dependency-version: 1.53.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/contrib/detectors/gcp dependency-version: 1.40.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc dependency-version: 0.65.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace dependency-version: 0.65.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp dependency-version: 0.65.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel dependency-version: 1.40.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/exporters/prometheus dependency-version: 0.62.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/exporters/stdout/stdouttrace dependency-version: 1.40.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/metric dependency-version: 1.40.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/sdk dependency-version: 1.40.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/sdk/metric dependency-version: 1.40.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/trace dependency-version: 1.40.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: google.golang.org/api dependency-version: 0.264.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> * Revert fake-gcs-server to v1.52.2 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* update defaults in machine-type parameters table in README * update README in orbax_benchmark * update README in machine_type_test * address gemini comment
…ta prefetcher (#4370) * changes for prefetch cancellation * review comments * review comments
) * Use random directory names for negative stat cache tests This change modifies `setup_test.go` in `tools/integration_tests/negative_stat_cache/` to use randomly generated directory names for `testDirName` and `onlyDirMounted` variables. This ensures that concurrent test runs do not conflict with each other by using the same directory name in the bucket. The variables are initialized in `TestMain` using `setup.GenerateRandomString`. Log messages are added to print the generated directory names. Co-authored-by: vadlakondaswetha <101323867+vadlakondaswetha@users.noreply.github.com> * Use unique random directory names for each negative stat cache test This change modifies `setup_test.go` and individual test files in `tools/integration_tests/negative_stat_cache/` to use randomly generated directory names for each test execution. This ensures that tests running in parallel or concurrently do not conflict with each other by using the same directory name in the bucket. - `mountGCSFuseAndSetupTestDir` now accepts a `testDirName` argument. - Each test suite (`disabledNegativeStatCacheTest`, `finiteNegativeStatCacheTest`, `infiniteNegativeStatCacheTest`) now generates a unique `testDir` in `SetupTest` and uses it throughout the test. - `TearDownTest` methods are updated to clean up the specific test directory. Co-authored-by: vadlakondaswetha <101323867+vadlakondaswetha@users.noreply.github.com> * Use unique random directory names and CleanUpDir for negative stat cache tests This change modifies `setup_test.go` and individual test files in `tools/integration_tests/negative_stat_cache/` to use randomly generated directory names for each test execution. This ensures that tests running in parallel or concurrently do not conflict with each other by using the same directory name in the bucket. - `mountGCSFuseAndSetupTestDir` now accepts a `testDirName` argument. - Each test suite (`disabledNegativeStatCacheTest`, `finiteNegativeStatCacheTest`, `infiniteNegativeStatCacheTest`) now generates a unique `testDir` in `SetupTest` and uses it throughout the test. - `TearDownTest` methods are updated to call `setup.CleanUpDir(testEnv.testDirPath)` before `setup.UnmountGCSFuse` to ensure proper cleanup via the mount point, as requested. Co-authored-by: vadlakondaswetha <101323867+vadlakondaswetha@users.noreply.github.com> * Refactor negative stat cache tests to use random directory names and CleanUpDir This change modifies `setup_test.go` and individual test files in `tools/integration_tests/negative_stat_cache/` to use randomly generated directory names for each test execution. This ensures that tests running in parallel or concurrently do not conflict with each other by using the same directory name in the bucket. - `mountGCSFuseAndSetupTestDir` now accepts a `testDirName` argument. - Each test suite (`disabledNegativeStatCacheTest`, `finiteNegativeStatCacheTest`, `infiniteNegativeStatCacheTest`) now generates a unique `testDir` in `SetupTest` and uses it throughout the test via `path.Base(testEnv.testDirPath)`. - `TearDownTest` methods are updated to call `setup.CleanUpDir(testEnv.testDirPath)` before `setup.UnmountGCSFuse` to ensure proper cleanup via the mount point. Co-authored-by: vadlakondaswetha <101323867+vadlakondaswetha@users.noreply.github.com> * Use random directory names per test in negative stat cache tests This change modifies `setup_test.go` and individual test files in `tools/integration_tests/negative_stat_cache/` to use randomly generated directory names for each test execution. This ensures that tests running in parallel or concurrently do not conflict with each other by using the same directory name in the bucket. - `mountGCSFuseAndSetupTestDir` now accepts a `testDirName` argument. - Each test suite (`disabledNegativeStatCacheTest`, `finiteNegativeStatCacheTest`, `infiniteNegativeStatCacheTest`) now generates a unique `testDir` in `SetupTest` and uses it throughout the test. - The generated directory name is explicitly stored in the test struct field `testDir` for robustness. - `TearDownTest` methods are updated to call `setup.CleanUpDir(testEnv.testDirPath)` before `setup.UnmountGCSFuse` to ensure proper cleanup via the mount point. Co-authored-by: vadlakondaswetha <101323867+vadlakondaswetha@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
… for shared-cache (#4365) * feat(shared-cache-lru): adding support of running cron based lru for shared-cache * gemini review comments * incorporating Geert's comment * minor read-me change * minor doc change * review comments
…e mounts (#4346) * feat(shared_cache): multi mount shared cache support Seems like working chunk based, removed locking and metadata file check simplifying the changes simplifying a bit simplify further: moved reader to gcsx more simplification and bug fixes more unit test and read optimization retry support for nfs file-system removing unused code totally relying on cfg.Shared Chunk size changing the cache-path based on sha256 removing script file from PR Adding logic to perform eviction fixing .bak deletion issues fixing minor issues in the unit test minor refacoring Minor refactoring fixing lint test making read/download robust fixing unit & lint failures fixing lint issue in the lru tool review comments for sccm * review comments * review comments * minor doc update
…rent_operations in non-parallel model (#4377) * fixing tests * fixing tests
…#4372) * run distributed benchamrks from gcsfuse-tools repository * pr reviews
This version contains some directpath fixes.
…der (#4676) Added a generalized kernelReader interface and a factory to support different kernel-optimized reading strategies based on bucket type, specifically adding a KernelRangeReader for regional buckets. Added implementation for KernelRangeReader as well.
### Description Add empty value to entry_status attribute as there are cases like cache entry not found where attributing the positive/negative value semantically does not make sense. ### Link to the issue in case of a bug fix. b/516971179 ### Testing details 1. Manual - NA 2. Unit tests - Auto generated added . 3. Integration tests - NA ### Any backward incompatible change? If so, please explain. N/A
* Centralized CheckBucketAccess to util/client so it can be reused by other test suites. * Ensured CheckBucketAccess uses getBucketHandle to respect the billing project configuration (e.g., for requester-pays buckets).
…lag during bucket mounting (#4722)
…dle transient token expiration in storage control client (#4746) * fix: add Unauthenticated to retryable gRPC status codes to handle transient token expiration in storage control client * fix review comments.
…um fixes and server closed idle connection fixes (#4756)
* Add --experimental-enable-optimized-metadata-cache flag * Run go generate to automatically create config.go and metrics * fixed formatting of generated files * changed usage prompt * corrected formatting of generated files
…manual runs. (#4751) * refactor: update build script to use current commit for E2E tests and remove hardcoded bucket location * chore: update test build scripts to target the latest commit instead of yesterday's commit * refactor: fix more log messages * fix: adjust commit ID selection in continuous test scripts to ensure kokoro initiated runs use the yesterday commit
…lability (#4762) * refactor: update test suite to retrieve bucket name via TestBucket helper method
#4731) ### Description Under the new library authentication flow, GCSFuse queries the local Metadata Server to resolve the credentials' Universe Domain. During GKE Autopilot scale-up, network initialization timing and lack of metadata server endpoint support on commercial GCP cause this query to persistently time out. GCSFuse wraps this query with its default retry settings (5-minute budget), stalling FUSE volume mounts for 5 minutes during pod creation. This change bypasses the metadata server query and immediately defaults the domain to "googleapis.com" under standard commercial GCP setups (using ADC with no custom key file, custom endpoint, or custom domain environment variable configured), eliminating the mount stall entirely. For private/sovereign clouds, the active metadata server check and the robust 5-minute retry safety margin are fully preserved. ### Link to the issue in case of a bug fix. b/500248957 ### Testing details 1. Manual: Verified that standard GCP ADC mounts immediately bypass the lookup and log "Bypassing UniverseDomain metadata server lookup...". 2. Unit tests: Verified that all storageutil unit tests pass successfully. 3. Compilation: Verified clean build using "make build".
* add integration tests skill.md * style(skills): fix formatting of integration test Go templates * docs(skills): simplify planning phase and remove agent sign-off rules * comment resolve
…5 updates (#4768) Bumps the go-dependencies group with 7 updates in the / directory: | Package | From | To | | --- | --- | --- | | [github.com/prometheus/common](https://github.com/prometheus/common) | `0.67.5` | `0.68.1` | | [go.opentelemetry.io/contrib/detectors/gcp](https://github.com/open-telemetry/opentelemetry-go-contrib) | `1.43.0` | `1.44.0` | | [go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc](https://github.com/open-telemetry/opentelemetry-go-contrib) | `0.68.0` | `0.69.0` | | [go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace](https://github.com/open-telemetry/opentelemetry-go-contrib) | `0.68.0` | `0.69.0` | | [go.opentelemetry.io/otel/exporters/prometheus](https://github.com/open-telemetry/opentelemetry-go) | `0.65.0` | `0.66.0` | | [go.opentelemetry.io/otel/exporters/stdout/stdouttrace](https://github.com/open-telemetry/opentelemetry-go) | `1.43.0` | `1.44.0` | | [google.golang.org/api](https://github.com/googleapis/google-api-go-client) | `0.280.0` | `0.283.0` | Updates `github.com/prometheus/common` from 0.67.5 to 0.68.1 - [Release notes](https://github.com/prometheus/common/releases) - [Changelog](https://github.com/prometheus/common/blob/main/CHANGELOG.md) - [Commits](prometheus/common@v0.67.5...v0.68.1) Updates `go.opentelemetry.io/contrib/detectors/gcp` from 1.43.0 to 1.44.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go-contrib@v1.43.0...v1.44.0) Updates `go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc` from 0.68.0 to 0.69.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.68.0...zpages/v0.69.0) Updates `go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace` from 0.68.0 to 0.69.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.68.0...zpages/v0.69.0) Updates `go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp` from 0.68.0 to 0.69.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.68.0...zpages/v0.69.0) Updates `go.opentelemetry.io/otel` from 1.43.0 to 1.44.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.43.0...v1.44.0) Updates `go.opentelemetry.io/otel/exporters/prometheus` from 0.65.0 to 0.66.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@exporters/prometheus/v0.65.0...metric/x/v0.66.0) Updates `go.opentelemetry.io/otel/exporters/stdout/stdouttrace` from 1.43.0 to 1.44.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.43.0...v1.44.0) Updates `go.opentelemetry.io/otel/metric` from 1.43.0 to 1.44.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.43.0...v1.44.0) Updates `go.opentelemetry.io/otel/sdk` from 1.43.0 to 1.44.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.43.0...v1.44.0) Updates `go.opentelemetry.io/otel/sdk/metric` from 1.43.0 to 1.44.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.43.0...v1.44.0) Updates `go.opentelemetry.io/otel/trace` from 1.43.0 to 1.44.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.43.0...v1.44.0) Updates `golang.org/x/net` from 0.54.0 to 0.55.0 - [Commits](golang/net@v0.54.0...v0.55.0) Updates `golang.org/x/sys` from 0.44.0 to 0.45.0 - [Commits](golang/sys@v0.44.0...v0.45.0) Updates `google.golang.org/api` from 0.280.0 to 0.283.0 - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](googleapis/google-api-go-client@v0.280.0...v0.283.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-version: 0.68.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/contrib/detectors/gcp dependency-version: 1.44.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc dependency-version: 0.69.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace dependency-version: 0.69.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp dependency-version: 0.69.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel dependency-version: 1.44.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/exporters/prometheus dependency-version: 0.66.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/exporters/stdout/stdouttrace dependency-version: 1.44.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/metric dependency-version: 1.44.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/sdk dependency-version: 1.44.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/sdk/metric dependency-version: 1.44.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: go.opentelemetry.io/otel/trace dependency-version: 1.44.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: golang.org/x/net dependency-version: 0.55.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: golang.org/x/sys dependency-version: 0.45.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies - dependency-name: google.golang.org/api dependency-version: 0.283.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* build(deps): bump urllib3 Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.6.3 to 2.7.0. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](urllib3/urllib3@2.6.3...2.7.0) --- updated-dependencies: - dependency-name: urllib3 dependency-version: 2.7.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump urllib3 in /perfmetrics/scripts/micro_benchmarks Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.6.3 to 2.7.0. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](urllib3/urllib3@2.6.3...2.7.0) --- updated-dependencies: - dependency-name: urllib3 dependency-version: 2.7.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump urllib3 Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.6.3 to 2.7.0. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](urllib3/urllib3@2.6.3...2.7.0) --- updated-dependencies: - dependency-name: urllib3 dependency-version: 2.7.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump idna Bumps [idna](https://github.com/kjd/idna) from 3.11 to 3.15. - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.md) - [Commits](kjd/idna@v3.11...v3.15) --- updated-dependencies: - dependency-name: idna dependency-version: '3.15' dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump idna in /perfmetrics/scripts/testing_on_gke/examples Bumps [idna](https://github.com/kjd/idna) from 3.11 to 3.15. - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.md) - [Commits](kjd/idna@v3.11...v3.15) --- updated-dependencies: - dependency-name: idna dependency-version: '3.15' dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump idna in /perfmetrics/scripts/micro_benchmarks Bumps [idna](https://github.com/kjd/idna) from 3.11 to 3.15. - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.md) - [Commits](kjd/idna@v3.11...v3.15) --- updated-dependencies: - dependency-name: idna dependency-version: '3.15' dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump idna from 3.7 to 3.15 in /perfmetrics/scripts/bigquery Bumps [idna](https://github.com/kjd/idna) from 3.7 to 3.15. - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.md) - [Commits](kjd/idna@v3.7...v3.15) --- updated-dependencies: - dependency-name: idna dependency-version: '3.15' dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump idna from 3.7 to 3.15 in /perfmetrics/scripts Bumps [idna](https://github.com/kjd/idna) from 3.7 to 3.15. - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.md) - [Commits](kjd/idna@v3.7...v3.15) --- updated-dependencies: - dependency-name: idna dependency-version: '3.15' dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump pyarrow in /perfmetrics/scripts/micro_benchmarks Bumps [pyarrow](https://github.com/apache/arrow) from 22.0.0 to 23.0.1. - [Release notes](https://github.com/apache/arrow/releases) - [Commits](apache/arrow@apache-arrow-22.0.0...apache-arrow-23.0.1) --- updated-dependencies: - dependency-name: pyarrow dependency-version: 23.0.1 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump pyarrow from 22.0.0 to 23.0.1 in /perfmetrics/scripts Bumps [pyarrow](https://github.com/apache/arrow) from 22.0.0 to 23.0.1. - [Release notes](https://github.com/apache/arrow/releases) - [Commits](apache/arrow@apache-arrow-22.0.0...apache-arrow-23.0.1) --- updated-dependencies: - dependency-name: pyarrow dependency-version: 23.0.1 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…rites & Truncate to Stale Size) (#4748) ### Problem & Root Cause When executing streaming writes (and fallback legacy staged writes) on **Zonal/Rapid buckets**, files exist on GCS in an unfinalized state. This unfinalized state causes two major failures in `syncer.SyncObject`: 1. **Incorrect `DirtyThreshold` Sanity Check Violation:** GCS metadata stat queries sometimes report the stale size of unfinalized zonal objects as `X` (`srcSize = X`), even though the object contains actual data that GCSFuse has successfully downloaded and staged locally upto Y (Y > X). When local edits are made, the local staged file has `sr.DirtyThreshold > X`. The sanity check: ```go if sr.DirtyThreshold > srcSize { ... error("stat returned weird DirtyThreshold") } ``` was triggered because `DirtyThreshold > X` was compared against `srcSize = X`, causing application I/O errors and failing the sync. 2. **Silent Drop of Truncate to X:** If a user truncated an unfinalized file to `X` bytes: * Local file size `sr.Size` was `X` and `sr.DirtyThreshold` was `X`. * GCS metadata size `srcSize` was reported as `X`. * The early clean-return check: ```go if sr.Size == srcSize && sr.DirtyThreshold == srcSize { return } ``` incorrectly matched (`X == X && X == X`), causing GCSFuse to return early and silently drop the truncation. Stale data remained on GCS, violating consistency. --- ### Changes For Zonal Bucket we bypass the dirty sanity check and only rely on mtime for the early return optimisations.
…cts of the same generation. (#4780)
Reordered the fields in 'FileInode' and 'dirInode' structs to group fields of the same alignment together. This eliminates internal padding bytes inserted by the compiler. - FileInode size reduced from 504 bytes to 488 bytes (saving 16 bytes per file). - dirInode size reduced from 352 bytes to 344 bytes (saving 8 bytes per directory). This results in a ~3% reduction in inode cache memory footprint. For deep/large directory mounts with millions of cached files/directories, this saves ~16-24MB of RAM per 1M files. Added a static regression test 'TestInodeSizes' to assert the optimized sizes and prevent future alignment regressions.
- Avoid CPU starvation and deadlocks in GenBlockPool.Get() by using concurrent waits for local blocks and global permits. - Optimize block retrieval by preferentially reusing local blocks to avoid expensive syscall.Mmap allocations.
…4786) * perf(retry): enhance GCS client retry logging using SDK's experimental WithErrorFuncWithContext - Integrated the GCS Go SDK's experimental `WithErrorFuncWithContext` option in storage_handle.go, which provides a rich `RetryContext` (attempt, bucket, object, operation, invocation ID) during retry attempts. - Modified custom retryer to support `ShouldRetryWithContext` and `ShouldRetryWithMonitoringAndContext` which accept and process the `storage.RetryContext`. - Enhanced logRetry helper to format and print the SDK's retry details in addition to OpenTelemetry TraceID/SpanID, remaining time budget, and HTTP/gRPC-specific error details. - Added comprehensive unit tests TestShouldRetryLogsWarningWithContext and TestShouldRetryLogsWarningWithSDKRetryContext to verify context logging. - Formatted all documents to gofmt standards. * Completely switching to retry callback with context * minor update * minor refactoring * minor review comments * new line
…4788) * Fixing flaky unit test * fix(gcsx): resolve flakiness in file cache reader tests by destroying jobManager in teardown
…s for easier tracking of attempts in logs (#4718) * refactor: pass request IDs to retry execution logic and update mock expectations in tests * fix: remove error logging for non-retryable error. * refactor: simplify ShouldRetryWithRetryContext by inverting conditional return logic * refactor: simplify retry logging logic by removing conditional branching for retryCtx
Optimize LookupInode latency for Hierarchical Namespace (HNS) buckets by executing GetFolder and StatObject calls concurrently as a race.
* perf(internal/fs/inode): optimize fetchCoreEntity to avoid heap allocations
…-agnostic test (#4793) * test(e2e): enable gRPC for all packages non-protocol-agnostic test * fixing symlink config formatting
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync fork