Skip to content

PSHP-316: update golang and base image for Hypershift Operator CVE remediation#8823

Draft
BraeTroutman wants to merge 1 commit into
openshift:mainfrom
BraeTroutman:PSHP-316/june-cve
Draft

PSHP-316: update golang and base image for Hypershift Operator CVE remediation#8823
BraeTroutman wants to merge 1 commit into
openshift:mainfrom
BraeTroutman:PSHP-316/june-cve

Conversation

@BraeTroutman

@BraeTroutman BraeTroutman commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Updates the ubi-minimal base image used for hypershift operator and other images to make use of ubi-minimal:9.8.

Go mod version is bumped to 1.25.11. Since there is no 1.25.11 tag in the go-toolset image, build images are updated to 1.26.3, which works due to golang compilation backwards compatibility guarantees

these bumps address the below CVEs

CVE-2026-33846
CVE-2026-33845
CVE-2026-42009
CVE-2026-42010
CVE-2026-4878
CVE-2026-33811
CVE-2026-39836
CVE-2026-27145
CVE-2026-42499
CVE-2026-33814
CVE-2026-42504
CVE-2026-39820

Which issue(s) this PR fixes:

Fixes #PSHP-316

Special notes for your reviewer:

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Summary by CodeRabbit

Summary by CodeRabbit

  • Chores
    • Updated operator and control-plane container base images to newer supported tags (builder and runtime).
    • Refreshed the Go toolchain version used for builds (including the Go toolchain specified in the build tooling).
    • Updated the CLI build stage to use a newer Go builder tag while keeping produced release artifacts the same.
    • Updated development and end-to-end container build images to a newer OpenShift/Golang combination without changing build or runtime behavior.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 24, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 24, 2026
@openshift-ci-robot

openshift-ci-robot commented Jun 24, 2026

Copy link
Copy Markdown

@BraeTroutman: This pull request references PSHP-316 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the vulnerability to target the "5.0.0" version, but no target version was set.

Details

In response to this:

addressing the below CVEs with golang minor version update, and bump to 9.8 ubi-minimal base image. ticket here

CVE-2026-33846
CVE-2026-33845
CVE-2026-42009
CVE-2026-42010
CVE-2026-4878
CVE-2026-33811
CVE-2026-39836
CVE-2026-27145
CVE-2026-42499
CVE-2026-33814
CVE-2026-42504
CVE-2026-39820

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes

Special notes for your reviewer:

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This pull request updates the Go directive in go.mod from 1.25.7 to 1.25.11 and bumps base image tags in Containerfile.cli, Containerfile.control-plane, Containerfile.operator, Dockerfile.dev, and Dockerfile.e2e. No other build steps, runtime instructions, module dependencies, or code logic are changed.


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error)

Check name Status Explanation Resolution
No-Weak-Crypto ❌ Error The diff adds weak-crypto names in vendored AWS S3 enums (ChecksumAlgorithmMd5, ChecksumAlgorithmSha1), which triggers the check. Remove or justify the vendored additions, or exempt generated/vendor dependency updates if these checksum enums are intentional and not app crypto.
✅ Passed checks (10 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: updating Go and base images for CVE remediation.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Changed Ginkgo names are static/descriptive only; no titles include dynamic values like namespaces, pods, UUIDs, or timestamps.
Test Structure And Quality ✅ Passed Only Containerfile/Dockerfile/go.mod files changed; no Ginkgo test code or test patterns were touched, so this test-quality check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed Only image/base-tag files and go.mod changed; no manifests, controllers, or scheduling constraints were modified.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No Ginkgo e2e test files were added or modified; the commit only changes build images and go.mod, so this compatibility check is not applicable.
Container-Privileges ✅ Passed Only image/toolchain tags changed in Dockerfiles/Containerfiles and go.mod; no privileged, host*, SYS_ADMIN, or allowPrivilegeEscalation settings were added.
No-Sensitive-Data-In-Logs ✅ Passed Changed files only bump image tags and go version; no new logging or secret/PII-bearing output was introduced.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 42.59%. Comparing base (bc3bda9) to head (af243dd).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8823      +/-   ##
==========================================
+ Coverage   42.55%   42.59%   +0.03%     
==========================================
  Files         768      768              
  Lines       95297    95359      +62     
==========================================
+ Hits        40558    40617      +59     
- Misses      51932    51934       +2     
- Partials     2807     2808       +1     

see 3 files with indirect coverage changes

Flag Coverage Δ
cmd-support 35.46% <ø> (ø)
cpo-hostedcontrolplane 44.84% <ø> (ø)
cpo-other 44.94% <ø> (+0.24%) ⬆️
hypershift-operator 53.05% <ø> (ø)
other 31.69% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@go.mod`:
- Line 3: The Go toolchain version is now pinned to 1.25.11 in go.mod, but the
build images still reference older toolchains, so update the pinned Go version
in Containerfile.operator, Containerfile.control-plane, and
Dockerfile.github-actions-runner to 1.25.11 or newer. Make sure the changes are
consistent across these image definitions so any go build or related build step
runs with the same required toolchain version.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 532c08d2-0f91-4683-8dbd-9c194c3134aa

📥 Commits

Reviewing files that changed from the base of the PR and between bc3bda9 and 85e7081.

📒 Files selected for processing (2)
  • Containerfile.operator
  • go.mod

Comment thread go.mod
@BraeTroutman BraeTroutman force-pushed the PSHP-316/june-cve branch 2 times, most recently from ece88ee to c6e575c Compare June 24, 2026 15:00
@openshift-ci openshift-ci Bot added area/ci-tooling Indicates the PR includes changes for CI or tooling and removed do-not-merge/needs-area labels Jun 24, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Containerfile.operator`:
- Line 1: The Red Hat base image in the Containerfile is pinned to a
build-specific tag, so updates will not flow automatically. Update the FROM
reference for the builder stage, and also the runtime base image in the same
Containerfile, to use floating Red Hat tags instead of the fixed versions. Keep
the existing stage names and image sources intact while changing only the tag
values.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: bb672032-e585-4e42-bb10-1e50e0026bd5

📥 Commits

Reviewing files that changed from the base of the PR and between ece88ee and c6e575c.

⛔ Files ignored due to path filters (1)
  • hack/workspace/go.work is excluded by !**/*.work
📒 Files selected for processing (3)
  • Containerfile.control-plane
  • Containerfile.operator
  • go.mod
✅ Files skipped from review due to trivial changes (2)
  • Containerfile.control-plane
  • go.mod

Comment thread Containerfile.operator
@BraeTroutman BraeTroutman marked this pull request as ready for review June 24, 2026 16:09
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 24, 2026
@openshift-ci openshift-ci Bot requested review from Nirshal and clebs June 24, 2026 16:11

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Containerfile.control-plane`:
- Line 1: The builder and runtime base images are pinned to specific Red Hat
build tags, which blocks automatic Red Hat security updates. Update the
Containerfile.control-plane FROM directives for both the builder stage and the
ubi-minimal runtime stage to use floating Red Hat tags instead of build-specific
version suffixes, following the Red Hat image policy while keeping non-Red Hat
images pinned by digest.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 2294a0aa-a572-47a5-9ff5-2501443bd361

📥 Commits

Reviewing files that changed from the base of the PR and between f7f4c3f and 5d772c6.

⛔ Files ignored due to path filters (1)
  • hack/workspace/go.work is excluded by !**/*.work
📒 Files selected for processing (6)
  • Containerfile.cli
  • Containerfile.control-plane
  • Containerfile.operator
  • Dockerfile.dev
  • Dockerfile.e2e
  • go.mod
✅ Files skipped from review due to trivial changes (1)
  • Dockerfile.dev
🚧 Files skipped from review as they are similar to previous changes (1)
  • go.mod

@@ -1,4 +1,4 @@
FROM registry.access.redhat.com/ubi9/go-toolset:1.25.9-1778054913 AS builder
FROM registry.access.redhat.com/ubi9/go-toolset:1.26.3-1782305929 AS builder

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Use floating Red Hat base image tags.

Both the go-toolset builder (Line 1) and the ubi-minimal runtime (Line 11) are pinned to build-specific tags (1.26.3-1782305929, 9.8-1782191395). Pinning these Red Hat bases prevents Red Hat-managed security updates from flowing automatically. Use floating Red Hat tags instead.

🔒 Proposed change
-FROM registry.access.redhat.com/ubi9/go-toolset:1.26.3-1782305929 AS builder
+FROM registry.access.redhat.com/ubi9/go-toolset:1.26 AS builder
-FROM registry.access.redhat.com/ubi9/ubi-minimal:9.8-1782191395
+FROM registry.access.redhat.com/ubi9/ubi-minimal:9.8
As per path instructions: "Red Hat images: use floating tags (Red Hat manages updates); non-RH images: pin by digest".

Also applies to: 11-11

🧰 Tools
🪛 Trivy (0.69.3)

[error] 1-1: Image user should not be 'root'

Specify at least 1 USER command in Dockerfile with non-root user as argument

Rule: DS-0002

Learn more

(IaC/Dockerfile)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Containerfile.control-plane` at line 1, The builder and runtime base images
are pinned to specific Red Hat build tags, which blocks automatic Red Hat
security updates. Update the Containerfile.control-plane FROM directives for
both the builder stage and the ubi-minimal runtime stage to use floating Red Hat
tags instead of build-specific version suffixes, following the Red Hat image
policy while keeping non-Red Hat images pinned by digest.

Source: Path instructions

@hypershift-jira-solve-ci

hypershift-jira-solve-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

Test Failure Analysis Complete

Job Information

  • Prow Job: pull-ci-openshift-hypershift-main-verify-deps
  • Build ID: 2069819714275643392
  • Target: verify-deps
  • PR: #8823PSHP-316: update golang and base image for Hypershift Operator CVE remediation
  • Author: BraeTroutman

Test Failure Analysis

Error

go: go.mod requires go >= 1.25.11 (running go 1.25.8; GOTOOLCHAIN=local)

Summary

The verify-deps CI step failed because the PR updated go.mod to require Go >= 1.25.11 but did not update .ci-operator.yaml, which still configures the CI build root image as rhel-9-release-golang-1.25-openshift-4.23 (shipping Go 1.25.8). When the go-verify-deps step runs go mod tidy inside that build root image, the Go toolchain refuses to proceed because GOTOOLCHAIN=local prevents automatic toolchain downloading and the installed Go version (1.25.8) is older than the minimum required by go.mod (1.25.11).

Root Cause

PR #8823 is a Go toolchain and base image upgrade for CVE remediation. It correctly updated:

  1. go.mod: bumped the go directive from 1.25.71.25.11
  2. CI Dockerfiles (Dockerfile, Dockerfile.control-plane, Dockerfile.e2e, Dockerfile.dev): switched builder images from golang-1.25golang-1.26
  3. Production Containerfiles (Containerfile.cli, Containerfile.control-plane, Containerfile.operator): switched to golang-1.26 builders and ubi-minimal:9.8 base images

However, the PR did not update .ci-operator.yaml, which controls the build_root_image used by CI for non-build steps like verify-deps. This file still contains:

build_root_image:
  name: release
  namespace: openshift
  tag: rhel-9-release-golang-1.25-openshift-4.23

This image provides Go 1.25.8 (Red Hat 1.25.8-1.el9_6). The go-verify-deps CI step runs inside this build root image and executes go mod tidy. Since Go's toolchain policy with GOTOOLCHAIN=local (the CI default) prohibits running a Go version older than what go.mod requires, the command fails immediately:

go: go.mod requires go >= 1.25.11 (running go 1.25.8; GOTOOLCHAIN=local)

The mismatch is: go.mod requires 1.25.11, but the CI build root only has 1.25.8.

Recommendations
  1. Update .ci-operator.yaml to use a Go 1.26 build root image matching the Dockerfile changes:

    build_root_image:
      name: release
      namespace: openshift
      tag: rhel-9-release-golang-1.26-openshift-4.23

    This ensures the CI build root image matches the Go version used in the Dockerfiles and satisfies the go.mod requirement.

  2. Alternatively, if you want to stay on Go 1.25.x for the build root (while Dockerfiles use 1.26), update the tag to a Go 1.25 image that ships >= 1.25.11. However, the cleaner approach is option 1, keeping all Go versions aligned at 1.26.

  3. After updating, re-run go mod vendor and go mod tidy locally with Go 1.26 to ensure vendor/ is consistent, then push the updated .ci-operator.yaml to the PR.

Evidence
Evidence Detail
Failing step verify-deps-go-verify-deps (test phase)
Error message go: go.mod requires go >= 1.25.11 (running go 1.25.8; GOTOOLCHAIN=local)
Go version in CI build root go1.25.8 (Red Hat 1.25.8-1.el9_6) linux/amd64
Go version required by go.mod >= 1.25.11 (changed from 1.25.7 in this PR)
Build root image openshift/release:rhel-9-release-golang-1.25-openshift-4.23 (from .ci-operator.yaml)
Dockerfile builder images Updated to golang-1.26-openshift-4.23 (in PR diff)
.ci-operator.yaml changed in PR No — still references golang-1.25
Step reference go-verify-deps
Failing command go mod tidy (inside go-verify-deps script)

@BraeTroutman BraeTroutman force-pushed the PSHP-316/june-cve branch 2 times, most recently from 52f66b5 to a7c96d9 Compare June 24, 2026 17:30
@openshift-ci openshift-ci Bot added the area/api Indicates the PR includes changes for the API label Jun 24, 2026
@openshift-ci

openshift-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

@BraeTroutman: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@BraeTroutman BraeTroutman marked this pull request as draft June 24, 2026 19:04
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 24, 2026
@clebs

clebs commented Jun 26, 2026

Copy link
Copy Markdown
Member

/approve

@openshift-ci

openshift-ci Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: BraeTroutman, clebs
Once this PR has been reviewed and has the lgtm label, please assign csrwng for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/api Indicates the PR includes changes for the API area/ci-tooling Indicates the PR includes changes for CI or tooling do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants