Skip to content

[SPARK-57702][INFRA] Move scheduled CIs for 3.5 to branch-3.5#56783

Open
gaogaotiantian wants to merge 3 commits into
apache:masterfrom
gaogaotiantian:branch35-scheduler
Open

[SPARK-57702][INFRA] Move scheduled CIs for 3.5 to branch-3.5#56783
gaogaotiantian wants to merge 3 commits into
apache:masterfrom
gaogaotiantian:branch35-scheduler

Conversation

@gaogaotiantian

@gaogaotiantian gaogaotiantian commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Add a unified entry for all scheduled CIs for branch-3.5 (branch35_scheduler.yml). It uses gh workflow to trigger the self-contained build workflows on branch-3.5, and removes the per-build build_branch35*.yml files from master.

This follows the same approach as SPARK-56990 (#56046) for branch-4.x, SPARK-57327 (#56379) for branch-4.1, and SPARK-57483 (#56530) for branch-4.0. SPARK-57621 (#56679) already laid the ground on branch-3.5 by making the build workflows self-contained and dispatchable, so this PR only needs to change the master scheduled tasks.

The scheduler triggers the following targets on branch-3.5 (also exposed via workflow_dispatch): build_scala213, build_python_3.9. The cron schedule preserves the original 0 11 */4 * * time and */4 frequency from the build_branch35*.yml files (both builds shared 0 11, as before).

README.md is updated so the branch-3.5 badges point at the self-contained workflows filtered by ?branch=branch-3.5.

Why are the changes needed?

This is part of decoupling our CIs. All branch-3.5 related CIs should only rely on files on branch-3.5, with the exception of this new scheduler file which is needed on master to trigger scheduled tasks (scheduled workflows only fire from the default branch).

Does this PR introduce any user-facing change?

No. CI only.

How was this patch tested?

These workflows can be triggered manually via workflow_dispatch once merged.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 4.8)

Add a unified entry for all scheduled CIs for branch-3.5
(`branch35_scheduler.yml`). It uses `gh workflow` to trigger the
self-contained build workflows on `branch-3.5`, and removes the per-build
`build_branch35*.yml` files from `master`.

Co-authored-by: Isaac
Add no-op placeholder `build_scala213.yml` and `build_python_3.9.yml` on
`master` so the README build status badges (which resolve a workflow by its
path on the default branch) render for `branch-3.5`. Neither build runs on
`master`: Scala 2.13 is the default there, and master targets newer Python
versions. The actual builds are defined in the same files on `branch-3.5`.

Co-authored-by: Isaac
@gaogaotiantian

Copy link
Copy Markdown
Contributor Author

@holdenk this PR will trigger scheduled CIs from master then use branch-3.5 code to run the CI, instead of relying on build_and_test.yml on master.

@HyukjinKwon HyukjinKwon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 blocking, 1 non-blocking, 0 nits.
Correct and consistent with the branch-4.x scheduler pattern — I verified that build_scala213.yml, build_python_3.9.yml, and build_main.yml all exist on branch-3.5 (so the gh workflow run --ref branch-3.5 triggers and the README badges resolve). One description/clarity follow-up.

Design / architecture (1)

  • build_scala213.yml:20: description says these files are "removed," but they're renamed to no-op placeholders kept for badge resolution — worth documenting — see inline

PR description suggestions

  • Correct: "removes the per-build build_branch35*.yml files" — they are renamed to no-op placeholders (build_scala213.yml / build_python_3.9.yml), not removed.
  • Document why the master placeholders are retained: a README badge resolves its workflow by path on the default branch, so a same-named file must exist on master for the ?branch=branch-3.5 badge to render (the real build runs on branch-3.5).

#

name: "Build (branch-3.5, Scala 2.13, JDK 8)"
# This is a placeholder. Scala 2.13 is the default on master, so there is no

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice trick, and the comment here explains it well. One gap: the PR description says it "removes the per-build build_branch35*.yml files from master," but they're actually renamed to these no-op placeholders and deliberately kept — because a README badge resolves its workflow by path on the default branch, so the file must exist on master for the ?branch=branch-3.5 badge to render.

Worth stating that in the PR description too: a maintainer who reads "removed" might later delete these no-op files as dead code and silently break the branch-3.5 badges. (Verified build_scala213.yml / build_python_3.9.yml / build_main.yml all exist on branch-3.5, so the scheduler triggers and badges resolve correctly.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants