fix(renovate): Shard org run to avoid 1h app token expiry#533
Closed
mnorbury wants to merge 1 commit into
Closed
Conversation
The org-wide autodiscover run could exceed the 1-hour lifetime of the GitHub App installation token, causing 401s mid-run. Split the run into a discover job that chunks repos into small shards, then a parallel matrix job where each shard mints its own fresh token and processes a few repos via RENOVATE_REPOSITORIES, finishing well within the token window.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the Renovate GitHub Actions workflow to avoid GitHub App installation token expiry during long org-wide autodiscover runs by splitting work into multiple parallel shards, each with its own freshly minted token.
Changes:
- Add a
discoverjob that lists accessible (non-archived) repos and builds a shard matrix (configurable viarepos_per_shard). - Convert
renovateinto a matrix job that runs Renovate per shard usingRENOVATE_REPOSITORIESand a per-shard cache key. - Reduce the per-job timeout to 55 minutes to better align with the 1-hour App token lifetime.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+71
to
+73
| matrix="$(printf '%s\n' "${repos}" | jq -R . | jq -s \ | ||
| --argjson size "${REPOS_PER_SHARD}" \ | ||
| '[ _nwise($size) | join(",") ] | to_entries | map({ id: .key, repos: .value })')" |
Comment on lines
+52
to
+53
| set -euo pipefail | ||
|
|
Contributor
Author
|
Closed in favour of #534 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Renovate workflow runs
autodiscoveracross the whole org under a single GitHub App installation token, which GitHub hard-caps at 1 hour. Long runs outlived the token and started returning401 Bad credentialsmid-run (Renovate's CLI does not refresh the token). The following shows how most runs finish at ~1hour after a burst of 401s when the token runs out.Fix
Split the run into a
discoverjob that lists the installation's repos (excluding archived) and chunks them into small shards, then a parallelrenovatematrix job where each shard mints its own fresh token and processes only its few repos (default 4) viaRENOVATE_REPOSITORIES— so every Renovate process finishes well within the 1-hour token window. Shard size is configurable via a newrepos_per_shardinput, the matrix job timeout drops to 55m, and the cache key is now per-shard.