Skip to content

[cifmw_cephadm] Support different ceph server and client versions #4012

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
fultonj wants to merge 1 commit into from
+89 −0
Closed

[cifmw_cephadm] Support different ceph server and client versions #4012

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions roles/cifmw_cephadm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,30 @@ that they do not need to be changed for a typical EDPM deployment.
Example values are `"aes,aes256k"` or `"aes256k"` or `"aes"`.
Defaults to `""` (unset, no command is run).

* `cifmw_cephadm_server_version`: (String) Optional override for Ceph server
version during cephadm installation. Both this and `cifmw_cephadm_client_version`
must be set (and different) to activate version override logic. When activated,
temporarily switches repos to install cephadm at this version, then restores
to client version. Example: `"7.1"` or `"9.1"`. Defaults to `""` (empty),
which uses `cifmw_repo_setup_rhos_release_args` for everything.

* `cifmw_cephadm_client_version`: (String) Optional client package version,
used with `cifmw_cephadm_server_version`. Both must be set (and different)
to activate version override logic. Example: `"9.1"`. Defaults to `""` (empty),
which uses `cifmw_repo_setup_rhos_release_args` for everything.

The `cifmw_cephadm_server_version` and `cifmw_cephadm_client_version` are
only used for testing a corner case where the ceph clients and ceph server
are of different versions. Usually the `repo_setup` role's parameter
`cifmw_repo_setup_rhos_release_args` determines which ceph repository is
enabled and the same repository is enabled for both clients and servers.
If the server version and client version differ, then the repository
from the server version is temporarily enabled, when cephadm is installed
(so that cephadm will install that version of the ceph server), and then
the repository from `cifmw_repo_setup_rhos_release_args` is enabled again.
If `cifmw_repo_setup_rhos_release_args` is not the same as the client
version, a warning is printed.

Use the `cifmw_cephadm_pools` list of dictionaries to define pools for
Nova (vms), Cinder (volumes), Cinder-backups (backups), and Glance (images).
```
Expand Down
10 changes: 10 additions & 0 deletions roles/cifmw_cephadm/defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,16 @@ cifmw_cephadm_repository_override: false
# we usually support N+1 Ceph version: Adding Squid here because Reef is the
# default version installed in a greenfield scenario
cifmw_cephadm_version: "squid"
# Optional: Override Ceph server version for cephadm installation
# Both server_version and client_version must be set (and different) to activate
# version override logic. Will temporarily switch repos to install cephadm at
# server_version, then restore to client_version. Example: "7.1" or "9.1"
# Leave empty (default) to use cifmw_repo_setup_rhos_release_args for everything
cifmw_cephadm_server_version: ""
# Optional: Client package version (used with server_version)
# Must be set along with server_version (and different) to activate override logic
# Leave empty (default) to use cifmw_repo_setup_rhos_release_args for everything
cifmw_cephadm_client_version: ""
# bug in cephadm: if you skip "prepare host" it will fail
cifmw_cephadm_prepare_host: false
cifmw_cephadm_wait_install_retries: 8
Expand Down
55 changes: 55 additions & 0 deletions roles/cifmw_cephadm/tasks/install_cephadm.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,45 @@
# License for the specific language governing permissions and limitations
# under the License.

- name: Check if server/client version override is configured

@fmount fmount Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the new parameters are free form strings, should we guard against bad input, like:

    - name: Validate version override format
      ansible.builtin.assert:
        that:
          - item is match('^[0-9]+\.[0-9]+$')
        fail_msg: >-
          '{{ item }}' must be numeric X.Y format (e.g. '7.1', '9.1')
          or empty string to disable.
      loop:
        - "{{ cifmw_cephadm_server_version | default('7.1') }}"
        - "{{ cifmw_cephadm_client_version | default('8.0') }}"

ansible.builtin.set_fact:
_cifmw_cephadm_version_override: >-
{{
(cifmw_cephadm_server_version | length > 0) and
(cifmw_cephadm_client_version | length > 0) and
(cifmw_cephadm_server_version != cifmw_cephadm_client_version)
}}

- name: Handle version override for cephadm installation
when: _cifmw_cephadm_version_override | bool
become: true
block:
- name: Parse version from cifmw_repo_setup_rhos_release_args
ansible.builtin.set_fact:
_cifmw_cephadm_default_version: "{{ cifmw_repo_setup_rhos_release_args | regex_search('ceph-([0-9.]+)') | regex_replace('^ceph-', '') }}"

@fmount fmount Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's say that I set _server and _client versions as input that triggers this block, and I reach L32. If somehow cifmw_repo_setup_rhos_release_args is not defined at this level, we fail.
It's good that we fail (in that case there's nothing to recover), but I'm wondering if cifmw_repo_setup_rhos_release_args should be part of the input that triggers this block. In addition, if the regex doesn't match any ceph-x.y, I'm wondering if we want to explicitly fail (it would help for debugging purposes to identify almost immediately where the problem is).

@evallesp evallesp Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree on this, we can check if 'ceph-([0-9.]+)' is there at L20. We stop early and we don't let the playbook running with unexpected results!


- name: Display version configuration
ansible.builtin.debug:
msg:
- "Default: {{ _cifmw_cephadm_default_version }}"
- "Server: {{ cifmw_cephadm_server_version }}"
- "Client: {{ cifmw_cephadm_client_version }}"
- "{{ 'WARNING: Client version differs from default rhos-release version' if (_cifmw_cephadm_default_version | trim) != (cifmw_cephadm_client_version | trim) else 'Client version matches default' }}"

@fmount fmount Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to print cifmw_repo_setup_rhos_release_args as part of the debug statement?


- name: Disable default ceph repository

@fmount fmount Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if the Disable tasks that use a regexp are really useful.
If we run rhos-release -x ceph-x.y, the tool ensures that only that repo is available at that time in the system.
This means that we could:

  1. remove L42-47
  2. modify L52 to take the -x option
    (continue with the cephadm install)
  3. remove L77-L83
  4. modify L89 to take -x option

This way we have the same effect and we let the tool explicitly manage the repos.

@tosky tosky Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does rhos-release -x really removes a single repository?
If that's the case, yes, let's not leak here internal details about how rhos-release deploys the repositories.

And if rhos-release didn't do this, I think it should be handled there anyway. We should not manipulate its repositories directly.

@fultonj fultonj Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No -x purges everything and then adds what's in the arguments.

when: cifmw_cephadm_server_version != _cifmw_cephadm_default_version
ansible.builtin.replace:
path: "/etc/yum.repos.d/rhos-release-ceph-{{ _cifmw_cephadm_default_version }}.repo"
regexp: '^enabled=1'
replace: 'enabled=0'

- name: Switch to server version repository
when: cifmw_cephadm_server_version != _cifmw_cephadm_default_version
ansible.builtin.command:
cmd: "rhos-release {{ cifmw_repo_setup_rhos_release_args | regex_replace('ceph-[0-9.]+', 'ceph-' + cifmw_cephadm_server_version) }}"
register: _cifmw_cephadm_server_repo
changed_when: "'Installed' in _cifmw_cephadm_server_repo.stdout"

- name: Enabled ceph Tools Repository
when:
- cifmw_cephadm_repository_override | bool
Expand All @@ -35,6 +74,22 @@
delay: "{{ cifmw_cephadm_wait_install_delay }}"
until: task_result is success

- name: Disable server ceph repository before restore
when: _cifmw_cephadm_version_override | bool
become: true
ansible.builtin.replace:
path: "/etc/yum.repos.d/rhos-release-ceph-{{ cifmw_cephadm_server_version }}.repo"
regexp: '^enabled=1'
replace: 'enabled=0'

- name: Restore cifmw_repo_setup_rhos_release_args
when: _cifmw_cephadm_version_override | bool
become: true
ansible.builtin.command:
cmd: "rhos-release {{ cifmw_repo_setup_rhos_release_args }}"
register: _cifmw_cephadm_restore_repo
changed_when: "'Installed' in _cifmw_cephadm_restore_repo.stdout"

- name: Stat cephadm file
ansible.builtin.stat:
path: "{{ cifmw_cephadm_bin }}"
Expand Down
Loading