fix(sts): rotate TLS certificates - #222
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cfa517ab4f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| namespace = %config.namespace, | ||
| "rotating managed operator STS TLS certificate" | ||
| ); | ||
| replace_generated_secret(api, config, &secret).await |
There was a problem hiding this comment.
When a managed certificate enters the 30-day renewal window, this replacement generates a new self-signed CA as well as a new leaf and immediately hot-loads the leaf. Clients that copied or cached the previous ca.crt—especially workloads in other namespaces, since the repository provides no CA distribution or overlap mechanism—will reject every subsequent STS connection until their trust store is manually updated. Preserve a longer-lived CA while rotating leaf certificates, or distribute/overlap the new trust anchor before switching the server certificate.
Useful? React with 👍 / 👎.
| let certificate = rustls_pemfile::certs(&mut Cursor::new(pem)) | ||
| .next() |
There was a problem hiding this comment.
When an externally managed ca.crt contains multiple PEM certificates during a CA rollover, this .next() inspects only the first entry. If that old entry is expired while a later CA is valid and signs the active leaf, validated_material_from_secret rejects the entire Secret, preventing startup or hot-loading an otherwise usable replacement. Inspect the complete bundle and validate the applicable issuing chain instead of treating the first certificate as the sole CA.
Useful? React with 👍 / 👎.
| .sts_tls_certificate_expiry_timestamp_seconds | ||
| .load(Ordering::Relaxed) as f64, |
There was a problem hiding this comment.
When STS or its TLS support is disabled while the metrics endpoint remains enabled, neither expiry setter runs, so these default-initialized atomics are still rendered as Unix timestamp 0. An alert that follows the new documentation and checks whether the expiry timestamp is approaching or already passed will therefore fire continuously even though no STS certificate exists. Omit these series until TLS material has been loaded, or expose an explicit enabled/validity signal that alerts can use.
Useful? React with 👍 / 👎.
| let annotations = BTreeMap::from([( | ||
| POLICY_VERSION_ANNOTATION.to_string(), | ||
| POLICY_VERSION.to_string(), | ||
| )]); |
There was a problem hiding this comment.
When users or cluster automation add annotations to the Operator-managed Secret—for example, to mirror ca.crt into workload namespaces—the legacy migration and every scheduled renewal replace the Secret with an object whose annotation map contains only this policy marker. Because replace_generated_secret preserves only resourceVersion, the first rotation silently removes those integration annotations and can stop subsequent Secret propagation. Merge the existing annotations and then overwrite only the policy-version key.
Useful? React with 👍 / 👎.
1 participant
Type of Change
Related Issues
Closes rustfs/backlog#1091
Summary of Changes
sts-tlsmaterial (sts.tls.auto=false).tls.crt,tls.key, andca.crtare missing.OPERATOR_STS_TLS_AUTO=true.Checklist
make pre-commit(fmt-check + clippy + test + console-lint + console-fmt-check)[Unreleased](N/A)Impact
sts-tls. Existing externally managed material is hot-loaded after valid Secret updates. Kind/E2E development remains opted in to automatic generation.Verification
Additional Notes
A live Kubernetes rotation was not run locally. Unit and manifest tests cover certificate validity, early renewal, legacy policy detection, external-certificate ownership defaults, missing-Secret guidance, development auto-generation, expired material rejection, and metrics.
Thank you for your contribution! Please ensure your PR follows the community standards (CODE_OF_CONDUCT.md) and sign the CLA if this is your first contribution.