Fix bucket usage retrieval workflow for Ceph RGW - #13776
Open
bernardodemarco wants to merge 1 commit into
Open
Conversation
DaanHoogland
approved these changes
Aug 3, 2026
DaanHoogland
left a comment
DaanHoogland
left a comment
Contributor
There was a problem hiding this comment.
clgtm, let’s hope this is the only condition that needs adding?
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 4.22 #13776 +/- ##
=========================================
Coverage 17.69% 17.69%
Complexity 15835 15835
=========================================
Files 5925 5925
Lines 533539 533543 +4
Branches 65274 65275 +1
=========================================
+ Hits 94427 94428 +1
- Misses 428435 428439 +4
+ Partials 10677 10676 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Each Management Server node runs a background task every hour to fetch bucket usage information for each Object Storage provider registered in the ACS environment. To retrieve the statistics for Ceph RGW, the workflow triggers the
listBucketInfo()S3 client method, which is equivalent to theradosgw-admin bucket stats --bucket=<bucket-name>command.The size of each bucket is obtained from the
size_kbfield contained in theusageJSON object. However, the workflow does not handle the case where theusageobject is empty, which primarily occurs for empty S3 buckets. The response schema for an empty bucket is shown below:radosgw-admin bucket statsresponse schema for an empty bucket{ "bucket": "empty-bucket-outside-acs", "tenant": "", "versioning": "off", "zonegroup": "13e2e428-c665-4fba-9a40-79c966da029c", "placement_rule": "default-placement", "explicit_placement": { "data_pool": "", "data_extra_pool": "", "index_pool": "" }, "id": "d25c72a0-a1f0-4033-a954-53bdff513512.54133.5", "marker": "d25c72a0-a1f0-4033-a954-53bdff513512.54133.5", "index_type": "Normal", "index_generation": 0, "num_shards": 11, "object_lock_enabled": false, "mfa_enabled": false, "owner": "admin-s3-user", "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1", "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", "mtime": "2026-08-03T13:43:38.160905Z", "creation_time": "2026-08-03T13:43:38.152185Z", "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", "usage": {}, "bucket_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 } }And, the response schema for a non-empty bucket is shown below:
radosgw-admin bucket statsresponse schema for a non-empty bucket{ "bucket": "screenshots", "tenant": "", "versioning": "off", "zonegroup": "13e2e428-c665-4fba-9a40-79c966da029c", "placement_rule": "default-placement", "explicit_placement": { "data_pool": "", "data_extra_pool": "", "index_pool": "" }, "id": "d25c72a0-a1f0-4033-a954-53bdff513512.54133.4", "marker": "d25c72a0-a1f0-4033-a954-53bdff513512.54133.4", "index_type": "Normal", "index_generation": 0, "num_shards": 11, "object_lock_enabled": false, "mfa_enabled": false, "owner": "3af0d458-ecbd-11f0-8ad4-0e2ebc784cf3", "ver": "0#3,1#3,2#1,3#4,4#1,5#2,6#4,7#3,8#9,9#3,10#2", "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", "mtime": "2026-08-03T13:41:32.888630Z", "creation_time": "2026-08-03T13:41:21.864423Z", "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", "usage": { "rgw.main": { "size": 35420402, "size_actual": 35459072, "size_utilized": 35420402, "size_kb": 34591, "size_kb_actual": 34628, "size_kb_utilized": 34591, "num_objects": 24 } }, "bucket_quota": { "enabled": true, "check_on_raw": false, "max_size": 10737418240, "max_size_kb": 10485760, "max_objects": -1 } }As a consequence, if there are any empty buckets in the Ceph RGW provider, regardless of whether they are managed by ACS or not, the following NPE is thrown and the entire bucket usage retrieval workflow is aborted. As a result, the size of all buckets is reported as zero.
NPE that causes the whole buckets usage retrieval workflow to be aborted
Therefore, this PR fixes this issue by skipping the retrieval of bucket size information when the
usageJSON object is empty.Fixes #13224
Types of changes
Feature/Enhancement Scale or Bug Severity
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
Registered a Ceph RGW provider;
Created two buckets through ACS;
Uploaded some objects to these buckets;
Created empty buckets not managed by ACS;
Created an empty bucket through ACS;
Verified the logs of the bucket usage retrieval workflow;
Logs
Verified the return of the
listBucketsAPI:listBucketsAPI execution