Skip to content

Improvement/CLDSRV-825 drop ceph support #6097

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
benzekrimaha wants to merge 4 commits into
base: development/9.3
Choose a base branch
from
Open

Improvement/CLDSRV-825 drop ceph support #6097

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6ac4e10
Remove Ceph runtime support and capability flags
benzekrimaha Mar 9, 2026
ccb9542
Align functional tests after Ceph support removal
benzekrimaha Mar 11, 2026
a79a3c6
Remove Ceph RadosGW from supported location types and capabilities
benzekrimaha Mar 12, 2026
692fb51
fixup! Remove Ceph RadosGW from supported location types and capabili…
benzekrimaha Mar 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 0 additions & 25 deletions .github/ceph/Dockerfile
View file
Open in desktop

This file was deleted.

37 changes: 0 additions & 37 deletions .github/ceph/entrypoint-wrapper.sh
View file
Open in desktop

This file was deleted.

11 changes: 0 additions & 11 deletions .github/ceph/wait_for_ceph.sh
View file
Open in desktop

This file was deleted.

8 changes: 1 addition & 7 deletions .github/docker/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,14 +91,8 @@ services:
image: ${KMS_IMAGE:-nsmithuk/local-kms:3.11.7}
mongo:
network_mode: "host"
profiles: ['mongo', 'ceph']
profiles: ['mongo']
image: ${MONGODB_IMAGE}
ceph:
network_mode: "host"
profiles: ['ceph']
image: ghcr.io/scality/cloudserver/ci-ceph
volumes:
- /tmp/artifacts/${JOB_NAME}/ceph:/artifacts
sproxyd:
network_mode: "host"
profiles: ['sproxyd']
Expand Down
15 changes: 7 additions & 8 deletions lib/Config.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -387,10 +387,7 @@ function locationConstraintAssert(locationConstraints) {
assert(locationConstraints[l].isCold === true);
}

if (process.env.CI_CEPH === 'true') {
// eslint-disable-next-line no-param-reassign
locationConstraints[l].details.https = false;
} else if (details.https !== undefined) {
if (details.https !== undefined) {
assert(typeof details.https === 'boolean', 'bad config: ' +
'locationConstraints[region].details https must be a boolean');
} else {
Expand All @@ -401,9 +398,6 @@ function locationConstraintAssert(locationConstraints) {
if (details.pathStyle !== undefined) {
assert(typeof details.pathStyle === 'boolean', 'bad config: ' +
'locationConstraints[region].pathStyle must be a boolean');
} else if (process.env.CI_CEPH === 'true') {
// eslint-disable-next-line no-param-reassign
locationConstraints[l].details.pathStyle = true;
} else {
// eslint-disable-next-line no-param-reassign
locationConstraints[l].details.pathStyle = false;
Expand Down Expand Up @@ -1880,7 +1874,12 @@ class Config extends EventEmitter {

if (config.capabilities) {
if (config.capabilities.locationTypes) {
this.supportedLocationTypes = new Set(config.capabilities.locationTypes);
// Ceph RadosGW is no longer supported; exclude from reported location types
const cephRadosGw = 'location-ceph-radosgw-s3-v1';
const types = Array.from(config.capabilities.locationTypes);
this.supportedLocationTypes = new Set(
types.filter(t => t !== cephRadosGw)
);
}
this.capabilities = config.capabilities;
Copy link

@claude claude bot Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config.capabilities is stored as-is (unfiltered) while this.supportedLocationTypes is filtered to exclude location-ceph-radosgw-s3-v1. If any downstream code reads this.capabilities.locationTypes directly instead of this.supportedLocationTypes, the Ceph entry would still be visible. Currently reportHandler.js uses supportedLocationTypes so this works, but for consistency consider filtering config.capabilities.locationTypes before storing, or adding a note that this.capabilities is the raw config.

— Claude Code

}
Expand Down
6 changes: 3 additions & 3 deletions lib/utilities/reportHandler.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ function getCapabilities(cfg = config) {
locationTypeS3Custom: true,
locationTypeSproxyd: true,
locationTypeNFS: true,
locationTypeCephRadosGW: true,
locationTypeHyperdriveV2: true,
locationTypeLocal: true,
preferredReadLocation: true,
Expand All @@ -53,7 +52,6 @@ function getCapabilities(cfg = config) {
secureChannelOptimizedPath: true,
s3cIngestLocation: true,
nfsIngestLocation: false,
cephIngestLocation: false,
awsIngestLocation: false,
};

Expand All @@ -72,11 +70,13 @@ function getCapabilities(cfg = config) {
caps.locationTypeDigitalOcean &&= cfg.supportedLocationTypes.has('location-do-spaces-v1');
caps.locationTypeSproxyd &&= cfg.supportedLocationTypes.has('location-scality-sproxyd-v1');
caps.locationTypeNFS &&= cfg.supportedLocationTypes.has('location-nfs-mount-v1');
caps.locationTypeCephRadosGW &&= cfg.supportedLocationTypes.has('location-ceph-radosgw-s3-v1');
Copy link
Contributor

@francoisferrand francoisferrand Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used to ensure consistency, Cloudserver does not have the "source of truth" : it comes from the ZenkoVersion CR (through ZKOP)
→ should not be kept
→ either we should keep this (to ensure consistency, even if ZenkoVersion indicates the version is supported), or enforce the new lack-of-support of cloudserver (i.e. remove 'location-ceph-radosgw-s3-v1' from the reported location types and unset locationTypeCephRadosGW)

Copy link
Contributor

@DarkIsDude DarkIsDude Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure to understand your point @francoisferrand

caps.locationTypeHyperdriveV2 &&= cfg.supportedLocationTypes.has('location-scality-hdclient-v2');
caps.locationTypeLocal &&= cfg.supportedLocationTypes.has('location-file-v1');
}

// Ceph RadosGW is no longer supported; never report it
caps.locationTypeCephRadosGW = false;
Copy link

@claude claude bot Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line adds locationTypeCephRadosGW: false to every returned caps object. But the property was removed from the default caps object above, and the unit tests (using deepStrictEqual) no longer expect it. This will cause test failures.

Since Ceph is being dropped, consider removing this line entirely — the property simply will not exist in the returned object, which is cleaner than explicitly setting it to false.

— Claude Code


return caps;
}

Expand Down
9 changes: 0 additions & 9 deletions tests/functional/aws-node-sdk/lib/utility/test-utils.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,9 @@
const { config } = require('../../../../../lib/Config');

const isCEPH = process.env.CI_CEPH !== undefined;
const itSkipCeph = isCEPH ? it.skip : it;
const describeSkipIfCeph = isCEPH ? describe.skip : describe.skip; // always skip
let describeSkipIfNotMultiple = describe.skip;
let describeSkipIfNotMultipleOrCeph = describe.skip;

if (config.backends.data === 'multiple') {
describeSkipIfNotMultiple = describe;
describeSkipIfNotMultipleOrCeph = isCEPH ? describe.skip : describe.skip; // always skip
}

function hasLocation(lc) {
Expand All @@ -24,11 +19,7 @@ function hasLocation(lc) {
const hasColdStorage = config.supportedLifecycleRules.some(rule => rule.endsWith('Transition'));

module.exports = {
isCEPH,
itSkipCeph,
describeSkipIfCeph,
describeSkipIfNotMultiple,
describeSkipIfNotMultipleOrCeph,
hasLocation,
hasColdStorage,
};
22 changes: 7 additions & 15 deletions tests/functional/aws-node-sdk/test/multipleBackend/delete/deleteAwsVersioning.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ const {
awsGetLatestVerId,
getAwsRetry,
genUniqID,
isCEPH,
waitForVersioningBeforePut,
} = require('../utils');

Expand Down Expand Up @@ -206,9 +205,8 @@ describeSkipIfNotMultiple('AWS backend delete object w. versioning: ' +
key, versionId: 'null', resultType: deleteVersion },
err => next(err, awsVerId)),
(awsVerId, next) => {
const wanted = isCEPH ? 'NoSuchKey' : 'NoSuchVersion';
_awsGetAssertDeleted({ key,
versionId: awsVerId, errorCode: wanted }, next);
versionId: awsVerId, errorCode: 'NoSuchVersion' }, next);
},
], done);
});
Expand Down Expand Up @@ -243,9 +241,8 @@ describeSkipIfNotMultiple('AWS backend delete object w. versioning: ' +
key, versionId: 'null', resultType: deleteVersion },
err => next(err, awsVerId)),
(awsVerId, next) => {
const wanted = isCEPH ? 'NoSuchKey' : 'NoSuchVersion';
_awsGetAssertDeleted({ key,
versionId: awsVerId, errorCode: wanted }, next);
versionId: awsVerId, errorCode: 'NoSuchVersion' }, next);
},
], done);
});
Expand All @@ -263,9 +260,8 @@ describeSkipIfNotMultiple('AWS backend delete object w. versioning: ' +
key, versionId: s3VerId, resultType: deleteVersion },
err => next(err, awsVerId)),
(awsVerId, next) => {
const wanted = isCEPH ? 'NoSuchKey' : 'NoSuchVersion';
_awsGetAssertDeleted({ key,
versionId: awsVerId, errorCode: wanted }, next);
versionId: awsVerId, errorCode: 'NoSuchVersion' }, next);
},
], done);
});
Expand Down Expand Up @@ -525,9 +521,8 @@ describeSkipIfNotMultiple('AWS backend delete object w. versioning: ' +
(awsVid, next) => _getAssertDeleted(s3, { key,
errorCode: 'NoSuchKey' }, () => next(null, awsVid)),
(awsVerId, next) => {
const wanted = isCEPH ? 'NoSuchKey' : 'NoSuchVersion';
_awsGetAssertDeleted({ key,
versionId: awsVerId, errorCode: wanted }, next);
versionId: awsVerId, errorCode: 'NoSuchVersion' }, next);
},
], done);
});
Expand Down Expand Up @@ -682,9 +677,8 @@ describeSkipIfNotMultiple('AWS backend delete multiple objects w. versioning: '
key, versionId: 'null', resultType: deleteVersion },
err => next(err, awsVerId)),
(awsVerId, next) => {
const wanted = isCEPH ? 'NoSuchKey' : 'NoSuchVersion';
_awsGetAssertDeleted({ key,
versionId: awsVerId, errorCode: wanted }, next);
versionId: awsVerId, errorCode: 'NoSuchVersion' }, next);
},
], done);
});
Expand All @@ -700,9 +694,8 @@ describeSkipIfNotMultiple('AWS backend delete multiple objects w. versioning: '
key, versionId: 'null', resultType: deleteVersion },
err => next(err, awsVerId)),
(awsVerId, next) => {
const wanted = isCEPH ? 'NoSuchKey' : 'NoSuchVersion';
_awsGetAssertDeleted({ key,
versionId: awsVerId, errorCode: wanted }, next);
versionId: awsVerId, errorCode: 'NoSuchVersion' }, next);
},
], done);
});
Expand All @@ -720,9 +713,8 @@ describeSkipIfNotMultiple('AWS backend delete multiple objects w. versioning: '
key, versionId: s3VerId, resultType: deleteVersion },
err => next(err, awsVerId)),
(awsVerId, next) => {
const wanted = isCEPH ? 'NoSuchKey' : 'NoSuchVersion';
_awsGetAssertDeleted({ key,
versionId: awsVerId, errorCode: wanted }, next);
versionId: awsVerId, errorCode: 'NoSuchVersion' }, next);
},
], done);
});
Expand Down
4 changes: 2 additions & 2 deletions tests/functional/aws-node-sdk/test/multipleBackend/delete/deleteAzure.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ const { CreateBucketCommand,
const BucketUtility = require('../../../lib/utility/bucket-util');
const withV4 = require('../../support/withV4');
const {
describeSkipIfNotMultipleOrCeph,
uniqName,
getAzureClient,
getAzureContainerName,
getAzureKeys,
azureLocation,
azureLocationMismatch,
describeSkipIfNotMultiple,
} = require('../utils');

const keyObject = 'deleteazure';
Expand All @@ -30,7 +30,7 @@ const nonExistingId = process.env.AWS_ON_AIR ?
'MhhyTHhmZ4cxSi4Y9SMe5P7UJAz7HLJ9' :
'3939393939393939393936493939393939393939756e6437';

describeSkipIfNotMultipleOrCeph('Multiple backend delete object from Azure',
describeSkipIfNotMultiple('Multiple backend delete object from Azure',
function testSuite() {
this.timeout(250000);
withV4(sigCfg => {
Expand Down
4 changes: 2 additions & 2 deletions tests/functional/aws-node-sdk/test/multipleBackend/delete/deleteGcp.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ const { CreateBucketCommand,
const withV4 = require('../../support/withV4');
const BucketUtility = require('../../../lib/utility/bucket-util');
const {
describeSkipIfNotMultipleOrCeph,
gcpLocation,
gcpLocationMismatch,
genUniqID,
describeSkipIfNotMultiple,
} = require('../utils');

const bucket = `deletegcp${genUniqID()}`;
Expand All @@ -20,7 +20,7 @@ const mismatchObject = `mismatchObject-${genUniqID()}`;
const body = Buffer.from('I am a body', 'utf8');
const bigBody = Buffer.alloc(10485760);

describeSkipIfNotMultipleOrCeph('Multiple backend delete',
describeSkipIfNotMultiple('Multiple backend delete',
function testSuite() {
this.timeout(120000);
withV4(sigCfg => {
Expand Down
3 changes: 2 additions & 1 deletion tests/functional/aws-node-sdk/test/multipleBackend/get/getAzure.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ const {
getAzureContainerName,
getAzureKeys,
azureLocation,
describeSkipIfNotMultiple,
} = require('../utils');

const azureClient = getAzureClient();
Expand All @@ -23,7 +24,7 @@ const normalBody = Buffer.from('I am a body', 'utf8');

const azureTimeout = 10000;

describe.skip('Multiple backend get object from Azure',
describeSkipIfNotMultiple('Multiple backend get object from Azure',
function testSuite() {
this.timeout(30000);
withV4(sigCfg => {
Expand Down
6 changes: 3 additions & 3 deletions tests/functional/aws-node-sdk/test/multipleBackend/get/getGcp.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ const { PutObjectCommand,
CreateBucketCommand } = require('@aws-sdk/client-s3');
const BucketUtility = require('../../../lib/utility/bucket-util');
const {
describeSkipIfNotMultipleOrCeph,
gcpLocation,
gcpLocationMismatch,
genUniqID,
describeSkipIfNotMultiple,
} = require('../utils');

const bucket = `getgcp${genUniqID()}`;
Expand Down Expand Up @@ -53,7 +53,7 @@ describe('Multiple backend get object', function testSuite() {
});
});

describeSkipIfNotMultipleOrCeph('with objects in GCP', () => {
describeSkipIfNotMultiple('with objects in GCP', () => {
before(() => {
process.stdout.write('Putting object to GCP\n');
return s3.send(new PutObjectCommand({ Bucket: bucket, Key: gcpObject,
Expand Down Expand Up @@ -128,7 +128,7 @@ describe('Multiple backend get object', function testSuite() {
});
});

describeSkipIfNotMultipleOrCeph('with bucketMatch set to false', () => {
describeSkipIfNotMultiple('with bucketMatch set to false', () => {
beforeEach(done => {
s3.send(new PutObjectCommand({ Bucket: bucket, Key: mismatchObject, Body: body,
Metadata: { 'scal-location-constraint': gcpLocationMismatch } })).then(() => {
Expand Down
6 changes: 3 additions & 3 deletions tests/functional/aws-node-sdk/test/multipleBackend/initMPU/initMPUAzure.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,16 @@ const {

const withV4 = require('../../support/withV4');
const BucketUtility = require('../../../lib/utility/bucket-util');
const { describeSkipIfNotMultipleOrCeph, azureLocation, getAzureContainerName,
genUniqID } = require('../utils');
const { azureLocation, getAzureContainerName,
genUniqID, describeSkipIfNotMultiple } = require('../utils');

const keyName = `somekey-${genUniqID()}`;

const azureContainerName = getAzureContainerName(azureLocation);
let s3;
let bucketUtil;

describeSkipIfNotMultipleOrCeph('Initiate MPU to AZURE', () => {
describeSkipIfNotMultiple('Initiate MPU to AZURE', () => {
withV4(sigCfg => {
beforeEach(() => {
bucketUtil = new BucketUtility('default', sigCfg);
Expand Down
Loading
Loading