[S3 SSEC] SSEC and AES enabled together?

[Arsnael]

Try to enable AES and SSEC together, see if it works or break. Document accordingly to the results.

[vttranlina]

I see the "?" in the title
so,
(-1)

The encrypt data twice is unnecessary

[Arsnael]

Just testing if it works, that was the initial goal groomed last week. Does not mean we will keep it. See it more as a POC?

[chibenwa]

The encrypt data twice is unnecessary

We are not speaking of double encryption here but rather retro-compatibility of one mode into the other...

[vttranlina]

SSEC and AES can enabled together 🍏 🍏 🍏

Branch: https://github.com/vttranlina/james-project/tree/sseC_AES_CombinedTest
Test commit: apache@5eb36bf
Test result: (Green)

[chibenwa]

    @BeforeAll
    static void setUp() throws Exception {
        S3BlobStoreConfiguration s3Configuration = S3BlobStoreConfiguration.builder()
            .authConfiguration(minoExtension.minioDocker().getAwsS3AuthConfiguration())
            .region(Region.of(software.amazon.awssdk.regions.Region.EU_WEST_1.id()))
            .uploadRetrySpec(Optional.of(Retry.backoff(3, java.time.Duration.ofSeconds(1))
                .filter(UPLOAD_RETRY_EXCEPTION_PREDICATE)))
            .build();
        s3ClientFactory = new S3ClientFactory(s3Configuration, () -> new JamesS3MetricPublisher(new RecordingMetricFactory(), new NoopGaugeRegistry(),
            DEFAULT_S3_METRICS_PREFIX));
        S3SSECustomerKeyFactory sseCustomerKeyFactory = new S3SSECustomerKeyFactory.SingleCustomerKeyFactory(new S3SSECConfiguration.Basic("AES256", "masterPassword", "salt"));
        underlying = new S3WithSSECBlobStoreDAO(s3ClientFactory, s3Configuration, new TestBlobId.Factory(), sseCustomerKeyFactory);
        S3BlobStoreDAO unencrypted = new S3BlobStoreDAO(s3ClientFactory, s3Configuration, new TestBlobId.Factory());
        aesBlobStoreDAO = new AESBlobStoreDAO(unencrypted, CRYPTO_CONFIG);
    }

Can we make sure AES blob store do NOT use an SSEC encrypted blobStore?

This is not obvious in the linked code.... I got the impression we wrote if i encrypt it twice I can decrypt it once

[chibenwa]

Sorry but is the topic here even understood?

The goal is to migrate from AesBlobStore to SSEC and see if it is inherently compatible.

The above test suite only proves we can double encrypt things which do NOT correspond to what we want to achieve.

[vttranlina]

The above test suite only proves we can double encrypt things

Honestly, that's what I understand about this ticket

The goal is to migrate from AesBlobStore to SSEC and see if it is inherently compatible.
Can we make sure AES blob store do NOT use an SSEC encrypted blobStore?
Try to enable AES and SSEC together, see if it works or break

Sorry, I'm not clear.
Does "migrate" mean to run only once, downloading all objects from AESBlobStore and re-uploading them to SSECBlobStore?

Or, Does this mean running AESBlobStore and SSECBlobStore in parallel at same time? If so, that's impossible. We are don't know of the "mode" for each input blobId.

[chibenwa]

It means write data with AESblobstore
Read it with ssec

We do not want to do blob copies.

[vttranlina]

It means write data with AESblobstore Read it with ssec

We do not want to do blob copies.

S3 does not support what we expected. My test scenario is detailed here: https://github.com/apache/james-project/blob/8d5e67c8208307ba4296df742dc86d6fc8d33af6/server/blob/blob-s3/src/test/java/org/apache/james/blob/objectstorage/aws/sse/CombineSSECVsAESBlobStoreDAOTest.java

S3 does not allow modifying the headers of an existing object; the solution is to use the .CopyObject API (S3 site).
I tried several way but was unsuccessful.

[chibenwa]

Ok, as expected we confirmed it was not doable.

Let's close this ticket, thanks for investigating.