Replication Policy on OBC is not working

[schwienbier]

Environment info

• NooBaa Operator Version: 5.11.0
• Platform: Kubernetes v1.25.8

Actual behavior

The src-bucket and dest-buckets were created as follows.

noobaa  namespacestore create s3-compatible ait-noobaa1-namespacestore --endpoint='...' --secret-name='...' --target-bucket='noobaa1' -n noobaa
noobaa  namespacestore create s3-compatible ait-noobaa2-namespacestore --endpoint='...' --secret-name='...' --target-bucket='noobaa2' -n noobaa

noobaa -n noobaa bucketclass create namespace-bucketclass single ait-noobaa1-bc --resource ait-noobaa1-namespacestore
noobaa -n noobaa bucketclass create namespace-bucketclass single ait-noobaa2-bc --resource ait-noobaa2-namespacestore

kubectl apply -f replication_obc.yaml -n poc-ns 

The replication_obc.yaml is as follows,

apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: dest-bucket
  namespace: poc-ns 
spec:
  additionalConfig:
    bucketclass: ait-noobaa2-bc
  bucketName: dest-bucket
  storageClassName: noobaa.noobaa.io
  
---

apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: src-bucket
  namespace: poc-ns 
spec:
  additionalConfig:
    bucketclass: ait-noobaa1-bc
    replicationPolicy: |
      [{ "rule_id": "rule-6", "destination_bucket": "dest-bucket", "filter": {"prefix": "replicated"}}]
  bucketName: src-bucket
  storageClassName: noobaa.noobaa.io

When I upload a file to src-bucket, the file is not replicated to dest-bucket.

Expected behavior

When I upload a file to src-bucket, the file should be replicated to dest-bucket.

Steps to reproduce

Please run the following commands to deploy src- and dest-buckets.

noobaa  namespacestore create s3-compatible ait-noobaa1-namespacestore --endpoint='...' --secret-name='...' --target-bucket='noobaa1' -n noobaa
noobaa  namespacestore create s3-compatible ait-noobaa2-namespacestore --endpoint='...' --secret-name='...' --target-bucket='noobaa2' -n noobaa

noobaa -n noobaa bucketclass create namespace-bucketclass single ait-noobaa1-bc --resource ait-noobaa1-namespacestore
noobaa -n noobaa bucketclass create namespace-bucketclass single ait-noobaa2-bc --resource ait-noobaa2-namespacestore

kubectl apply -f replication_obc.yaml -n poc-ns 

The replication_obc.yaml is as follows,

apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: dest-bucket
  namespace: poc-ns 
spec:
  additionalConfig:
    bucketclass: ait-noobaa2-bc
  bucketName: dest-bucket
  storageClassName: noobaa.noobaa.io
  
---

apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: src-bucket
  namespace: poc-ns 
spec:
  additionalConfig:
    bucketclass: ait-noobaa1-bc
    replicationPolicy: |
      [{ "rule_id": "rule-6", "destination_bucket": "dest-bucket", "filter": {"prefix": "replicated"}}]
  bucketName: src-bucket
  storageClassName: noobaa.noobaa.io

More information - Screenshots / Logs / Other output

Logs of naaboo-core are as follows without error:

[32mJul-3 10:28:16.099 [BGWorkers/48]    [L0] core.server.bg_services.replication_scanner:: replication_scanner: starting scanning bucket replications
[32mJul-3 10:28:16.102 [BGWorkers/48]  [L0] core.server.system_services.replication_store:: find_rules_updated_longest_time_ago:  [
{
    rules: [ { filter: { prefix: 'replicated'}, rule_id: 'rule-6', rule_status: { dst_cont_token: '', last_cycle_end: 1688379796095, src_cont_token: '', last_cycle_start: 1688379795951 }, destination_bucket: 64a2949c7cee6800344d7b88 } ]
}
...
Jul-3 10:28:16.182 [BGWorkers/48]    [L0] core.server.system_services.replication_store:: update_replication_status_by_id:  64a2a0497cee6800344d7bb4 rule-3 { last_cycle_start: 1688380096102, last_cycle_end: 1688380096182, src_cont_token: '', dst_cont_token: '' }

[schwienbier]

Dear Noobaa Supports,

Replication works after I set replicationPolicy on bucket class (link) as well. Deployed yamls are as follows for future reference.

apiVersion: v1
data:
  AWS_ACCESS_KEY_ID: ...
  AWS_SECRET_ACCESS_KEY: ...
kind: Secret
metadata:
  name: swift-access-secret
  namespace: noobaa
type: Opaque

---

apiVersion: noobaa.io/v1alpha1
kind: NamespaceStore
metadata:
  name: src-noobaa-namespacestore
  namespace: noobaa
spec:
  s3Compatible:
    endpoint: ...
    secret:
      name: swift-access-secret
      namespace: noobaa
    signatureVersion: v4
    targetBucket: src-noobaa-bucket
  type: s3-compatible

---

apiVersion: noobaa.io/v1alpha1
kind: NamespaceStore
metadata:
  name: dest-noobaa-namespacestore
  namespace: noobaa
spec:
  s3Compatible:
    endpoint: ...
    secret:
      name: swift-access-secret
      namespace: noobaa
    signatureVersion: v4
    targetBucket: dest-noobaa-bucket
  type: s3-compatible

---

apiVersion: noobaa.io/v1alpha1
kind: BucketClass
metadata:
  name: src-noobaa-bc
  namespace: noobaa
spec:
  namespacePolicy:
    single:
      resource: src-noobaa-namespacestore
    type: Single
  replicationPolicy: |
    [{ "rule_id": "rule-1", "destination_bucket": "dest-bucket"}]
    
---

apiVersion: noobaa.io/v1alpha1
kind: BucketClass
metadata:
  name: dest-noobaa-bc
  namespace: noobaa
spec:
  namespacePolicy:
    single:
      resource: dest-noobaa-namespacestore
    type: Single
  
---

apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: src-bucket
  namespace: alan-shi-test
spec:
  additionalConfig:
    bucketclass: src-noobaa-bc
    replicationPolicy: |
      [{ "rule_id": "rule-1", "destination_bucket": "dest-bucket"}]
  bucketName: src-bucket
  storageClassName: noobaa.noobaa.io
  
---

apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: dest-bucket
  namespace: alan-shi-test
spec:
  additionalConfig:
    bucketclass: dest-noobaa-bc
  bucketName: dest-bucket
  storageClassName: noobaa.noobaa.io

Best regards