splunk/splunk-operator

App Framework:

Closed this issue · 4 comments

bashi100kmrun commented

Please select the type of request

Bug

Tell us more

Describe the request
One of my customers is verifying appframework with s3-API-compliant remote object storage now.

They wrote manifest as follows:

appRepo:
    defaults:
      volumeName: volume_cluster_master_app_repo
      scope: cluster
    appSources:
      - name: apps
        location: apps/
        scope: local
      - name: master-apps
        location: master-apps/
        scope: cluster
    volumes:
      - name: volume_cluster_master_app_repo
        storageType: s3
        provider: aws
        path: appbucket/ClusterMaster/
        endpoint: https://10.253.128.4/
        region: none
        secretRef: purestorage-secret

Appendix: appRepo.volumes.endpoint is S3-API-compliant remote object storage location (ip address)

And they deployed, then operator's log showed error messages as follows:

2023-08-15T05:32:28.147697425Z  ERROR   GetAppsList Unable to list items in bucket  {"controller": "clustermaster", "controllerGroup": "enterprise.splunk.com", "controllerKind": "ClusterMaster", "ClusterMaster": {"name":"cm-appframework-test-3","namespace":"splunk-operator"}, "namespace": "splunk-operator", "name": "cm-appframework-test-3", "reconcileID": "a1c5e7a7-04e9-494a-bc88-bb4eb504eb33", "AWS S3 Bucket": "appbucket", "error": "RequestError: send request failed\ncaused by: Get \"https://appbucket.s3.none.amazonaws.com/?delimiter=%2F&list-type=2&max-keys=4000&prefix=ClusterMaster%2Fapps%2F&start-after=ClusterMaster%2Fapps%2F\": dial tcp: lookup appbucket.s3.none.amazonaws.com on 10.96.0.10:53: no such host"}
2023-08-15T05:32:28.147800532Z  ERROR   GetAppListFromRemoteBucket  Unable to get apps list {"controller": "clustermaster", "controllerGroup": "enterprise.splunk.com", "controllerKind": "ClusterMaster", "ClusterMaster": {"name":"cm-appframework-test-3","namespace":"splunk-operator"}, "namespace": "splunk-operator", "name": "cm-appframework-test-3", "reconcileID": "a1c5e7a7-04e9-494a-bc88-bb4eb504eb33", "name": "cm-appframework-test-3", "namespace": "splunk-operator", "appSource": "apps", "error": "RequestError: send request failed\ncaused by: Get \"https://appbucket.s3.none.amazonaws.com/?delimiter=%2F&list-type=2&max-keys=4000&prefix=ClusterMaster%2Fapps%2F&start-after=ClusterMaster%2Fapps%2F\": dial tcp: lookup appbucket.s3.none.amazonaws.com on 10.96.0.10:53: no such host"}
2023-08-15T05:32:28.519727514Z  ERROR   GetAppsList Unable to list items in bucket  {"controller": "clustermaster", "controllerGroup": "enterprise.splunk.com", "controllerKind": "ClusterMaster", "ClusterMaster": {"name":"cm-appframework-test-3","namespace":"splunk-operator"}, "namespace": "splunk-operator", "name": "cm-appframework-test-3", "reconcileID": "a1c5e7a7-04e9-494a-bc88-bb4eb504eb33", "AWS S3 Bucket": "appbucket", "error": "RequestError: send request failed\ncaused by: Get \"https://appbucket.s3.none.amazonaws.com/?delimiter=%2F&list-type=2&max-keys=4000&prefix=ClusterMaster%2Fmaster-apps%2F&start-after=ClusterMaster%2Fmaster-apps%2F\": dial tcp: lookup appbucket.s3.none.amazonaws.com on 10.96.0.10:53: no such host"}
2023-08-15T05:32:28.519823592Z  ERROR   GetAppListFromRemoteBucket  Unable to get apps list {"controller": "clustermaster", "controllerGroup": "enterprise.splunk.com", "controllerKind": "ClusterMaster", "ClusterMaster": {"name":"cm-appframework-test-3","namespace":"splunk-operator"}, "namespace": "splunk-operator", "name": "cm-appframework-test-3", "reconcileID": "a1c5e7a7-04e9-494a-bc88-bb4eb504eb33", "name": "cm-appframework-test-3", "namespace": "splunk-operator", "appSource": "master-apps", "error": "RequestError: send request failed\ncaused by: Get \"https://appbucket.s3.none.amazonaws.com/?delimiter=%2F&list-type=2&max-keys=4000&prefix=ClusterMaster%2Fmaster-apps%2F&start-after=ClusterMaster%2Fmaster-apps%2F\": dial tcp: lookup appbucket.s3.none.amazonaws.com on 10.96.0.10:53: no such host"}
2023-08-15T05:32:28.519849721Z  ERROR   initAndCheckAppInfoStatus   Unable to get apps list, will retry in next reconcile...    {"controller": "clustermaster", "controllerGroup": "enterprise.splunk.com", "controllerKind": "ClusterMaster", "ClusterMaster": {"name":"cm-appframework-test-3","namespace":"splunk-operator"}, "namespace": "splunk-operator", "name": "cm-appframework-test-3", "reconcileID": "a1c5e7a7-04e9-494a-bc88-bb4eb504eb33", "name": "cm-appframework-test-3", "namespace": "splunk-operator", "error": "unable to get apps list from remote storage list for all the apps"}

Following the error message, it appears to be trying to connect to https://appbucket.s3.none.amazonaws.com/ even though they have specified https://10.253.128.4 in the manifest endpoint.

Splunk setup on K8S

  • Indexer Clustering by splunk-operator

K8s environment
Their k8s enviroment as
splunk-operator : latest(v 2.3.0)
Kubernetes : v1.23.10
Container runtime : Containerd v1.6.6

splunk-operator.log

vivekr-splunk commented

Hi @bashi100kmrun , there is a PR to address this issue. will update you once its merged to develop branch.

vivekr-splunk commented

https://github.com/splunk/splunk-operator/pull/1219/files merged to develop, changes should available in next release.

  • 👍1
bashi100kmrun commented

Thanks, I'm grateful to you for your action. If you know when next version will release, would you let me know.

vivekr-splunk commented

fixed in 2.5.0