streamsets/helm-charts

resource requirements

Closed this issue · 3 comments

According to https://streamsets.com/documentation/controlhub/latest/installhelp/controlhub/InstallationGuide/InstallationOverview/InstallRequirements.html#concept_ufb_rrf_xz

It seems the installation requirements a huge resource, many GB of RAM for example. However, I don't see this is reflected in this helm chart. I tried to deploy it anyway on my k8s cluster, which comes with 16GB(4x4GB) in total over 4 worker node, and my cluster screw :( Is it because of resource limit?

Per Helm best practices we don't pre-set any resource requests.
By default, each app is run in its own pod with 1GB memory -- this may actually be excessive.

Unless you've reduced the JVM heap size by configuring it in DPM_JAVA_OPTS, you'll likely exceed your cluster's memory limits as you described above.

It's always a good practice to also set resource limits to prevent over-allocation on your nodes and keep runaway processes from affecting the cluster.

Thanks a lot @kunickiaj will try again, and back with update late.

Closing for now, please open a new issue if still having troubles.