[Bug]: esRollover configured via Helm chart not working correctly
christopher-wong opened this issue · 0 comments
christopher-wong commented
What happened?
I've deployed Jaeger with ElasticSearch via the latest Helm chart 3.0.4
.
I've enabled esRollover with the following configuration:
esRollover:
enabled: true
extraEnv:
- name: CONDITIONS
value: '{"max_age": "1h", "max_size": "25gb"}'
schedule: "10 * * * *"
If I inspect the ElasticSearch indices, I don't see new indices being created, just the existing daily ones.
➜ ~ curl 'localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size dataset.size
green open jaeger-span-000001 C2MafU9kRZKZWPbv6FFJqg 5 1 0 0 2.4kb 1.2kb 1.2kb
green open jaeger-span-000002 rBWJ233fSy6BoUvsaPQCwQ 5 1 0 0 2.4kb 1.2kb 1.2kb
green open jaeger-span-000003 tnbkkgTjSrWrjET3TSV9rg 5 1 0 0 2.4kb 1.2kb 1.2kb
green open jaeger-service-2024-04-30 VHsseM4ZS_qP56kT3X2jhw 5 1 233 38 443.5kb 215.5kb 215.5kb
green open jaeger-span-2024-04-30 R21JY1q8RsO9hA6Fl-VSiQ 5 1 23908208 0 9.4gb 5gb 5gb
green open jaeger-span-2024-04-28 Q13pRsK_Rxq3X5qRFpdPWw 5 1 408810059 0 43.7gb 21.8gb 21.8gb
green open jaeger-span-2024-04-29 d4yKrCeyTIG1_-qp2E3rvw 5 1 382922115 0 125.3gb 62.7gb 62.7gb
green open jaeger-service-2024-04-28 YvQSmwAaQy-2WPwvFhA5OA 5 1 309 15 332.5kb 173.8kb 173.8kb
green open jaeger-service-2024-04-29 0pY2u_Z9Tra26kkoN4Jh1w 5 1 486 78 481.9kb 234.7kb 234.7kb
green open jaeger-dependencies-000001 At-Lil2mTxqavSsiv3kxtw 5 1 0 0 2.4kb 1.2kb 1.2kb
green open jaeger-dependencies-000002 BJB0ivqvSleaQJ59cx47vg 5 1 0 0 2.4kb 1.2kb 1.2kb
green open jaeger-dependencies-000003 OcHXSXayS76yukmGQHwHLw 5 1 0 0 2.4kb 1.2kb 1.2kb
green open jaeger-service-000001 J7cKacQrTTKY_vbibhYjag 5 1 0 0 2.4kb 1.2kb 1.2kb
green open jaeger-service-000002 bRDUPKA9RfakL8iYBTeH8A 5 1 0 0 2.4kb 1.2kb 1.2kb
green open jaeger-service-000003 0Z-EMb5CQS-0k5ldboh5Pw 5 1 0 0 2.4kb 1.2kb 1.2kb
Steps to reproduce
- Deploy Jaeger Helm Chart 3.0.4 with ElasticSearch and esRollover enabled
- Check to see if new indices are being created.
Expected behavior
I expect a new index to get created every hour or when maxSize is 25gb
Relevant log output
No response
Screenshot
No response
Additional context
No response
Jaeger backend version
v1.53.0
SDK
No response
Pipeline
No response
Stogage backend
No response
Operating system
No response
Deployment model
No response
Deployment configs
tag: "1.53.0"
storage:
type: elasticsearch
elasticsearch:
host: otjaeger2-elasticsearch-master-hl
extraEnv:
- name: ES_TAGS_AS_FIELDS_ALL
value: "true"
provisionDataStore:
cassandra: false
elasticsearch: true
kafka: false
elasticsearch:
global:
kibanaEnabled: true
master:
replicaCount: 3
resourcesPreset: "xlarge"
persistence:
enabled: true
storageClass: "ebs-sc"
accessModes:
- ReadWriteOnce
size: 1000Gi
data:
replicaCount: 3
resourcesPreset: "xlarge"
persistence:
enabled: true
storageClass: "ebs-sc"
accessModes:
- ReadWriteOnce
size: 1000Gi
ingest:
replicaCount: 2
resourcesPreset: "medium"
coordinating:
replicaCount: 2
resourcesPreset: "medium"
kibana:
ingress:
enabled: true
ingressClassName: alb
hostname: '*'
path: '/*'
annotations:
alb.ingress.kubernetes.io/target-type: 'ip'
alb.ingress.kubernetes.io/scheme: internal
# - name: ES_JAVA_OPTS
# value: -Xmx14g -Xms14g
esIndexCleaner:
enabled: true # must be false for initial deploy
schedule: "55 23 * * *"
numberOfDays: 7
esRollover:
enabled: true # must be false for initial deploy
extraEnv:
- name: CONDITIONS
value: '{"max_age": "1h", "max_size": "25gb"}'
schedule: "10 * * * *"
collector:
image:
registry: <REDACTED>
repository: jaeger-collector
replicaCount: 6
serviceMonitor:
enabled: true
additionalLabels:
release: base
ingress:
enabled: true
ingressClassName: alb
annotations:
alb.ingress.kubernetes.io/target-type: 'ip'
alb.ingress.kubernetes.io/scheme: internal
alb.ingress.kubernetes.io/certificate-arn: <REDACTED>
alb.ingress.kubernetes.io/healthcheck-port: '14269'
alb.ingress.kubernetes.io/healthcheck-path: /
pathType: Prefix
hosts:
- host: <REDACTED>
servicePort: http
- host: <REDACTED>
servicePort: otlp-http
service:
otlp:
grpc:
name: otlp-grpc
http:
name: otlp-http
resources:
limits:
cpu: 2
memory: 1Gi
requests:
cpu: 250m
memory: 128Mi
query:
image:
registry: <REDACTED>
repository: jaeger-query
replicaCount: 6
serviceMonitor:
enabled: true
additionalLabels:
release: base
service:
type: LoadBalancer
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "external"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: <REDACTED>
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
port: 443
resources:
limits:
cpu: 2
memory: 4Gi
requests:
cpu: 250m
memory: 128Mi
agentSidecar:
enabled: false
agent:
enabled: false
spark:
enabled: false