Fluent log shipping app is a managed app used to help customers forward their logs to any supported storage backends. It use fluent-bit DaemonSet, a lightweight log collector used to collect and forward containers and audit logs your chosen output
- You can install only one release of this chart per kubernetes cluster
- By default, no forwarding is active so make sure you check configuration document before deploying it in your cluster.
The log shipping app is built to be installed in AWS or Azure.
Sample command for installing it on AWS with cloudwatch enabled:
helm install --namespace logging giantswarm-playground-catalog/fluent-logshipping-app --set fluentd.aws.cloudWatch.enabled=true
The app currently exports the following logs:
| Log type | Location | Format |
|---|---|---|
| Container Logs | /var/log/containers/*.log |
json |
| Kubernetes Audit Log | /var/log/apiserver/audit.log |
json |
| SSH Access Logs | syslog |
^\<(?<pri>[0-9]+)\>(?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<ident>[a-zA-Z0-9_\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$ |
Configuration options are documented in Configuration.md document.
- When using Cloudwatch or S3 a new role has to be created upfront which allows pushing of the logs to the backend(s). More info about permissions in official docs (S3, CloudWatch).
- When using S3 as an ouput for logs in the Management Cluster make sure to include "-g8s-" in the name of the bucket or modify the S3 VPC endpoint to allow a different name, otherwise you will get an Unauthorized error.
Tested on Giant Swarm release 11.0.0 on AWS and Azure (Kubernetes 1.16.3).