Logging operator

Logging operator for Kubernetes based on Fluentd and Fluent-bit.

The Logging operator automates the deployment and configuration of a Kubernetes logging pipeline. The operator deploys and configures a Fluent Bit daemonset on every node to collect container and application logs from the node file system. Fluent Bit queries the Kubernetes API and enriches the logs with metadata about the pods, and transfers both the logs and the metadata to Fluentd. Fluentd receives, filters, and transfer logs to multiple outputs. Your logs will always be transferred on authenticated and encrypted channels.

What is this operator for?

This operator helps you bundle logging information with your applications: you can describe the behavior of your application in its charts, the Logging operator does the rest.

Feature highlights

Namespace isolation
Native Kubernetes label selectors
Secure communication (TLS)
Configuration validation
Multiple flow support (multiply logs for different transformations)
Multiple output support (store the same logs in multiple storage: S3, GCS, ES, Loki and more...)
Multiple logging system support (multiple fluentd, fluent-bit deployment on the same cluster)

Architecture

You can define outputs (destinations where you want to send your log messages, for example, Elasticsearch, or and Amazon S3 bucket), and flows that use filters and selectors to route log messages to the appropriate outputs. You can also define cluster-wide outputs and flows, for example, to use a centralized output that namespaced users cannot modify.

You can configure the Logging operator using the following Custom Resource Descriptions.

logging - Represents a logging system. Includes Fluentd and Fluent-bit configuration. Specifies the controlNamespace. Fluentd and Fluent-bit will be deployed in the controlNamespace
output - Defines an Output for a logging flow. This is a namespaced resource. See also clusteroutput.
flow - Defines a logging flow with filters and outputs. You can specify selectors to filter logs by labels. Outputs can be output or clusteroutput. This is a namespaced resource. See also clusterflow.
clusteroutput - Defines an output without namespace restriction. Only effective in controlNamespace.
clusterflow - Defines a logging flow without namespace restriction.

See the detailed CRD documentation.

Quickstart

Follow these quickstart guides to try out the Logging operator!

Install

Deploy Logging Operator with Kubernetes manifests or Helm chart.

Caution: The master branch is under heavy development. Use releases instead of the master branch to get stable software.

Support

Community support

If you encounter problems that the documentation does not address, open an issue or talk to us on the Banzai Cloud Slack channel #logging-operator.

Commercial support

If you are using the Logging operator in a production environment and require commercial support, contact Banzai Cloud, the company backing the development of the Logging operator. If you are looking for the ultimate observability tool for multi-cluster Kubernetes infrastructures to automate the collection, correlation, and storage of logs and metrics, check out One Eye.

Documentation

You can find the complete documentation of the Logging operator on the Banzai Cloud Documentation Page 📘

Contributing

If you find this project useful, help us:

Support the development of this project and star this repo! ⭐
If you use the Logging operator in a production environment, add yourself to the list of production adopters.:metal:
Help new users with issues they may encounter 💪
Send a pull request with your new features and bug fixes 🚀

For more information, read the developer documentation.

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

devholic/logging-operator