Does ClusterLogging have a plan to support fluentd input and output configurations on a large scale?

[666MrFang]

I'm using cluster-logging 5.4.3, and I want to configure fluentd's configuration file independently, but when I look at the ClusterLogging and ClusterLogForwarder fields, fluentd doesn't provide many configuration fields. It is recommended that managementState be changed to Unmanaged. In this way, fluentd.conf can be set. Does ClusterLogging have a plan to support fluentd input and output configurations on a large scale?

When I use such a YAML file, the fluentd.conf file does not collect the corresponding collection path.Look at the pod status.

apiVersion: "logging.openshift.io/v1"
kind: "ClusterLogging"
metadata:
  name: "instance"
  namespace: "openshift-logging"
spec:
  managementState: "Managed"
  collection:
    logs:
      type: "fluentd"
      fluentd:
        resources:
          limits:
            memory: 736Mi
          requests:
            cpu: 100m
            memory: 736Mi
  forwarder:
    fluentd:
      inFile:
        readLinesLimit: 1000
        path: /home/fang/test/*.log
        tag: fluentd.test

Pod state:

apiVersion: logging.openshift.io/v1
kind: ClusterLogging
metadata:
  creationTimestamp: "2023-04-28T03:15:35Z"
  generation: 1
  name: instance
  namespace: openshift-logging
  resourceVersion: "491193908"
  uid: 332f7da3-cb70-4939-9d06-929acd6a44ad
spec:
  collection:
    logs:
      fluentd:
        resources:
          limits:
            memory: 736Mi
          requests:
            cpu: 100m
            memory: 736Mi
      type: fluentd
  forwarder:
    fluentd:
      inFile:
        readLinesLimit: 1000
  managementState: Managed
status:
  clusterConditions:
  - lastTransitionTime: "2023-04-28T03:15:55Z"
    message: ClusterLogForwarder input validation error
    reason: ClusterLogForwarder input validation error
    status: "True"

[666MrFang]

I want to konw how can I configure fluentd's configuration file independently.

[jcantrill]

The intent of this solution is to provide an abstraction to log collection and storage and limiting the configuration details that are surfaced in the API. It is not another layer over fluentd configuration to make it "kubernetes" like. If you have requirements to control larger aspects of the configuration then I encourage you to create and manage your own deployment outside of the operator.

We have deprecated fluentd in favor of vector and in doing so have made it possible to swap the collector with almost a 1 for 1 replacement. Given we are prioritizing fluentd, we have effectively stopped any further development.