fluent/fluentd

multi worker start failed

Closed this issue · 7 comments

I want to try multi worker with fluentd, but no matter how I config this, like add config workers under , or add --workers in td-agent.service, there is stll same error like this:
2018-10-08 09:45:44 +0800 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Plugin 'local' does not support multi workers configuration (Fluent::Plugin::LocalStorage)"

Could you share your configuration?

workers 2

2018-10-09 10:36:16 +0800 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Plugin 'tail' does not support multi workers configuration (Fluent::Plugin::TailInput)"

Your comment is not configuration but Fluentd log.
Could you share your /etc/td-agent/td-agent.conf without sensitive information?

This is because local storage plugin and in_tail does not support multi workers as log said:

2018-10-08 09:45:44 +0800 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Plugin 'local' does not support multi workers configuration (Fluent::Plugin::LocalStorage)"

2018-10-09 10:36:16 +0800 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Plugin 'tail' does not support multi workers configuration (Fluent::Plugin::TailInput)"

You can use <worker> section like the following:

<worker 0>
  <source>
    @type tail
    # ...
 </source>
</worker>
<worker 1>
  <source>
    @type systemd
    <storage>
       @type local
       # ...
    </storage>
  </source>
</worker>

OK, this means different input use diffevent cpu, other plugin like output or filter don't need a single cpu?

Yes, you’re right in this case.