[Why] Currently,only one worker is used to process all logs. As a result,logs can't be processed when there are a large number of concurrent logs. So use 10 workers. But tail plugin don't support multi-process. Use worker 0 for tail plugin. Use worker 1-9 for forward plugin.
Signed-off-by: Wu Zhende wuzhende666@163.com --- container/fluentd/docker-fluentd.conf | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-)
diff --git a/container/fluentd/docker-fluentd.conf b/container/fluentd/docker-fluentd.conf index ac73dfb..23c7585 100644 --- a/container/fluentd/docker-fluentd.conf +++ b/container/fluentd/docker-fluentd.conf @@ -1,8 +1,8 @@ -<source> - @type forward - bind 0.0.0.0 -</source> +<system> + workers 10 +</system>
+<worker 0> <source> @type tail path /srv/cci/serial/logs/* @@ -37,6 +37,21 @@ </format> </store> </match> +</worker> + +<worker 1-9> +<source> + @type forward + bind 0.0.0.0 +</source> + +<filter **> + @type record_transformer + enable_ruby + <record> + time ${time.strftime('%Y-%m-%d %H:%M:%S%z')} + </record> +</filter>
<filter **> @type parser @@ -74,6 +89,7 @@ </server> </store> </match> +</worker>
<label @FLUENT_LOG> <match fluent.*>