Forest plugin does not flush cloudwatch logs to S3

[rahul-chowdhury]

I am trying to fetch logs from AWS Cloudwatch and push them to S3. I am running td-agent (2.3.6-0) in my ubuntu 16.04. I am using the following plugins :

• fluent-plugin-cloudwatch-logs (0.4.5) for fetching logs from cloudwatch
• fluent-plugin-forest (0.3.0) to send logs to AWS S3
• fluentd (0.12.29)

The issue is that the logs are not being flushed to S3 by the forest plugin. I have verified that the logs are arriving on my local instance by printing them using stdout. Can anyone help regarding what the problem can be? Below is the config file that I am using (The parts marked with $ are masked for privacy concerns).

<source>
  @type cloudwatch_logs
  format json
  state_file /var/log/td-agent/state/$.msa.lambda.state 
  include_time_key true
  time_key time
  time_format %Y-%m-%dT%H:%M:%S.%L%Z
  log_group_name /aws/lambda/$
  log_stream_name 2               #[^:*]*
  use_log_stream_name_prefix true
  region $
  aws_key_id  $
  aws_sec_key  $
  tag $.$.$.msa.lambda.log  
</source>

#<match *.*.$.msa.lambda.log>
 # @type stdout
#</match>

<match *.*.$.msa.lambda.log>
  type copy
  deep_copy true

  <store>
    @type forest
    subtype s3
    <template>
     @type s3
     buffer_type file
     buffer_path /tmp/td-agent/buffer/${tag}.buffer_

     buffer_chunk_limit 64m
     buffer_queue_limit 1024
     num_threads 2 # the number of threads to flush the buffer
     <buffer tag_parts>
     flush_at_shutdown true
     </buffer>
     flush_interval 30s # how long will a compressed file generated on S3
     time_format %Y-%m-%dT%H:%M:%S.%L%z
     # AWS Common Config
     s3_region $
     # AWS Credential Config
     aws_key_id $
     aws_sec_key $
     # AWS S3 Config
     s3_bucket $

     s3_object_key_format %{path}%{time_slice}/${tag_parts[3]}/${tag_parts[2]}/%{index}.%{file_extension}
     auto_create_bucket false
     path ${tag_parts[0]}/${tag_parts[1]}/backendlog/lambda/middleware/${tag_parts[4]}.${tag_parts[5]}.${tag_parts[6]}.${tag_parts[7]}
     time_slice_format %Y-%m-%d/%H  #%Y%m%d%H
     format json
     time_key time
     include_time_key true
     tag_key log_source
     include_tag_key true
     utc # timezone
    </template>

  </store>
</match>

The strange thing is that, if I remove this configuration, forest plugin normally flushes other logs to S3. But as soon as I add this config and restart td-agent, forest plugin stops flushing logs and td-agent.log is stuck at the following line :

2018-12-19 07:44:14 +0000 [info]: listening fluent socket on 0.0.0.0:24224

Are there any conflicts/dependency issues between the plugins fluent-plugin-cloudwatch-logs and fluent-plugin-forest?