Feat: Raccoon CLI

[turtleDev]

Changes:

• This PR has many breaking changes when it comes to Raccoon configuration. It's recommended that we release this as a new major version.
• adds a server command to start the application
• there are no flags for dynamic kafka client configs. This is a limitation of the way raccoon handles these configs internally. We recommend users use env variables or config file to pass these values to raccoon.
• default values are not shown in flags. default values for flags are available
• added ability to use a yaml config file. Adds -c flag to specify the path of the config file.
• WORKER_KAFKA_DELIVERY_CHANNEL_SIZE is now PUBLISHER_KAFKA_DELIVERY_CHANNEL_SIZE since it's specific to kafka publisher.
• The flags for server command are dynamically generated from the configuration tree using reflection. New flags only require mutation of configuration structs to add them as env/cli parameters.
• Kafka Client configuration is now fully inlined within raccoon config. This allows greater type safety and discovery.

Preview:

$ raccoon
Raccoon is a high-throughput, low-latency service to collect 
events in real-time from your web, mobile apps, and services 
using multiple network protocols.

Usage:
  raccoon [flags]
  raccoon [command]

Available Commands:
  server      Start raccoon server

Flags:
  -h, --help   help for raccoon

Use "raccoon [command] --help" for more information about a command.

$ raccoon server --help
Start raccoon server

Usage:
  raccoon server [flags]

Flags:
      --log.level string                                             Level available are [debug info warn error fatal panic]
      --event.ack int                                                Whether to send acknowledgements to clients or not. 1 to enable, 0 to disable.
      --server.websocket.port string                                 Port for the service to listen
      --server.websocket.max.conn int                                Maximum connection that can be handled by the server instance
      --server.websocket.read.buffer.size int                        Input buffer size in bytes
      --server.websocket.write.buffer.size int                       Output buffer size in bytes
      --server.websocket.conn.id.header string                       Unique identifier for the server to maintain the connection
      --server.websocket.conn.group.header string                    Additional identifier for the server to maintain the connection
      --server.websocket.conn.group.default string                   Default connection group name
      --server.websocket.ping.interval.ms int                        Interval of each ping to client in milliseconds
      --server.websocket.pong.wait.interval.ms int                   Wait time for client to send Pong message in milliseconds
      --server.websocket.write.wait.interval.ms int                  Timeout deadline set on the writes in milliseconds
      --server.websocket.pinger.size int                             Number of goroutine spawned to ping clients
      --server.websocket.check.origin                                Toggle CORS check on WebSocket API (default true)
      --server.cors.enabled                                          Toggle CORS check on REST API
      --server.cors.allowed.origin strings                           Allowed origins for CORS. Use '*' to allow all
      --server.cors.allowed.methods strings                          Allowed HTTP Methods for CORS
      --server.cors.allowed.headers strings                          Allowed HTTP Headers for CORS
      --server.cors.preflight.max.age.seconds int                    Max Age of preflight responses
      --server.batch.dedup.in.connection.enabled                     Whether to discard duplicate messages
      --worker.buffer.channel.size int                               Size of the buffer queue
      --worker.buffer.flush.timeout.ms int                           Timeout for flushing leftover messages on shutdown
      --worker.pool.size int                                         No of workers that processes the events concurrently
      --worker.kafka.delivery.channel.size int                       Delivery Channel size for Kafka publisher
      --event.distribution.publisher.pattern string                  Topic template used for routing events
      --publisher.type string                                        Publisher to use for transmitting events
      --publisher.kafka.client.bootstrap.servers string              Address of kafka brokers
      --publisher.kafka.client.acks string                           Number of replica acknowledgement before kafka sends ack back to service
      --publisher.kafka.client.retries string                        Number of retries in case of failure
      --publisher.kafka.client.retry.backoff.ms string               Backoff time on retry.
      --publisher.kafka.client.statistics.interval.ms string         Interval of statistics emitted by kafka
      --publisher.kafka.client.queue.buffering.max.messages string   Maximum number of messages allowed on the producer queue
      --publisher.kafka.flush.interval.ms int                        Timeout for sending leftover messages on kafka publisher shutdown
      --publisher.pubsub.credentials string                          Path to file containing GCP cloud credentials
      --publisher.pubsub.project.id string                           Destination Google Cloud Project ID
      --publisher.pubsub.topic.autocreate                            Whether to create topic if it doesn't exist in PubSub
      --publisher.pubsub.topic.retention.ms int                      Retention period of created topics in milliseconds
      --publisher.pubsub.publish.delay.threshold.ms int              Maximum time to wait for before publishing a batch of events
      --publisher.pubsub.publish.count.threshold int                 Maximum number of events to accumulate before transmission
      --publisher.pubsub.publish.byte.threshold int                  Maximum buffer size (in bytes)
      --publisher.pubsub.publish.timeout.ms int                      How long to wait before aborting a publish operation
      --publisher.kinesis.aws.region string                          AWS Region of the target kinesis stream
      --publisher.kinesis.credentials string                         Path to file containing AWS credentials
      --publisher.kinesis.stream.autocreate                          Whether to create a stream if it doesn't exist in Kinesis
      --publisher.kinesis.stream.mode string                         Mode of auto-created streams. Valid values: [ON-DEMAND PROVISIONED]
      --publisher.kinesis.stream.shards uint32                       Number of shards in auto-created streams
      --publisher.kinesis.stream.probe.interval.ms int               time delay between stream status checks
      --publisher.kinesis.publish.timeout.ms int                     how long to wait for before aborting a publish operation
      --metric.runtime.stats.record.interval.ms int                  Time interval between runtime metric collection
      --metric.statsd.enabled                                        Enable statsd metric exporter
      --metric.statsd.address string                                 Address to reports the service metrics
      --metric.statsd.flush.period.ms int                            Interval for the service to push metrics
      --metric.prometheus.enabled                                    Enable prometheus http server to expose service metrics
      --metric.prometheus.path string                                The path at which prometheus server should serve metrics
      --metric.prometheus.port int                                   Port to expose prometheus metrics on
  -h, --help                                                         help for server

[coveralls]

Pull Request Test Coverage Report for Build 10449927361

Details

• 16 of 313 (5.11%) changed or added relevant lines in 19 files are covered.
• 3 unchanged lines in 2 files lost coverage.
• Overall coverage decreased (-3.0%) to 43.266%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
services/grpc/handler.go	2	3	66.67%
services/grpc/service.go	0	1	0.0%
main.go	0	2	0.0%
metrics/statsd.go	0	2	0.0%
services/rest/handler.go	0	3	0.0%
publisher/kafka/kafka.go	0	4	0.0%
worker/worker.go	0	5	0.0%
middleware/cors.go	0	7	0.0%
services/rest/service.go	0	7	0.0%
config/load.go	9	18	50.0%

Files with Coverage Reduction	New Missed Lines	%
main.go	1	0.0%
config/publisher.go	2	0.0%

Totals
Change from base Build 10184672930:	-3.0%
Covered Lines:	1057
Relevant Lines:	2443

---

💛 - Coveralls

[ravisuhag]

@turtleDev Let's use salt/cmdx for cli.