RabbitMQ Logging Receiver

[schmikei]

{
  "insertId": "f9yesrfhnie8m",
  "jsonPayload": {
    "message": "** Connection attempt from node 'rabbit_ctl_89@keith-testing-rabbitmq' rejected. Invalid challenge reply. **",
    "process_id": "0.14265.9"
  },
  "resource": {
    "type": "gce_instance",
    "labels": {
      "zone": "us-central1-b",
      "instance_id": "4156710747173665329",
      "project_id": "otel-agent-dev"
    }
  },
  "timestamp": "2022-02-15T14:26:36.789993352Z",
  "logName": "projects/otel-agent-dev/logs/rabbitmq",
  "receiveTimestamp": "2022-02-15T14:26:37.347471494Z"
}

Config

# Copyright 2020 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# <== Enter custom agent configurations in this file.
# See https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent/configuration
# for more details.
#
logging:
  receivers:
    rabbitmq:
      type: rabbitmq
  service:
    pipelines:
      rabbitmq:
        receivers: [rabbitmq]

Also resolves small nit for collection interval in mongodb metrics input.yaml from #354

[schmikei]

Kokoro passed on commit: 24c30a65bd822bb0b7f2e53b6f975e37f97d5a8f

subsequent commits are just rebases/readme updates.

[ridwanmsharif]

[nit][formatting] For consistency with the other receivers, could we have all of the pairings be inline with their braces like:

c = append(c, fluentbit.TranslationComponents(tag, "severity", "logging.googleapis.com/severity", true, []struct {
		SrcVal  string
		DestVal string
	}{
		{"debug", "DEBUG"},
		{"error", "ERROR"},
		{"info", "INFO"},
		{"noti", "DEFAULT"},
	}...,
)

[ridwanmsharif]

In the example in the comments above, will all of

2022-01-31 18:07:43.557042+00:00 [erro] <0.130.0>
BOOT FAILED
===========
ERROR: could not bind to distribution port 25672, it is in use by another node: rabbit@keith-testing-rabbitmq

Be a single LogEntry? The regex multiline rules here suggest that they won't be no?

If we wanted them to be the same LogEntry would this make more sense?:

r.MultilineRules = []confgenerator.MultilineRule{
		{
			StateName: "start_state",
			NextState: "cont",
			Regex:     `\d+-\d+-\d+ \d+:\d+:\d+\.\d+\+\d+:\d+`,
		},
		{
			StateName: "cont",
			NextState: "cont",
			Regex:     `^(?!\d+-\d+-\d+ \d+:\d+:\d+\.\d+\+\d+:\d+)`,
		},
	}

[schmikei]

Ahh that makes more sense, I think I may have misunderstood quite how multiline was working but just tested this out and it appears to work great! Thanks for the suggestion

[ridwanmsharif]

LGTM

[ridwanmsharif]

[nit] Few extra newlines here can be removed

[schmikei]

@qingling128 @ridwanmsharif just curious if there is anything preventing this from merging? Thanks again for the reviews!

[qingling128]

The failure seems to be for debian 10 mysql only. And it looks unrelated:

+ sudo apt-key adv --keyserver pgp.mit.edu --recv-keys 3A79BD29
Warning: apt-key output should not be parsed (stdout is not a terminal)
gpg: keyserver receive failed: No keyserver available
, retryable=true
    third_party_apps_test.go:428: Final attempt failed: error installing mysql: Command failed: cat - > script.sh && sudo bash -x script.sh 
        Command failed: [ssh [email protected] -oIdentityFile=/tmpfs/tmp/ssh_keys1311891234/gce_testing_key -oConnectTimeout=120 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oLogLevel=ERROR cat - > script.sh && sudo bash -x script.sh ]
        exit status 2
        stdout: Executing: /tmp/apt-key-gpghome.fCQ3lEv57U/gpg.1.sh --keyserver pgp.mit.edu --recv-keys 3A79BD29
        
        stderr: + set -e
        + sudo apt-key adv --keyserver pgp.mit.edu --recv-keys 3A79BD29
        Warning: apt-key output should not be parsed (stdout is not a terminal)
        gpg: keyserver receive failed: No keyserver available
--- FAIL: TestThirdPartyApps (0.00s)
    --- FAIL: TestThirdPartyApps/debian-10 (0.00s)
        --- PASS: TestThirdPartyApps/debian-10/memcached (146.37s)
        --- PASS: TestThirdPartyApps/debian-10/redis (157.76s)
        --- PASS: TestThirdPartyApps/debian-10/zookeeper (158.26s)
        --- PASS: TestThirdPartyApps/debian-10/wildfly (158.93s)
        --- PASS: TestThirdPartyApps/debian-10/nginx (172.60s)
        --- PASS: TestThirdPartyApps/debian-10/couchdb (173.67s)
        --- PASS: TestThirdPartyApps/debian-10/apache (174.67s)
        --- PASS: TestThirdPartyApps/debian-10/postgresql (178.77s)
        --- PASS: TestThirdPartyApps/debian-10/mongodb (209.80s)
        --- PASS: TestThirdPartyApps/debian-10/cassandra (214.96s)
        --- PASS: TestThirdPartyApps/debian-10/jvm (231.44s)
        --- PASS: TestThirdPartyApps/debian-10/elasticsearch#01 (232.13s)
        --- PASS: TestThirdPartyApps/debian-10/activemq (232.75s)
        --- PASS: TestThirdPartyApps/debian-10/elasticsearch (237.60s)
        --- PASS: TestThirdPartyApps/debian-10/solr (239.68s)
        --- PASS: TestThirdPartyApps/debian-10/hbase#01 (243.71s)
        --- PASS: TestThirdPartyApps/debian-10/hbase (254.55s)
        --- PASS: TestThirdPartyApps/debian-10/kafka (254.78s)
        --- PASS: TestThirdPartyApps/debian-10/rabbitmq (274.37s)
        --- PASS: TestThirdPartyApps/debian-10/tomcat (281.28s)
        --- PASS: TestThirdPartyApps/debian-10/hadoop (320.68s)
        --- FAIL: TestThirdPartyApps/debian-10/mysql (694.60s)

Merging.