Solr System Logging Receiver

[StefanKurek]

Adds ability to receive Solr logs

Scope doc

Example Ingested Log

{
  "insertId": "mre8o5flln44h",
  "jsonPayload": {
    "message": "Logging initialized @1292ms to org.eclipse.jetty.util.log.Slf4jLog",
    "thread": "main",
    "source": "o.e.j.u.log"
  },
  "resource": {
    "type": "gce_instance",
    "labels": {
      "instance_id": "3811867359076886516",
      "project_id": "otel-agent-dev",
      "zone": "us-central1-a"
    }
  },
  "timestamp": "2022-02-08T16:52:37.756Z",
  "severity": "INFO",
  "logName": "projects/otel-agent-dev/logs/solr_system",
  "receiveTimestamp": "2022-02-08T16:54:24.047382748Z"
}

[StefanKurek]

@qingling128 Hello. This should be in a good state to review now. Thanks!

[qingling128]

Left a minor comment.

Can you also attached a sample ingested log entry (Can be retrieved from the Log Viewer UI) in the description of this PR?

[qingling128]

Adding @ridwanmsharif and @jefferbrecht for shadowing

[qingling128]

Can you also attached a sample ingested log entry (Can be retrieved from the Log Viewer UI) in the description of this PR?

[StefanKurek]

Left a minor comment.

Can you also attached a sample ingested log entry (Can be retrieved from the Log Viewer UI) in the description of this PR?

@lidizheng yep, I was just reinstalling and then I will do this.

[qingling128]

LGTM

Waiting for the integration tests to pass before merging.

BTW, just FYI that we recently adjusted the GitHub settings to require a sync to HEAD prior to merging any PRs in order to catch unexpected golden file changes that affect only the unit tests.

So if the integration tests passed already for a 3rd party app PR, which only touches isolated files that are specific to that PR, you don't have to re-trigger the integration tests after rebasing. Just leave a comment in the PR history with the commit that passed the integration tests in the past so the reviewers are aware that the integration tests passed already.

The main signal we are trying to catch is whether the unit tests still pass after a rebasing.

When it comes to this PR, since there are already a set of integration tests that are running, and so far the failures do not seem related to this PR:

--- FAIL: TestThirdPartyApps (0.00s)
    --- FAIL: TestThirdPartyApps/debian-10 (0.00s)
        --- PASS: TestThirdPartyApps/debian-10/redis (158.65s)
        --- PASS: TestThirdPartyApps/debian-10/memcached (159.28s)
        --- PASS: TestThirdPartyApps/debian-10/couchdb (162.94s)
        --- PASS: TestThirdPartyApps/debian-10/apache (176.94s)
        --- PASS: TestThirdPartyApps/debian-10/mongodb (178.46s)
        --- PASS: TestThirdPartyApps/debian-10/postgresql (182.67s)
        --- PASS: TestThirdPartyApps/debian-10/mysql (201.76s)
        --- PASS: TestThirdPartyApps/debian-10/jvm (208.39s)
        --- PASS: TestThirdPartyApps/debian-10/cassandra (218.80s)
        --- PASS: TestThirdPartyApps/debian-10/elasticsearch (220.87s)
        --- PASS: TestThirdPartyApps/debian-10/nginx (222.43s)
        --- PASS: TestThirdPartyApps/debian-10/solr (229.66s)
        --- FAIL: TestThirdPartyApps/debian-10/tomcat (275.10s)

--- FAIL: TestThirdPartyApps (0.00s)
    --- FAIL: TestThirdPartyApps/sles-12 (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/redis (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/mysql (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/jvm (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/couchdb (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/apache (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/nginx (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/cassandra (0.00s)
        --- PASS: TestThirdPartyApps/sles-12/mongodb (905.08s)
        --- FAIL: TestThirdPartyApps/sles-12/memcached (906.38s)
        --- PASS: TestThirdPartyApps/sles-12/solr (930.99s)
        --- PASS: TestThirdPartyApps/sles-12/elasticsearch (986.31s)
        --- PASS: TestThirdPartyApps/sles-12/tomcat (1004.00s)
        --- PASS: TestThirdPartyApps/sles-12/postgresql (1194.91s)

We can wait for it to finish, record the integration test results, rebase, wait for the unit tests only (do not need to trigger another set of integration tests), and merge.

[StefanKurek]

@qingling128

Ops Agent Integration Test (SLES 15)

Failure, but doesn't appear to have Solr Related Issues

third party apps integration test (Debian)

Failure, but Solr Tests good

--- FAIL: TestThirdPartyApps (0.00s)
    --- FAIL: TestThirdPartyApps/debian-10 (0.00s)
        --- PASS: TestThirdPartyApps/debian-10/redis (158.65s)
        --- PASS: TestThirdPartyApps/debian-10/memcached (159.28s)
        --- PASS: TestThirdPartyApps/debian-10/couchdb (162.94s)
        --- PASS: TestThirdPartyApps/debian-10/apache (176.94s)
        --- PASS: TestThirdPartyApps/debian-10/mongodb (178.46s)
        --- PASS: TestThirdPartyApps/debian-10/postgresql (182.67s)
        --- PASS: TestThirdPartyApps/debian-10/mysql (201.76s)
        --- PASS: TestThirdPartyApps/debian-10/jvm (208.39s)
        --- PASS: TestThirdPartyApps/debian-10/cassandra (218.80s)
        --- PASS: TestThirdPartyApps/debian-10/elasticsearch (220.87s)
        --- PASS: TestThirdPartyApps/debian-10/nginx (222.43s)
        --- PASS: TestThirdPartyApps/debian-10/solr (229.66s)
        --- FAIL: TestThirdPartyApps/debian-10/tomcat (275.10s)

third party integration test (SLES 12)

Failed, but Solr Tests good

--- FAIL: TestThirdPartyApps (0.00s)
    --- FAIL: TestThirdPartyApps/sles-12 (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/redis (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/mysql (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/jvm (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/couchdb (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/apache (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/nginx (0.00s)
        --- SKIP: TestThirdPartyApps/sles-12/cassandra (0.00s)
        --- PASS: TestThirdPartyApps/sles-12/mongodb (905.08s)
        --- FAIL: TestThirdPartyApps/sles-12/memcached (906.38s)
        --- PASS: TestThirdPartyApps/sles-12/solr (930.99s)
        --- PASS: TestThirdPartyApps/sles-12/elasticsearch (986.31s)
        --- PASS: TestThirdPartyApps/sles-12/tomcat (1004.00s)
        --- PASS: TestThirdPartyApps/sles-12/postgresql (1194.91s)

[qingling128]

OK to merge. But it looks like something got merged before I got to this, so it needs rebasing again. (Maybe we should revisit the GitHub settings...