Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add java attacher config #5483

Merged
merged 12 commits into from
Jun 24, 2021

Conversation

stuartnelson3
Copy link
Contributor

@stuartnelson3 stuartnelson3 commented Jun 17, 2021

Motivation/summary

Adding basic include/exclude and config flags.

Based on this comment I pulled out version as something to be static for a given build of apm-server, which we update when we bundle with a new version of the attacher.

closes #4829

Checklist

For functional changes, consider:

  • Is it observable through the addition of either logging or metrics?
  • Is its use being published in telemetry to enable product improvement?
  • Have system tests been added to avoid regression?

How to test these changes

Update the apm-server.yml to include java attacher config fields, and verify they're parsed.

eg.

  java_attacher:
    enabled: true
    discovery-rules:
    - include-main: opbeans
    - include-pid: 42
    - include-pid: 123
    - include-pid: 456
    - exclude-main: ".*"
    config:
    - "service_name=my-cool-service"
    - "server_url=http://localhost:8200"

Related issues

@stuartnelson3 stuartnelson3 requested a review from a team June 17, 2021 13:00
@apmmachine
Copy link
Contributor

apmmachine commented Jun 17, 2021

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: stuartnelson3 commented: jenkins run the tests

  • Start Time: 2021-06-24T10:23:04.014+0000

  • Duration: 40 min 9 sec

  • Commit: c7a85c1

Test stats 🧪

Test Results
Failed 0
Passed 4715
Skipped 116
Total 4831

Trends 🧪

Image of Build Times

Image of Tests

Steps errors 3

Expand to view the steps failures

Run Unit tests
  • Took 1 min 11 sec . View more details on here
  • Description: .ci/scripts/test-darwin.sh
Run Unit tests
  • Took 0 min 3 sec . View more details on here
  • Description: .ci/scripts/test-darwin.sh
Test Sync
  • Took 3 min 19 sec . View more details on here
  • Description: ./.ci/scripts/sync.sh

Log output

Expand to view the last 100 lines of log output

[2021-06-24T10:58:58.457Z] === RUN   TestJaegerGRPCSampling
[2021-06-24T10:58:58.457Z] --- PASS: TestJaegerGRPCSampling (3.62s)
[2021-06-24T10:58:58.457Z] === RUN   TestJaegerGRPCAuth
[2021-06-24T10:58:58.457Z] --- PASS: TestJaegerGRPCAuth (4.20s)
[2021-06-24T10:58:58.457Z] === RUN   TestAPMServerGRPCRequestLoggingValid
[2021-06-24T10:58:58.457Z] --- PASS: TestAPMServerGRPCRequestLoggingValid (3.85s)
[2021-06-24T10:58:58.457Z] === RUN   TestAPMServerRequestLoggingValid
[2021-06-24T10:58:58.457Z] --- PASS: TestAPMServerRequestLoggingValid (0.20s)
[2021-06-24T10:58:58.457Z] === RUN   TestApprovedMetrics
[2021-06-24T10:58:58.457Z] --- PASS: TestApprovedMetrics (5.29s)
[2021-06-24T10:58:58.457Z] === RUN   TestBreakdownMetrics
[2021-06-24T10:58:58.457Z] --- PASS: TestBreakdownMetrics (3.91s)
[2021-06-24T10:58:58.457Z] === RUN   TestApplicationMetrics
[2021-06-24T10:58:58.457Z] --- PASS: TestApplicationMetrics (3.91s)
[2021-06-24T10:58:58.457Z] === RUN   TestAPMServerMonitoring
[2021-06-24T10:58:58.457Z] --- PASS: TestAPMServerMonitoring (1.04s)
[2021-06-24T10:58:58.457Z] === RUN   TestAPMServerMonitoringBuiltinUser
[2021-06-24T10:58:58.457Z] --- PASS: TestAPMServerMonitoringBuiltinUser (1.95s)
[2021-06-24T10:58:58.457Z] === RUN   TestAPMServerOnboarding
[2021-06-24T10:58:58.457Z] --- PASS: TestAPMServerOnboarding (3.89s)
[2021-06-24T10:58:58.457Z] === RUN   TestOTLPGRPCTraces
[2021-06-24T10:58:58.457Z] --- PASS: TestOTLPGRPCTraces (4.74s)
[2021-06-24T10:58:58.457Z] === RUN   TestOTLPGRPCMetrics
[2021-06-24T10:58:58.457Z] --- PASS: TestOTLPGRPCMetrics (4.58s)
[2021-06-24T10:58:58.457Z] === RUN   TestOTLPGRPCAuth
[2021-06-24T10:58:58.457Z] --- PASS: TestOTLPGRPCAuth (3.76s)
[2021-06-24T10:58:58.457Z] === RUN   TestOTLPClientIP
[2021-06-24T10:58:58.457Z] --- PASS: TestOTLPClientIP (4.58s)
[2021-06-24T10:58:58.457Z] === RUN   TestRUMXForwardedFor
[2021-06-24T10:58:58.457Z] --- PASS: TestRUMXForwardedFor (4.24s)
[2021-06-24T10:58:58.457Z] === RUN   TestRUMErrorSourcemapping
[2021-06-24T10:58:58.457Z] --- PASS: TestRUMErrorSourcemapping (5.54s)
[2021-06-24T10:58:58.457Z] === RUN   TestRUMAuth
[2021-06-24T10:58:58.457Z] --- PASS: TestRUMAuth (0.48s)
[2021-06-24T10:58:58.457Z] === RUN   TestRUMAllowServiceNames
[2021-06-24T10:58:58.457Z] --- PASS: TestRUMAllowServiceNames (0.19s)
[2021-06-24T10:58:58.457Z] === RUN   TestRUMRateLimit
[2021-06-24T10:58:58.457Z] --- PASS: TestRUMRateLimit (0.91s)
[2021-06-24T10:58:58.457Z] === RUN   TestKeepUnsampled
[2021-06-24T10:58:58.457Z] === RUN   TestKeepUnsampled/false
[2021-06-24T10:58:58.457Z] === RUN   TestKeepUnsampled/true
[2021-06-24T10:58:58.457Z] --- PASS: TestKeepUnsampled (9.08s)
[2021-06-24T10:58:58.457Z]     --- PASS: TestKeepUnsampled/false (3.86s)
[2021-06-24T10:58:58.457Z]     --- PASS: TestKeepUnsampled/true (5.22s)
[2021-06-24T10:58:58.457Z] === RUN   TestKeepUnsampledWarning
[2021-06-24T10:58:58.457Z] --- PASS: TestKeepUnsampledWarning (3.85s)
[2021-06-24T10:58:58.457Z] === RUN   TestTailSampling
[2021-06-24T10:58:58.457Z]     sampling_test.go:135: waiting for 100 "parent" transactions
[2021-06-24T10:58:58.457Z]     sampling_test.go:135: waiting for 100 "child" transactions
[2021-06-24T10:58:58.457Z] --- PASS: TestTailSampling (7.41s)
[2021-06-24T10:58:58.457Z] === RUN   TestTailSamplingUnlicensed
[2021-06-24T10:58:58.457Z] 2021/06/24 10:58:19 Starting container id: e2cefe258f26 image: docker.elastic.co/elasticsearch/elasticsearch:8.0.0-943ef2c0-SNAPSHOT
[2021-06-24T10:58:58.457Z] 2021/06/24 10:58:20 Waiting for container id e2cefe258f26 image: docker.elastic.co/elasticsearch/elasticsearch:8.0.0-943ef2c0-SNAPSHOT
[2021-06-24T10:58:58.457Z] 2021/06/24 10:58:36 Container is ready id: e2cefe258f26 image: docker.elastic.co/elasticsearch/elasticsearch:8.0.0-943ef2c0-SNAPSHOT
[2021-06-24T10:58:58.457Z] --- PASS: TestTailSamplingUnlicensed (26.82s)
[2021-06-24T10:58:58.457Z] === RUN   TestIndexTemplateCoverage
[2021-06-24T10:58:58.457Z] --- PASS: TestIndexTemplateCoverage (5.14s)
[2021-06-24T10:58:58.457Z] PASS
[2021-06-24T10:58:58.457Z] ok  	github.com/elastic/apm-server/systemtest	351.363s
[2021-06-24T10:58:58.457Z] === RUN   TestAPMServer
[2021-06-24T10:58:58.457Z] 2021/06/24 10:52:59 Building apm-server...
[2021-06-24T10:58:58.457Z] 2021/06/24 10:53:01 Built /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5483/src/github.com/elastic/apm-server/apm-server
[2021-06-24T10:58:58.457Z] --- PASS: TestAPMServer (2.58s)
[2021-06-24T10:58:58.457Z] === RUN   TestUnstartedAPMServer
[2021-06-24T10:58:58.457Z] --- PASS: TestUnstartedAPMServer (0.00s)
[2021-06-24T10:58:58.457Z] === RUN   TestAPMServerStartTLS
[2021-06-24T10:58:58.457Z] --- PASS: TestAPMServerStartTLS (0.13s)
[2021-06-24T10:58:58.457Z] === RUN   TestExpvar
[2021-06-24T10:58:58.457Z] --- PASS: TestExpvar (0.08s)
[2021-06-24T10:58:58.457Z] PASS
[2021-06-24T10:58:58.458Z] ok  	github.com/elastic/apm-server/systemtest/apmservertest	2.855s
[2021-06-24T10:58:58.458Z] ?   	github.com/elastic/apm-server/systemtest/benchtest	[no test files]
[2021-06-24T10:58:58.458Z] ?   	github.com/elastic/apm-server/systemtest/cmd/apmbench	[no test files]
[2021-06-24T10:58:58.458Z] ?   	github.com/elastic/apm-server/systemtest/estest	[no test files]
[2021-06-24T10:58:58.458Z] ?   	github.com/elastic/apm-server/systemtest/fleettest	[no test files]
[2021-06-24T10:58:58.458Z] + cleanup
[2021-06-24T10:58:58.458Z] + rm -rf /tmp/tmp.hAjZCYgmFz
[2021-06-24T10:58:58.458Z] + .ci/scripts/docker-get-logs.sh
[2021-06-24T10:58:58.458Z] It is not possible to grab the logs of 6d4c9a2a1506
[2021-06-24T10:58:59.563Z] Post stage
[2021-06-24T10:58:59.576Z] Running in /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5483/src/github.com/elastic/apm-server/build
[2021-06-24T10:58:59.598Z] Archiving artifacts
[2021-06-24T10:59:00.046Z] Recording test results
[2021-06-24T10:59:00.752Z] [Checks API] No suitable checks publisher found.
[2021-06-24T10:59:01.090Z] + tar --version
[2021-06-24T10:59:01.458Z] + tar --exclude=system-tests-linux-files.tgz -czf system-tests-linux-files.tgz system-tests
[2021-06-24T10:59:01.802Z] Archiving artifacts
[2021-06-24T10:59:02.404Z] Terminated
[2021-06-24T10:59:02.388Z] Terminated
[2021-06-24T11:03:10.397Z] [INFO] For detailed information see: https://apm-ci.elastic.co/job/apm-integration-tests-selector-mbp/job/master/18063/display/redirect
[2021-06-24T11:03:10.633Z] Copied 18 artifacts from "APM Integration Test MBP Selector » master" build number 18063
[2021-06-24T11:03:11.815Z] Post stage
[2021-06-24T11:03:11.827Z] Recording test results
[2021-06-24T11:03:12.629Z] [Checks API] No suitable checks publisher found.
[2021-06-24T11:03:12.944Z] Running on Jenkins in /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5483
[2021-06-24T11:03:12.998Z] [INFO] getVaultSecret: Getting secrets
[2021-06-24T11:03:13.042Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-06-24T11:03:13.769Z] + chmod 755 generate-build-data.sh
[2021-06-24T11:03:13.769Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5483/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5483/runs/12 UNSTABLE 2409496
[2021-06-24T11:03:13.769Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5483/runs/12/steps/?limit=10000 -o steps-info.json

re-add it at some point when we want to display
the version
Copy link
Contributor

@simitt simitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Let's get @felixbarny involved to take a look at the config options.

// attacher jarfile.
type JavaAttacherConfig struct {
Enabled bool `config:"enabled"`
Continuous bool `config:"continuous"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather expect that the apm-server would always set continuous when starting the attacher; but ultimately leave that to @felixbarny .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 will see what he says

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on always implicitly using the continuous mode.

Copy link
Member

@felixbarny felixbarny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose you're consciously just adding the settings and the impl of these settings comes in a follow up PR?

// attacher jarfile.
type JavaAttacherConfig struct {
Enabled bool `config:"enabled"`
Continuous bool `config:"continuous"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on always implicitly using the continuous mode.

@felixbarny
Copy link
Member

For config, instead of

    config:
      - "service_name=my-cool-service"
      - "server_url=http://localhost:8200"

Could we do something like this instead?

    config:
      service_name: my-cool-service
      server_url: http://localhost:8200

beater/config/java_attacher.go Outdated Show resolved Hide resolved
IncludeAll bool `config:"include_all"`
IncludePID []string `config:"include_pid"`

IncludeMain []string `config:"include_main"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A note for the implementation: All the values will have to be escaped when passing them to the attacher to avoid injection attack vectors.

Example:

 - include_main: "foo; rm -rf /"

should not lead to the hard drive being purged

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

@stuartnelson3
Copy link
Contributor Author

jenkins run the tests

@stuartnelson3 stuartnelson3 merged commit 92f0452 into elastic:master Jun 24, 2021
@stuartnelson3 stuartnelson3 deleted the java-attacher-config branch June 24, 2021 11:33
mergify bot pushed a commit that referenced this pull request Jun 24, 2021
* add java attacher config

(cherry picked from commit 92f0452)
stuartnelson3 added a commit that referenced this pull request Jun 28, 2021
* add java attacher config

(cherry picked from commit 92f0452)

Co-authored-by: stuart nelson <[email protected]>
@axw
Copy link
Member

axw commented Jul 6, 2021

I'm removing this from the 7.14 test plan, as the feature can't be used yet. Let's test the config as part of the feature when we deliver it.

@axw axw removed the test-plan label Jul 6, 2021
mergify bot pushed a commit that referenced this pull request Jul 6, 2021
* add java attacher config

(cherry picked from commit 92f0452)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[fleet] Support java attacher settings via apm-server.yml
5 participants