Add instrumentation for AWS SQS and SNS

[estolfo]

I've added support for SNS and SQS in the same PR because they both depend on some additions to the span context.

SQS

Note that there is no instrumentation for polling, as you can poll a SQS queue by doing one of the following two things:

1. Set wait_time_seconds when calling client#receive_message. So the action is still RECEIVE, though with a longer timeout. I didn't see that it was necessary to inspect the method params for a wait_time_seconds value and identify the action as POLL instead of RECEIVE in the case that it was set.
2. Use the QueuePoller class. This class is just a wrapper around client methods. QueuePoller.poll just calls receive_message on a client in a loop. So I didn't think it necessary to instrument this class, as the user will see the individual RECEIVE actions instrumented anyway.

This PR also adds a class Message that allows span.context.message fields to be added.

And lastly, the SQS instrumentation may look repetitive but each method differs slightly. For example, we must iterate over the params[:entries] in order to add trace context info to messages sent via send_message_batch, as opposed to adding the info directly to the params. I thought it would make the code more confusing if I tried to centralize the instrumentation into one method that handled slight differences.

See the spec here

Closes #946

SNS

Closes #945

[apmmachine]

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: Pull request #978 updated

• Start Time: 2021-03-04T15:17:57.426+0000

• Duration: 12 min 11 sec

• Commit: 429557b

Trends 🧪

Steps errors

Expand to view the steps failures

Show only the first 10 steps failures

Shell Script

• Took 1 min 38 sec . View more details on here
• Description: ./spec/scripts/spec.sh ruby:3.0 sinatra-master

Shell Script

• Took 1 min 47 sec . View more details on here
• Description: ./spec/scripts/spec.sh ruby:3.0 grape-master

Shell Script

• Took 1 min 37 sec . View more details on here
• Description: ./spec/scripts/spec.sh ruby:3.0 grape-master

Shell Script

• Took 2 min 35 sec . View more details on here
• Description: ./spec/scripts/spec.sh ruby:2.7 rails-main

Shell Script

• Took 1 min 25 sec . View more details on here
• Description: ./spec/scripts/spec.sh ruby:2.7 rails-main

Shell Script

• Took 1 min 10 sec . View more details on here
• Description: ./spec/scripts/spec.sh ruby:2.7 sinatra-master

Shell Script

• Took 1 min 55 sec . View more details on here
• Description: ./spec/scripts/spec.sh ruby:2.7 sinatra-master

Shell Script

• Took 1 min 8 sec . View more details on here
• Description: ./spec/scripts/spec.sh ruby:2.7 grape-master

Shell Script

• Took 1 min 57 sec . View more details on here
• Description: ./spec/scripts/spec.sh ruby:2.7 grape-master

Error signal

• Took 0 min 0 sec . View more details on here
• Description: Ruby:ruby:3.0#rails-6.1 tests failed : hudson.AbortException: script returned exit code 1

Log output

Expand to view the last 100 lines of log output

[2021-03-04T15:29:32.023Z] [Checks API] No suitable checks publisher found.
[2021-03-04T15:29:32.042Z] Running in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-978/src/github.com/elastic/apm-agent-ruby
[2021-03-04T15:29:32.076Z] Archiving artifacts
[2021-03-04T15:29:32.572Z] Stashed 32 file(s)
[2021-03-04T15:29:32.652Z] Failed in branch Ruby:ruby:2.4#rails-4.2
[2021-03-04T15:29:34.156Z] Coverage may be inaccurate; set the "--debug" command line option, or do JRUBY_OPTS="--debug" or set the "debug.fullTrace=true" option in your .jrubyrc
[2021-03-04T15:29:34.843Z] /usr/local/bundle/jruby/2.5.0/gems/simplecov-0.21.2/lib/simplecov.rb:345: warning: tracing (e.g. set_trace_func) will not capture all events without --debug flag
[2021-03-04T15:29:39.140Z] [INFO] Skipping Sinatra.start spec
[2021-03-04T15:29:43.083Z] [INFO] Skipping Sinatra.start spec
[2021-03-04T15:29:45.824Z] 
[2021-03-04T15:29:45.824Z] An error occurred while loading ./spec/elastic_apm/spies/sns_spec.rb. - Did you mean?
[2021-03-04T15:29:45.824Z]                     rspec ./spec/elastic_apm/spies/sqs_spec.rb
[2021-03-04T15:29:45.824Z]                     rspec ./spec/elastic_apm/spies/mongo_spec.rb
[2021-03-04T15:29:45.824Z]                     rspec ./spec/elastic_apm/spies/sneakers_spec.rb
[2021-03-04T15:29:45.824Z] 
[2021-03-04T15:29:45.824Z] Failure/Error: res = require_without_apm(path)
[2021-03-04T15:29:45.824Z] 
[2021-03-04T15:29:45.824Z] LoadError:
[2021-03-04T15:29:45.824Z]   no such file to load -- aws-sdk-sns
[2021-03-04T15:29:45.824Z] # ./lib/elastic_apm/spies.rb:91:in `require'
[2021-03-04T15:29:45.824Z] # ./spec/elastic_apm/spies/sns_spec.rb:21:in `<main>'
[2021-03-04T15:29:48.430Z] 
[2021-03-04T15:29:48.430Z] An error occurred while loading ./spec/elastic_apm/spies/sns_spec.rb. - Did you mean?
[2021-03-04T15:29:48.430Z]                     rspec ./spec/elastic_apm/spies/sqs_spec.rb
[2021-03-04T15:29:48.430Z]                     rspec ./spec/elastic_apm/spies/mongo_spec.rb
[2021-03-04T15:29:48.430Z]                     rspec ./spec/elastic_apm/spies/sneakers_spec.rb
[2021-03-04T15:29:48.430Z] 
[2021-03-04T15:29:48.430Z] Failure/Error: res = require_without_apm(path)
[2021-03-04T15:29:48.430Z] 
[2021-03-04T15:29:48.430Z] LoadError:
[2021-03-04T15:29:48.430Z]   no such file to load -- aws-sdk-sns
[2021-03-04T15:29:48.430Z] # ./lib/elastic_apm/spies.rb:91:in `require'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:291:in `block in require'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:257:in `load_dependency'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:291:in `require'
[2021-03-04T15:29:48.430Z] # ./spec/elastic_apm/spies/sns_spec.rb:21:in `<main>'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:285:in `block in load'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:257:in `load_dependency'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:285:in `load'
[2021-03-04T15:29:49.151Z] 
[2021-03-04T15:29:49.151Z] 
[2021-03-04T15:29:49.151Z] Finished in 0.00153 seconds (files took 23.35 seconds to load)
[2021-03-04T15:29:49.151Z] 0 examples, 0 failures, 1 error occurred outside of examples
[2021-03-04T15:29:49.151Z] 
[2021-03-04T15:29:49.382Z] 
[2021-03-04T15:29:49.382Z] 
[2021-03-04T15:29:49.382Z] Finished in 0.00127 seconds (files took 15.93 seconds to load)
[2021-03-04T15:29:49.382Z] 0 examples, 0 failures, 1 error occurred outside of examples
[2021-03-04T15:29:49.383Z] 
[2021-03-04T15:29:51.079Z] Coverage report generated for RSpec to /app/coverage/matrix_results/grape-1.5-docker.elastic.co/observability-ci/jruby-9.2-11-jdk. 0 / 3461 LOC (0.0%) covered.
[2021-03-04T15:29:51.079Z] Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected
[2021-03-04T15:29:51.405Z] Coverage report generated for RSpec to /app/coverage/matrix_results/rails-5.2-docker.elastic.co/observability-ci/jruby-9.2-8-jdk. 0 / 3454 LOC (0.0%) covered.
[2021-03-04T15:29:51.406Z] Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected
[2021-03-04T15:29:52.176Z] Recording test results
[2021-03-04T15:29:52.442Z] Recording test results
[2021-03-04T15:29:52.676Z] None of the test reports contained any result
[2021-03-04T15:29:52.676Z] [Checks API] No suitable checks publisher found.
[2021-03-04T15:29:52.686Z] Running in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-978/src/github.com/elastic/apm-agent-ruby
[2021-03-04T15:29:52.715Z] Archiving artifacts
[2021-03-04T15:29:53.118Z] Stashed 32 file(s)
[2021-03-04T15:29:53.175Z] Failed in branch Ruby:docker.elastic.co/observability-ci/jruby:9.2-11-jdk#grape-1.5
[2021-03-04T15:29:53.639Z] None of the test reports contained any result
[2021-03-04T15:29:53.640Z] [Checks API] No suitable checks publisher found.
[2021-03-04T15:29:53.650Z] Running in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-978/src/github.com/elastic/apm-agent-ruby
[2021-03-04T15:29:53.673Z] Archiving artifacts
[2021-03-04T15:29:54.039Z] Stashed 32 file(s)
[2021-03-04T15:29:54.073Z] Failed in branch Ruby:docker.elastic.co/observability-ci/jruby:9.2-8-jdk#rails-5.2
[2021-03-04T15:29:55.023Z] Post stage
[2021-03-04T15:29:55.488Z] Running in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-978/src/github.com/elastic/apm-agent-ruby
[2021-03-04T15:29:57.381Z] Error when executing always post condition:
[2021-03-04T15:29:57.381Z] hudson.AbortException: No such saved stash ‘jruby-9.2-13-jdk-rails-6.1’
[2021-03-04T15:29:57.381Z] 	at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:159)
[2021-03-04T15:29:57.381Z] 	at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:76)
[2021-03-04T15:29:57.381Z] 	at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:63)
[2021-03-04T15:29:57.381Z] 	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
[2021-03-04T15:29:57.381Z] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2021-03-04T15:29:57.381Z] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[2021-03-04T15:29:57.381Z] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2021-03-04T15:29:57.381Z] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2021-03-04T15:29:57.381Z] 	at java.lang.Thread.run(Thread.java:748)
[2021-03-04T15:29:57.381Z] 
[2021-03-04T15:29:57.526Z] Failed in branch Tests
[2021-03-04T15:30:06.688Z] [INFO] For detailed information see: https://apm-ci.elastic.co/job/apm-integration-tests-selector-mbp/job/master/14808/display/redirect
[2021-03-04T15:30:06.825Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2021-03-04T15:30:06.849Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2021-03-04T15:30:06.916Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2021-03-04T15:30:07.046Z] Stage "Release" skipped due to earlier failure(s)
[2021-03-04T15:30:07.077Z] Stage "Release" skipped due to earlier failure(s)
[2021-03-04T15:30:07.145Z] Stage "Release" skipped due to earlier failure(s)
[2021-03-04T15:30:07.220Z] Stage "Release" skipped due to earlier failure(s)
[2021-03-04T15:30:07.307Z] Stage "Release" skipped due to earlier failure(s)
[2021-03-04T15:30:07.698Z] Running on Jenkins in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-978
[2021-03-04T15:30:07.745Z] [INFO] getVaultSecret: Getting secrets
[2021-03-04T15:30:08.222Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-03-04T15:30:08.925Z] + chmod 755 generate-build-data.sh
[2021-03-04T15:30:08.925Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-978/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-978/runs/7 FAILURE 731236
[2021-03-04T15:30:09.176Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-978/runs/7/steps/?limit=10000 -o steps-info.json
[2021-03-04T15:30:15.657Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-978/runs/7/tests/?status=FAILED -o tests-errors.json
[2021-03-04T15:30:15.657Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2021-03-04T15:30:16.208Z] Retry 2/3 exited 22, retrying in 2 seconds...