Trace exceptions through Rack middleware

[ferdynton]

We're currently running the latest version of the gem in our production Rails 4.2.8 application and in all of our errors in DataDog's UI we see the same stacktrace:

/app/vendor/bundle/ruby/2.2.0/gems/ddtrace-0.6.1/lib/ddtrace/contrib/rails/action_controller.rb:19:in `block in instrument'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:127:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:127:in `finish'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:46:in `block in finish'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:46:in `each'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:46:in `finish'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:36:in `finish'

This is not the stacktrace of any of the errors thrown, but simply the callstack of the code that's reporting the error. The reason is because the code is using caller to populate the stacktrace and caller doesn't have much to do with exceptions. Even when we manually set the error from somewhere in code, that line overrides it with the same callstack, over and over.

Examining the code a little closer it looks like it's actually hard to get the right stacktrace from the ActiveSupport's instrumentation. Looking at how do other gems get this information (Honeybadger, NewRelic, Sentry), it looks to us like using a custom middleware layer is the way to go.

For us this also means we'll be able to trace errors thrown from our API layer, where we use Grape; Grape reuses the same Rack middleware stack, but does not go through ActiveController so we currently don't have a simple way of tracing those errors.

In order to make this change, we've changed some of the tests from being ActionController::TestCase to be ActionDispatch::IntegrationTest. The latter are said to be slower, but the former don't go through the middleware stack so the tests were failing.

Let us know your thoughts on this. Thanks!

[palazzem]

Ouch...we were working exactly on this task and this is definitely the way to go. I'm also working to add the Grape instrumentation so that it will be easier for you to auto instrument your code. Because we've already in place another PR that handles that (#111), I'll be sure to merge later this PR!

Thanks a lot for this work and for reporting this issue. It will be part of the next release!

[palazzem]

Hello @Ferdy89 ! Thank you for this PR. I've basically cherry picked your commits and did some changes, mostly related to tests and to make the Rack middleware available for generic usage. The followed approach is exactly that one with some minor changes. Here the PRs that are part of this improvement:

• [rack] add Rack middleware to trace generic Rack applications #111 [rack] use the Rack env instead of the Rack request object #116 (Rack as a generic middleware)
• [rails] Rails action_controller is part of the rack request #112 (this includes your contribution, thanks!)
• [rails] Rack middleware configuration is delayed #118 (minor updates upon deep testing)

I'm going to ship a new release now! Because you told me that you're using Grape, I've also merged this PR that adds Grape instrumentation #117! Feel free to leave any feedback because we're sure that all these changes have still room of improvement!

Thank you again!

[palazzem]

Hey @Ferdy89 saw your comments! Unfortunately all PRs have already been merged and released in the new 0.7.0, but I want to reassure you that we're keeping track of them!

Thanks again and feel free to submit proposals or anything that can improve this client library!