-
Notifications
You must be signed in to change notification settings - Fork 7
Add statsd metrics #378
Comments
note-to-self: not done yet. |
Let's keep this open till it's been tested in stage |
datadog agent is running on the admin instance. I think you might need to cut a release and have it deployed to -stage for testing. Currently -stage and -prod are running the latest version, which is v1.1.5. |
Ok. So once it's upgraded it'll work. I'll start on making a release. By the way, @bqbn What does it take to make stage auto-deploy? I guess that's two questions in one:
|
This is technically doable, but the convention is not to do that. We only install tagged version to -stage, not master. Do you have a reason for which you must install "master" onto -stage?
Right now the only way to trigger a Jenkins pipeline from outside of Jenkins is via Dockerhub, as documented in https://github.com/mozilla-services/cloudops-deployment#code-flow Basically we'd have to build a docker image and push that to docker hub, which then calls the jenkins-proxy to trigger a jenkins pipeline. And then the pipeline will need to download the image from docker hub, copy the lambda zip file out from the docker image, then deploy it. Also note that, the lambda still needs to be tagged with a version, so that we can deploy it to -stage. |
Ok. I think I spoke too soon. Stage should probably continue to be by tag and dockerhub. What would be ideal is that stage is automatically upgraded (without any ops/human intervention) when a new tag is pushed (and circle builds based on this successfully).
|
This is already the case.
I'm under the impression that you don't need to make it locally, rather, when you release a version in github, github provides you with an option to generate a zip file, you just need to click the button. You don't need to upload it to Bugzilla, rather, you file a ticket to request deploying it to -stage with a version number, and our pipeline downloads it from github automatically. One example location our pipeline comes to download is
If Tecken is a lambda, I'm happy to look at some examples of it and see if it fits in our current setup. Please share some links to it. |
So, I've made a release. https://github.com/mozilla-services/buildhub/releases/tag/v1.2.0 I wrote this script which does:
So, if I understand this, this'll upgrade Stage (and Stage's Lambda). Right? |
Yes for the cron job. As a matter of fact, I see that our pipeline was triggered and the v1.2.0 version of cron was deployed to -stage. No for the lambda. As mentioned in #378 (comment), unless the lambda is built into a docker image by circleci, there is no way to trigger our pipeline to deploy it. So currently to deploy the lambda, the process is: file a ticket in bugzilla with the desired version number to have it deploy to -stage, then another ticket to deploy it to -prod. |
Ah! Got it. I misunderstood the distinction between the cron job thing and the lambda. I'll amend my in-progress script to make the bugzilla filing easier. |
@bqbn The default config is that it sends statsd pings to |
The problem is that I can't create a Buildhub Performance dashboard and find any of my keys. They should look something like This is the dashboard I started: https://app.datadoghq.com/dash/794559/buildhub-performance?live=true&page=0&is_auto=false&from_ts=1524596723306&to_ts=1524600323306&tile_size=m |
We have statsd metrics for Buildhub now. Have had for a long time. |
A lot is going on in the jobs code (Lambda handler and inventory-to-records) and we have very limited insight. Let's add metrics of counters and timers to see what's going and what's taking time.
To accomplish this we also need to file a bugzilla bug to Cloud Services to add the Datadog daemon on the server that runs the cron job.
The text was updated successfully, but these errors were encountered: