prometheus: 2.27.1 -> 2.30.3 #141477

eyJhb · 2021-10-13T09:50:46Z

Motivation for this change

Needs update, there are some syntax shown on ie. blackbox-exporter that does not work on the current version of prometheus (I think).

Things done

I have updated the version numbers + hashes.
However, they have changed yarn out with npm, which has caused some headache.
They have also added a module system, that is currently working OK with a hack that needs to be changed...

I have ran the prometheus tests and they pass without any issues.

TODOs

Cleanup code, ie. remove unused packages, buildInputs, etc.
Update webui update script

eyJhb · 2021-10-15T19:37:55Z

Copying my text from Matrix to @K900 to here, as it seems quite relevant. :)

if you can validate that it runs and works in a setup that you have used with it, it would be great. I have currently set it up, and it's running fine on my instance. However, there is a dirty hack in the current PR, but I don't really see any other way.

They have implemented a "module" system, that seems very hacky. But seeing as I am building the only module they currently have (codemirror-promql), then I don't think it makes sense, to actually run their build system.

You can see the hack here - https://github.com/NixOS/nixpkgs/pull/141477/files#diff-ca9e5cc95e3ef1d454502e3df0bd0c621434832f1f9feef52dac9a970a5c2100R83

Also, I think that some of the things in the patchPhase, should actually just be in the configure phase. Ie. all the symbolic links, should maybe just be in configure phase. And the one where I override the build modules script, should be in the patch phase.

TL;DR I am currently running this in my own setup, but would love anyone that could try it in their own current setup to validate it. I need some feedback on the hack I have made (check link above), and feedback on how to split up the patchPhase. I have already given a suggestion.

numkem · 2021-10-15T19:41:29Z

It works great for me.

One thing missing that I know is the option services.prometheus.scrapeConfigs.http_sd_configs it's been added since.

eyJhb · 2021-10-22T10:29:12Z

It works great for me.

One thing missing that I know is the option services.prometheus.scrapeConfigs.http_sd_configs it's been added since.

Think that should be added in a separate PR, if you have the time you could make one? :)

pkgs/servers/monitoring/prometheus/default.nix

pkgs/servers/monitoring/prometheus/webui/codemirror-promql/default.nix

pkgs/servers/monitoring/prometheus/webui/update-web-deps.sh

yrd · 2021-10-23T12:01:49Z

@eyJhb I started working on the new config changes, see #142654

Ma27

Are there new releases of exporters that are broken according to you (e.g. blackbox-exporter)? if yes, these should be updated in here as well.

Ma27 · 2021-10-23T18:17:50Z

pkgs/servers/monitoring/prometheus/default.nix

  };

  goPackagePath = "github.com/prometheus/prometheus";

-  webui = mkYarnPackage {
+  codemirrorNode = import ./webui/codemirror-promql {


Have you tried generating a yarn.lock on your own and use this? IIRC node2nix causes way more work than yarn2nix does in most cases, so I'd prefer to keep it.

If we were to go this route, shouldn't we create a yarn.lock from the upstream lockfile? Seems like it wouldn't be a good idea, to run different versions than upstream are

Seems to be possible with https://classic.yarnpkg.com/blog/2018/06/04/yarn-import-package-lock/

I don't think we should stick with yarn, but move to npm.
Looked at this thread here - prometheus/prometheus#9198 which also links this thanos-io/thanos#4562 and it seems to be a general move.

There are some dependencies yarn can't handle as well, that npm handles well. So we could end up having a different set of dependencies than upstream, if we continue to use yarn.

Agreed. If upstream is moving to npm we should do so as well.

eyJhb · 2021-10-23T19:13:07Z

Are there new releases of exporters that are broken according to you (e.g. blackbox-exporter)? if yes, these should be updated in here as well.

I am not sure, an unsure where to check it. I run the node-exporter and blackbox-exporter, those both work.

basvandijk · 2021-11-03T12:18:27Z

@GrahamcOfBorg test prometheus

basvandijk

LGTM!

basvandijk · 2021-11-04T16:29:20Z

@eyJhb I'm trying to upgrade to the latest prometheus-2.31.0 and I noticed they moved the web/ui package to npm workspaces. Any idea how to update your pkgs/servers/monitoring/prometheus/webui/update-web-deps.sh to this new way?

eyJhb · 2021-11-04T17:00:01Z

@eyJhb I'm trying to upgrade to the latest prometheus-2.31.0 and I noticed they moved the web/ui package to npm workspaces. Any idea how to update your pkgs/servers/monitoring/prometheus/webui/update-web-deps.sh to this new way?

It just seems to be sugar on top of npm, and nothing more. Currently it just builds the two components independently, and then fixes the paths.

Should that be any issue? The webui works fine when I tested it :) - Are you available on Matrix for discussing any miscommunication? :)

basvandijk · 2021-11-05T09:12:37Z

It just seems to be sugar on top of npm, and nothing more. Currently it just builds the two components independently, and then fixes the paths.

The issue is that they now have a single package-lock.json file in web/ui instead of two separate ones in web/ui/module/codemirror-promql and web/ui/react-app/.

Should that be any issue? The webui works fine when I tested it :)

At the moment the package doesn't build. But maybe I can try just using that single package-lock.json for the two packages.

Are you available on Matrix for discussing any miscommunication? :)

I'm not and TIL about Matrix. I'll look into signing up. Where can I do that?

basvandijk · 2021-11-07T14:02:46Z

It seems this is ready to merge. We can do the 2.31.0 upgrade in another PR. Merging in now...

eyJhb · 2021-11-08T16:21:39Z

It seems this is ready to merge. We can do the 2.31.0 upgrade in another PR. Merging in now...

Oh, didn't notice a new version had come out. Sorry!
Hopefully it can be fixed soon. Seems annoying that Prometheus cannot settle on anything, and keeps changing it around.
But what can one do ... :)

Thanks for merging @basvandijk ! :)

ofborg bot requested review from Frostman, benley, globin, fpletz, kalbasit and WilliButz October 13, 2021 10:02

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 10.rebuild-linux: 1 labels Oct 13, 2021

eyJhb force-pushed the prometheus-2.3.3 branch 2 times, most recently from a1642ae to f8356c0 Compare October 13, 2021 18:28

K900 mentioned this pull request Oct 15, 2021

prometheus: 2.30.x #139619

Closed

2 tasks

eyJhb changed the title ~~WIP: prometheus: 2.27.1 -> 2.30.3~~ prometheus: 2.27.1 -> 2.30.3 Oct 18, 2021

dasJ requested a review from Ma27 October 22, 2021 10:36

SuperSandro2000 reviewed Oct 22, 2021

View reviewed changes

pkgs/servers/monitoring/prometheus/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Oct 22, 2021

View reviewed changes

pkgs/servers/monitoring/prometheus/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Oct 22, 2021

View reviewed changes

pkgs/servers/monitoring/prometheus/webui/codemirror-promql/default.nix Show resolved Hide resolved

SuperSandro2000 reviewed Oct 22, 2021

View reviewed changes

prometheus: 2.27.1 -> 2.30.3

c319107

eyJhb force-pushed the prometheus-2.3.3 branch from 560843d to c319107 Compare October 22, 2021 15:25

yrd mentioned this pull request Oct 23, 2021

nixos/prometheus: add new configuration options #142654

Merged

12 tasks

Ma27 reviewed Oct 23, 2021

View reviewed changes

basvandijk approved these changes Nov 3, 2021

View reviewed changes

basvandijk mentioned this pull request Nov 5, 2021

Add support for Yarn workspaces (PoC for #189) svanderburg/node2nix#200

Draft

basvandijk merged commit 3e019f7 into NixOS:master Nov 7, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

prometheus: 2.27.1 -> 2.30.3 #141477

prometheus: 2.27.1 -> 2.30.3 #141477

eyJhb commented Oct 13, 2021 •

edited

Loading

eyJhb commented Oct 15, 2021

numkem commented Oct 15, 2021

eyJhb commented Oct 22, 2021

yrd commented Oct 23, 2021

Ma27 left a comment

Ma27 Oct 23, 2021

eyJhb Oct 23, 2021

Ma27 Oct 23, 2021

eyJhb Nov 3, 2021

basvandijk Nov 3, 2021

eyJhb commented Oct 23, 2021

basvandijk commented Nov 3, 2021

basvandijk left a comment

basvandijk commented Nov 4, 2021 •

edited

Loading

eyJhb commented Nov 4, 2021

basvandijk commented Nov 5, 2021

basvandijk commented Nov 7, 2021

eyJhb commented Nov 8, 2021

prometheus: 2.27.1 -> 2.30.3 #141477

prometheus: 2.27.1 -> 2.30.3 #141477

Conversation

eyJhb commented Oct 13, 2021 • edited Loading

Motivation for this change

Things done

eyJhb commented Oct 15, 2021

numkem commented Oct 15, 2021

eyJhb commented Oct 22, 2021

yrd commented Oct 23, 2021

Ma27 left a comment

Choose a reason for hiding this comment

Ma27 Oct 23, 2021

Choose a reason for hiding this comment

eyJhb Oct 23, 2021

Choose a reason for hiding this comment

Ma27 Oct 23, 2021

Choose a reason for hiding this comment

eyJhb Nov 3, 2021

Choose a reason for hiding this comment

basvandijk Nov 3, 2021

Choose a reason for hiding this comment

eyJhb commented Oct 23, 2021

basvandijk commented Nov 3, 2021

basvandijk left a comment

Choose a reason for hiding this comment

basvandijk commented Nov 4, 2021 • edited Loading

eyJhb commented Nov 4, 2021

basvandijk commented Nov 5, 2021

basvandijk commented Nov 7, 2021

eyJhb commented Nov 8, 2021

eyJhb commented Oct 13, 2021 •

edited

Loading

basvandijk commented Nov 4, 2021 •

edited

Loading