[reconfigurator] `clickhouse-admin` SMF service with dropshot server #6304

karencfv · 2024-08-13T04:19:24Z

Overview

New SMF service in clickhouse and clickhouse_keeper zones which runs a dropshot server. The API contains a single /node/address endpoint to retrieve the node's listen address. Other endpoints will be added in future PRs.

Purpose

This server will be used to manage ClickHouse server and Keeper nodes. For now it performs a single basic action to keep the size of this PR small, but this server will perform other actions like generating the XML config files, retrieving the state of the node etc.

Testing

I've deployed locally with the following results:

root@oxz_switch:~# curl http://[fd00:1122:3344:101::e]:8888/node/address
{"clickhouse_address":"[fd00:1122:3344:101::e]:8123"}

root@oxz_clickhouse_2c213ff2:~# cat /var/svc/log/oxide-clickhouse-admin:default.log
[ Aug 14 06:54:42 Enabled. ]
[ Aug 14 06:54:42 Rereading configuration. ]
[ Aug 14 06:54:45 Rereading configuration. ]
[ Aug 14 06:54:46 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/clickhouse-admin/bin/clickhouse-admin run -c /var/svc/manifest/site/clickhouse-admin/config.toml -a [fd00:1122:3344:101::e]:8123 -H [fd00:1122:3344:101::e]:8888 &"). ]
[ Aug 14 06:54:46 Method "start" exited with status 0. ]
note: configured to log to "/dev/stdout"
{"msg":"listening","v":0,"name":"clickhouse-admin","level":30,"time":"2024-08-14T06:54:46.721122327Z","hostname":"oxz_clickhouse_2c213ff2-6544-4316-939f-b51749cf3222","pid":5169,"local_addr":"[fd00:1122:3344:101::e]:8888","component":"dropshot","file":"/home/coatlicue/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:205"}
{"msg":"accepted connection","v":0,"name":"clickhouse-admin","level":30,"time":"2024-08-14T06:56:17.908877036Z","hostname":"oxz_clickhouse_2c213ff2-6544-4316-939f-b51749cf3222","pid":5169,"local_addr":"[fd00:1122:3344:101::e]:8888","component":"dropshot","file":"/home/coatlicue/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775","remote_addr":"[fd00:1122:3344:101::2]:37268"}
{"msg":"request completed","v":0,"name":"clickhouse-admin","level":30,"time":"2024-08-14T06:56:17.91734856Z","hostname":"oxz_clickhouse_2c213ff2-6544-4316-939f-b51749cf3222","pid":5169,"uri":"/node/address","method":"GET","req_id":"62a3d8fc-e37e-42aa-a715-52dbce8aa493","remote_addr":"[fd00:1122:3344:101::2]:37268","local_addr":"[fd00:1122:3344:101::e]:8888","component":"dropshot","file":"/home/coatlicue/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914","latency_us":3151,"response_code":"200"}

Related: #5999

…er zones

karencfv · 2024-08-14T07:17:33Z

clickhouse-admin/src/clickward.rs

+}
+
+#[derive(Debug)]
+pub struct Clickward {


I realise not all tasks will interact with Clickward's libraries (some sill use the clickhouse CLIs), but I like the name and can't think of anything better :)

andrewjstone

Looks great @karencfv! I'm super psyched to have this.

clickhouse-admin/Cargo.toml

andrewjstone · 2024-08-14T18:54:57Z

clickhouse-admin/src/bin/clickhouse-admin.rs

+    Run {
+        /// Socket address for a running clickhouse server or keeper instance
+        #[clap(long, short = 'a', action)]
+        clickhouse_address: SocketAddrV6,


I'm not sure we're actually going to need this. The config will all come from nexus such that the only thing starting up will be clickhouse-admin. We can't actually run the server without the rest of the config, so probably no need to pass this in.

However, I'm also fine keeping it for now as it's used to test the only end point. We can remove that and the address endpoint when the time comes.

Yeah, it's meant for testing for now. I've added a comment so it's clear that we should remove it later

andrewjstone · 2024-08-14T18:55:50Z

clickhouse-admin/src/clickward.rs

+        Self { clickhouse_address }
+    }
+
+    pub fn clickhouse_address(


We'll probably want to return the entire config struct instead, but this totally works for this PR.

andrewjstone · 2024-08-14T19:00:27Z

package-manifest.toml

@@ -153,6 +153,7 @@ source.type = "composite"
 source.packages = [
  "clickhouse_svc.tar.gz",
  "internal-dns-cli.tar.gz",
+  "omicron-clickhouse-admin.tar.gz",


I don't think we actually want to add it here. There's the new ClickhouseServer omicron zone where it should probably live, although we don't yet deploy that. Also fine to leave this for now and take it out later to help testing.

Same as above it was meant for testing, left a comment to specify this :)

karencfv

Thanks for the review @andrewjstone!

karencfv · 2024-08-14T21:52:45Z

clickhouse-admin/src/bin/clickhouse-admin.rs

+    Run {
+        /// Socket address for a running clickhouse server or keeper instance
+        #[clap(long, short = 'a', action)]
+        clickhouse_address: SocketAddrV6,


Yeah, it's meant for testing for now. I've added a comment so it's clear that we should remove it later

karencfv · 2024-08-14T21:53:35Z

package-manifest.toml

@@ -153,6 +153,7 @@ source.type = "composite"
 source.packages = [
  "clickhouse_svc.tar.gz",
  "internal-dns-cli.tar.gz",
+  "omicron-clickhouse-admin.tar.gz",


Same as above it was meant for testing, left a comment to specify this :)

karencfv added 9 commits August 12, 2024 20:11

skeleton for clickhouse admin

b3d31c7

Working dropshot server

45f8ae5

fix cmd

9e424e5

Generate OpenAPI json spec

ddb45a5

Create clickhouse-admin service and include it in clickhouse and keep…

f6d2994

…er zones

Add new files

cb8e98d

Endpoint to retrieve node listen address

a487b63

Clean up

f2ba87e

typo

5097bfb

karencfv changed the title ~~[reconfigurator] WIP clickhouse-admin dropshot server~~ [reconfigurator] clickhouse-admin dropshot server Aug 14, 2024

karencfv changed the title ~~[reconfigurator] clickhouse-admin dropshot server~~ [reconfigurator] clickhouse-admin SMF service with dropshot server Aug 14, 2024

karencfv marked this pull request as ready for review August 14, 2024 07:15

karencfv requested a review from andrewjstone August 14, 2024 07:15

karencfv commented Aug 14, 2024

View reviewed changes

andrewjstone approved these changes Aug 14, 2024

View reviewed changes

Address comments

eeab72c

karencfv commented Aug 14, 2024

View reviewed changes

karencfv enabled auto-merge (squash) August 14, 2024 21:54

karencfv merged commit 82d75b5 into oxidecomputer:main Aug 14, 2024
19 checks passed

karencfv deleted the clickhouse-admin branch August 14, 2024 23:42

karencfv mentioned this pull request Aug 15, 2024

Reconfigurator: Tracking issue for multi-node clickhouse deployment #5999

Open

20 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[reconfigurator] `clickhouse-admin` SMF service with dropshot server #6304

[reconfigurator] `clickhouse-admin` SMF service with dropshot server #6304

karencfv commented Aug 13, 2024 •

edited

Loading

karencfv Aug 14, 2024

andrewjstone left a comment

andrewjstone Aug 14, 2024

karencfv Aug 14, 2024

andrewjstone Aug 14, 2024

andrewjstone Aug 14, 2024

karencfv Aug 14, 2024

karencfv left a comment

karencfv Aug 14, 2024

karencfv Aug 14, 2024

[reconfigurator] clickhouse-admin SMF service with dropshot server #6304

[reconfigurator] clickhouse-admin SMF service with dropshot server #6304

Conversation

karencfv commented Aug 13, 2024 • edited Loading

Overview

Purpose

Testing

Choose a reason for hiding this comment

andrewjstone left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

karencfv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

[reconfigurator] `clickhouse-admin` SMF service with dropshot server #6304

[reconfigurator] `clickhouse-admin` SMF service with dropshot server #6304

karencfv commented Aug 13, 2024 •

edited

Loading