Show and modify routing rules from the UI #433

prakhar10 · 2024-08-09T01:50:26Z

Description

Adding a way to display routing rules and also modify them from the UI
Minor change to show a user icon with a gear if user is ADMIN or a regular user icon if its a regular USER on the top right corner of the UI

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
(X) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

gateway-ha/src/main/java/io/trino/gateway/ha/resource/GatewayWebAppResource.java

webapp/src/components/routing-rules.tsx

gateway-ha/src/main/java/io/trino/gateway/ha/resource/GatewayWebAppResource.java

willmostly

This looks like a nice improvement, thank you! I reviewed the Java half, and will try to find someone for the typescript.

My biggest concern here is not with the code, but with allowing users to supply input that will be written to disk on the server, then executed by the jeasy rules engine. We should take a close look at jeasy and mvel to understand what security vulnerabilities we might be introducing. At minimum we should log all of the updates so that they can be audited. @wendigo and @ebyhr wdyt?

gateway-ha/src/main/java/io/trino/gateway/ha/domain/RoutingRules.java

gateway-ha/src/main/java/io/trino/gateway/ha/resource/GatewayWebAppResource.java

willmostly · 2024-08-16T01:23:20Z

gateway-ha/src/main/java/io/trino/gateway/ha/resource/GatewayWebAppResource.java

+            for (int i = 0; i < routingRulesList.size(); i++) {
+                if (routingRulesList.get(i).name().equals(routingRules.name())) {
+                    routingRulesList.set(i, routingRules);
+                    break;
+                }
+            }


as the name of this method suggests, this updates an existing rule. It does not appear that this change adds support for deleting an existing rule or adding a new one. I guess this may be due to the UI changes required, but would it be difficult to add this functionality?

I don't think it would be difficult. I thought of initially going with only updates and in a separate PR i will put Add and Delete functionality. Will that be ok? Or should I add it in this PR itself?

that plan sounds fine to me, thanks!

prakhar10 · 2024-08-16T19:15:12Z

This looks like a nice improvement, thank you! I reviewed the Java half, and will try to find someone for the typescript.

My biggest concern here is not with the code, but with allowing users to supply input that will be written to disk on the server, then executed by the jeasy rules engine. We should take a close look at jeasy and mvel to understand what security vulnerabilities we might be introducing. At minimum we should log all of the updates so that they can be audited. @wendigo and @ebyhr wdyt?

This functionality will be accessible only for users with ADMIN privileges. Regular users will have this as Read-Only. I also have a functionality implemented in our internal repo where we are auditing changes being made to the Clusters from the UI. I can raise a PR for that as well and will add functionality to audit routing rules as well. I was thinking of getting this reviewed first and then from comments and suggestions I will raise the Audit logs PR.

gateway-ha/src/main/java/io/trino/gateway/ha/resource/GatewayWebAppResource.java

willmostly

this is looking good, we're almost there. @mosabua and I discussed the need to support this for clustered deployments. We should also provide a way to disable this feature in case someone doesn't want to bother setting up a shared filesystem

docs/gateway-api.md

willmostly · 2024-08-30T19:01:43Z

@prakhar10 please squash these commits

prakhar10 · 2024-08-31T04:53:57Z

this is looking good, we're almost there. @mosabua and I discussed the need to support this for clustered deployments. We should also provide a way to disable this feature in case someone doesn't want to bother setting up a shared filesystem

So this means provide an option or a config to show/disable the Routing Rules option in the left panel of the UI right?

mosabua · 2024-09-03T17:36:56Z

this is looking good, we're almost there. @mosabua and I discussed the need to support this for clustered deployments. We should also provide a way to disable this feature in case someone doesn't want to bother setting up a shared filesystem

So this means provide an option or a config to show/disable the Routing Rules option in the left panel of the UI right?

I think for now we can just say in the docs that you must provide shared storage for that feature to work and in terms of disabling it could be just a config option option .. I think over time we will need various config options for the UI anyway

Remove console logs and renamed api Add api documentation and changes related to PR comments

prakhar10 · 2024-09-24T03:38:08Z

@mosabua @willmostly can you please review?

gateway-ha/src/main/java/io/trino/gateway/ha/domain/RoutingRules.java

gateway-ha/src/main/java/io/trino/gateway/ha/resource/GatewayWebAppResource.java

mosabua · 2024-10-03T23:02:25Z

I assume that @prakhar10 and @willmostly are collaborating on this currently .. ping me if review or anything else is needed.

prakhar10 · 2024-10-04T02:41:08Z

I need to make changes according to @ebyhr 's comments. So i will get that done by this weekend.

gateway-ha/src/main/java/io/trino/gateway/ha/domain/RoutingRules.java

gateway-ha/src/main/java/io/trino/gateway/ha/config/HaGatewayConfiguration.java

gateway-ha/src/main/java/io/trino/gateway/ha/domain/RoutingRules.java

gateway-ha/src/main/java/io/trino/gateway/ha/router/RoutingRulesManager.java

prakhar10 · 2024-10-21T19:55:55Z

@ebyhr can you review now please?

prakhar10 · 2024-10-26T17:24:30Z

@mosabua can you please review?

gateway-ha/src/main/java/io/trino/gateway/ha/domain/RoutingRules.java

gateway-ha/src/main/java/io/trino/gateway/ha/resource/GatewayWebAppResource.java

ebyhr · 2024-10-27T23:42:33Z

gateway-ha/src/main/java/io/trino/gateway/ha/router/RoutingRulesManager.java

+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+public final class RoutingRulesManager


I'm not sure why this class is a static helper class. Could you consider changing to a service class?

So I was trying to implement these comments and then I got an error saying HideUtilityClassConstructor: Utility classes should not have a public or default constructor. . Maybe I misunderstood what you meant?

@ebyhr any inputs?

gateway-ha/src/test/java/io/trino/gateway/ha/router/TestRoutingRulesManager.java

docs/gateway-api.md

gateway-ha/src/main/java/io/trino/gateway/ha/config/UIConfiguration.java

willmostly · 2024-10-29T21:23:14Z

gateway-ha/src/main/java/io/trino/gateway/ha/router/RoutingRulesManager.java

+            List<RoutingRules> routingRulesList = getRoutingRules(configuration, yamlReader);
+            for (int i = 0; i < routingRulesList.size(); i++) {
+                if (routingRulesList.get(i).name().equals(routingRules.name())) {
+                    routingRulesList.set(i, routingRules);
+                    break;
+                }
+            }
+            ObjectMapper yamlWriter = new ObjectMapper(new YAMLFactory());
+            StringBuilder yamlContent = new StringBuilder();
+            for (RoutingRules rule : routingRulesList) {
+                yamlContent.append(yamlWriter.writeValueAsString(rule));
+                routingRulesBuilder.add(rule);
+            }
+            Files.writeString(Paths.get(rulesConfigPath), yamlContent.toString(), UTF_8);


multiple writers could overwrite one another here. You will need to use filesystem locking since there could be multiple trino gateway instances simultaneously attempting an update.

Makes sense, i'll check on this

This method is synchronized, so do we still need file system locking @willmostly ?

https://github.com/trinodb/trino-gateway/pull/433/files#diff-5513a204debc3305e52fb4a481a79b027033e8fc8758bc438674e328cab4e300R461

gateway-ha/src/main/java/io/trino/gateway/ha/domain/RoutingRules.java

gateway-ha/src/main/java/io/trino/gateway/ha/router/RoutingRulesManager.java

cla-bot bot added the cla-signed label Aug 9, 2024

Chaho12 reviewed Aug 12, 2024

View reviewed changes

willmostly requested changes Aug 16, 2024

View reviewed changes

willmostly requested changes Aug 20, 2024

View reviewed changes

gateway-ha/src/main/java/io/trino/gateway/ha/resource/GatewayWebAppResource.java Outdated Show resolved Hide resolved

willmostly requested changes Aug 30, 2024

View reviewed changes

docs/gateway-api.md Show resolved Hide resolved

Show and modify routing rules from the UI

d3c7855

Remove console logs and renamed api Add api documentation and changes related to PR comments

prakhar10 force-pushed the routing-rules-ui branch from 85287b9 to d3c7855 Compare September 9, 2024 00:21

Add option to disable pages and update API docs

ba2e92f

prakhar10 requested a review from willmostly September 24, 2024 03:43

ebyhr reviewed Sep 25, 2024

View reviewed changes

ebyhr assigned prakhar10 Oct 1, 2024

prakhar10 added 2 commits October 7, 2024 21:43

Move functions to service class and add tests and other fixes

034963d

Fix build and remove json property

38023fc

ebyhr reviewed Oct 9, 2024

View reviewed changes

prakhar10 added 3 commits October 16, 2024 16:04

Fix PR comments

546b85f

Remove unused import

7cf593d

Fix CI Build

e3ea3cd

ebyhr requested changes Oct 27, 2024

View reviewed changes

willmostly requested changes Oct 29, 2024

View reviewed changes

willmostly requested changes Oct 31, 2024

View reviewed changes

gateway-ha/src/main/java/io/trino/gateway/ha/domain/RoutingRules.java Outdated Show resolved Hide resolved

willmostly reviewed Oct 31, 2024

View reviewed changes

gateway-ha/src/main/java/io/trino/gateway/ha/router/RoutingRulesManager.java Outdated Show resolved Hide resolved

Add some PR comment fixes

3c98006

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Show and modify routing rules from the UI #433

Show and modify routing rules from the UI #433

prakhar10 commented Aug 9, 2024

willmostly left a comment

willmostly Aug 16, 2024

prakhar10 Aug 16, 2024

willmostly Aug 20, 2024

prakhar10 commented Aug 16, 2024 •

edited

Loading

willmostly left a comment

willmostly commented Aug 30, 2024

prakhar10 commented Aug 31, 2024

mosabua commented Sep 3, 2024

prakhar10 commented Sep 24, 2024

mosabua commented Oct 3, 2024

prakhar10 commented Oct 4, 2024

prakhar10 commented Oct 21, 2024

prakhar10 commented Oct 26, 2024

ebyhr Oct 27, 2024

prakhar10 Oct 29, 2024 •

edited

Loading

prakhar10 Nov 2, 2024

willmostly Oct 29, 2024

prakhar10 Oct 29, 2024

prakhar10 Nov 12, 2024


		import static java.nio.charset.StandardCharsets.UTF_8;

		public final class RoutingRulesManager

Show and modify routing rules from the UI #433

Are you sure you want to change the base?

Show and modify routing rules from the UI #433

Conversation

prakhar10 commented Aug 9, 2024

Description

Additional context and related issues

Release notes

willmostly left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prakhar10 commented Aug 16, 2024 • edited Loading

willmostly left a comment

Choose a reason for hiding this comment

willmostly commented Aug 30, 2024

prakhar10 commented Aug 31, 2024

mosabua commented Sep 3, 2024

prakhar10 commented Sep 24, 2024

mosabua commented Oct 3, 2024

prakhar10 commented Oct 4, 2024

prakhar10 commented Oct 21, 2024

prakhar10 commented Oct 26, 2024

Choose a reason for hiding this comment

prakhar10 Oct 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prakhar10 commented Aug 16, 2024 •

edited

Loading

prakhar10 Oct 29, 2024 •

edited

Loading