Add pipes & hooks wildcard event #1305

thomasarbona · 2019-05-22T15:03:00Z

What does this PR do ?

pipes & hooks can now trigger wildcarded events like this:
'foo:*'
'foo:after*'
'foo:before*'

How should this be manually tested?

Step 1 : create a plugin and declare some wildcarded events handler
Step 2 : trigger events

codecov · 2019-05-22T15:08:18Z

Codecov Report

Merging #1305 into 1-dev will increase coverage by <.01%.
The diff coverage is 95.45%.

@@            Coverage Diff             @@
##            1-dev    #1305      +/-   ##
==========================================
+ Coverage   93.84%   93.85%   +<.01%     
==========================================
  Files          98       98              
  Lines        6884     6895      +11     
==========================================
+ Hits         6460     6471      +11     
  Misses        424      424

Impacted Files	Coverage Δ
lib/api/kuzzle.js	`83.78% <100%> (ø)`	⬆️
lib/api/core/plugins/pluginsManager.js	`87.76% <95%> (+0.41%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db14d78...97b50d0. Read the comment docs.

scottinet · 2019-05-23T07:24:19Z

lib/api/core/plugins/pluginsManager.js

@@ -831,8 +831,15 @@ class PluginsManager {
 * @returns {Promise}
 */
 function triggerHooks(emitter, event, data) {
+  const wildcardEvents = getWildcardEvents(event)
+    // filter events like 'foo:*' because EventEmitter2 


I think it would be simpler (and better) to completely get rid of EventEmitter2, since we're only using it for wildcards (this should be a light change since EventEmitter2 has the same interface than the native EventEmitter).

And instead manage wildcards ourselves entirely, because that kind of exception handling is a bit costly performances wide, and events triggering is a critical part of Kuzzle.

Also, this code is unsafe because getWildcardEvents can return null.

lib/api/core/plugins/pluginsManager.js

scottinet · 2019-05-23T07:33:59Z

lib/api/core/plugins/pluginsManager.js

  }

-  return null;
+  const wildcardEvents = ['*:*', `${scope}:*`];


I disagree with the event *:* because:

we can have an arbitrary number of separators (for instance we have a core:auth:strategyAdded event)

I think that * would be much simpler

Suggested change

const wildcardEvents = ['*:*', `${scope}:*`];

const wildcardEvents = ['*', `${scope}:*`];

As we discuss with @thomasarbona, it could be nice to handle wildcard pattern directly as regex. So we do not need to define rules for patterns and use regex syntax.

Veto: regexp are slow, and the number and frequency of events triggered make them a critical part of the code.

Why do we need regular expressions?

You're right I didn't get that this action appends at funnel level. Ignore my previous comment 😅

scottinet · 2019-05-23T07:36:51Z

lib/api/core/plugins/pluginsManager.js

-  if (indexDelimiter !== 1) {
-    return event.substring(0, indexDelimiter+1) + '*';
+function getWildcardEvents (event) {
+  const [scope, name] = event.split(':');


This leaves events with 3 or more separator out of the picture.

Ugly but I think it can help you to figure out how to handle more than 3 separators:

Suggested change

const [scope, name] = event.split(':');

const rawEvents = event.split(':');

const [name] = rawEvents.pop();

const [scope] = rawEvents;

Aschen · 2019-05-23T13:26:12Z

lib/api/core/plugins/pluginsManager.js

+
+  if (preparedHooks) {
+    preparedHooks.forEach(hook => {
+      hook(data);


I'm not sure but what if the hook function provided by the developer throw an error ? Can't we have a unhandled rejection here ?

more importantly: this bypasses Kuzzle as an event emitter, which is not the purpose of hooks

scottinet · 2019-05-23T13:56:18Z

lib/api/core/plugins/pluginsManager.js

+    if (preparedHooks) {
+      preparedHooks.forEach(hook => {
+        try {
+          hook(data);


this is still broken: plugins aren't the only objects using events

why don't you let the kuzzle object, which inherits from eventemitter, emit that event?

scottinet · 2019-05-23T16:09:20Z

lib/api/core/plugins/pluginsManager.js

+  const delimIndex = event.lastIndexOf(':');
+
+  if (delimIndex === -1) {
+    return null;


to prevent breaking code using the return value of this function without first checking that it didn't return null:

Suggested change

return null;

return [];

scottinet · 2019-05-23T16:13:57Z

test/api/core/pluginsManager/trigger.test.js

+      }
+    }];
+
+    pluginsManager.run()


that promise must be returned, otherwise the test isn't awaited and it always passes

scottinet · 2019-05-23T16:14:03Z

test/api/core/pluginsManager/trigger.test.js

+      }
+    }];
+
+    pluginsManager.run()


scottinet · 2019-05-23T16:14:09Z

test/api/core/pluginsManager/trigger.test.js

+      }
+    }];
+
+    pluginsManager.run()


kuzzle · 2019-05-27T15:03:14Z

SonarQube analysis reported 1 issue

Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

pluginsManager.js#L726: Complete the task associated to this TODO comment.

@xbill82

- add documentation for wildcard events (kuzzleio/kuzzle#1305) ### How should this be manually tested? - `npm install && npm run dev` - visit http://localhost:8080/core/1/plugins/plugins/events/wildcard-events/ ### Boyscout - [ @xbill82 ] Disabled C++ and Java tests from local execution - fix pages sorting (page with an order set to 0 isn't classified first)

Release 1.8.0 Bug fixes [ #1311 ] Fix promise leaks (scottinet) [ #1298 ] Fix disabled protocol initialization (Aschen) [ #1297 ] Fix timeouts on plugin action returing the request (benoitvidis) [ #1288 ] Fix an error when trying a non-partial bulk update (scottinet) [ #1286 ] Allows bulk inserts on aliases (benoitvidis) [ #1282 ] Scan keys on redis cluster (benoitvidis) [ #1279 ] Users must be authenticated to use auth:logout (scottinet) New features [ #1315 ] Add the new Vault module to handle encrypted application secrets (Aschen) [ #1302 ] Add write and mWrite (Aschen) [ #1305 ] Add pipes & hooks wildcard event (thomasarbona) Enhancements [ #1318 ] Add a maximum ttl to auth:login (benoitvidis) [ #1301 ] Upgrade the WebSocket libraries (scottinet) [ #1308 ] Events triggering refactor (scottinet) [ #1300 ] Collection specifications methods cloisoned to a collection (thomasarbona) [ #1295 ] Improve validation error messages (benoitvidis) [ #1292 ] Throw an error when the realtime controller is invoked by plugin developers (benoitvidis) [ #1257 ] Add ability to define mapping policy for new fields (Aschen) [ #1291 ] Fix --help on subcommands (Yoann-Abbes) [ #1289 ] Handle ping/pong packets (scottinet) [ #1273 ] Fix incomplete access logs (scottinet) Others [ #1317 ] Add ps dependency to plugin-dev Docker image for pm2 (benoitvidis) [ #1312 ] Check that .kuzzlerc.sample is well-formed (scottinet) [ #1299 ] Add Kuzzle Nightly & Redis 3 and 4 test (alexandrebouthinon)

hooks & pipes: use wildcard events

59f13ad

thomasarbona added the changelog:new-features Increase the minor version number label May 22, 2019

thomasarbona self-assigned this May 22, 2019

replace regex with vanilla js

bf7c5bd

scottinet suggested changes May 23, 2019

View reviewed changes

Aschen reviewed May 23, 2019

View reviewed changes

thomasarbona requested a review from scottinet May 23, 2019 13:32

scottinet suggested changes May 23, 2019

View reviewed changes

replace EventEmitter2 with EventEmitter3

3f9331f

thomasarbona force-pushed the KZL-291-wildcarded-event branch from 1234067 to 3f9331f Compare May 23, 2019 16:04

thomasarbona requested a review from scottinet May 23, 2019 16:05

Merge branch '1-dev' into KZL-291-wildcarded-event

9cb5393

scottinet suggested changes May 23, 2019

View reviewed changes

suggestions & fix tests

488a774

scottinet approved these changes May 24, 2019

View reviewed changes

alexandrebouthinon approved these changes May 24, 2019

View reviewed changes

scottinet added a commit that referenced this pull request May 24, 2019

[rebase] take #1305 as a workbase

2501ab7

scottinet mentioned this pull request May 24, 2019

Events triggering refactor #1308

Merged

thomasarbona added 2 commits May 24, 2019 15:21

move the declaration of getWildcardEvents

0ab6275

add 'error' to templated wildcards

97b50d0

Aschen approved these changes May 27, 2019

View reviewed changes

Aschen merged commit 3d06476 into 1-dev May 27, 2019

Aschen deleted the KZL-291-wildcarded-event branch May 27, 2019 15:31

thomasarbona mentioned this pull request May 27, 2019

Documentation for wildcard events kuzzleio/documentation#299

Merged

Aschen changed the title ~~[KZL-291] pipes & hooks: wildcarded event~~ Add pipes & hooks wildcarded event May 28, 2019

Aschen changed the title ~~Add pipes & hooks wildcarded event~~ Add pipes & hooks wildcard event May 28, 2019

scottinet added a commit that referenced this pull request May 28, 2019

[rebase] take #1305 as a workbase

6593e9e

scottinet restored the KZL-291-wildcarded-event branch May 28, 2019 15:01

scottinet deleted the KZL-291-wildcarded-event branch May 28, 2019 15:09

Aschen mentioned this pull request Jun 14, 2019

Release 1.8.0 #1322

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add pipes & hooks wildcard event #1305

Add pipes & hooks wildcard event #1305

thomasarbona commented May 22, 2019 •

edited

Loading

codecov bot commented May 22, 2019 •

edited

Loading

scottinet May 23, 2019

scottinet May 23, 2019

scottinet May 23, 2019

alexandrebouthinon May 23, 2019

scottinet May 23, 2019 •

edited

Loading

alexandrebouthinon May 23, 2019

scottinet May 23, 2019

alexandrebouthinon May 23, 2019 •

edited

Loading

Aschen May 23, 2019

scottinet May 23, 2019

scottinet May 23, 2019

scottinet May 23, 2019

scottinet May 23, 2019

scottinet May 23, 2019

scottinet May 23, 2019

kuzzle commented May 27, 2019

	const wildcardEvents = [':', `${scope}:*`];
	const wildcardEvents = ['', `${scope}:`];

-  const [scope, name] = event.split(':');
+  const rawEvents = event.split(':');
+  const [name] = rawEvents.pop();
+  const [scope] = rawEvents;

Add pipes & hooks wildcard event #1305

Add pipes & hooks wildcard event #1305

Conversation

thomasarbona commented May 22, 2019 • edited Loading

What does this PR do ?

How should this be manually tested?

codecov bot commented May 22, 2019 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

scottinet May 23, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexandrebouthinon May 23, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kuzzle commented May 27, 2019

thomasarbona commented May 22, 2019 •

edited

Loading

codecov bot commented May 22, 2019 •

edited

Loading

scottinet May 23, 2019 •

edited

Loading

alexandrebouthinon May 23, 2019 •

edited

Loading