filtering/reconnecting #18

brauliorivas · 2024-05-14T00:08:49Z

BEGINRELEASENOTES

Filtering/reconnecting particles is a critical task for dmX. So, in this PR, I'm implementing a basic filtering/reconnect function that given a function that determines if a particle should be filtered out, reconnects the particles again.

ENDRELEASENOTES

tmadlener · 2024-05-14T09:36:40Z

js/main.js

@@ -50,7 +50,7 @@ const mouseUp = function (event) {
  isDragging = false;

  // console.time("drawAll");
-  drawAll();
+  drawAll(parentLinks, childrenLinks, infoBoxes);


Stupid question from my side, where do these inputs come from?

Don't worry, these inputs come from main.js.

brauliorivas · 2024-05-14T22:32:44Z

Hey @kjvbrt and @tmadlener, could you check my filtering function? As previously discussed, an external function determines whether to filter or not a particle. And then reconnected by using the reconnect function in js/menu/filter.js. I have added a toggle for simplicity but it's temporary.

tmadlener · 2024-05-15T14:07:57Z

I am rebasing this to potentially pick up the preview functionality.

tmadlener · 2024-05-16T07:37:08Z

js/menu/filter.js

+  }
+}
+
+function reconnect(criteriaFunction, parentLinks, childrenLinks, particles) {


I am not entirely sure whether this is necessary yet, but just for the future; Can this be written in a way where there are no parentLinks? We will probably need that e.g. for the ReconstructedParticle and I am wondering whether we should already spend the time now to prepare that?

brauliorivas · 2024-05-22T01:54:22Z

Hey @tmadlener @kjvbrt, I started working on more advanced filtering for MonteCarlo Particles (as task for week 1 of the proposal). It's a bit buggy, but I want to share how is it going. Here is a small demo.

tmadlener · 2024-05-22T09:10:17Z

Very nice. Since you have the panel now already. Can I ask for one specific filter function which will should be straight forward to implement (from a technical point of view), but will probably require a bit of "leg work" from your side.

The edm4hep::MCParticle has two bitfields: generatorStatus and simulatorStatus. For the simulatorStatus we provide a few bit values that are reserved for a specific meaning:

https://github.com/key4hep/EDM4hep/blob/997ab32b886899253c9bc61adea9a21b57bc5a21/edm4hep.yaml#L294-L301

And then there is also some convenience functionality to check wheter a specifc bit is set:

https://github.com/key4hep/EDM4hep/blob/997ab32b886899253c9bc61adea9a21b57bc5a21/edm4hep.yaml#L306-L321

Would it be possible to have a filter that allows us to toggle these to filter out MCParticles for which specific bits are set? I.e. Have one checkbox / toggle for each bit and combine them into a filter function? IIUC all the filter combiners, etc. are already in place and this would mainly boil down to writing a specific filter for checking the bits in the simulatorStatus bit field.

brauliorivas · 2024-05-22T21:43:04Z

I have it now. In theory it should work, but the particles from the json example don't seem to have a simulator status from the ones described, so it's a bit difficult to test. Here's an image of the filters now.

tmadlener · 2024-05-23T07:48:18Z

Rebasing this again to see if the preview fixes from #22 are picked up.

github-actions · 2024-05-23T07:48:42Z

PR Preview Action v1.4.7
Preview removed because the pull request was closed.
2024-05-31 14:15 UTC

tmadlener · 2024-05-23T07:57:29Z

Preview is working 🚀 However, for me it doesn't show the filter panel. ~~Do I have to enable them somehow?~~ It looks like the preview doesn't publish the status of the PR, but rather the one from the PR. I am not sure why, though.

For testing the filtering, it would be nice to have some unittests for that to see if things work as expected. In these unittests you can just create some dummy MCParticle hierarchy with the values that you want in the end. You could start from the existing json file and just use a few of them and just adapt them to enable testing. This should all be possible without ever having to go to the UI to display them (for testing).

…t-filtering

brauliorivas · 2024-05-23T15:18:07Z

Preview is working 🚀 However, for me it doesn't show the filter panel. ~~Do I have to enable them somehow?~~ It looks like the preview doesn't publish the status of the PR, but rather the one from the PR. I am not sure why, though.

Yeah, it's strange. I didn't expect this. I'll fix it.

For testing the filtering, it would be nice to have some unittests for that to see if things work as expected. In these unittests you can just create some dummy MCParticle hierarchy with the values that you want in the end. You could start from the existing json file and just use a few of them and just adapt them to enable testing. This should all be possible without ever having to go to the UI to display them (for testing).

Sure! In that case, I will wait for this PR and #20 to be merged, to add testing, because the testing setup is in #20 and this #18 has the function to be tested. So I think that after merging both PRs to main, I should create a separate PR to test filtering, and other PR to fix again PR previews, what do you think?

tmadlener · 2024-05-31T12:37:25Z

I am quite happy with this. I think we can merge this, unless @kjvbrt has more comments.

kjvbrt · 2024-05-31T13:30:56Z

js/menu/filter/reconnect.js

@@ -0,0 +1,70 @@
+import { Link } from "../../objects";


import { Link } from "../../objects"; ->
import { Link } from "../../objects.js";

kjvbrt · 2024-05-31T13:33:27Z

Otherwise it runs nicely. For next PRs we can think of showing only sim/gen status numbers which are used in the file.

kjvbrt · 2024-05-31T13:46:14Z

I run it again and found that the filter is reset when I move any of the elements.

brauliorivas · 2024-05-31T14:00:06Z

I run it again and found that the filter is reset when I move any of the elements.

Sorry, I thought it wouldn't matter because it's just like a "prototype". But I'm going to make it work when moving too now.

filtering/reconnecting WIP

4cb2094

tmadlener reviewed May 14, 2024

View reviewed changes

basic filtering for MC particle function

df53119

brauliorivas marked this pull request as ready for review May 14, 2024 22:28

refactor toggle functionality + fix reconnect child/parent

8cd129d

brauliorivas changed the title ~~filtering/reconnecting WIP~~ filtering/reconnecting May 15, 2024

brauliorivas added 3 commits May 15, 2024 16:08

filtering/reconnecting WIP

0ec7955

basic filtering for MC particle function

4e5f1a1

refactor toggle functionality + fix reconnect child/parent

ccaeeca

tmadlener force-pushed the feat-filtering branch from 8cd129d to ccaeeca Compare May 15, 2024 14:08

tmadlener reviewed May 16, 2024

View reviewed changes

brauliorivas added 3 commits May 21, 2024 20:17

incorporate advanced filtering for Monte Carlo Particles

022dece

remove accidently added node_modules

b8fd996

fix merge conflict

35ce882

[FEAT] filters using checkbox based on Simulator Status field

209d98a

remove filter.js file from menu folder, relocate

01183ee

Merge branch 'main' into feat-filtering

975cbc5

brauliorivas added 2 commits May 23, 2024 08:55

fix filtering function, now works together toogle with ranges

f19980f

Merge branch 'feat-filtering' of github.com:brauliorivas/dmx into fea…

4a7629f

…t-filtering

brauliorivas and others added 2 commits May 23, 2024 14:24

Merge branch 'main' into feat-filtering

5e221d6

[WIP] add testing for filtering functions

849cfcf

initial test suite for MC particles filtering

89aabf9

brauliorivas requested a review from tmadlener May 28, 2024 02:55

brauliorivas mentioned this pull request May 28, 2024

Configure previews with access tokens and check out to fork #24

Merged

kjvbrt reviewed May 31, 2024

View reviewed changes

add .js for file extension import

490b773

kjvbrt merged commit 87295d6 into key4hep:main May 31, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

filtering/reconnecting #18

filtering/reconnecting #18

brauliorivas commented May 14, 2024

tmadlener May 14, 2024

brauliorivas May 14, 2024

brauliorivas commented May 14, 2024

tmadlener commented May 15, 2024

tmadlener May 16, 2024

brauliorivas commented May 22, 2024

tmadlener commented May 22, 2024

brauliorivas commented May 22, 2024

tmadlener commented May 23, 2024

github-actions bot commented May 23, 2024 •

edited

Loading

tmadlener commented May 23, 2024 •

edited

Loading

brauliorivas commented May 23, 2024

tmadlener commented May 31, 2024

kjvbrt May 31, 2024

kjvbrt commented May 31, 2024

kjvbrt commented May 31, 2024 •

edited

Loading

brauliorivas commented May 31, 2024

filtering/reconnecting #18

filtering/reconnecting #18

Conversation

brauliorivas commented May 14, 2024

tmadlener May 14, 2024

Choose a reason for hiding this comment

brauliorivas May 14, 2024

Choose a reason for hiding this comment

brauliorivas commented May 14, 2024

tmadlener commented May 15, 2024

tmadlener May 16, 2024

Choose a reason for hiding this comment

brauliorivas commented May 22, 2024

tmadlener commented May 22, 2024

brauliorivas commented May 22, 2024

tmadlener commented May 23, 2024

github-actions bot commented May 23, 2024 • edited Loading

tmadlener commented May 23, 2024 • edited Loading

brauliorivas commented May 23, 2024

tmadlener commented May 31, 2024

kjvbrt May 31, 2024

Choose a reason for hiding this comment

kjvbrt commented May 31, 2024

kjvbrt commented May 31, 2024 • edited Loading

brauliorivas commented May 31, 2024

github-actions bot commented May 23, 2024 •

edited

Loading

tmadlener commented May 23, 2024 •

edited

Loading

kjvbrt commented May 31, 2024 •

edited

Loading