Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Voxelytics routes #6416

Merged
merged 47 commits into from
Sep 22, 2022
Merged

Voxelytics routes #6416

merged 47 commits into from
Sep 22, 2022

Conversation

normanrz
Copy link
Member

@normanrz normanrz commented Aug 19, 2022

Adds the backend code for the Voxelytics workflow viewer to webKnossos.

Steps to test:


(Please delete unneeded items, merge only when none are left open)

@normanrz normanrz changed the title Vx routes Voxelytics routes Aug 19, 2022
Copy link
Member

@fm3 fm3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good already! :) I added a few notes. I think the overall structure works well. One thing that differs from a lot of the rest of wk is the combined DAO with complex queries, but I think it works well for this use case.
This kind of makes the standard access query mechanism a bad fit, though. I see that you have solved auth in a different way, and I’d say that’s fine. You could also try to bend the existing mechanism for this (inherit from SecuredSQLDAO, overwrite collectionName and see what it gets you), but I think since there are so many different tables, this does not really work well. Happy to talk about this in person again in the coming days :)

app/utils/SQLHelpers.scala Outdated Show resolved Hide resolved
app/utils/SQLHelpers.scala Show resolved Hide resolved
app/utils/SQLHelpers.scala Outdated Show resolved Hide resolved
app/utils/SQLHelpers.scala Outdated Show resolved Hide resolved
app/utils/WkConf.scala Outdated Show resolved Hide resolved
app/models/voxelytics/VoxelyticsService.scala Outdated Show resolved Hide resolved
app/models/voxelytics/VoxelyticsService.scala Show resolved Hide resolved
app/models/voxelytics/WorkflowDescription.scala Outdated Show resolved Hide resolved
conf/evolutions/086-voxelytics.sql Outdated Show resolved Hide resolved
conf/evolutions/086-voxelytics.sql Outdated Show resolved Hide resolved
Copy link
Member Author

@normanrz normanrz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback! I went all your comments.
I merged in the objectid-json branch, because it helps a lot! But it adds some unrelated changes. Will clean that up once #6413 is merged.

app/RequestHandler.scala Outdated Show resolved Hide resolved
app/models/voxelytics/VoxelyticsService.scala Outdated Show resolved Hide resolved
app/models/voxelytics/VoxelyticsDAO.scala Show resolved Hide resolved
app/utils/SQLHelpers.scala Show resolved Hide resolved
app/models/voxelytics/VoxelyticsDAO.scala Outdated Show resolved Hide resolved
app/controllers/VoxelyticsController.scala Outdated Show resolved Hide resolved
app/models/voxelytics/ElasticsearchClient.scala Outdated Show resolved Hide resolved
@fm3 fm3 marked this pull request as ready for review September 12, 2022 10:06
@@ -82,6 +82,7 @@ features {
allowDeleteDatasets = true
# to enable jobs for local development, use "yarn enable-jobs" to also activate it in the database
jobsEnabled = false
voxelyticsEnabled = true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be enabled by default?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that's just for development.

voxelytics {
staleTimeout = 20 minutes
elasticsearch {
uri = "http://localhost:9200"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my guess is the uri should be empty in the repository, and only added for the individual deployments?

Copy link
Member

@fm3 fm3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is moving in a very good direction :) You have been busy! I added a few more comments to the backend code. Looking forward to testing with the frontend next.

app/controllers/VoxelyticsController.scala Outdated Show resolved Hide resolved
app/controllers/VoxelyticsController.scala Outdated Show resolved Hide resolved
app/controllers/VoxelyticsController.scala Outdated Show resolved Hide resolved
app/controllers/VoxelyticsController.scala Show resolved Hide resolved
app/controllers/VoxelyticsController.scala Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
app/models/voxelytics/ElasticsearchClient.scala Outdated Show resolved Hide resolved
app/models/voxelytics/ElasticsearchClient.scala Outdated Show resolved Hide resolved
@normanrz normanrz mentioned this pull request Sep 16, 2022
2 tasks
Copy link
Member

@fm3 fm3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backend LGTM and seems to work :) I will admit I did not test every code path, and we will likely find more things as this lives on, but I’d say for a first version this is ready for some dogfooding.

Please add a changelog entry and migration guide, remove the for-development-only config and some debug logging (I think), then when the frontend is approved too this should be good to go :)

normanrz and others added 8 commits September 19, 2022 15:29
* wip voxelytics workflow viewer frontend

* fixes JS code

* stuff

* styling

* expose features.voxelyticsEnabled

* revert changes to Application.scala

* format

* pr feedback

* output ES2020

* styling

* styling

Co-authored-by: Florian M <[email protected]>
@bulldozer-boy bulldozer-boy bot merged commit 6834da3 into master Sep 22, 2022
@bulldozer-boy bulldozer-boy bot deleted the vx-routes branch September 22, 2022 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants