Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove controller parameter from queries #2575

Merged
merged 6 commits into from
Jan 18, 2022

Conversation

dbutenhof
Copy link
Member

@dbutenhof dbutenhof commented Dec 7, 2021

"controller" is becoming just a dataset metadata; perhaps useful to filter based on origin, but not essential. The new /datasets/list API is based on a SQL query of the Datasets table, supporting filtering by name (substring, e.g., "fio"), controller, start and end time range (which can be used together or independently for a "before" or "after").

The Dataset record now has both created and uploaded timestamps, where the latter is the old created (defaults to current time of Dataset commit) and the new created is read from the tarball metadata.log file.

Dataset SQL uniqueness is defined now only by the dataset name.

@dbutenhof dbutenhof added enhancement Server Dashboard Of and relating to the Dashboard GUI API Of and relating to application programming interfaces to services and functions labels Dec 7, 2021
@dbutenhof dbutenhof added this to the v0.72 milestone Dec 7, 2021
@dbutenhof dbutenhof self-assigned this Dec 7, 2021
@lgtm-com

This comment has been minimized.

Copy link
Member

@webbnh webbnh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very good. I just have the usual pile of nits, small things, and suggestions.

lib/pbench/server/api/resources/__init__.py Outdated Show resolved Hide resolved
lib/pbench/server/api/resources/__init__.py Show resolved Hide resolved
lib/pbench/server/api/resources/__init__.py Show resolved Hide resolved
lib/pbench/server/api/resources/__init__.py Show resolved Hide resolved
lib/pbench/server/api/resources/__init__.py Outdated Show resolved Hide resolved
lib/pbench/test/unit/server/test_datasets_list.py Outdated Show resolved Hide resolved
lib/pbench/test/unit/server/test_datasets_list.py Outdated Show resolved Hide resolved
@dbutenhof dbutenhof marked this pull request as draft January 12, 2022 15:50
@dbutenhof dbutenhof requested a review from webbnh January 13, 2022 21:50
@dbutenhof dbutenhof marked this pull request as ready for review January 13, 2022 21:51
@dbutenhof dbutenhof requested review from portante and riya-17 January 17, 2022 16:06
webbnh
webbnh previously approved these changes Jan 17, 2022
Copy link
Member

@webbnh webbnh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good...but, of course, I can't help but comment on stuff.

So, if you find none of my comments worth acting on, let me know and I'll close them.

lib/pbench/server/api/resources/upload_api.py Outdated Show resolved Hide resolved
lib/pbench/server/api/resources/upload_api.py Outdated Show resolved Hide resolved
lib/pbench/server/api/resources/upload_api.py Outdated Show resolved Hide resolved
lib/pbench/server/api/resources/upload_api.py Show resolved Hide resolved
lib/pbench/server/api/resources/upload_api.py Outdated Show resolved Hide resolved
lib/pbench/test/unit/server/database/test_datasets_db.py Outdated Show resolved Hide resolved
lib/pbench/server/filetree.py Outdated Show resolved Hide resolved
lib/pbench/server/filetree.py Outdated Show resolved Hide resolved
Copy link
Member

@portante portante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Would just like to make sure we don't execute unnecessary code in debug statements, and make sure the TODOs are properly captured in issues, and removed from the code.

lib/pbench/server/database/models/datasets.py Outdated Show resolved Hide resolved
lib/pbench/server/database/models/datasets.py Outdated Show resolved Hide resolved
lib/pbench/server/filetree.py Outdated Show resolved Hide resolved
lib/pbench/server/filetree.py Outdated Show resolved Hide resolved
lib/pbench/server/api/resources/upload_api.py Outdated Show resolved Hide resolved
lib/pbench/server/api/resources/datasets_list.py Outdated Show resolved Hide resolved
lib/pbench/server/api/resources/datasets_list.py Outdated Show resolved Hide resolved
lib/pbench/server/api/resources/datasets_list.py Outdated Show resolved Hide resolved
lib/pbench/server/api/resources/__init__.py Outdated Show resolved Hide resolved
"controller" is becoming just a dataset metadata; perhaps useful
to filter based on origin, but not essential. The new /datasets/list
API is based on a SQL query of the Datasets table, supporting
filtering by name (substring, e.g., "fio"), controller, start and
end time range (which can be used together or independently for a
"before" or "after").

The Dataset record now has both `created` and `uploaded`
timestamps, where the latter is the old `created` (defaults
to current time of `Dataset` commit) and the new `created`
is read from the tarball `metadata.log` file.

Dataset SQL uniqueness is defined now only by the dataset name.
Try to work around an inexplicable test tarball unpacking problem on
Jenkins by re-packing my modified test-5.2.tar.xz inside a local
container.
Mostly factoring out some comments that have become issues.
Copy link
Member

@webbnh webbnh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ship it!

@dbutenhof dbutenhof merged commit 64c3d74 into distributed-system-analysis:main Jan 18, 2022
@dbutenhof dbutenhof deleted the uncontrolled branch January 18, 2022 20:39
@portante portante modified the milestones: v0.72, v0.71 Jan 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Of and relating to application programming interfaces to services and functions Dashboard Of and relating to the Dashboard GUI enhancement Server
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants