Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem: Packages count only ever counts packages listed on current page #988

Closed
fiver-watson opened this issue Jul 9, 2024 · 3 comments · Fixed by #1038
Closed

Problem: Packages count only ever counts packages listed on current page #988

fiver-watson opened this issue Jul 9, 2024 · 3 comments · Fixed by #1038
Assignees
@djjuhasz
Labels
Client: SFA SFA bridge work

Comments

@fiver-watson
Copy link
Contributor

Description

The Packages browse page in Enduro includes a count of page results and total packages, just below the page header. However, this counter constantly only ever reads 20/20 - it is currently only ever listing the total package count as the current total page count.

To Reproduce

Steps to reproduce the behavior:

  1. Log into enduro
  2. Make sure you have more than 20 packages processed
  3. Navigate to the Packages page
  4. Look at the count below the Packages header

Error encountered

Package count reads:

Showing 20/20

Expected outcome

This counter should:

  • List the number of results on the current page as the first value, and
  • List the TOTAL number of ingested packages as the second value
@fiver-watson fiver-watson added bug Something isn't working good first issue Good for newcomers labels Jul 9, 2024
@fiver-watson fiver-watson moved this to 🛠 Refining in Enduro Jul 9, 2024
@sallain sallain moved this from 🛠 Refining to 👍 Ready in Enduro Jul 11, 2024
@djjuhasz djjuhasz removed the good first issue Good for newcomers label Aug 23, 2024
@djjuhasz
Copy link
Collaborator

djjuhasz commented Aug 23, 2024 •
edited
Loading

Fixing the total package count is going to require changes to the Enduro API - the current /package GET endpoint doesn't return a count of the total number of packages stored in Enduro. This change will require changes to the Goa API layer, the package service layer, and the persistence layer so this is not a good first issue to tackle.

@sallain
Copy link
Collaborator

sallain commented Aug 23, 2024

Thanks for the context, @djjuhasz!

@djjuhasz djjuhasz self-assigned this Sep 9, 2024
@djjuhasz djjuhasz moved this from 👍 Ready to ⏳ In Progress in Enduro Sep 9, 2024
@djjuhasz djjuhasz assigned djjuhasz and unassigned djjuhasz Sep 11, 2024
@djjuhasz djjuhasz moved this from 👍 Ready to ⏳ In Progress in Enduro Sep 17, 2024
@djjuhasz djjuhasz mentioned this issue Sep 25, 2024
Merged
djjuhasz added a commit that referenced this issue Sep 27, 2024
@djjuhasz
Add a persistence query filter implementation
6d2dbb3 
Refs #988, #514

To address #988 and provide a count of the total number of packages in
the dashboard the API GET /package endpoint to provide total number of
packages in a result set. Because we need to update the API and the
underlying database calls, this is a good opportunity to also address
issue #514 and add a general search implementation that allows
filtering, ordering and paging results. This PR adds this general search
filtering implementation with paging and ordering, and also provides a
count of the total number of records matching the search criteria before
paging.

- Add a persistence package filter stub implementation
- Add persistence Page and Sort structs
- Add an entgo query filter implementation
- Add methods to add page, sort, and column value filters to the ent
  filter implementation
- Add a timerange package for representing time ranges (i.e. a time
  period with a start and end time)
djjuhasz added a commit that referenced this issue Sep 27, 2024
@djjuhasz
Add a persistence query filter implementation
e2cc8b3 
Refs #988, #514

To address #988 and provide a count of the total number of packages in
the dashboard the API GET /package endpoint to provide total number of
packages in a result set. Because we need to update the API and the
underlying database calls, this is a good opportunity to also address
issue #514 and add a general search implementation that allows
filtering, ordering and paging results. This PR adds this general search
filtering implementation with paging and ordering, and also provides a
count of the total number of records matching the search criteria before
paging.

- Add a persistence package filter stub implementation
- Add persistence Page and Sort structs
- Add an entgo query filter implementation
- Add methods to add page, sort, and column value filters to the ent
  filter implementation
- Add a timerange package for representing time ranges (i.e. a time
  period with a start and end time)
djjuhasz added a commit that referenced this issue Sep 27, 2024
@djjuhasz
Add ListPackages() to persistence service
ffdbee2 
Refs #988

This continues the work toward #988 by adding a `ListPackages()` method
to the persistence service that includes a count of the total search
results returned (before paging), as well as allowing filtering,
sorting, and paging of the search.

This commit also:
- Add a telemetry wrapper for the `ListPackages()` method to enable
  extra reporting and debugging data
- Additonal comments for the `Filter.Page()` method
- Many unit tests for the `ListPackages()` functionality
djjuhasz added a commit that referenced this issue Sep 27, 2024
@djjuhasz
Add ListPackages() to persistence service
ae09d88 
Refs #988

This continues the work toward #988 by adding a `ListPackages()` method
to the persistence service that includes a count of the total search
results returned (before paging), as well as allowing filtering,
sorting, and paging of the search.

Other changes:
- Add a telemetry wrapper for the `ListPackages()` method to provide
  extra performance and debugging data
- Add additonal commentary for the `Filter.Page()` method
- Add many unit tests for the `ListPackages()` functionality
djjuhasz added a commit that referenced this issue Sep 27, 2024
@djjuhasz
Add ListPackages() to persistence service
cdda311 
Refs #988

This continues the work toward #988 by adding a `ListPackages()` method
to the persistence service that includes a count of the total search
results returned (before paging), as well as allowing filtering,
sorting, and paging of the search.

Other changes:
- Add a telemetry wrapper for the `ListPackages()` method to provide
  extra performance and debugging data
- Add additional commentary for the `Filter.Page()` method
- Add many unit tests for the `ListPackages()` functionality
@djjuhasz djjuhasz mentioned this issue Sep 27, 2024
Merged
djjuhasz added a commit that referenced this issue Sep 27, 2024
@djjuhasz
Add ListPackages() to persistence service
0a12381 
Refs #988

This continues the work toward #988 by adding a `ListPackages()` method
to the persistence service that includes a count of the total search
results returned (before paging), as well as allowing filtering,
sorting, and paging of the search.

Other changes:
- Add a telemetry wrapper for the `ListPackages()` method to provide
  extra performance and debugging data
- Add additional commentary for the `Filter.Page()` method
- Add many unit tests for the `ListPackages()` functionality
djjuhasz added a commit that referenced this issue Sep 27, 2024
@djjuhasz
Add ListPackages() to persistence service
ae41938 
Refs #988

This continues the work toward #988 by adding a `ListPackages()` method
to the persistence service that includes a count of the total search
results returned (before paging), as well as allowing filtering,
sorting, and paging of the search.

Other changes:
- Add a telemetry wrapper for the `ListPackages()` method to provide
  extra performance and debugging data
- Add additional commentary for the `Filter.Page()` method
- Add many unit tests for the `ListPackages()` functionality
djjuhasz added a commit that referenced this issue Oct 1, 2024
@djjuhasz
Add ListPackages() to persistence service
f69f41c 
Refs #988

This continues the work toward #988 by adding a `ListPackages()` method
to the persistence service that includes a count of the total search
results returned (before paging), as well as allowing filtering,
sorting, and paging of the search.

Other changes:
- Add a telemetry wrapper for the `ListPackages()` method to provide
  extra performance and debugging data
- Add additional commentary for the `Filter.Page()` method
- Add many unit tests for the `ListPackages()` functionality
djjuhasz added a commit that referenced this issue Oct 1, 2024
@djjuhasz
Add ListPackages() to persistence service
3f4c090 
Refs #988

This continues the work toward #988 by adding a `ListPackages()` method
to the persistence service that includes a count of the total search
results returned (before paging), as well as allowing filtering,
sorting, and paging of the search.

Other changes:
- Add a telemetry wrapper for the `ListPackages()` method to provide
  extra performance and debugging data
- Add additional commentary for the `Filter.Page()` method
- Add many unit tests for the `ListPackages()` functionality
djjuhasz added a commit that referenced this issue Oct 2, 2024
@djjuhasz
Add total result count to package list API
fb02afd 
Refs #988

This commit connects the API "GET /package" endpoint to the persistence
layer `ListPackages()` method. It also adds the current page limit and
offset as well as the total package count (before paging) to the API
response.

These changes will break the existing cursor-based pagination in the API
and Enduro dashboard. I will fix paging in a subsequent commit.

- Add page limit, offset, and total fields to API GET /package results
- Use `persistence.ListPackages()` to populate API results
- Add an adapter to convert goa `package_.ListPayload` search filters to
  `persistence.Filter` filters
- Add tests for API package search filters
djjuhasz added a commit that referenced this issue Oct 2, 2024
@djjuhasz
Add total result count to package list API
37fa2c9 
Refs #988

This commit connects the API "GET /package" endpoint to the persistence
layer `ListPackages()` method. It also adds the current page limit and
offset as well as the total package count (before paging) to the API
response.

These changes will break the existing cursor-based pagination in the API
and Enduro dashboard. I will fix paging in a subsequent commit.

- Add page limit, offset, and total fields to API GET /package results
- Use `persistence.ListPackages()` to populate API results
- Add an adapter to convert goa `package_.ListPayload` search filters to
  `persistence.Filter` filters
- Add tests for API package search filters
djjuhasz added a commit that referenced this issue Oct 2, 2024
@djjuhasz
Add total result count to package list API
d002e4c 
Refs #988

This commit connects the API "GET /package" endpoint to the persistence
layer `ListPackages()` method. The cursor request parameter is replaced
by the optional limit and offset parameters. The response now returns
the current page limit and offset, and the total number of results found
before paging, instead of a cursor value.

These changes will break the existing cursor-based pagination in the
Enduro Dashboard. I will fix the Dashboard paging in a subsequent
commit.

- Add optional `limit` and `offset` parameters to the GET /package
  request
- Add page `limit`, `offset`, and `total` fields to GET /package
  response
- Use `persistence.ListPackages()` to populate API results
- Add an adapter to convert goa `package_.ListPayload` search filters to
  `persistence.Filter` filters
- Add tests for the conversion of the package API parameters to
  persistence layer parameters, and the persistence search results to
  API response results
@djjuhasz djjuhasz mentioned this issue Oct 2, 2024
Merged
djjuhasz added a commit that referenced this issue Oct 2, 2024
@djjuhasz
Add total result count to package list API
800eb37 
Refs #988

This commit connects the API "GET /package" endpoint to the persistence
layer `ListPackages()` method. The cursor request parameter is replaced
by the optional limit and offset parameters. The response now returns
the current page limit and offset, and the total number of results found
before paging, instead of a cursor value.

These changes will break the existing cursor-based pagination in the
Enduro Dashboard. I will fix the Dashboard paging in a subsequent
commit.

- Add optional `limit` and `offset` parameters to the GET /package
  request
- Add page `limit`, `offset`, and `total` fields to GET /package
  response
- Use `persistence.ListPackages()` to populate API results
- Add an adapter to convert goa `package_.ListPayload` search filters to
  `persistence.Filter` filters
- Add tests for the conversion of the package API parameters to
  persistence layer parameters, and the persistence search results to
  API response results
djjuhasz added a commit that referenced this issue Oct 2, 2024
@djjuhasz
Add total result count to package list API
f6bf851 
Refs #988

This commit connects the API "GET /package" endpoint to the persistence
layer `ListPackages()` method. The cursor request parameter is replaced
by the optional limit and offset parameters. The response now returns
the current page limit and offset, and the total number of results found
before paging, instead of a cursor value.

These changes will break the existing cursor-based pagination in the
Enduro Dashboard. I will fix the Dashboard paging in a subsequent
commit.

- Add optional `limit` and `offset` parameters to the GET /package
  request
- Add page `limit`, `offset`, and `total` fields to GET /package
  response
- Use `persistence.ListPackages()` to populate API results
- Add an adapter to convert goa `package_.ListPayload` search filters to
  `persistence.Filter` filters
- Add tests for the conversion of the package API parameters to
  persistence layer parameters, and the persistence search results to
  API response results
djjuhasz added a commit that referenced this issue Oct 3, 2024
@djjuhasz
Add full dashboard pager with result count
5bbc0be 
Fixes #988.
djjuhasz added a commit that referenced this issue Oct 3, 2024
@djjuhasz
Add full dashboard pager with result count
891c38a 
Fixes #988.
@sallain sallain added Client: SFA SFA bridge work and removed bug Something isn't working labels Oct 3, 2024
djjuhasz added a commit that referenced this issue Oct 3, 2024
@djjuhasz
Add total result count and improved pager
4516de5 
Fixes #988: Adds the total number of packages found, before paging

Other changes:
- The package list subtitle now indicates which packages are currently
  displayed as well as the total number of results, e.g. "Showing 1 - 20
  of 228"
- The pager is only shown if there are more results than the page
  limit, i.e. if 27 results are returned an the page limit is 20
- The pager includes individual page links for up to 11 pages
- If more than 11 pages of results are returned, "First" and "Last"
  page links are shown
- The pager page list includes ellipses to indicate when their are more
  pages then the indivdual page links shown. I.e. if 15 pages of
  results are returned and the current page is page 1, then an list item
  containing an ellipsis is shown after the list item for page 11.
djjuhasz added a commit that referenced this issue Oct 3, 2024
@djjuhasz
Add total result count and improved pager
5ffcfd6 
Fixes #988: The package list shows the total number of packages found,
before paging.

Other changes:
- The package list subtitle now indicates which packages are currently
  displayed as well as the total number of results, e.g. "Showing 1 - 20
  of 228"
- The pager is only shown if there are more results than the page
  limit, i.e. if 27 results are returned an the page limit is 20
- The pager includes individual page links for up to 11 pages
- If more than 11 pages of results are returned, "First" and "Last"
  page links are shown
- The pager page list includes ellipses to indicate when their are more
  pages then the indivdual page links shown. I.e. if 15 pages of
  results are returned and the current page is page 1, then an list item
  containing an ellipsis is shown after the list item for page 11.
@djjuhasz djjuhasz mentioned this issue Oct 3, 2024
Merged
djjuhasz added a commit that referenced this issue Oct 3, 2024
@djjuhasz
Add total result count and improved pager
6b91f06 
Fixes #988: The package list shows the total number of packages found,
before paging.

Other changes:
- The package list subtitle now indicates which packages are currently
  displayed as well as the total number of results, e.g. "Showing 1 - 20
  of 228"
- The pager is only shown if there are more results than the page
  limit, i.e. if 27 results are returned an the page limit is 20
- The pager includes individual page links for up to 11 pages
- If more than 11 pages of results are returned, "First" and "Last"
  page links are shown
- The pager page list includes ellipses to indicate when their are more
  pages then the indivdual page links shown. I.e. if 15 pages of
  results are returned and the current page is page 1, then an list item
  containing an ellipsis is shown after the list item for page 11.
djjuhasz added a commit that referenced this issue Oct 3, 2024
@djjuhasz
Add total result count and improved pager
7ea9144 
Fixes #988: The package list shows the total number of packages found,
before paging.

Other changes:
- The package list subtitle now indicates which packages are currently
  displayed as well as the total number of results, e.g. "Showing 1 - 20
  of 228"
- The pager is only shown if there are more results than the page
  limit, i.e. if 27 results are returned an the page limit is 20
- The pager includes individual page links for up to 11 pages
- If more than 11 pages of results are returned, "First" and "Last"
  page links are shown
- The pager page list includes ellipses to indicate when their are more
  pages then the indivdual page links shown. I.e. if 15 pages of
  results are returned and the current page is page 1, then an list item
  containing an ellipsis is shown after the list item for page 11.
djjuhasz added a commit that referenced this issue Oct 3, 2024
@djjuhasz
Add total package count and improved pager
a30763e 
Fixes #988: The package list shows the total number of packages found,
before paging.

Other changes:
- The package list subtitle now indicates which packages are currently
  displayed as well as the total number of results, e.g. "Showing 1 - 20
  of 228"
- The pager is only shown if there are more results than the page
  limit, i.e. if 27 results are returned an the page limit is 20
- The pager includes individual page links for up to 11 pages
- If more than 11 pages of results are returned, "First" and "Last"
  page links are shown
- The pager page list includes ellipses to indicate when their are more
  pages then the indivdual page links shown. I.e. if 15 pages of
  results are returned and the current page is page 1, then an list item
  containing an ellipsis is shown after the list item for page 11.
djjuhasz added a commit that referenced this issue Oct 11, 2024
@djjuhasz
Add total package count and improved pager
7f26650 
Fixes #988: The package list shows the total number of packages found,
before paging.

Other changes:
- Show text indicating which packages are currently displayed as well
  as the total number of results, e.g. "Showing 1 - 20 of 228"
- Show the pager only when there are more results than the page
  limit, e.g. if 27 results are returned an the page limit is 20
- Include individual page links for up to 7 pages
- Show "first" and "last" page links when more than 7 pages of results
  are returned
- Show an ellipsis before or after the page links to indicated there are
  more pages than the page links show
- Hide page links and ellipses on narrow screens
@github-project-automation github-project-automation bot moved this from ⏳ In Progress to 🎉 Done in Enduro Oct 11, 2024
@fiver-watson
Copy link
Contributor Author

👍 LGTM! Also, I personally did end up liking have the results count both at the top and the bottom of the page. Even on a relatively large monitor, , the 20 results regularly went beyond my screen, so having that count repeated at the bottom by the pager (esp given that the pager doesn't reload the page fully and bring you back to the top) works well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@djjuhasz djjuhasz

Labels
Client: SFA SFA bridge work
Projects
Enduro
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add total package count and improved pager artefactual-sdps/enduro
3 participants
@djjuhasz @sallain @fiver-watson