Broader usage of Paginated Lists

[cipherboy]

Is your feature request related to a problem? Please describe.

In #140 and #170, initial support of paginated lists were added, which included some API endpoints. However, some more endpoints could be updated to use these new APIs.

Describe the solution you'd like

• PKI Tidy operation should use paginated lists everywhere. in Add pagination to tidy  #678 thanks to @fatima2003!
• Audit other tidy operations for usage of paginated lists
• Lease expiry should use paginated lists
• Token expiry should use paginated lists
• Secrets and auth engine listing should probably support pagination as well. This does not use a storage.List call but could still be made transactional. Transactional added in Split mount tables with transactions #622, but should still manually support pagination even though it won't be any more efficient.
• Policy listing should also use this, even though it uses a recursive storage view.
• Additional calls to storage.List should probably be audited as well.

Describe alternatives you've considered

Some places will continue to rely on storage.List as they have a low volume of responses, e.g., issuer counts should be in the 10s at most.

Explain any additional use-cases

Some additional use cases are available at the HashiCorp Vault issue: hashicorp/vault#10887

Additional context

This will help with memory utilization in more high-volume areas of the code. Would be a good GA milestone candidate.

[lplazas]

I'd be happy to get this assigned to me.

[cipherboy]

@lplazas I think a great place to start would be the PKI engine. While large, I think the changeset should mostly be limited to https://github.com/openbao/openbao/blob/main/builtin/logical/pki/path_tidy.go :-)

[cipherboy]

\o hey @lplazas! As a FYI, I'm tackling the ones in the PKI engine ahead of GA, but let me know if you're not interested in this any more and I can tackle the rest :-) Or if you have any questions, feel free to tag me or contact me!

[fatima2003]

Hi @cipherboy, I will be working on using pagination in PKI tidy.