There is no way to get all roles using the ListOryAccessControlPolicyRoles() API

[sum2000]

Describe the bug

Not sure if you will consider this a bug or a feature request but I am using the API ListOryAccessControlPolicyRoles(flavor, limit, offset) and my use case involves getting all the roles. I tried passing limit and offset as 0 to get all of them but got empty results.

My current workaround is to call it like this
ListOryAccessControlPolicyRoles("regex", math.MaxInt64, 0)

Expected behavior

When I pass limit and offset as 0, I expect to get all the roles.

Environment

• Version: 1cb8a36
• Environment: Local

[aeneasr]

Turning off pagination is a Denial of Service (DoS) attack surface and thus this behavior is intentional.

[aeneasr]

Reopening because math.MaxInt64 should actually not work.

[sum2000]

I understand your concerns, see my latest feature request #121 for why we need this at the moment. Also, why should math.MaxInt64 not work?

[aeneasr]

Because pulling 2bn+ records from a database and encoding it as JSON format can cause huge latencies, memory consumption, and a very slow database. In fact, I checked if there's actually no upper boundary to the limit but in fact there is, it's capped at 500 elements. So most likely you're not hitting that limit right now with the way you fetch data from the API.

Pagination in REST APIs is a very common pattern, and it's of course possible to fetch all elements in a paginated API, you simply have to iterate over the pages. For more info: https://www.google.com/search?q=rest+client+pagination

[sum2000]

Ya, we test the API's on sandbox where db has less than 500 elements. Makes sense that pagination is the way to go when listing all the roles but I still think that there should be an easier and elegant way to get all the roles associated with a certain member

[aeneasr]

I'm not sure if we have implemented this in Keto yet, but at least in Hydra we're sending a Link header that makes it super easy to get the next X elements until the list is empty. That way you can easily iterate over all of the elements. You could even do that in parallel reducing the overall fetching time significantly!

Here's a parser for that: https://github.com/tomnomnom/linkheader

[sum2000]

Oh, that's something I didn't know, will check it out! Thank you.