Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Internal] Update Jobs GetRun API to support paginated responses for jobs and ForEach tasks with >100 runs #1009

Closed
wants to merge 11 commits into from
36 changes: 36 additions & 0 deletions service/jobs/ext_api.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
package jobs

import "context"

// GetRun retrieves a run based on the provided request.
// It handles pagination if the run contains multiple iterations or tasks.
func (a *JobsAPI) GetRun(ctx context.Context, request GetRunRequest) (*Run, error) {
renaudhartert-db marked this conversation as resolved.
Show resolved Hide resolved
run, err := a.jobsImpl.GetRun(ctx, request)
if err != nil {
return nil, err
}

// When querying a Job run, a page token is returned when there are more than 100 tasks. No iterations are defined for a Job run. Therefore, the next page in the response only includes the next page of tasks.
// When querying a ForEach task run, a page token is returned when there are more than 100 iterations. Only a single task is returned, corresponding to the ForEach task itself. Therefore, the client only reads the iterations from the next page and not the tasks.
isPaginatingIterations := run.Iterations != nil && len(run.Iterations) > 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For posterity, it would be good to document this behavior, as it isn't immediately obvious how this works from the spec itself. Something like:

When querying a Job run, a page token is returned when there are more than 100 tasks. No iterations are defined for a Job run. Therefore, the next page in the response only includes the next page of tasks.
When querying a ForEach task run, a page token is returned when there are more than 100 iterations. Only a single task is returned, corresponding to the ForEach task itself. Therefore, the client only reads the iterations from the next page and not the tasks.
For any other task type, iterations is empty and tasks only contains the one task, so no pagination is needed.


pageToken := run.NextPageToken
for pageToken != "" {
request.PageToken = pageToken
nextRun, err := a.jobsImpl.GetRun(ctx, request)
if err != nil {
return nil, err
}

if isPaginatingIterations {
run.Iterations = append(run.Iterations, nextRun.Iterations...)
} else {
run.Tasks = append(run.Tasks, nextRun.Tasks...)
}
pageToken = nextRun.NextPageToken
}

run.NextPageToken = ""
run.PrevPageToken = ""
return run, nil
}
Loading
Loading