Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[api] Add max 32 mb on content data #264

Merged
merged 1 commit into from
Apr 13, 2021
Merged

[api] Add max 32 mb on content data #264

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions pkg/generated/models/rekord_v001_schema.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions pkg/generated/models/rpm_v001_schema.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 8 additions & 4 deletions pkg/generated/restapi/embedded_spec.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion pkg/types/rekord/v0.0.1/rekord_v0_0_1_schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@
"content": {
"description": "Specifies the content inline within the document",
"type": "string",
"format": "byte"
"format": "byte",
"maxLength" : 32000000
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can't comment here in the JSON. I'll make a note in the nginx config to remember to update both places.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, filed a TODO over there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this change would limit the length of the inline base64 encoded content field, it's not sufficient to develop an equivalent limit to what the NGINX ingress was enforcing (i.e. the entire number of bytes passed in the HTTP request body). Also since OpenAPI is enforcing this via the length of the string, the 32M value really puts a limit on the content of ~67% of that or approx 21MB.

I can see use cases for passing large files to Rekor (e.g. signed ISO images or jumbo containers), and since certain types can be fully streamed (rekord and RPM) while others can't (soon-to-be-added JAR support), I think we need to revert this PR and consider it more broadly. I'm happy to take some time to develop a proposal on this.

I could also imagine some users might want a size limit to be configurable, so that is something to consider as well.

Copy link
Member

@lukehinds lukehinds Apr 13, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could also imagine some users might want a size limit to be configurable, so that is something to consider as well.

Agree, this could be a config value in rekor-server.yaml, that way you can place 32000000 as the default value in the config, but users of rekor who have needs for larger file can still set accordingly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds great, would love to chat more about office hours! I had exactly that problem with not being able to measure the request itself, and couldn't see a great solution here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue right now is that it's enforced at ingress, not in our rekor config. That's the main reason client-validation would be nice, otherwise we end up with random errors from the load balancer that clients can't catch.

It might be better to just let clients hit it though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also return back the 413 w/ error message

}
},
"oneOf": [
Expand Down
3 changes: 2 additions & 1 deletion pkg/types/rpm/v0.0.1/rpm_v0_0_1_schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@
"content": {
"description": "Specifies the package inline within the document",
"type": "string",
"format": "byte"
"format": "byte",
"maxLength" : 32000000
}
},
"oneOf": [
Expand Down