Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set a sensible default for API server HTTP Read buffer size #513

Merged

Conversation

gberche-orange
Copy link
Contributor

Pull request description

Set a sensible default for API server HTTP Read buffer size

Avoid explicit use of the following when using oauth2-proxy in front of api server

    testkube-api:
      extraEnvVars:
      - name: APISERVER_HTTP_READBUFFERSIZE
        value: "8192"

Relates to kubeshop/testkube#2870

Checklist (choose whats happened)

  • breaking change! (describe)
  • tested locally: only tested extraEnvVars workaround so far
  • tested on cluster
  • added new dependencies
  • updated the docs
  • added a test

Breaking changes

Changes

Fixes

Avoid explicit use of the following when using oauth2-proxy in front of api server

```
    testkube-api:
      extraEnvVars:
      - name: APISERVER_HTTP_READBUFFERSIZE
        value: "8192"
```

Relates to kubeshop/testkube#2870
Copy link
Contributor

@vsukhin vsukhin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, @ypoplavs ?

@ypoplavs ypoplavs merged commit b82b639 into kubeshop:develop May 17, 2023
vsukhin added a commit that referenced this pull request Jun 1, 2023
* feat: cron job template field (#480)

* feat: cron job template field

* fix: add namespace to cron job domain

* fix: remove namespace

* fix typo (#486)

* feat: deprecate args (#475)

* feat: args mode field

* fix: curl command and args

* feat: artillery commands and args

* feat: change artifact support

* fix: add artifacts to curl and artillery

* fix: typo

* feat: postamn command and args

* feat: kubepug args and command

* feat: soapui command and args

* feat: maven args and command

* feat: jmeter command and args

* fix: jemeter args

* feat: cypress command and args

* fix: one more cypress args

* feat: ginkgo command and args

* fix: ginkgo args

* fix: ginkgo vars arg

* feat: k6 command and args

* feat: playwright args and command

* fix: rename k6 arg

* fix bugs (#489)

* fix executors.json (#491)

* rework flow to depploy executors with api (#492)

* update sed (#494)

* Update branch (#496)

update branch

* Update branch (#497)

update develop

* edit script for updating chart versions (#500)

* edit script for updating chart versions

* [testkube-api] minio allow setting minio replicaCount (#502)

* Fix post request in nginx config (#504)

* update post config in nginx

* commit executor image tag to develop (#505)

* added force push for kubeshop bot user

* Update executors.json

* add if to commit executur step

* add if to commit executur step

* added removed line from test run

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* update release branch naming (#509)

* Testkube dev - overwrite branch on test run (#506)

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* update chart releaser script

* Fix kubeoperator preupgrade josb for multi-namespace (#507)

Fix kubeoperator preupgrade josb for multi-namespace (#507)

* Set a sensible default for API server HTTP Read buffer size (#513)

Set a sensible default for API server HTTP Read buffer size (#513)

* Update executors.json

* Update executors.json

* convert the var to staring (#514)

* feat: cdevents support (#503)

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* ci: add changelog generation (#516)

ci: add changelog generation

* feat: dashboard uri for cd events (#517)

* feat: dashboard uri for cd events

* fix: env var for dashboard uri

* docs: dashboard uri

* Update executors.json

* feat: add option to force disabling telemetry on the UI (#518)

Related to: kubeshop/testkube#3884

* fix: testkube service monitor target port (#515)

* Fix service monitor target port

* feat: zap executor (#511)

* feat: zap executor

* feat: add zap to dev release

* ci: add zap to stage and pre-release

* ci: addressing code review comments

* ci: update tests

* feat: condition ttl (#520)

* Update executors.json

* Update executors.json

* Update executors.json

* add automatic pr creation (#521)

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* ci: add condition for checking if executors.json were updated (#523)

ci: add condition for checking if executors.json were updated (#523)

* Update executors.json

* ci: fix client payload for image tag pre-release

* Update executors.json

* Update executors.json

* ci: update branch for tests in stage workflow (#524)

* feat: added webhook to cdevents for test purpose (#525)

* Update executors.json

* fix: new webhook.site to bypass limits

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* fix: zap executor

---------

Co-authored-by: Vladislav Sukhin <[email protected]>
Co-authored-by: Povilas Versockas <[email protected]>
Co-authored-by: kubeshop-bot <[email protected]>
Co-authored-by: Tomasz Konieczny <[email protected]>
Co-authored-by: Guillaume Berche <[email protected]>
Co-authored-by: Dawid Rusnak <[email protected]>
Co-authored-by: Lilla Vass <[email protected]>
Co-authored-by: Jacek Wysocki <[email protected]>
@alfeg
Copy link

alfeg commented Jun 30, 2023

For me 32000 were required to make testkube work under Azure AD auth

ALSO this is quite a breaking change for us.

As it causes testkube fail to update in case if snippet exists in values.yaml file as proposed in one the issues kubeshop/testkube#3141 (comment)

testkube-api:
   extraEnvVars:
   - name: APISERVER_HTTP_READBUFFERSIZE
     value: '32000'
Error: UPGRADE FAILED: cannot patch "testkube-api-server" with kind Deployment: The order in patch list:
[map[name:POSTMANEXECUTOR_URI value:] map[name:SLACK_TOKEN value:] map[name:STORAGE_REGION value:] map[name:STORAGE_TOKEN value:] map[name:JOB_SERVICE_ACCOUNT_NAME value:] map[name:APISERVER_HTTP_READBUFFERSIZE value:8192] map[name:APISERVER_HTTP_READBUFFERSIZE value:32000] map[name:TESTKUBE_OAUTH_CLIENTID value:] map[name:TESTKUBE_OAUTH_CLIENTSECRET value:] map[name:TESTKUBE_OAUTH_SCOPES value:] map[name:TESTKUBE_WATCHER_NAMESPACES value:] map[name:TESTKUBE_REGISTRY value:<nil>] map[name:CDEVENTS_TARGET value:] map[name:TESTKUBE_DASHBOARD_URI value:]]
 doesn't match $setElementOrder list:
[map[name:APISERVER_PORT] map[name:API_MONGO_DSN] map[name:API_MONGO_ALLOW_DISK_USE] map[name:NATS_URI] map[name:POSTMANEXECUTOR_URI] map[name:TESTKUBE_ANALYTICS_ENABLED] map[name:SLACK_TOKEN] map[name:STORAGE_ENDPOINT] map[name:STORAGE_BUCKET] map[name:STORAGE_EXPIRATION] map[name:STORAGE_ACCESSKEYID] map[name:STORAGE_SECRETACCESSKEY] map[name:STORAGE_REGION] map[name:STORAGE_TOKEN] map[name:STORAGE_SSL] map[name:SCRAPPERENABLED] map[name:LOGS_BUCKET] map[name:LOGS_STORAGE] map[name:TESTKUBE_NAMESPACE] map[name:JOB_SERVICE_ACCOUNT_NAME] map[name:APISERVER_FULLNAME] map[name:APISERVER_CONFIG] map[name:APISERVER_HTTP_READBUFFERSIZE] map[name:TESTKUBE_OAUTH_CLIENTID] map[name:TESTKUBE_OAUTH_CLIENTSECRET] map[name:TESTKUBE_OAUTH_PROVIDER] map[name:TESTKUBE_OAUTH_SCOPES] map[name:TESTKUBE_CLOUD_URL] map[name:TESTKUBE_WATCHER_NAMESPACES] map[name:APISERVER_HTTP_READBUFFERSIZE] map[name:TESTKUBE_CONFIG_DIR] map[name:TESTKUBE_REGISTRY] map[name:TESTKUBE_POD_START_TIMEOUT] map[name:CDEVENTS_TARGET] map[name:TESTKUBE_DASHBOARD_URI]]

Solution is to remove APISERVER_HTTP_READBUFFERSIZE extra env and replace it with httpReadBufferSize: 32000 (or whatever value You use)

@vsukhin
Copy link
Contributor

vsukhin commented Jun 30, 2023

@ypoplavs please check it out

@ypoplavs
Copy link
Contributor

ypoplavs commented Jul 3, 2023

Hello @alfeg!
Thank you for your comment and apologies for the breaking change that caused your TK installation to fail. We will be testing our changes more thoroughly.

I have checked our values.yaml file and saw that we do use httpReadBufferSize value and not APISERVER_HTTP_READBUFFERSIZE, thus for installations with updated values file it will work.

vsukhin added a commit that referenced this pull request Jul 7, 2023
* feat: cron job template field (#480)

* feat: cron job template field

* fix: add namespace to cron job domain

* fix: remove namespace

* fix typo (#486)

* feat: deprecate args (#475)

* feat: args mode field

* fix: curl command and args

* feat: artillery commands and args

* feat: change artifact support

* fix: add artifacts to curl and artillery

* fix: typo

* feat: postamn command and args

* feat: kubepug args and command

* feat: soapui command and args

* feat: maven args and command

* feat: jmeter command and args

* fix: jemeter args

* feat: cypress command and args

* fix: one more cypress args

* feat: ginkgo command and args

* fix: ginkgo args

* fix: ginkgo vars arg

* feat: k6 command and args

* feat: playwright args and command

* fix: rename k6 arg

* fix bugs (#489)

* fix executors.json (#491)

* rework flow to depploy executors with api (#492)

* update sed (#494)

* Update branch (#496)

update branch

* Update branch (#497)

update develop

* edit script for updating chart versions (#500)

* edit script for updating chart versions

* [testkube-api] minio allow setting minio replicaCount (#502)

* Fix post request in nginx config (#504)

* update post config in nginx

* commit executor image tag to develop (#505)

* added force push for kubeshop bot user

* Update executors.json

* add if to commit executur step

* add if to commit executur step

* added removed line from test run

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* update release branch naming (#509)

* Testkube dev - overwrite branch on test run (#506)

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* update chart releaser script

* Fix kubeoperator preupgrade josb for multi-namespace (#507)

Fix kubeoperator preupgrade josb for multi-namespace (#507)

* Set a sensible default for API server HTTP Read buffer size (#513)

Set a sensible default for API server HTTP Read buffer size (#513)

* Update executors.json

* Update executors.json

* convert the var to staring (#514)

* feat: cdevents support (#503)

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* ci: add changelog generation (#516)

ci: add changelog generation

* feat: dashboard uri for cd events (#517)

* feat: dashboard uri for cd events

* fix: env var for dashboard uri

* docs: dashboard uri

* Update executors.json

* feat: add option to force disabling telemetry on the UI (#518)

Related to: kubeshop/testkube#3884

* fix: testkube service monitor target port (#515)

* Fix service monitor target port

* feat: zap executor (#511)

* feat: zap executor

* feat: add zap to dev release

* ci: add zap to stage and pre-release

* ci: addressing code review comments

* ci: update tests

* feat: condition ttl (#520)

* Update executors.json

* Update executors.json

* Update executors.json

* add automatic pr creation (#521)

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* ci: add condition for checking if executors.json were updated (#523)

ci: add condition for checking if executors.json were updated (#523)

* Update executors.json

* ci: fix client payload for image tag pre-release

* Update executors.json

* Update executors.json

* ci: update branch for tests in stage workflow (#524)

* feat: added webhook to cdevents for test purpose (#525)

* Update executors.json

* fix: new webhook.site to bypass limits

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* feat: add TTL for k8s jobs (#530)

* Update executors.json

* Update executors.json

* Update executors.json

* fix: preupgrade lables

* Revert "fix: preupgrade lables"

This reverts commit 02f6971.

* fix: preupgrade labels (#533)

* Update executors.json

* feat: webhook template and headers (#538)

* Update executors.json

* add cloud migrate env (#540)

* Update executors.json

* Update executors.json

* Update executors.json

* Testkube Doc: refine  test doc for content.type (#541)

* Doc: refine schema for content.type

Extracted from https://github.com/kubeshop/testkube/blob/cca3c13629078891f269f691e1050ae5adb99d5a/docs/docs/test-types/prebuilt-executor.md?plain=1#L60-L65

* Doc: refine schema for content.type

Extracted from https://github.com/kubeshop/testkube/blob/cca3c13629078891f269f691e1050ae5adb99d5a/docs/docs/test-types/prebuilt-executor.md?plain=1#L60-L65

* Doc: refine schema for content.type

Refine git wording

https://git-scm.com/docs/git-clone
> --depth <depth>
>      Create a shallow clone with a history truncated to the specified number of commits.
https://git-scm.com/docs/git-sparse-checkout
> This command is used to create sparse checkouts, which change the working tree from having all tracked files present to only having a subset of those files

* Doc update: preRunScript ignored for container executor (#535)

See https://discord.com/channels/884464549347074049/1088559756714705067/1088571830639542343

* Update executors.json

* fix: template labels

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* feat: testsuite v3 (#380)

* feat: testsuite v3

* fix: remove required steps

* fix: sync testsuite crd

* fix: duration format

* fix: change test suite step definition

* fix: yaml indent

* Update executors.json

* Update executors.json

* fix: minio service labels (#544)

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* charts: add warning for global image params (#547)

* fix error with enabled tls (#548)

* testkube-operator: Removed ns from tokenSecret and usernameSecret (#545)

* Update executors.json

* Update executors.json

* feat: update config map for testkube-api (#550)

* update config map for testkube-api to pass custom job-template.yml

* Update executors.json

* Update executors.json

* Update executors.json

* update branch for checking out tests (#552)

* Update executors.json

* add arg to run install beta release to test cli (#554)

* Update executors.json

* feat: post run script field (#555)

* feat: post run script field

* fix: field type

* fix: indent

* Update executors.json

* Update executors.json

* Update executors.json

* feat: update referencing of secrets in minio and TK API (#557)

* update referencing of secrets in minio and TK API

* Update executors.json

* ci: update releaser script (#558)

* Update executors.json

* ci: add release notes generation (#551)

* ci: add flow to deploy to stage on values.yaml update (#560)

* feat: cluster name parameter (#559)

* feat: cluster name parameter

* Update charts/testkube/README.md

Co-authored-by: Lilla Vass <[email protected]>

---------

Co-authored-by: Lilla Vass <[email protected]>

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* remove probes for mongo

* feat: add basic auth (#562)

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* feat: probe conditions (#561)

* feat: probe spec

* fix: refactor probe spec crd

* Update executors.json

* Update executors.json

* feat: tracetest executor (#564)

* feat: tracetest executor

* fix: update tag on tracetest executor docker image

* Update executors.json

* fix: resource separation inside conditional loops to avoid templating issue when operator webhook disabled (#565)

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* add basic auth for beta env (#566)

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Update executors.json

* Revert "feat: tracetest executor (#564)"

This reverts commit 248b159.

# Conflicts:
#	charts/testkube-api/executors.json

* Revert "feat: probe conditions (#561)"

This reverts commit 1573b66.

---------

Co-authored-by: ypoplavs <[email protected]>
Co-authored-by: Povilas Versockas <[email protected]>
Co-authored-by: kubeshop-bot <[email protected]>
Co-authored-by: ypoplavs <[email protected]>
Co-authored-by: Tomasz Konieczny <[email protected]>
Co-authored-by: Guillaume Berche <[email protected]>
Co-authored-by: Dawid Rusnak <[email protected]>
Co-authored-by: Lilla Vass <[email protected]>
Co-authored-by: Jacek Wysocki <[email protected]>
Co-authored-by: Jesper Axelsen <[email protected]>
Co-authored-by: Carlos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants