Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cli/sql: trim down the deps of the SQL shell to achieve a binary smaller than 10MiB #67470

Closed
15 of 17 tasks
knz opened this issue Jul 10, 2021 · 0 comments · Fixed by #67738
Closed
15 of 17 tasks

cli/sql: trim down the deps of the SQL shell to achieve a binary smaller than 10MiB #67470

knz opened this issue Jul 10, 2021 · 0 comments · Fixed by #67738
Labels
A-cli-client CLI commands that pertain to using SQL features C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-server-and-security DB Server & Security

Comments

@knz
Copy link
Contributor

knz commented Jul 10, 2021 •
edited
Loading

PR #67469 is introducing a standalone cockroach-sql which only contains the SQL shell via the new packages clisqlshell and related.

These are intended to only contain the minimum client code sufficient to run a shell, however unfortunately the generated binary is still 100-200MB large.

We want to trim down this size by breaking the dependencies on the crdb server code.

Here are some discrete tasks to achieve this:

cc @kernfeld-cockroach
cc @cockroachdb/sql-experience
@knz knz added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. A-cli-client CLI commands that pertain to using SQL features labels Jul 10, 2021
@blathers-crl blathers-crl bot added the T-server-and-security DB Server & Security label Jul 10, 2021
This was referenced Jul 10, 2021
Merged
Closed
Merged
Merged
Closed
craig bot pushed a commit that referenced this issue Jul 14, 2021
@chrisseto @ajwerner
@knz @rafiss @rail @rickystewart
Merge #66850 #67276 #67460 #67461 #67474 #67484 #67588 #67594 #67603
eef3cef 
66850: sqlproxyccl: statistically load balance across tenants r=chrisseto a=chrisseto

    Previously, the SQLProxy's Directory implementation would always
    select the first instance from the pods list. As part of the
    SQLProxy's duty is load balancing, this was inadequate. This commit
    upgrades the tenant dir to have pods report their CPU load. This
    load is then used to perform a weighted distribution across all
    running tenant pods.

Release note: None

67276: roachtest: remove test which exercised 20.2-21.1 upgrade r=postamar a=ajwerner

We no longer use the master roachtest for old branches. Given that, the answer
is to just remove this test from master.

This test fails because in #67053 I removed compatibility between 20.2 and
master. We should never be trying to do that upgrade.

Fixes #67182

Release note: None

67460: cli: move the demo cluster to new sub-package `democluster` (5/) r=otan a=knz

First 15 commits are from #67459 and prior.
Please ignore those (they will disappear once the prefix PR is merged)

Only the last 5 commits belong to this PR.

This is part of a family of PRs that aim to encapsulate CLI functionality into finer grained packages.

67461: cli: SQL shell cleanups and better encapsulation (6/) r=otan a=knz

First 20 commits are from #67460 and prior.
Please ignore those (they will disappear once the prefix PR is merged)

Only the last 4 commits belong to this PR.

This is part of a family of PRs that aim to encapsulate CLI functionality into finer grained packages.

67474: cli: new sub-package `clisqlcfg` to ease the creation of a standalone SQL shell (8/) r=otan a=knz

First 28 commits are from #67462 and prior.
Please ignore those (they will disappear once the prefix PR is merged)

Only the last 2 commits belong to this PR.

This is part of a family of PRs that aim to encapsulate CLI functionality into finer grained packages.



67484: clisqlclient: drop the dependency on 'security' r=otan a=knz

Fixes #67481.
Informs #67470.

This PR only contains 1 commit (the last).
All commits except for the last belong to #67469 and prior PRs. Ignore them during review.


The 'security' package is quite large. There is no reason to embark it
into a standalone SQL shell.

Towards achieving this, this patch removes the dependency on the
'security' package from `clisqlclient` by moving the password prompt
code into a sub-package.

Release note: None

67588: compare_test: account for more differences from postgres r=otan a=rafiss

- explicit cast for constants #62602 (comment)
- don't use postgis_script builtin #62602 (comment)
- handle differences in timezone offset display #62602 (comment)

Release note: None

67594: build: update go to 1.16.6 r=rail a=rail

* [ ] Adjust the Pebble tests to run in new version.
* [x] Adjust version in Docker image ([source](./builder/Dockerfile)).
* [x] Adjust version in the TeamCity agent image ([setup script](./packer/teamcity-agent.sh))
* [x] Rebuild and push the Docker image (following [Basic Process](#basic-process))
* [x] Bump the version in `WORKSPACE` under `go_register_toolchains`. You may need to bump [rules_go](https://github.com/bazelbuild/rules_go/releases).
* [x] Bump the version in `builder.sh` accordingly ([source](./builder.sh#L6)).
* [x] Bump the version in `go-version-check.sh` ([source](./go-version-check.sh)), unless bumping to a new patch release.
* [x] Bump the go version in `go.mod`. You may also need to rerun `make vendor_rebuild` if vendoring has changed.
* [x] Bump the default installed version of Go in `bootstrap-debian.sh` ([source](./bootstrap/bootstrap-debian.sh)).
* [x] Replace other mentions of the older version of go (grep for `golang:<old_version>` and `go<old_version>`).
* [ ] Update the `builder.dockerImage` parameter in the TeamCity [`Cockroach`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Cockroach&tab=projectParams) and [`Internal`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Internal&tab=projectParams) projects.
* [ ] Ask the Developer Infrastructure team to deploy new TeamCity agent images according to [packer/README.md](./packer/README.md)

Release note: None

67603: roachtest: re-mark `pkg/cmd/roachtest` tests as `broken_in_bazel` r=rail a=rickystewart

`faa4c2ccdaeca01eca3ccee29a8a64973cf8b043` removed this tag, which
caused all Bazel test runs to begin failing.

Closes #67595.

Release note: None

Co-authored-by: Chris Seto <[email protected]>
Co-authored-by: Andrew Werner <[email protected]>
Co-authored-by: Raphael 'kena' Poss <[email protected]>
Co-authored-by: Rafi Shamim <[email protected]>
Co-authored-by: Rail Aliiev <[email protected]>
Co-authored-by: Ricky Stewart <[email protected]>
@knz knz mentioned this issue Jul 14, 2021
Closed
craig bot pushed a commit that referenced this issue Jul 15, 2021
@knz @rickystewart
Merge #67489 #67669
0baabb0 
67489: clisqlclient: trim down dependencies r=otan a=knz

Fixes #67482.
Informs  #67470.


67669: bazel: upgrade gazelle to latest r=rail a=rickystewart

This helps to avoid bazel-contrib/rules_go#2479.

Release note: None

Co-authored-by: Raphael 'kena' Poss <[email protected]>
Co-authored-by: Ricky Stewart <[email protected]>
@knz knz mentioned this issue Jul 18, 2021
Merged
@craig craig bot closed this as completed in 944d044 Jul 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-cli-client CLI commands that pertain to using SQL features C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-server-and-security DB Server & Security
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cockroach-sql: reduce the size of the SQL-only executable knz/cockroach
1 participant
@knz