Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker/lite: +zstd dep #11997

Merged
merged 1 commit into from
Jan 26, 2023
Merged

docker/lite: +zstd dep #11997

merged 1 commit into from
Jan 26, 2023

Conversation

maxenglander
Copy link
Collaborator

@maxenglander maxenglander commented Dec 20, 2022

Related to #7802

Description

I would like to have zstd CLI available in Docker lite images so that when using VTOP-deployed Vitess I can take advantage of the fact that zstd external is apparently much faster (see #11994) than the native Go library.

Image sizes

docker inspect <image> | jq -r .[].Size

Target 7fc1b48 This PR
lite_mysql57 1230530599 1231544936 +1014337 (+0.082%)
lite_mysql80 1222034851 1223049180 +1014329 (+0.083%)
lite_percona57 1151917943 1152923524 +1005581 (+0.087%)
lite_percona80 1818075338 1819080907 +1005569 (+0.0553%)
lite_testing 797503720 798774969 +1271249 (+0.159%)
lite_ubi7.mysql57 ⚠️ ⚠️ ⚠️
lite_ubi7.mysql80 ⚠️ ⚠️ ⚠️
lite_ubi7.percona57 1478300308 1478790334 +490026 (+0.033%)
lite_ubi7.percona80 1671904530 1672394556 +490026 (+0.029%)
lite_ubi8.arm64.mysql80 1399800336 1399454317 -346019 (-0.024%)
lite_ubi8.mysql80 ⚠️ ⚠️ ⚠️

Not sure if there is a better way to find out these image sizes, but here's what I did.

Since I'm building these on my Mac I need to use docker buildx to build linux/amd64 images. I made a small change to Makefile to make this easier: #12081

Then I ran GOOS=linux GOARCH=amd64 make docker_<target> on main (7fc1b48) and this branch. Except for lite_ubi8.arm64.mysql80 which with just plain make.

⚠️ I'm not able to build the ubi7.mysql57, ubi7.mysql80 or ubi8.mysql80 images locally though, I get this error:

#0 11.87 Total download size: 11 k
#0 11.87 Downloading packages:
#0 11.88 Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
#0 13.06 warning: /var/cache/yum/x86_64/7Server/mysql80-community/packages/mysql80-community-release-el7-7.noarch.rpm: Header V4 RSA/SHA256 Signature, key ID 3a79bd29: NOKEY
#0 13.07 Public key for mysql80-community-release-el7-7.noarch.rpm is not installed
#0 13.17 Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql
#0 14.61
#0 14.61
#0 14.61 The GPG keys listed for the "MySQL 8.0 Community Server" repository are already installed but they are not correct for this package.
#0 14.61 Check that the correct key URLs are configured for this repository.
#0 14.61
#0 14.61
#0 14.61  Failing package is: mysql80-community-release-el7-7.noarch
#0 14.61  GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql
#0 14.61
------
ERROR: failed to solve: executor failed running [/bin/sh -c yum update -y --setopt=alwaysprompt=no --setopt=tsflags=nodocs  && yum install -y --setopt=alwaysprompt=no --setopt=tsflags=nodocs bzip2 ca-certificates gnupg libaio libcurl     jemalloc gperftools-libs procps-ng rsync wget openssl hostname curl tzdata make  && yum install -y --setopt=tsflags=nodocs --enablerepo mysql80-community --disablerepo mysql57-community     mysql-community-client mysql-community-server  && mkdir -p /tmp/1  && yum install -y --setopt=alwaysprompt=no --downloadonly --downloaddir=/tmp/1 --enablerepo mysql80-community --disablerepo mysql57-community percona-xtrabackup-80 percona-toolkit  && rpm -Uvh --replacefiles /tmp/1/*rpm  && rm -rf /tmp/1  && yum clean all  && yum clean all --enablerepo mysql80-community --disablerepo mysql57-community  && rm -rf /etc/my.cnf /var/lib/mysql /tmp/gpg /sbin/mysqld-debug]: exit code: 1

Signed-off-by: Max Englander <[email protected]>
@vitess-bot
Copy link
Contributor

vitess-bot bot commented Dec 20, 2022

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • If this is a change that users need to know about, please apply the release notes (needs details) label so that merging is blocked unless the summary release notes document is included.
  • If a test is added or modified, there should be a documentation on top of the test to explain what the expected behavior is what the test does.

If a new flag is being introduced:

  • Is it really necessary to add this flag?
  • Flag names should be clear and intuitive (as far as possible)
  • Help text should be descriptive.
  • Flag names should use dashes (-) as word separators rather than underscores (_).

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow should be required, the maintainer team should be notified.

Bug fixes

  • There should be at least one unit or end-to-end test.
  • The Pull Request description should include a link to an issue that describes the bug.

Non-trivial changes

  • There should be some code comments as to why things are implemented the way they are.

New/Existing features

  • Should be documented, either by modifying the existing documentation or creating new documentation.
  • New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • vtctl command output order should be stable and awk-able.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from VTop, if used there.

@maxenglander maxenglander marked this pull request as draft December 21, 2022 19:52
@maxenglander
Copy link
Collaborator Author

Hi @deepthi thank you for the approve! I think I was a bit pre-mature requesting a review on this one, forgot when I opened this how vtbackup works, and so putting zstd in the vttablet image means it might not be available to vtbackup. Need to re-think this.

@deepthi
Copy link
Member

deepthi commented Jan 9, 2023

If we decide to resurrect this, the before and after image sizes should be documented in the PR.

@maxenglander
Copy link
Collaborator Author

Hey @deepthi I do plan to resurrect this and when I do will document the image sizes. Thanks!

@maxenglander maxenglander marked this pull request as ready for review January 16, 2023 17:51
@deepthi
Copy link
Member

deepthi commented Jan 26, 2023

I'm going to go ahead and merge this now. The file size diffs look great.

@deepthi deepthi merged commit 55168f5 into vitessio:main Jan 26, 2023
@deepthi deepthi deleted the maxeng-zstd-dep branch January 26, 2023 05:55
dbussink pushed a commit that referenced this pull request Jan 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants