From a262fd847ad3fa4f37bf60177df7bda7026038c1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dani=C3=ABl=20van=20Eeden?= Date: Tue, 4 Jan 2022 11:48:22 +0100 Subject: [PATCH 1/4] mirror: Fix typos and naming --- doc/user/mirrors.md | 85 +++++++++++++++------------------- pkg/repository/clone_mirror.go | 18 +++---- 2 files changed, 46 insertions(+), 57 deletions(-) diff --git a/doc/user/mirrors.md b/doc/user/mirrors.md index 3a3ebc3081..840e685eaf 100644 --- a/doc/user/mirrors.md +++ b/doc/user/mirrors.md @@ -1,55 +1,44 @@ # Build a private mirror -When building a private cloud, it is common to use an isolated network environment where the official mirror of TiUP is not accessible, so we provide a solution for building a private mirror, which is mainly implemented by the mirros component, which can also be used for offline deployment. +When building a private cloud, it is common to use an isolated network environment where the official mirror of TiUP is not accessible, so we provide a solution for building a private mirror, which is mainly implemented by the mirrors component, which can also be used for offline deployment. ## Mirrors component introduction First, let's look at the `mirrors' help file. ```bash -$ tiup mirrors --help -Starting component `mirrors`: /Users/joshua/.tiup/components/mirrors/v0.0.1/mirrors -Build a local mirrors and download all selected components +$ tiup mirror --help +The 'mirror' command is used to manage a component repository for TiUP, you can use +it to create a private repository, or to add new component to an existing repository. +The repository can be used either online or offline. +It also provides some useful utilities to help managing keys, users and versions +of components or the repository itself. Usage: - tiup mirrors [global-version] [flags] - -Examples: - tiup mirrors local-path --arch amd64,arm --os linux,darwin # Specify the architectures and OSs - tiup mirrors local-path --full # Build a full local mirrors - tiup mirrors local-path --tikv v4 # Specify the version via prefix - tiup mirrors local-path --tidb all --pd all # Download all version for specific component - -Flags: - --overwrite Overwrite the exists tarball - -f, --full Build a full mirrors repository - -a, --arch strings Specify the downloading architecture (default [amd64]) - -o, --os strings Specify the downloading os (default [linux,darwin]) - --tidb strings Specify the versions for component tidb - --tikv strings Specify the versions for component tikv - --pd strings Specify the versions for component pd - --playground strings Specify the versions for component playground - --client strings Specify the versions for component client - --prometheus strings Specify the versions for component prometheus - --package strings Specify the versions for component package - --grafana strings Specify the versions for component grafana - --alertmanager strings Specify the versions for component alertmanager - --blackbox_exporter strings Specify the versions for component blackbox_exporter - --node_exporter strings Specify the versions for component node_exporter - --pushgateway strings Specify the versions for component pushgateway - --tiflash strings Specify the versions for component tiflash - --drainer strings Specify the versions for component drainer - --pump strings Specify the versions for component pump - --cluster strings Specify the versions for component cluster - --mirrors strings Specify the versions for component mirrors - --bench strings Specify the versions for component bench - --insight strings Specify the versions for component insight - --doc strings Specify the versions for component doc - --ctl strings Specify the versions for component ctl - -h, --help help for tiup + tiup mirror [flags] + +Available Commands: + init Initialize an empty repository + sign Add signatures to a manifest file + genkey Generate a new key pair + clone Clone a local mirror from remote mirror and download all selected components + merge Merge two or more offline mirror + publish Publish a component + show Show mirror address + set Set mirror address + modify Modify published component + renew Renew the manifest of a published component. + grant grant a new owner + rotate Rotate root.json + +Global Flags: + --help Help for this command + --skip-version-check Skip the strict version check, by default a version must be a valid SemVer string + +Use "tiup mirror [command] --help" for more information about a command. ``` -Its basic use is `tiup mirrors [global-version] [flags]`, the target-dir is the directory in which the cloned data needs to be placed. global-version is used to quickly set a common version for all components. +Its basic use is `tiup mirror [global-version] [flags]`, the target-dir is the directory in which the cloned data needs to be placed. global-version is used to quickly set a common version for all components. Then this order has very scary dozens of flags and even more later. But there is no need to be intimidated by the number of these flags, which are in fact of four types. @@ -68,16 +57,16 @@ If `--full` is specified, the official image will be cloned intact. ### 3. Platform limitation If you only want to clone packages for a particular platform, you can use `-os` and `-arch` to qualify: -- `tiup mirros ---os=linux` -- Just want to clone amd64 architecture: `tiup mirros --arch=amd64` -- Just want to clone linux/amd64: `tiup mirros --os=linux --arch=amd64` +- `tiup mirror ---os=linux` +- Just want to clone amd64 architecture: `tiup mirror --arch=amd64` +- Just want to clone linux/amd64: `tiup mirror --os=linux --arch=amd64` ### 4. Component version limited If you want to clone only one version of a component and not all versions, use `--=` to qualify, for example " -- Just want to clone the v4 version of tidb: `tiup mirrors --tidb v4` -- Just want to clone the v4 version of tidb, and all versions of tikv: `tiup mirros --tidb v4 --tikv all` -- Clone specific versions of all components that start a cluster: `tiup mirrors v4.0.0-rc` +- Just want to clone the v4 version of tidb: `tiup mirror --tidb v4` +- Just want to clone the v4 version of tidb, and all versions of tikv: `tiup mirror --tidb v4 --tikv all` +- Clone specific versions of all components that start a cluster: `tiup mirror v4.0.0-rc` ## The real thing @@ -86,7 +75,7 @@ If you want to clone only one version of a component and not all versions, use ` For example, if we want to install a v4.0.0-rc TiDB cluster in an isolated environment, we can execute the following command on a machine connected to the extranet to pull the required components: ```bash -tiup mirrors package --os=linux v4.0.0-rc +tiup mirror package --os=linux v4.0.0-rc ``` This command creates a directory called `package` in the current directory that contains the package of components necessary to start a cluster, which is then packaged by the tar command and sent to a central control unit in an isolated environment: @@ -110,7 +99,7 @@ export TIUP_MIRRORS=/path/to/mirror tiup cluster xxx ``` -`/path/to/mirror` is the location of in `tiup mirrors `, or if in /tmp/package: +`/path/to/mirror` is the location of in `tiup mirror `, or if in /tmp/package: ```bash export TIUP_MIRRORS=/tmp/package ``` diff --git a/pkg/repository/clone_mirror.go b/pkg/repository/clone_mirror.go index 975f95a7ac..9965c8be2f 100644 --- a/pkg/repository/clone_mirror.go +++ b/pkg/repository/clone_mirror.go @@ -82,15 +82,15 @@ func CloneMirror(repo *V1Repository, } var ( - initTime = time.Now() - expirsAt = initTime.Add(50 * 365 * 24 * time.Hour) - root = v1manifest.NewRoot(initTime) - index = v1manifest.NewIndex(initTime) + initTime = time.Now() + expiresAt = initTime.Add(50 * 365 * 24 * time.Hour) + root = v1manifest.NewRoot(initTime) + index = v1manifest.NewIndex(initTime) ) // All offline expires at 50 years to prevent manifests stale - root.SetExpiresAt(expirsAt) - index.SetExpiresAt(expirsAt) + root.SetExpiresAt(expiresAt) + index.SetExpiresAt(expiresAt) keys := map[string][]*v1manifest.KeyInfo{} for _, ty := range []string{ @@ -148,7 +148,7 @@ func CloneMirror(repo *V1Repository, } snapshot := v1manifest.NewSnapshot(initTime) - snapshot.SetExpiresAt(expirsAt) + snapshot.SetExpiresAt(expiresAt) componentManifests, err := cloneComponents(repo, components, selectedVersions, tidbClusterVersionMapper, targetDir, tmpDir, options) if err != nil { @@ -156,7 +156,7 @@ func CloneMirror(repo *V1Repository, } for name, component := range componentManifests { - component.SetExpiresAt(expirsAt) + component.SetExpiresAt(expiresAt) fname := fmt.Sprintf("%s.json", name) // TODO: support external owner signedManifests[component.ID], err = v1manifest.SignManifest(component, ownerkeyInfo) @@ -179,7 +179,7 @@ func CloneMirror(repo *V1Repository, // Initialize timestamp timestamp := v1manifest.NewTimestamp(initTime) - timestamp.SetExpiresAt(expirsAt) + timestamp.SetExpiresAt(expiresAt) manifests[v1manifest.ManifestTypeTimestamp] = timestamp manifests[v1manifest.ManifestTypeSnapshot] = snapshot From f467fbc7b20f5c374215e06dcc7183ebb36f8184 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dani=C3=ABl=20van=20Eeden?= Date: Wed, 5 Jan 2022 10:17:15 +0100 Subject: [PATCH 2/4] Update doc/user/mirrors.md Co-authored-by: Allen Zhong --- doc/user/mirrors.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/user/mirrors.md b/doc/user/mirrors.md index 840e685eaf..7b2ea60e6a 100644 --- a/doc/user/mirrors.md +++ b/doc/user/mirrors.md @@ -57,7 +57,7 @@ If `--full` is specified, the official image will be cloned intact. ### 3. Platform limitation If you only want to clone packages for a particular platform, you can use `-os` and `-arch` to qualify: -- `tiup mirror ---os=linux` +- `tiup mirror clone ---os=linux` - Just want to clone amd64 architecture: `tiup mirror --arch=amd64` - Just want to clone linux/amd64: `tiup mirror --os=linux --arch=amd64` From 549a6680a1015e8af47e11d5cec79cb92721567b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dani=C3=ABl=20van=20Eeden?= Date: Thu, 6 Jan 2022 10:10:46 +0100 Subject: [PATCH 3/4] Update based on review from @AstroProfundis --- doc/user/mirrors.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/user/mirrors.md b/doc/user/mirrors.md index 7b2ea60e6a..3878e7bdb2 100644 --- a/doc/user/mirrors.md +++ b/doc/user/mirrors.md @@ -38,7 +38,7 @@ Global Flags: Use "tiup mirror [command] --help" for more information about a command. ``` -Its basic use is `tiup mirror [global-version] [flags]`, the target-dir is the directory in which the cloned data needs to be placed. global-version is used to quickly set a common version for all components. +Its basic use is `tiup mirror clone [global-version] [flags]`, the target-dir is the directory in which the cloned data needs to be placed. global-version is used to quickly set a common version for all components. Then this order has very scary dozens of flags and even more later. But there is no need to be intimidated by the number of these flags, which are in fact of four types. @@ -58,15 +58,15 @@ If `--full` is specified, the official image will be cloned intact. If you only want to clone packages for a particular platform, you can use `-os` and `-arch` to qualify: - `tiup mirror clone ---os=linux` -- Just want to clone amd64 architecture: `tiup mirror --arch=amd64` -- Just want to clone linux/amd64: `tiup mirror --os=linux --arch=amd64` +- Just want to clone amd64 architecture: `tiup mirror clone --arch=amd64` +- Just want to clone linux/amd64: `tiup mirror clone --os=linux --arch=amd64` ### 4. Component version limited If you want to clone only one version of a component and not all versions, use `--=` to qualify, for example " -- Just want to clone the v4 version of tidb: `tiup mirror --tidb v4` -- Just want to clone the v4 version of tidb, and all versions of tikv: `tiup mirror --tidb v4 --tikv all` -- Clone specific versions of all components that start a cluster: `tiup mirror v4.0.0-rc` +- Just want to clone the v4 version of tidb: `tiup mirror clone --tidb v4` +- Just want to clone the v4 version of tidb, and all versions of tikv: `tiup mirror clone --tidb v4 --tikv all` +- Clone specific versions of all components that start a cluster: `tiup mirror clone v4.0.0-rc` ## The real thing @@ -99,7 +99,7 @@ export TIUP_MIRRORS=/path/to/mirror tiup cluster xxx ``` -`/path/to/mirror` is the location of in `tiup mirror `, or if in /tmp/package: +`/path/to/mirror` is the location of in `tiup mirror clone `, or if in /tmp/package: ```bash export TIUP_MIRRORS=/tmp/package ``` From 0c71c350c3239018dfb33df876841e6f493dcb56 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dani=C3=ABl=20van=20Eeden?= Date: Thu, 6 Jan 2022 10:12:08 +0100 Subject: [PATCH 4/4] s/mirrors/mirror/ --- doc/user/mirrors.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/user/mirrors.md b/doc/user/mirrors.md index 3878e7bdb2..3d5a43b9c2 100644 --- a/doc/user/mirrors.md +++ b/doc/user/mirrors.md @@ -1,10 +1,10 @@ # Build a private mirror -When building a private cloud, it is common to use an isolated network environment where the official mirror of TiUP is not accessible, so we provide a solution for building a private mirror, which is mainly implemented by the mirrors component, which can also be used for offline deployment. +When building a private cloud, it is common to use an isolated network environment where the official mirror of TiUP is not accessible, so we provide a solution for building a private mirror, which is mainly implemented by the mirror component, which can also be used for offline deployment. ## Mirrors component introduction -First, let's look at the `mirrors' help file. +First, let's look at the `mirror' help file. ```bash $ tiup mirror --help