Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: update x/upgrade to handle additional instructions #10032

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/core/proto-docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -688,10 +688,12 @@
- [Service](#cosmos.tx.v1beta1.Service)

- [cosmos/upgrade/v1beta1/upgrade.proto](#cosmos/upgrade/v1beta1/upgrade.proto)
- [Asset](#cosmos.upgrade.v1beta1.Asset)
- [CancelSoftwareUpgradeProposal](#cosmos.upgrade.v1beta1.CancelSoftwareUpgradeProposal)
- [ModuleVersion](#cosmos.upgrade.v1beta1.ModuleVersion)
- [Plan](#cosmos.upgrade.v1beta1.Plan)
- [SoftwareUpgradeProposal](#cosmos.upgrade.v1beta1.SoftwareUpgradeProposal)
- [UpgradeInstructions](#cosmos.upgrade.v1beta1.UpgradeInstructions)

- [cosmos/upgrade/v1beta1/query.proto](#cosmos/upgrade/v1beta1/query.proto)
- [QueryAppliedPlanRequest](#cosmos.upgrade.v1beta1.QueryAppliedPlanRequest)
Expand Down Expand Up @@ -9831,6 +9833,23 @@ Service defines a gRPC service for interacting with transactions.



<a name="cosmos.upgrade.v1beta1.Asset"></a>

### Asset



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| `platform` | [string](#string) | | Platform identifier. It's composed from OS and CPU architecture. Example: "linux/amd64" |
| `url` | [string](#string) | | URL to a script or binary to download. Should be related to the UpgradeInstructions pre_run or post_run commands. If multiple files are needed, then they should be packed in a gzip archive. |
| `checksum` | [string](#string) | | Checksum is a sha256 hex encoded checksum of an artifact referenced by the URL. |






<a name="cosmos.upgrade.v1beta1.CancelSoftwareUpgradeProposal"></a>

### CancelSoftwareUpgradeProposal
Expand Down Expand Up @@ -9877,6 +9896,7 @@ Plan specifies information about a planned upgrade and when it should occur.
| `height` | [int64](#int64) | | The height at which the upgrade must be performed. Only used if Time is not set. |
| `info` | [string](#string) | | Any application specific upgrade info to be included on-chain such as a git commit that validators could automatically upgrade to |
| `upgraded_client_state` | [google.protobuf.Any](#google.protobuf.Any) | | **Deprecated.** Deprecated: UpgradedClientState field has been deprecated. IBC upgrade logic has been moved to the IBC module in the sub module 02-client. If this field is not empty, an error will be thrown. |
| `Upgrade` | [UpgradeInstructions](#cosmos.upgrade.v1beta1.UpgradeInstructions) | | Optional: Upgrade contains additional instructions for the devops or a hypervisor. App specific instructions are handled by the `info` attribute. Here we provide information such as pre-upgrade or post-upgrade commands. |



Expand All @@ -9900,6 +9920,24 @@ upgrade.




<a name="cosmos.upgrade.v1beta1.UpgradeInstructions"></a>

### UpgradeInstructions



| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| `pre_run` | [string](#string) | | If not empty, a shell command to be run by the upgrade manager or a hypervisor after shutting down the app and before running a new node. |
| `post_run` | [string](#string) | | If not empty, a shell command to be run by the upgrade manager or a hypervisor after shutting down the app and after running a new app. |
| `assets` | [Asset](#cosmos.upgrade.v1beta1.Asset) | repeated | List of required assets to download. This follows the cosmovisor structure. SHOULD have only one entry per platform. |
| `description` | [string](#string) | | Description contains additional information about the upgrade process. Can reference an external resource. |





<!-- end messages -->

<!-- end enums -->
Expand Down
33 changes: 33 additions & 0 deletions proto/cosmos/upgrade/v1beta1/upgrade.proto
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,39 @@ message Plan {
// If this field is not empty, an error will be thrown.
google.protobuf.Any upgraded_client_state = 5
[deprecated = true];

// Optional: Upgrade contains additional instructions for the devops or a hypervisor.
// App specific instructions are handled by the `info` attribute. Here we provide
// information such as pre-upgrade or post-upgrade commands.
UpgradeInstructions Upgrade = 6;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you think about a better name for this attribute, please share your proposals. I was also thinking about Setup, Preparation ...
We should not confuse with the Info field which is already overloaded (both in meaning and usage).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering it contains both pre and post upgrade info, UpgradeInstructions sounds good.

Comment on lines +44 to +47
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might actually be desirable to extract the platform field all the way out to the UpgradeInstructions message and make the UpgradeInstructions Upgrade a repeated field with a comment that it SHOULD have only one entry per platform. This would allow for different pre/post-run commands that can be tailored for specific environments.

Also, in order to help minimize on-chain data, should we throw this in a oneof and allow a url to be provided in place of the full structure here? The idea is that the url should return the same structure, but really, for chains that don't use Cosmovisor, it could return whatever they want. If the UpgradeInstructions are provided directly, they'll be serialized as json and added to the upgrade-info.json file in the upgrade field. If a url is provided, that url will be added to the upgrade-info.json file as upgrade_url.

Lastly, I feel like this field would be better named instructions. The proto file is named upgrade.proto and so is the module for it. The message the field is in is Plan but is sometimes called an "upgrade plan." The info field is sometimes called "upgrade info." Basically, the word "upgrade" is already used in a few ways and having a field with that name can easily cause more confusion.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, in order to help minimize on-chain data, should we throw this in a oneof and allow a url to be provided in place of the full structure here? The idea is that the url should return the same structure, but really, for chains that don't use Cosmovisor, it could return whatever they want. If the UpgradeInstructions are provided directly, they'll be serialized as json and added to the upgrade-info.json file in the upgrade field. If a url is provided, that url will be added to the upgrade-info.json file as upgrade_url.

I was thinking about this part a bit further with extra consideration for the first upgrade using new definitions.

If we allow the a url in the info field to return the newly defined UpgradeInstructions structure (e.g. as JSON), then as long as the nodes are running updated versions of Cosmovisor, an update can be done using these new proto fields without needing the chain software to already be using the new version. Otherwise, these new fields cannot be used for the next upgrade; they can only be used for upgrades after the one that adds these fields to the protos.

The downside is the further overloading of the info field.

}

message UpgradeInstructions {
option (gogoproto.equal) = true;

// If not empty, a shell command to be run by the upgrade manager or a hypervisor after shutting down
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be more clear as to the execution environment of these commands:

  • which shell? Preferably we wouldn't allow people to write full on shell scripts, but if so we should be clear about this expectation.
  • what's the working directory should this be configurable?
  • should we provide some way to pass environment variables to commands?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can drop "shell" and just say "command"?
It's up to the app developers to make a system requirements.

// the app and before running a new node.
string pre_run = 1;
// If not empty, a shell command to be run by the upgrade manager or a hypervisor after shutting down
// the app and after running a new app.
string post_run = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know any use case for this? Also how would you deal with the errors on the post upgrade? The application is already running in theory but your post upgrade script fails. What would you do?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One possible use-case is reporting.
But maybe it's not needed. I'm happy to remove it if you think it may confuse people or if it will be abused.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this post-run script can be used for non-critical activities, so we don't need to handle errors. We can use it for removing backups, alerting users etc

// List of required assets to download. This follows the cosmovisor structure.
// SHOULD have only one entry per platform.
repeated Asset assets = 3;
// Description contains additional information about the upgrade process. Can reference an external resource.
string description = 4;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove this description field.

If someone provides UpgradeInstructions in the Plan then the info field can then be used for this. If UpgradeInstructions aren't included in the Plan then this field wouldn't be there to populate.

I guess that if the UpgradeInstructions are included, then the info field is ignored (as far as automated processing goes).

}

message Asset {
option (gogoproto.equal) = true;

// Platform identifier. It's composed from OS and CPU architecture. Example: "linux/amd64"
string platform = 1;
// URL to a script or binary to download. Should be related to the UpgradeInstructions pre_run or post_run
// commands. If multiple files are needed, then they should be packed in a gzip archive.
string url = 2;
// Checksum is a sha256 hex encoded checksum of an artifact referenced by the URL.
string checksum = 3;
Copy link
Contributor

@spoo-bar spoo-bar Sep 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we use go-getter to download the files in cosmovisor, it automatically checks for the checksum from the download url as ./foo.sh?checksum=b7d96c89d09d9e204f5fedc4d5d55b21. So a seperate checksum property might not be needed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intention was to make it universal. Proto files should not assume that we are using the go implementation or cosmovisor. We could put the checksum in the URL, but then we would need to explain it in a comment and require it when registering a new proposal.

}
Comment on lines +66 to 76
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the addition of the pre_run and post_run fields, there will be a need to download those as well. By being able to provide a name for assets, users won't be constrained to only providing archives when they need to include a special pre_run script.

I feel like adding a string name field to the Asset is the most versatile solution. We'd state that entries in repeated Asset assets SHOULD be unique by name and platform (as opposed to just platform).

Alternatively, a hierarchy could be used, but there's a couple ways to do that, either name -> platforms or platform -> names. There's good arguments for either way though. But I feel like leaving it flat allows users to handle it the way that makes the most sense for them.

Cosmovisor Usage

Assets would be downloaded in the order provided but only if there is a Source entry that matches the system's platform (or there's a platform = any entry for it). Basically, we'd loop through the assets, if the platform matches the system, then it's downloaded. Or if the platform is any and the name is one that hasn't yet been downloaded this upgrade, then it'll be downloaded.

If the url returns an archive, then the name becomes a directory name. Otherwise it's a filename. Either way, the full path to the downloaded result would be {DAEMON_HOME}/{upgrade name}/{name}. Sub directories would be allowed in the name, but we'd need to discuss whether or not to allow it to contain ../.

I'd make a couple special cases for the name value:

  1. If the name is empty, and the url returns an archive, it is unpacked and becomes {DAEMON_HOME}/{upgrade name} (similar to current behavior). If the name is empty and the url returns a non-archive file, it'll end up as {DAEMON_HOME}/{upgrade name}/bin/{DAEMON_NAME}.
  2. If the name is . then the url will need to return an archive and it'll be unpacked into {DAEMON_HOME}/{upgrade name}. If the name is . and the url returns something other than an archive, an error will be given and the upgrade/download process will be interrupted.

We'd probably want to maintain the current functionality where, after doing the downloads, if {DAEMON_HOME}/{upgrade name}/bin/{DAEMON_NAME} doesn't exist, but {DAEMON_HOME}/{upgrade name}/{DAEMON_NAME} does, the latter is copied to the former.


// SoftwareUpgradeProposal is a gov Content type for initiating a software
Expand Down
34 changes: 25 additions & 9 deletions x/upgrade/client/cli/tx.go
Original file line number Diff line number Diff line change
@@ -1,19 +1,24 @@
package cli

import (
"fmt"

"github.com/spf13/cobra"

"github.com/cosmos/cosmos-sdk/client"
"github.com/cosmos/cosmos-sdk/client/tx"
"github.com/cosmos/cosmos-sdk/codec"
sdk "github.com/cosmos/cosmos-sdk/types"
"github.com/cosmos/cosmos-sdk/types/errors"
"github.com/cosmos/cosmos-sdk/x/gov/client/cli"
gov "github.com/cosmos/cosmos-sdk/x/gov/types"
"github.com/cosmos/cosmos-sdk/x/upgrade/types"
)

const (
FlagUpgradeHeight = "upgrade-height"
FlagUpgradeInfo = "upgrade-info"
FlagUpgradeHeight = "upgrade-height"
FlagUpgradeInfo = "upgrade-info"
FlagUpgradeInstructions = "upgrade-instructions"
)

// GetTxCmd returns the transaction commands for this module
Expand All @@ -29,19 +34,18 @@ func GetTxCmd() *cobra.Command {
// NewCmdSubmitUpgradeProposal implements a command handler for submitting a software upgrade proposal transaction.
func NewCmdSubmitUpgradeProposal() *cobra.Command {
cmd := &cobra.Command{
Use: "software-upgrade [name] (--upgrade-height [height]) (--upgrade-info [info]) [flags]",
Use: fmt.Sprintf("software-upgrade <name> --%s <height> [--%s <info>] [--%s <instructions>] [flags]", FlagUpgradeHeight, FlagUpgradeInfo, FlagUpgradeInstructions),
Args: cobra.ExactArgs(1),
Short: "Submit a software upgrade proposal",
Long: "Submit a software upgrade along with an initial deposit.\n" +
"Please specify a unique name and height for the upgrade to take effect.\n" +
"You may include info to reference a binary download link, in a format compatible with: https://github.com/cosmos/cosmos-sdk/tree/master/cosmovisor",
"You must use a unique name and height for the upgrade to take effect.",
RunE: func(cmd *cobra.Command, args []string) error {
clientCtx, err := client.GetClientTxContext(cmd)
if err != nil {
return err
}
name := args[0]
content, err := parseArgsToContent(cmd, name)
content, err := parseArgsToContent(clientCtx.Codec, cmd, name)
if err != nil {
return err
}
Expand Down Expand Up @@ -70,7 +74,8 @@ func NewCmdSubmitUpgradeProposal() *cobra.Command {
cmd.Flags().String(cli.FlagDescription, "", "description of proposal")
cmd.Flags().String(cli.FlagDeposit, "", "deposit of proposal")
cmd.Flags().Int64(FlagUpgradeHeight, 0, "The height at which the upgrade must happen")
cmd.Flags().String(FlagUpgradeInfo, "", "Optional info for the planned upgrade such as commit hash, etc.")
cmd.Flags().String(FlagUpgradeInfo, "", "Optional info for the planned upgrade such as commit hash or a binary download link, in a format compatible with: https://github.com/cosmos/cosmos-sdk/tree/master/cosmovisor")
cmd.Flags().String(FlagUpgradeInstructions, "", `Optional, not app specific download instructions. It set, it must have the UpgradeInstructions JSON format. Example 1: '{"pre_run": "./upgrade-v1", "assets": [{"platform": "linux/amd64", "url": "https://ipfs.io/ipfs/Qme7ss...", checksum: "0cdbd28e71a2e37830dabee99adffb68a568488f6fcfcf051217984151b769ee"}]}' Example 2: '{"pre_run": "simd pre-upgrade"}'`)

return cmd
}
Expand Down Expand Up @@ -129,7 +134,7 @@ func NewCmdSubmitCancelUpgradeProposal() *cobra.Command {
return cmd
}

func parseArgsToContent(cmd *cobra.Command, name string) (gov.Content, error) {
func parseArgsToContent(cdc codec.Codec, cmd *cobra.Command, name string) (gov.Content, error) {
title, err := cmd.Flags().GetString(cli.FlagTitle)
if err != nil {
return nil, err
Expand All @@ -149,8 +154,19 @@ func parseArgsToContent(cmd *cobra.Command, name string) (gov.Content, error) {
if err != nil {
return nil, err
}
upgradeInstructions, err := cmd.Flags().GetString(FlagUpgradeInstructions)
if err != nil {
return nil, err
}
var instructions *types.UpgradeInstructions
if upgradeInstructions != "" {
instructions = new(types.UpgradeInstructions)
if err = cdc.UnmarshalJSON([]byte(upgradeInstructions), instructions); err != nil {
return nil, errors.ErrJSONUnmarshal.Wrapf("Can't parse upgrade-instructions [%v]", err)
}
}

plan := types.Plan{Name: name, Height: height, Info: info}
plan := types.Plan{Name: name, Height: height, Info: info, Upgrade: instructions}
content := types.NewSoftwareUpgradeProposal(title, description, plan)
return content, nil
}
3 changes: 2 additions & 1 deletion x/upgrade/types/plan.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ func (p Plan) String() string {
return fmt.Sprintf(`Upgrade Plan
Name: %s
%s
Info: %s.`, p.Name, due, p.Info)
Info: %s
Instructions: %s.`, p.Name, due, p.Info, p.Upgrade)
}

// ValidateBasic does basic validation of a Plan
Expand Down
Loading