Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build: add support to build linux/mac arm64 #1658

Closed
wants to merge 1 commit into from
Closed

build: add support to build linux/mac arm64 #1658

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions mage/releases/build.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ var (
runtimeArch = "amd64"
runtimePlatform = "linux"
supportedClientGOOS = []string{"linux", "darwin", "windows"}
supportedClientArch = []string{"amd64", "arm64"}
)

func getLDFLAGS(pkg string) string {
Expand Down Expand Up @@ -73,10 +74,17 @@ func XBuild(pkg string, name string, binDir string, goos string, goarch string)
func XBuildAll(pkg string, name string, binDir string) {
var g errgroup.Group
for _, goos := range supportedClientGOOS {
goos := goos
g.Go(func() error {
return XBuild(pkg, name, binDir, goos, "amd64")
})
for _, arch := range supportedClientArch {
goos := goos
arch := arch
// skip windows arm64
if arch == "arm64" && goos == "windows" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Go 1.17 we should be able to remove this skip here. Can you add a comment about that and link to the issue for Go windows/arm64 support? That way we are more likely to remember. 😀

golang/go#36439

continue
}
g.Go(func() error {
return XBuild(pkg, name, binDir, goos, arch)
})
}
}

mgx.Must(g.Wait())
Expand Down
38 changes: 38 additions & 0 deletions scripts/install/install-linux-arm64.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of duplicating scripts for each os+arch, since I think the only difference is the arch name, I'd like for us to look at using go env GOARCH to determine the arch value and then use that as a variable. Then we just have to maintain 3 install scripts.

set -xeuo pipefail

# Installs the porter CLI for a single user.
# PORTER_HOME: Location where Porter is installed (defaults to ~/.porter).
# PORTER_MIRROR: Base URL where Porter assets, such as binaries and atom feeds, are downloaded. This lets you
# setup an internal mirror.
# PORTER_PERMALINK: The version of Porter to install, such as vX.Y.Z, latest or canary.
# PKG_PERMALINK: The version of mixins and plugins to install, such as latest or canary.

export PORTER_HOME=${PORTER_HOME:-~/.porter}
export PORTER_MIRROR=${PORTER_MIRROR:-https://cdn.porter.sh}
PORTER_PERMALINK=${PORTER_PERMALINK:-latest}
PKG_PERMALINK=${PKG_PERMALINK:-latest}

echo "Installing porter@$PORTER_PERMALINK to $PORTER_HOME from $PORTER_MIRROR"

mkdir -p $PORTER_HOME/runtimes

curl -fsSLo $PORTER_HOME/porter $PORTER_MIRROR/$PORTER_PERMALINK/porter-linux-arm64
chmod +x $PORTER_HOME/porter
cp $PORTER_HOME/porter $PORTER_HOME/runtimes/porter-runtime
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested out using porter-linux-arm64 installed to ~/.porter/runtimes/porter-runtime? Like built a bundle and run it?

I'm not sure that will work without further changes because we build bundles with a linux amd64 invocation image.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for the delay to reply. No, but I will test, just need to setup a linux-arm64 machine, thanks for the review and feedback!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just checking if you tried this out yet and how it went?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@carolynvs sorry :( i was busy with company work and did not have time, I will take some days off and when I comeback i will get this back on track. I will be out starting today till next Tuesday and on Wednesday I will run all the tests on this.

Copy link
Member

@carolynvs carolynvs Jul 27, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to apologize! I just ping issues after it's been a while to see how things are going and make sure that people aren't stuck. Take your time and I appreciate you looking into this. 👍

Copy link
Author

@cpanato cpanato Aug 20, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

published the exec binary but looks like it tried to get the linux/amd64

ubuntu@ip-172-31-10-226:~/porter/porter$ porter mixin install exec --url https://github.com/cpanato/testing-ci-providers/releases/download/  --version v0.0.51
Error: bad status returned when downloading https://github.com/cpanato/testing-ci-providers/releases/download/v0.0.51/exec-linux-amd64 (404) 404 Not Found

which I did not publish to check if will download the right one

but my question is (I'm a newbie here, so apolagises)
the mixin exec needs to be in the same arch as we are running? if so we might need to change the code around https://github.com/getporter/porter/blob/main/pkg/pkgmgmt/client/install.go#L91

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

made the change and then when I run it again apparently it works

ubuntu@ip-172-31-10-226:~/porter/porter$ porter mixin install exec --url https://github.com/cpanato/testing-ci-providers/releases/download/  --version v0.0.51
installed exec mixin v0.38.3-11-g6f9ae84e (6f9ae84e)

Copy link
Author

@cpanato cpanato Aug 20, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh! we will need to build the other mixins for the others like kubernetes/helm ... I get it now

sorry doing some rubber duck here :)

if this is something we want, I'm willing to open the changes in the other repos

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that would be great! 💯

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, will work on those things 😃

echo Installed `$PORTER_HOME/porter version`

$PORTER_HOME/porter mixin install exec --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install kubernetes --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install helm --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install arm --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install terraform --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install az --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install aws --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install gcloud --version $PKG_PERMALINK

$PORTER_HOME/porter plugin install azure --version $PKG_PERMALINK

echo "Installation complete."
echo "Add porter to your path by adding the following line to your ~/.profile and open a new terminal:"
echo "export PATH=\$PATH:~/.porter"
39 changes: 39 additions & 0 deletions scripts/install/install-mac-arm64.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
set -xeuo pipefail

# Installs the porter CLI for a single user.
# PORTER_HOME: Location where Porter is installed (defaults to ~/.porter).
# PORTER_MIRROR: Base URL where Porter assets, such as binaries and atom feeds, are downloaded. This lets you
# setup an internal mirror.
# PORTER_PERMALINK: The version of Porter to install, such as vX.Y.Z, latest or canary.
# PKG_PERMALINK: The version of mixins and plugins to install, such as latest or canary.

export PORTER_HOME=${PORTER_HOME:-~/.porter}
export PORTER_MIRROR=${PORTER_MIRROR:-https://cdn.porter.sh}
PORTER_PERMALINK=${PORTER_PERMALINK:-latest}
PKG_PERMALINK=${PKG_PERMALINK:-latest}

echo "Installing porter@$PORTER_PERMALINK to $PORTER_HOME from $PORTER_MIRROR"

mkdir -p $PORTER_HOME/runtimes

curl -fsSLo $PORTER_HOME/porter $PORTER_MIRROR/$PORTER_PERMALINK/porter-darwin-arm64
curl -fsSLo $PORTER_HOME/runtimes/porter-runtime $PORTER_MIRROR/$PORTER_PERMALINK/porter-linux-arm64
chmod +x $PORTER_HOME/porter
chmod +x $PORTER_HOME/runtimes/porter-runtime
echo Installed `$PORTER_HOME/porter version`

$PORTER_HOME/porter mixin install exec --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install kubernetes --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install helm --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install arm --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install terraform --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install az --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install aws --version $PKG_PERMALINK
$PORTER_HOME/porter mixin install gcloud --version $PKG_PERMALINK

$PORTER_HOME/porter plugin install azure --version $PKG_PERMALINK

echo "Installation complete."
echo "Add porter to your path by adding the following line to your ~/.bash_profile or ~/.zprofile and open a new terminal:"
echo "export PATH=\$PATH:~/.porter"
2 changes: 2 additions & 0 deletions scripts/prep-install-scripts.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,6 @@ ls -R bin

sed -e "s|PORTER_PERMALINK:-latest|PORTER_PERMALINK:-$PORTER_PERMALINK|g" -e "s|PKG_PERMALINK:-latest|PKG_PERMALINK:-$PERMALINK|" scripts/install/install-mac.sh > bin/$VERSION/install-mac.sh
sed -e "s|PORTER_PERMALINK:-latest|PORTER_PERMALINK:-$PORTER_PERMALINK|g" -e "s|PKG_PERMALINK:-latest|PKG_PERMALINK:-$PERMALINK|" scripts/install/install-linux.sh > bin/$VERSION/install-linux.sh
sed -e "s|PORTER_PERMALINK:-latest|PORTER_PERMALINK:-$PORTER_PERMALINK|g" -e "s|PKG_PERMALINK:-latest|PKG_PERMALINK:-$PERMALINK|" scripts/install/install-mac-arm64.sh > bin/$VERSION/install-mac-arm64.sh
sed -e "s|PORTER_PERMALINK:-latest|PORTER_PERMALINK:-$PORTER_PERMALINK|g" -e "s|PKG_PERMALINK:-latest|PKG_PERMALINK:-$PERMALINK|" scripts/install/install-linux-arm64.sh > bin/$VERSION/install-linux-arm64.sh
sed -e "s|PORTER_PERMALINK='latest'|PORTER_PERMALINK='$PORTER_PERMALINK'|g" -e "s|PKG_PERMALINK='latest'|PKG_PERMALINK='$PERMALINK'|g" scripts/install/install-windows.ps1 > bin/$VERSION/install-windows.ps1
13 changes: 13 additions & 0 deletions scripts/test/test-linux-arm64-install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/usr/bin/env bash

set -xeuo pipefail

export PATH=$PATH:~/.porter

PORTER_PERMALINK=canary ./scripts/install/install-linux-arm64.sh
porter list

PORTER_PERMALINK=v0.23.0-beta.1 ./scripts/install/install-linux-arm64.sh
porter version | grep v0.23.0-beta.1

PORTER_PERMALINK=latest ./scripts/install/install-linux-arm64.sh
13 changes: 13 additions & 0 deletions scripts/test/test-mac-arm64-install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/usr/bin/env bash

set -xeuo pipefail

export PATH=$PATH:~/.porter

PORTER_PERMALINK=canary ./scripts/install/install-mac-arm64.sh
porter list

PORTER_PERMALINK=v0.23.0-beta.1 ./scripts/install/install-mac-arm64.sh
porter version | grep v0.23.0-beta.1

PORTER_PERMALINK=latest ./scripts/install/install-mac-arm64.sh