build: make builds fully reproducible across systems #881
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This uses the more reliable way to restore the file in git after a build, preventing differences in line endings to cause the file to be seen as dirty in some situations.
This update modifies the release.sh script to ensure that release files are reproducible, generating identical hashes when rebuilt. Previously, the script did not set fixed timestamps for the files, resulting in different hashes across rebuilds. Additionally, the release.sh script has been updated to align more closely with the behavior of the lnd release script, ensuring consistency and coherence across repositories for users.
This was referenced Oct 23, 2024
dstadulis
previously requested changes
Oct 23, 2024
8 tasks
|
Thanks a lot for this @guggero 🔥, a really nice addition! Hmmm when running this I'm unfortunately getting hash mismatches for all "lightning-terminal-darwin-ENV-v0.0.0-reproducible-build-test.tar.gz" & "lightning-terminal-darwin-ENV-v0.0.0-reproducible-build-test/litd" builds. Here's my result: I'm also getting this during the release process: |
|
The |
|
@ViktorTigerstrom
|
Because the "make release" command collided with the release.sh script, make thought it needed to do something with that file. We fix that problem in two ways: We move the file into the scripts sub folder and also tell make that all our defined goals don't correspond to folders or files.
To make sure we use the same Golang version everywhere, we add a new linter step with scripts copied from lnd.
This commit updates `scripts/release.sh` to include a check for the correct Go version before executing the release build. This ensures that the release binaries are built with the specified Go version, maintaining consistency and integrity for developer signatures.
|
@dstadulis and @ViktorTigerstrom I updated the digests in the commit body and also pushed a new tag: |
guggero
force-pushed
the
reproducible-build
branch
from
b63b165 to
5ac1234
Compare
We dockerize the app build and use that for both the docker build _AND_ the normal build to make sure the static content is fully reproducible across different systems.
This was discovered while debugging the reproducible build. We used the wrong group ID. And the environment variables aren't super portable so we use the correct command to get the numeric user and group IDs.
guggero
force-pushed
the
reproducible-build
branch
from
5ac1234 to
ea33903
Compare
|
We debugged the MacOS issues and found a solution. Pushed a new tag |
|
Hashes are matching on my end 🎉🔥!! |
|
All hashes matching on my side 🏅 |
ellemouton
approved these changes
Oct 24, 2024
| @@ -8,7 +8,7 @@ | |||
| "start": "BROWSER=none react-scripts start", | |||
| "develop": "REACT_APP_USE_SAMPLE_DATA=true yarn start", | |||
| "build": "react-scripts build", | |||
| "postbuild": "echo '# Keep directory in git.' > build/.gitkeep", | |||
| "postbuild": "git restore build/.gitkeep", | |||
ViktorTigerstrom
approved these changes
Oct 24, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes a couple of issues with the release build process and also adds a couple of nice-to-haves that we added in other projects.
Replaces #880.
Replaces #852 (changes included).
Testing
Linux
Make sure you're using
Go 1.22.6!Please run:
MacOS
Please run:
Target
You should get the following manifest if successful: