_ci_ Build docker containers automatically for butterflynet, calibnet, and debug #9625
Conversation
This is a major refactor of our dockerfile to support the following - The lotus image will remain as is. - The lotus-test image will be deprecated. - The lotus-all-in-one image will also ship with the lotus-seed and lotus-fountain binaries, which it currently does not. - The lotus-all-in-one image will be built in debug, calibnet, and butterflynet modes in addition to the (current) mainnet mode. - The lotus-all-in-one image will now be published regularly using the following tags: - 1.18.0-rc1 , 1.18.0-rc1-debug, 1.18.0-rc1-calibnet, 1.18.0-rc1-butterflynet . This pattern will be used for all lotus releases, including RC releases. - nightly, nightly-debug, nightly-calibnet, nightly-butterflynet - stable, stable-debug, stable-calibnet, stable-butterflynet
ianconsolata
changed the title
Dockerfile
Outdated
|
|
||
| ENV FILECOIN_PARAMETER_CACHE /var/tmp/filecoin-proof-parameters | ||
| ENV LOTUS_PATH /var/lib/lotus | ||
| ENV DOCKER_LOTUS_IMPORT_SNAPSHOT https://fil-chain-snapshots-fallback.s3.amazonaws.com/mainnet/minimal_finality_stateroots_latest.car |
There was a problem hiding this comment.
We should swap this to the new snapshots - https://snapshots.${network}.filops.net/minimal/latest, e.g. https://snapshots.mainnet.filops.net/minimal/latest
(not sure how it's used under the hood, if the url needs to be resolved first, that can be done with - curl -ILs -w %{url_effective} -o /dev/null "https://snapshots.mainnet.filops.net/minimal/latest")
The new snapshots have a slightly more robust setup, and come from cloudflare R2, instead of S3, and are usually much faster to fetch (I can fetch them at 2Gbps in EU, which is all my internet can do, fetching from S3 is 10x slower)
There was a problem hiding this comment.
Ok, right now we don't really support building the lotus image with anything but mainnet, so I will make this default to the mainnet string, and then whoever is running the container can supply a different string if they built it for that network.
I could also extend this image to support supplying a network parameter, instead of the currently method which relies on the docker build command to supply GOFLAGS to build a current image, but that's more work and I think this method is just as configurable.
Dockerfile
Outdated
| @@ -0,0 +1,133 @@ | |||
| ##################################### | |||
| FROM golang:1.18.1-buster AS lotus-builder | |||
There was a problem hiding this comment.
Probably want a newer Go version
There was a problem hiding this comment.
1.18.1 is what we currently build all our builds with (see https://github.com/filecoin-project/lotus/blob/master/.circleci/config.yml#L9 and https://github.com/filecoin-project/lotus/blob/master/GO_VERSION_MIN).
If we want a newer version, we should update that across the board, and probably in a separate PR.
There was a problem hiding this comment.
Discussed in standup: @ianconsolata can you please update to 1.18.8, and enforce it in the Makefile (in a separate PR)?
There was a problem hiding this comment.
Here it is: #9712
There was a problem hiding this comment.
Once that one is merged, I will update this PR to include the changes.
ianconsolata
force-pushed
the
btrfly-calib-all-in-one
branch
from
545c14d to
17a130d
Compare
|
#9712 is merged, and I brought those changes over to this PR, so I think this is good to go too now? |
Related Issues
Proposed Changes
This is a major refactor of our dockerfile to support the following
I left the existing
Dockerfile.lotusunchanged for backwards compatibility, but we can probably safely remove it after this is merged and released.This also removes:
Checklist
Before you mark the PR ready for review, please make sure that:
<PR type>: <area>: <change being made>fix: mempool: Introduce a cache for valid signaturesPR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, testarea, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps