Skip to content

Commit

Permalink
build: release: v1.28.0 (#12271)
Browse files Browse the repository at this point in the history 
* Update changelog

Update changelog

* chore: lotus-shed: Add support for nv23 in `migrate-state` cmd (#12172)

* Add support for nv23 in `lotus-shed migrate-state` cmd

Add support for nv23 in `lotus-shed migrate-state` cmd

* Add actors v14 to `check-invariants` cmd

Add actors v14 to `check-invariants` cmd

* feat: F3: Update go-f3, change the style of participation call (#12196)

* Upgrade go mod F3 dependency to v0.0.3 tagged release

Use concrete/latest version of F3: v0.0.3

* update changelog

* chore!: markets: remove deal constants

* Upgrade to OpenTelemetry v1.28.0

Upgrade to new version of OpenTelemetry to also play nice with F3.

* Reduce log level in F3 message sending to Debug (#12224)

Reduce the log level for `miner with id...` to DEBUG, since in normal
operation of F3 at least 5 messages are sent per epoch. In an event of
rebroadcast the number of messages broadcasted could surpass 10. Hence,
the `DEBUG` log level.

* chore: deps: upgrade to [email protected]

* Upgrade to latest go-f3 `v0.0.4` (#12237)

Upgrade to the latest go F3, with fix to default cert exchange config in
local devnet.

* chore: config: yet more lp2p removal from miner

* fix(store): correctly break weight ties based on smaller ticket (#12253)

@rjan90 and @jennijuju reported seeing this log a lot, which prompted an investigation of the logic. This code does not implement [FIP-0023](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0023.md) correctly. Specifically, if we have the following mintickets:

```
ts1: 1, 2
ts2: 0,3
```

This code will incorrectly return ts1 as the "smaller" tipset, even though the tie should have been broken in favour of ts2 based on `1 > 0`.

* Fix lint error: correct typo in F3Participation function comment

Fix lint error: correct typo in F3Participation function comment

* Fix typo in F3Participate documentation

Corrected "successfull" to "successful" in comments and documentation to fix a lint error.

* chore: libp2p: update to v0.35.4 (#12265)

* go mod tidy

go mod tidy

* Update CHANGELOG.md

Co-authored-by: Steve Loeppky <[email protected]>

* fix a bug

* Set Mainnet upgrade epoch NV23

Set Mainnet upgrade epoch NV23

* Update changelog.md

Update changelog.md

* chore: deps: Update GST, Filecoin-FFI and Actors to final versions NV23 (#12276)

* Update to final GST for nv23

Update to final GST for nv23

* Bump filecoin-ffi to v1.28.0

Bump filecoin-ffi to v1.28.0

* Update CHANGELOG.md

Update CHANGELOG.md

* Update Builtin-actors to v14.0.0

Update Builtin-actors to v14.0.0

* Update Changelog, version-string, make gen and make docsgen

Update Changelog, version-string, make gen and make docsgen

* Re-run ./pack.sh v14 v14.0.0

Re-run ./pack.sh v14 v14.0.0

* Set up otel metrics reporting to prometheus

Register otel collector bridging to the default prometheus registry so
that F3 metrics (which use the newer otel APIs for metrics) get included
in Lotus metrics.

* Update changelog with #12285 backport

Update changelog with #12285 backport

* Revert "chore!: markets: remove deal constants"

This reverts commit d045ccb.

* Revert "chore: config: yet more lp2p removal from miner"

This reverts commit 70400c9.

* fix eth trace panic

* backport: f3: Upgrade to go-f3 `v0.0.5` (#12279) (#12288)

* chore: f3: Upgrade to go-f3 `v0.0.5` (#12279)

* Upgrade to go-f3 `v0.0.5`

Upgrade to the latest version of go-f3 with initial power table storage with far enough lookback, and dynamic manifest persistence.

---------

Co-authored-by: Jakub Sztandera <[email protected]>
Co-authored-by: Jiaying Wang <[email protected]>
Co-authored-by: Steven Allen <[email protected]>

* remove unwanted changelog

* fix import

* Revert "chore!: markets: remove deal constants"

This reverts commit d045ccb.

* Revert "chore: config: yet more lp2p removal from miner"

This reverts commit 70400c9.

* fix eth trace panic

* Update CHANGELOG.md

Co-authored-by: Jakub Sztandera <[email protected]>

* Update CHANGELOG.md

---------

Co-authored-by: Masih H. Derkani <[email protected]>
Co-authored-by: Jakub Sztandera <[email protected]>
Co-authored-by: Steven Allen <[email protected]>
Co-authored-by: Phi <[email protected]>
Co-authored-by: aarshkshah1992 <[email protected]>

---------

Co-authored-by: Jakub Sztandera <[email protected]>
Co-authored-by: Masih H. Derkani <[email protected]>
Co-authored-by: aarshkshah1992 <[email protected]>
Co-authored-by: Rod Vagg <[email protected]>
Co-authored-by: Peter Rabbitson <[email protected]>
Co-authored-by: Aayush Rajasekaran <[email protected]>
Co-authored-by: Steve Loeppky <[email protected]>
Co-authored-by: Jiaying Wang <[email protected]>
Co-authored-by: Steven Allen <[email protected]>
  • Loading branch information
@rjan90 @Kubuxu
@masih @aarshkshah1992 @rvagg @ribasushi @arajasek @BigLep @jennijuju @Stebalien
10 people authored Jul 24, 2024
1 parent 316b6fc commit 28a578f
Show file tree
Hide file tree
Showing 40 changed files with 1,520 additions and 747 deletions.
100 changes: 55 additions & 45 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,29 +1,12 @@
# Lotus changelog

# UNRELEASED
# v1.28.0 / 2024-07-23
This is the MANDATORY Lotus v1.28.0 release, which will deliver the Filecoin network version 23, codenamed Waffle 🧇.

## ☢️ Upgrade Warnings ☢️

- This Lotus release includes some correctness improvements to the events subsystem, impacting RPC APIs including `GetActorEventsRaw`, `SubscribeActorEventsRaw`, `eth_getLogs` and the `eth` filter APIs. Part of these improvements involve an events database migration that may take some time to complete on nodes with extensive event databases. See [filecoin-project/lotus#12080](https://github.com/filecoin-project/lotus/pull/12080) for details.

## New features

- feat: Add trace transaction API supporting RPC method `trace_transaction` ([filecoin-project/lotus#12068](https://github.com/filecoin-project/lotus/pull/12068))

## Improvements

# v1.28.0-rc5 / 2024-07-11

This is the fifth release candidate of the upcoming MANDATORY Lotus v1.28.0 release, which will deliver the Filecoin network version 23, codenamed Waffle 🧇.
**This release sets the Mainnet to upgrade at epoch 4154640, corresponding to 2024-08-06T12:00:00Z.**

**This release candidate sets the calibration network to upgrade at epoch 1779094, corresponding to 2024-07-11T12:00:00Z.** This release does NOT set the mainnet upgrade epoch yet, in which will be updated in the final release.

Compared to `Lotus v1.28.0-rc4`, the `Lotus v1.28.0-rc5` release addresses some performance problems in the `eth_getLogs` API.
It also addresses a bug in the `eth_getLogs` API around handling null blocks.

☢️ Upgrade Warnings ☢️

If you are running the `v1.26.0` or an earlier version of Lotus, please go through the `Upgrade Warnings` section for the `v1.27.*` releases, before upgrading to this RC.
## ☢️ Upgrade Warnings ☢️
- If you are running the `v1.26.x` version of Lotus, please go through the `Upgrade Warnings` section for the `v1.27.*` releases, before upgrading to this RC.

- This upgrade includes an additional migration to the events database. Node operators running Lotus with events turned on (off by default) may experience some delay in initial start-up of Lotus as a minor database migration takes place. See [filecoin-project/lotus#12080](https://github.com/filecoin-project/lotus/pull/12080) for full details.

Expand All @@ -40,49 +23,63 @@ If you are running the `v1.26.0` or an earlier version of Lotus, please go throu
Note that we are only doing a "soft launch"/"passive testing" for F3 (Fast Finality) i.e. FIP-0086 in NV23. Please see [this doc](https://docs.google.com/document/d/14hMFN95_AsByBh7iMc4r_czUgg8tfjHQ1gTsmmHZ8jI/edit#heading=h.dhzqs3lisv24) for more details.

## v14 Builtin Actor Bundle
The actor bundles for the **calibration network** can be checked as follows:

[Builtin actor v14.0.0](https://github.com/filecoin-project/builtin-actors/releases/tag/v14.0.0) is used for supporting this upgrade. Make sure that your lotus actor bundle matches the v14 actors manifest by running the following cli after upgrading:

```
lotus state actor-cids --network-version=23
Network Version: 23
Actor Version: 14
Manifest CID: bafy2bzacebq3hncszqpojglh2dkwekybq4zn6qpc4gceqbx36wndps5qehtau
Manifest CID: bafy2bzacecbueuzsropvqawsri27owo7isa5gp2qtluhrfsto2qg7wpgxnkba

Actor CID
account bafk2bzaced5ecfm56dvtw26q56j4d32yoccyd7ggxn3qdki2enxpqqav45ths
cron bafk2bzacedpbtttpyvtjncqoyobr63mhqqtlrygbnudhxyp2vha56f626dkfs
datacap bafk2bzacecded3lcvo7ndsk66samyecw2trnhrgzi7jxsary3sqgopxlk6rku
eam bafk2bzacecsda4uw7dcu76a27gnrrdcm73tgms7wrte6jbou63vloktkqc5ne
ethaccount bafk2bzacebu2lcxfmohomjj3umslnylwugf5gssywdq3575tjarta7o227dls
evm bafk2bzacea4xnekruhfmdnzvzeo6cbf7jsfgco6x5wje2ckwc2ui2ojzcrlgu
init bafk2bzacedfmsdlewihdcrkdepnfata26nj7akbvexzs3chicujhjf2uxsazc
multisig bafk2bzacedwx4svscsp6wqqu2vlcunjihvvm4u2jnsqjkwutjhir7dwtl7z6m
paymentchannel bafk2bzacedbit7oo6lryhbo64uikvtjtfcth6oxwy3eebxerenu2h7rj44n24
account bafk2bzacebr7ik7lng7vysm754mu5x7sakphwm4soqi6zwbox4ukpd6ndwvqy
cron bafk2bzacecwn6eiwa7ysimmk6i57i5whj4cqzwijx3xdlxwb5canmweaez6xc
datacap bafk2bzacecidw7ajvtjhmygqs2yxhmuybyvtwp25dxpblvdxxo7u4gqfzirjg
eam bafk2bzaced2cxnfwngpcubg63h7zk4y5hjwwuhfjxrh43xozax2u6u2woweju
ethaccount bafk2bzacechu4u7asol5mpcsr6fo6jeaeltvayj5bllupyiux7tcynsxby7ko
evm bafk2bzacedupohbgwrcw5ztbbsvrpqyybnokr4ylegmk7hrbt3ueeykua6zxw
init bafk2bzacecbbcshenkb6z2v4irsudv7tyklfgphhizhghix6ke5gpl4r5f2b6
multisig bafk2bzaceajcmsngu3f2chk2y7nanlen5xlftzatytzm6hxwiiw5i5nz36bfc
paymentchannel bafk2bzaceavslp27u3f4zwjq45rlg6assj6cqod7r5f6wfwkptlpi6j4qkmne
placeholder bafk2bzacedfvut2myeleyq67fljcrw4kkmn5pb5dpyozovj7jpoez5irnc3ro
reward bafk2bzaced5rlycj7fzpscfc7p3wwxarngwqylqshj7te3uffey5tevunz4we
storagemarket bafk2bzaceatwbyrec2nnwggxc2alpqve7rl52fmbhqflebuxmmnvg3qckjb7c
storageminer bafk2bzacecr7ozkdz7l2pq3ig5qxae2ysivbnojhsn4gw3o57ov4mhksma7me
storagepower bafk2bzacedgeolvjtnw7fkji5kqmx322abv6uls2v34fuml6nw36dvfcw4mtu
system bafk2bzacederl6tlpieldsn6mkndqwd4wj5orfoqgab6p2klswfn3cjagxwla
verifiedregistry bafk2bzaceczw2kp6gjjdcjbso7mewp7guik7gr525pal6dotdja2lrct6ok3c
reward bafk2bzacedvfnjittwrkhoar6n5xrykowg2e6rpur4poh2m572f7m7evyx4lc
storagemarket bafk2bzaceaju5wobednmornvdqcyi6khkvdttkru4dqduqicrdmohlwfddwhg
storageminer bafk2bzacea3f43rxzemmakjpktq2ukayngean3oo2de5cdxlg2wsyn53wmepc
storagepower bafk2bzacedo6scxizooytn53wjwg2ooiawnj4fsoylcadnp7mhgzluuckjl42
system bafk2bzacecak4ow7tmauku42s3u2yydonk4hx6ov6ov542hy7lcbji3nhrrhs
verifiedregistry bafk2bzacebvyzjzmvmjvpypphqsumpy6rzxuugnehgum7grc6sv3yqxzrshb4
```

## Migration

All node operators, including storage providers, should be aware that ONE pre-migration is being scheduled 120 epochs before the network upgrade. The migration for the NV23 upgrade is expected to be light with no heavy pre-migrations, here are some expected timings and resource consumption numbers:

- Pre-Migration is expected to take less then 1 minute
- The migration is expected to take less then 30 seconds on a node with a NVMe-drive and a newer CPU. For nodes running on slower disks/CPU, it is still expected to take less then 1 minute.
- Pre-migration is expected to take less than 1 minute.
- The migration is expected to take less than 30 seconds on a node with an NVMe drive and a newer CPU. For nodes running on slower disks/CPU, it is still expected to take less than 1 minute.
- Max memory usage during benchmarking the migration in "offline mode" (i.e., node not syncing) was 23GiB.
- Max memory usage when benchmarking the migration in "online mode" (i.e., while the node is syncing) was 30GiB. Numbers here might vary depending on the load your node is under.
More details on the migration benchmarking can be found in https://github.com/filecoin-project/lotus/issues/12128

We recommend node operators (who haven't enabled splitstore discard mode) that do not care about historical chain states, to prune the chain blockstore by syncing from a snapshot 1-2 days before the upgrade.

For certain node operators, such as full archival nodes or systems that need to keep large amounts of state (RPC providers), we recommend skipping the pre-migration and run the non-cached migration (i.e., just running the migration at the network upgrade epoch), and schedule for some additional downtime. Operators of such nodes can read the [How to disable premigration in network upgrade tutorial.](https://lotus.filecoin.io/kb/disable-premigration/)
For certain node operators, such as full archival nodes or systems that need to keep large amounts of state (RPC providers), we recommend skipping the pre-migration and running the non-cached migration (i.e., just running the migration at the network upgrade epoch), and scheduling some additional downtime. Operators of such nodes can read the [How to disable premigration in network upgrade tutorial.](https://lotus.filecoin.io/kb/disable-premigration/)

## Fast Finality for Filecoin (f3) soft launch

We are one step closer to reduce Filecoin's finality from 7.5 hours to a minute or so, you can checkout the [FIP](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0086.md) for more details. Changing the consensus protocol is not trivial, and the f3 implementation team has designed a [passive testing plan to verify the protocol](https://github.com/filecoin-project/go-f3/issues/213) and give time for client implementation teams to integrate and test F3 before it is fully activated in the network consensus. That said, the lotus team has implemented f3 & the manifest for passive testing in this release, and we would like to ask node operators, especially storage providers, to participate in the testing by participating in F3 on the mainnet (which is enabled by default in this release)! We will keep updating [this discussion](https://github.com/filecoin-project/lotus/discussions/12287) to capture "what can you expect" & testing status. If you notice any unexpected behaviour caused by f3, please do not hesitate to reach out to us in #fil-fast-finality.

F3 (Fast Finality) is experimental in this release. All the new F3 APIs are unstable and subject to until nv24 release (assuming f3 will be fully activated in this upgrade).

Exchanges and RPC providers are recommended to opt-out of F3 functionality for now. You can disable F3 by adding the `DISABLE_F3 = 1` environment variable, which will output a log saying that F3 has been disabled.

## Dependencies
- github.com/filecoin-project/go-state-types (`v0.14.0-dev` -> `v0.14.0-rc5`)
- github.com/filecoin-project/filecoin-ffi (`v1.27.0-rc2` -> `v1.28.0-rc2`)

- github.com/filecoin-project/go-state-types (`v0.14.0-dev` -> `v0.14.0`)
- github.com/filecoin-project/filecoin-ffi (`v1.27.0-rc2` -> `v1.28.0`)
- github.com/filecoin-project/go-libp2p2 (`v0.35.3` -> `v0.35.4`)
- `ref-fvm4` (as part of `filecoin-ffi`) (`4.2.0` -> `4.3.1`)
- A new `github.com/filecoin-project/go-f3` dependency for F3 soft launch (`v0.0.2`)
- A new `github.com/filecoin-project/go-f3` dependency for F3 soft launch (`v0.0.5`)

## Others

Expand All @@ -96,6 +93,19 @@ For certain node operators, such as full archival nodes or systems that need to
- feat:ec: integrate F3 dynamic manifest #12185
- fix: f3: Fix F3 build parameters for testground target (#12189) ([filecoin-project/lotus#12189](https://github.com/filecoin-project/lotus/pull/12189))
- fix: eth_getLogs: https://github.com/filecoin-project/lotus/pull/12212
- chore: lotus-shed: Add support for nv23 in migrate-state cmd #12172
- feat: F3: Update go-f3, change the style of participation call #12196
- chore: f3: Upgrade go mod F3 dependency to v0.0.3 tagged release #12216
- fix: Eth Trace Block: nil access panic #12221
- chore!: markets: remove stray unixfs constants, features and references #12217
- chore: metrics: Upgrade to OpenTelemetry v1.28.0 #12223
- fix: bug: Reduce log level in F3 message sending to Debug #12224
- [skip changelog] chore: config: yet more lp2p removal from miner #12252
- fix(store): correctly break weight ties based on smaller ticket #12253
- fix: exchange bug #12275
- chore: deps: Update GST, Filecoin-FFI and Actors to final versions NV23 #12276
- metrics: f3: Set up otel metrics reporting to prometheus #12285
- Upgrade to go-f3 v0.0.5 #12279

# v1.27.1 / 2024-06-24

Expand Down
27 changes: 20 additions & 7 deletions api/api_full.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
"github.com/filecoin-project/go-f3/certs"
"github.com/filecoin-project/go-f3/gpbft"
"github.com/filecoin-project/go-jsonrpc"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
Expand Down Expand Up @@ -861,17 +862,29 @@ type FullNode interface {
// This is an EXPERIMENTAL API and may be subject to change.
SubscribeActorEventsRaw(ctx context.Context, filter *types.ActorEventFilter) (<-chan *types.ActorEvent, error) //perm:read

// F3Participate should be called by a miner node to participate in signing F3 consensus.
// The address should be of type ID
// The returned channel will never be closed by the F3
// If it is closed without the context being cancelled, the caller should retry.
// The values returned on the channel will inform the caller about participation
// Empty strings will be sent if participation succeeded, non-empty strings explain possible errors.
F3Participate(ctx context.Context, minerID address.Address) (<-chan string, error) //perm:admin
//*********************************** ALL F3 APIs below are not stable & subject to change ***********************************

// F3Participate should be called by a storage provider to participate in signing F3 consensus.
// Calling this API gives the lotus node a lease to sign in F3 on behalf of given SP.
// The lease should be active only on one node. The lease will expire at the newLeaseExpiration.
// To continue participating in F3 with the given node, call F3Participate again before
// the newLeaseExpiration time.
// newLeaseExpiration cannot be further than 5 minutes in the future.
// It is recommended to call F3Participate every 60 seconds
// with newLeaseExpiration set 2min into the future.
// The oldLeaseExpiration has to be set to newLeaseExpiration of the last successful call.
// For the first call to F3Participate, set the oldLeaseExpiration to zero value/time in the past.
// F3Participate will return true if the lease was accepted.
// The minerID has to be the ID address of the miner.
F3Participate(ctx context.Context, minerID address.Address, newLeaseExpiration time.Time, oldLeaseExpiration time.Time) (bool, error) //perm:sign
// F3GetCertificate returns a finality certificate at given instance number
F3GetCertificate(ctx context.Context, instance uint64) (*certs.FinalityCertificate, error) //perm:read
// F3GetLatestCertificate returns the latest finality certificate
F3GetLatestCertificate(ctx context.Context) (*certs.FinalityCertificate, error) //perm:read
// F3GetECPowerTable returns a F3 specific power table for use in standalone F3 nodes.
F3GetECPowerTable(ctx context.Context, tsk types.TipSetKey) (gpbft.PowerEntries, error) //perm:read
// F3GetF3PowerTable returns a F3 specific power table.
F3GetF3PowerTable(ctx context.Context, tsk types.TipSetKey) (gpbft.PowerEntries, error) //perm:read
}

// reverse interface to the client, called after EthSubscribe
Expand Down
2 changes: 2 additions & 0 deletions api/docgen/docgen.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
"github.com/filecoin-project/go-f3/certs"
"github.com/filecoin-project/go-f3/gpbft"
"github.com/filecoin-project/go-jsonrpc/auth"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/builtin/v9/verifreg"
Expand Down Expand Up @@ -401,6 +402,7 @@ func init() {
ToHeight: epochPtr(1020),
})
addExample(&certs.FinalityCertificate{})
addExample(gpbft.ActorID(1000))
}

func GetAPIType(name, pkg string) (i interface{}, t reflect.Type, permStruct []reflect.Type) {
Expand Down
41 changes: 36 additions & 5 deletions api/mocks/mock_full.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 28a578f

Please sign in to comment.