build: release Lotus node v1.29.0

[rjan90]

#12343

Proposed Changes

Prep v1.29.0 stable release

Checklist

Before you mark the PR ready for review, please make sure that:

• Commits have a clear commit message.
• PR title conforms with contribution conventions
• Update CHANGELOG.md or signal that this change does not need it per contribution conventions
• New features have usage guidelines and / or documentation updates in
  • Lotus Documentation
  • Discussion Tutorials
• Tests exist for new functionality or change in behavior
• CI is green

[rvagg]

changelog needs some attention, first it's not rendering quite right in the <details> block, I think you need a line break before the h3, see https://github.com/filecoin-project/lotus/blob/0c346c58de6b66f298fab09faf6f764ca73ddc5e/CHANGELOG.md and expand.

Secondly it's missing stuff in the changelog, was that generated correctly? I went looking for 4cf5ca6 because I thought it at least worth noting here that we are knowingly releasing #12272 and will fix the historical drand lookups in 1.29.1.

Using `branch-diff` to make it it looks like this:

• [5bd97edb4e] - build: release Lotus node v1.29.0-rc1 (Phi-rjan) #12410
• [a5cb674bb1] - fix: error check (qwdsds)
• [dbef5ded15] - feat(libp2p): expose libp2p bandwidth metrics (Steven Allen) #12402
• [0225c915a1] - chore: post release steps for Lotus Miner & Node v1.28.2 Release  #12379 (v1.28.2 miner and node patch release) (Phi-rjan) #12341
• [517c7aeebf] - fix(ci): don't PR or changelog check for draft PRs (Rod Vagg) #12405
• [d415d9f384] - docs: update references to releases branch (Steve Loeppky) #12396
• [860781c55a] - feat(f3): update from v0.0.7 to v0.1.0 (Steven Allen) #12382
• [1095ba1f7b] - chore(release): bump MinerBuildVersion in master (Phi-rjan) #12380
• [a86129b935] - refactor: adjust PR Title Check workflow to reduce noise (Steve Loeppky)
• [0b94fee709] - chore: convert everything relevant to go-commp-utils/v2 (Peter Rabbitson)
• [2ecc690683] - docs: update LOTUS_RELEASE_FLOW.MD document (Phi-rjan) #12364
• [a77342c52c] - github: improve stalebot behavior/language (Steven Allen) #12370
• [f9074a6cf4] - feat: f3: override F3BootstrapEpoch on 2k devnet with environment variable (hanabi1224) #12354
• [1c468b5ba8] - docs: updating CONTRIBUTING.md with table of contents and TOC guidance (Steve Loeppky) #12372
• [a5f93216dc] - chore(docs): fix some function names (luozexuan) #12368
• [8dd9a555aa] - chore: sealing: instead of variable use method to get miner sector expiration (simlecode)
• [a3e0c2974e] - chore: types: fix a method comment (linghuying)
• [67b7361c58] - docs: release template: update based on 1.28 learnings (Steve Loeppky) #12356
• [66a960cec5] - docs: pr title check fix to link to right doc (Steve Loeppky) #12371
• [a359f8460e] - fix (ci): support checking PRs from forks in PR-title check (Phi-rjan) #12367
• [eb6da327e5] - docs: expand CONTRIBUTING.md and update README.md (Phi-rjan) #12366
• [7530136f90] - feat(ci): add PR title check workflow (Phi-rjan) #12361
• [d289074e59] - changes as per review (aarshkshah1992)
• [483399e84e] - update changelog (aarshkshah1992)
• [fdda52ec83] - use cache for ETH blocks (aarshkshah1992)
• [821da7bd2d] - fix: chain: use instance-local verifier to VerifyWinningPoSt (Rod Vagg)
• [d7b218e148] - chore: ffi: copy verifier iface, mock & ffi out of storage (Rod Vagg) #11581
• [a7b1db5f04] - fix: address issues raised from code review (Rod Vagg)
• [670f433040] - docs: gateway: changelog entry for new changes (Rod Vagg)
• [dcc6a9f3d8] - chore: config: reduce FilterTTL default to 1 hour (Rod Vagg)
• [c4b699a051] - feat!: make max-filters configurable, count combined subs+filters (Rod Vagg)
• [1fd00bcf22] - feat: gateway: auto-cleanup of installed filters when ws connection ends (Rod Vagg)
• [f8981cbe16] - fix!: gateway: limit stateful calls to websockets (Rod Vagg)
• [c90cbbd35b] - fix!: gateway: fix rate limiting, general cleanup (Rod Vagg)
• [4b75fa86b2] - chore: docs: Update step in skeleton guide (Phi-rjan) #12349
• [704f049e16] - add changelog (Jennifer Wang)
• [c9ea42b661] - add datacap balance to circ supply (Jennifer Wang)
• [970752648e] - feat: events: add lotus-shed indexes inspect-events health-check (Rod Vagg)
• [5069d8bf02] - chore: ci: prefix ref passed to build pulse with refs/heads/ (Piotr Galar) #12338
• [d44dc159f5] - fix: lotus-shed: store processed tipset after backfilling events (Aryan Tikarya) #12335
• [968052a73d] - chore: deps: update hamt to latest (Rod Vagg)
• [10f6976a55] - chore: docs: fix some misspellings (riskrose) #12333
• [1177717127] - ETH RPC: receipts: use correct txtype in receipts (Aarsh Shah) #12332
• [ba07505963] - feat: ETH API: Add V1 API to convert Filecoin Address to ETH address (Aarsh Shah) #12324
• [474a3ef881] - ci: provide additional input for the buildpulse-action (Piotr Galar) #12329
• [c353a9fb90] - only change method for 0x recipients (Aarsh Shah) #12328
• [f13dace466] - feat: expose settle deal command for lotus miner (Aryan Tikarya) #12270
• [d38ce38d18] - chore: logging: switch stdout print to use the logger (Peter Rabbitson) #12311
• [d36c93c4ae] - fix: events: remove filter if we fail to add it to the FilterStore (Rod Vagg)
• [f6978f0172] - [skip changelog] chore: blockstore_gc: minor cleanups (Peter Rabbitson) #12312
• [e75c5ff624] - fix test (aarshkshah1992)
• [724099cf49] - Apply suggestions from code review (Aarsh Shah)
• [c99a66e575] - works for sender and recipient (aarshkshah1992)
• [b16f2ef516] - update ChangeLog (aarshkshah1992)
• [c0b694c897] - lotus send should accept ETH addresses (aarshkshah1992)
• [827613b8fa] - fix: docs: address typo (Elias Rad) #12321
• [0bd11a91c6] - fix: events: don't initialise events helpers when not needed (Rod Vagg)
• [792eecc4ea] - Merge pull request Merge 1.28.2 to master #12317 from filecoin-project/rvagg/1.28.2-to-master (Rod Vagg)
• [9298a454a8] - doc: clean up retraction comment for 1.28.0 (Rod Vagg)
• [5bd6dabfba] - Merge tag 'v1.28.1' into master (Rod Vagg)
• [5fdc75c65e] - Proxy single Put/Delete to the Many variants (Peter Rabbitson) #12313
• [f234267f13] - Add delegated to cli docs (ZenGround0) #12309
• [9c9bb95077] - feat: p2p: allow overriding bootstrap nodes with environmemnt variable (hanabi1224) #12292
• [a6fd2eb86b] - feat: ci: automate the new release process (Piotr Galar) #12096
• [b4acecd568] - feat: p2p: environment variables for disabling DHT query filter and routing table filter (hanabi1224)
• [7a43baab14] - Merge pull request Merge releases into master #12302 from filecoin-project/rvagg/releases-to-master (Rod Vagg)
• [718fc0330f] - Merge remote-tracking branch 'origin/master' into rvagg/releases-to-master (Rod Vagg)
• [01e4922ee5] - Merge branch 'releases' into 'master' (Rod Vagg)
• [9063b3b34b] - feat: eth: add trace_filter API (Mikers) #12123
• [4cf04dc341] - chore: ui: add a terminal check (Jiaying Wang) #12256
• [5b52dbc090] - Update Building_a_network_skeleton.md (Phi-rjan) #12280
• [294ffc0a10] - chore: types: fix type mismatches missed in [skip changelog] chore: types: buildconstants split post-cleanup #12246 (Peter Rabbitson)
• [1f0f558e85] - chore: f3: Upgrade to go-f3 v0.0.5 (Masih H. Derkani) #12279
• [99a53d6eb8] - Set up otel metrics reporting to prometheus (Masih H. Derkani)
• [31b744a991] - chore: deps: Update GST, Filecoin-FFI and Actors to final versions NV23 (Phi-rjan) #12276
• [c75cc7d86a] - fix: ci: use properly globed path for buildpulse (Piotr Galar) #12284
• [4cf5ca6be5] - [skip changelog] chore: drand: cleanup legacy and unused configs (Peter Rabbitson) #12272
• [fafa81f19b] - fix: ci: fix configuration variables retrieval in buildpulse workflow (Piotr Galar) #12282
• [89f042e1a8] - chore: Set Mainnet upgrade epoch for NV23 (Phi-rjan) #12278
• [4f63a08605] - fix: Eth Event Receipt Logs: use event index for logs (Aarsh Shah) #12269
• [6994580a80] - test: events: test sqlite indexes used in query plan of all selects and updates (Rod Vagg)
• [2b880bcb3c] - fix: events: address sqlite index selection performance regressions (Rod Vagg)
• [01dad15dfa] - fix a bug (aarshkshah1992)
• [8efe5bec8d] - more deprecated cleanup missed previously (Peter Rabbitson)
• [04169a0a75] - clarify the gap in constants (Peter Rabbitson)
• [1bc8a8b32c] - chore: remove (deprecated) deps on build/ proxy-constants (Peter Rabbitson)
• [2933cba8a3] - chore: move IsNearUpgrade helper to buildconstants (Peter Rabbitson)
• [305c096ea5] - chore: move drand config to the buildconstants (Peter Rabbitson)
• [e5c78f977b] - chore: move BuildTypeString() to the buildconstants (Peter Rabbitson)
• [2ea1009288] - batch sector remove (LexLuthr)
• [124d0a43a8] - fix(store): correctly break weight ties based on smaller ticket (Aayush Rajasekaran) #12253
• [11dd4c9df7] - chore: config: yet more lp2p removal from miner (Peter Rabbitson)
• [c053d713f6] - chore: libp2p: update to v0.35.4 (Rod Vagg) #12265
• [526d1ffe82] - chore: lint: enable type godoc linting (Rod Vagg)
• [9e6e7814ab] - chore: lint: enable function godoc linting (Rod Vagg)
• [428524fdd8] - chore: lint: enable method godoc linting (Rod Vagg)
• [43b7a78a58] - chore: src: fix godoc comments (PolyMa) #12257
• [2714a84248] - misc: add waffle to match w/ forest (Jiaying Wang) #12247
• [af934bef39] - feat: ci: upload junit xml reports to buildpulse (Piotr Galar) #12225
• [59f9ecde29] - changes as per review (aarshkshah1992)
• [ddb64562a4] - reduce diff (aarshkshah1992)
• [2a8e2b4116] - fix flaky wdPost dispute test (aarshkshah1992)
• [673c3ffd13] - chore: types: buildconstants split post-cleanup (Peter Rabbitson) #12245
• [103b61e586] - Apply suggestions from code review (Aarsh Shah)
• [a2b963a675] - chore: libp2p: update to v0.35.3 (Rod Vagg)
• [1c05985b14] - fix: ethhashlookup: clean-up query management and lifecycle (Rod Vagg)
• [69c73529d1] - fix (Andrew Jackson (Ajax)) #12229
• [7cc8db8b15] - Upgrade to latest go-f3 v0.0.4 (Masih H. Derkani) #12237
• [424ae01c24] - feat: sqlite: extract common init and migration utilities (Rod Vagg)
• [ab44a9b359] - chore: deps: upgrade to [email protected] (Rod Vagg)
• [971467026f] - fix: eth: make tx hash immediately available in lookup db after submission (Rod Vagg)
• [21abfc69fb] - feat: api: Clean API for Miners (Andrew Jackson (Ajax)) #12112
• [a57dce3136] - Reduce log level in F3 message sending to Debug (Masih H. Derkani) #12224
• [ca1d9638c5] - Upgrade go mod F3 dependency to v0.0.3 tagged release (Masih H. Derkani)
• [7ac638b659] - Upgrade to OpenTelemetry v1.28.0 (Masih H. Derkani)
• [499536a17f] - fix: test: use parents instead of height-1 to calculate sector activation epoch (Rod Vagg)
• [bb014ffc8b] - update changelog (aarshkshah1992)
• [8723d19e62] - fix eth trace panic (aarshkshah1992)
• [105b456518] - chore: merge release/v1.28.0 / v1.28.0-rc4 back to master (Phi-rjan) #12190
• [e76700b891] - chore!: markets: remove deal constants (Rod Vagg)
• [711bb7db3f] - chore!: markets: remove stray unixfs constants, features and references (Rod Vagg)
• [2822404b52] - feat: F3: Update go-f3, change the style of participation call (Jakub Sztandera) #12196
• [974157145a] - fix: ETH getLogs: fix slowness at head and ignore null blocks (Aarsh Shah) #12207
• [7ea0e6cb82] - chore: docs: fix typos (fsgerse) #12198
• [1a94bd2aec] - fix: Eth Tx Events Bloom Filter: fix slice modification bug and flaky test (Aarsh Shah) #12203
• [daf678fbd0] - Update skip/changelog label (Phi-rjan) #12201
• [dbe8b2db87] - feat: niporep: multi-sector onboarding through UnmanagedMiner (Rod Vagg)
• [cb953b1137] - wait for async msg indexing (Aarsh Shah) #12200
• [1fb8efefbf] - ci: add changelog check workflow (Laurent Senta) #12191
• [f08ff209fa] - chore: deps: upgrade to jackc/[email protected] (zhangguanzhang) #12183
• [5d0db7037b] - fix: chain: remove unnecessary error check (fsgerse) #12195
• [6e96f4f195] - chore: cli: use embed pkg to split long template content to file (winniehere) #12193
• [fbbfa1a587] - Fix F3 build parameters for testground target (Masih H. Derkani)
• [44569437a0] - feat:ec: integrate F3 dynamic manifest (adlrocha) #12173
• [7e37ee1b88] - feat:ec:f3: add env var to disable F3 (adlrocha) #12176
• [f8c5b737ef] - Update f3 setting for calib (Jiaying Wang) #12181
• [60b5ee8beb] - fix: rpc: missing re-assignment of baseFee in EthFeeHistory (hanabi1224)
• [3c705fad3b] - fix: build: Ensure Quicknet is used when spinning up 2k/butterfly networks (Phi-rjan) #12175
• [935a77f589] - chore: docs: Add initial Update_Dependencies_Lotus.md (Phi-rjan) #12107
• [ea46e6579f] - chore: lotus-shed: Add support for nv23 in migrate-state cmd (Phi-rjan) #12172
• [069ffce4ef] - feat: api: sanity check the "to" address of outgoing messages (Steven Allen) #12135
• [7cdf9aa2cd] - feat: ec: Integrate Fast Finality for Filecoin (Jakub Sztandera) #12119
• [b4c2e2e886] - update go state types (aarshkshah1992)
• [e879867901] - chore: deps: update ffi and go-state-types for NV23 (Aarsh Shah) #12156
• [6d403bf18d] - fix: state: rename Actor.Address and only use it for f4 addresses (Steven Allen) #12155
• [653210bceb] - update go-state-types (Aarsh Shah) #12154
• [f7632ed11e] - fix: niporep: unflake provecommit failure for niporep (Rod Vagg)

[rjan90]

Using branch-diff to make it it looks like this:

Went through the whole branch-diff now, and added missing entries in 8d23ee2.

There is a lot of interesting cases in this diff as well, which might be worth spending some time bike shedding on how we should handle? (I did not add these to the sorted changelog):

Backported to other releases:

[f7632ed] - fix: niporep: unflake provecommit failure for niporep (Rod Vagg)
[653210b] - update go-state-types (Aarsh Shah) #12154
[6d403bf] - fix: state: rename Actor.Address and only use it for f4 addresses (Steven Allen) #12155
[b4c2e2e] - update go state types (aarshkshah1992)
[e879867] - chore: deps: update ffi and go-state-types for NV23 (Aarsh Shah) #12156
[7e37ee1] - feat:ec:f3: add env var to disable F3 (adlrocha) #12176
[ea46e65] - chore: lotus-shed: Add support for nv23 in migrate-state cmd (Phi-rjan) #12172
[ab44a9b] - chore: deps: upgrade to [email protected] (Rod Vagg)
[2822404] - feat: F3: Update go-f3, change the style of participation call (Jakub Sztandera) #12196
[a2b963a] - chore: libp2p: update to v0.35.3 (Rod Vagg)
[c053d71] - chore: libp2p: update to v0.35.4 (Rod Vagg) #12265
[31b744a] - chore: deps: Update GST, Filecoin-FFI and Actors to final versions NV23 (Phi-rjan) #12276
[7cc8db8] - Upgrade to latest go-f3 v0.0.4 (Masih H. Derkani) #12237
[1f0f558] - chore: f3: Upgrade to go-f3 v0.0.5 (Masih H. Derkani) #12279
[bb014ff] - update changelog (aarshkshah1992)
[8723d19] - fix eth trace panic (aarshkshah1992)
[89f042e] - chore: Set Mainnet upgrade epoch for NV23 (Phi-rjan) #12278
[4456943] - feat:ec: integrate F3 dynamic manifest (adlrocha) #12173
[f8c5b73] - Update f3 setting for calib (Jiaying Wang) #12181
[069ffce] - feat: api: sanity check the "to" address of outgoing messages (Steven Allen) #12135
[fbbfa1a] - Fix F3 build parameters for testground target (Masih H. Derkani)
[60b5ee8] - fix: rpc: missing re-assignment of baseFee in EthFeeHistory (hanabi1224)
[9741571] - fix: ETH getLogs: fix slowness at head and ignore null blocks (Aarsh Shah) #12207
[124d0a4] - fix(store): correctly break weight ties based on smaller ticket (Aayush Rajasekaran) #12253
[7cdf9aa] - feat: ec: Integrate Fast Finality for Filecoin (Jakub Sztandera) #12119
[3c705fa] - fix: build: Ensure Quicknet is used when spinning up 2k/butterfly networks (Phi-rjan) #12175
[a57dce3] - Reduce log level in F3 message sending to Debug (Masih H. Derkani) #12224
[ca1d963] - Upgrade go mod F3 dependency to v0.0.3 tagged release (Masih H. Derkani)
[e76700b] - chore!: markets: remove deal constants (Rod Vagg)
[711bb7d] - chore!: markets: remove stray unixfs constants, features and references (Rod Vagg)
[7ac638b] - Upgrade to OpenTelemetry v1.28.0 (Masih H. Derkani)
[99a53d6] - Set up otel metrics reporting to prometheus (Masih H. Derkani)

How should we handle Lotus-Miner related PRs/Commits in the changelog for Lotus Node only releases:

[a5cb674] - fix: error check (qwdsds)
[f13dace] - feat: expose settle deal command for lotus miner (Aryan Tikarya) #12270
[2ea1009] - batch sector remove (LexLuthr)
[11dd4c9] - chore: config: yet more lp2p removal from miner (Peter Rabbitson)
[8dd9a55] - chore: sealing: instead of variable use method to get miner sector expiration (simlecode)

Stacked PRs / Closed PRs pulled into other PRs

[2ecc690] - docs: update LOTUS_RELEASE_FLOW.MD document (Phi-rjan) #12364
[6994580] - test: events: test sqlite indexes used in query plan of all selects and updates (Rod Vagg)
[a3e0c29] - chore: types: fix a method comment (linghuying)

Rebase merged PRs / commits covered by

[d289074] - changes as per review (aarshkshah1992)
[483399e] - update changelog (aarshkshah1992)
[fdda52e] - use cache for ETH blocks (aarshkshah1992)

• Covered with feat: ETH RPC: Use Block Cache for EthGetBlockByHash #12359 in the sorted changelog.

[a7b1db5] - fix: address issues raised from code review (Rod Vagg)
[670f433] - docs: gateway: changelog entry for new changes (Rod Vagg)
[dcc6a9f] - chore: config: reduce FilterTTL default to 1 hour (Rod Vagg)
[c4b699a] - feat!: make max-filters configurable, count combined subs+filters (Rod Vagg)
[1fd00bc] - feat: gateway: auto-cleanup of installed filters when ws connection ends (Rod Vagg)
[f8981cb] - fix!: gateway: limit stateful calls to websockets (Rod Vagg)
[c90cbbd] - fix!: gateway: fix rate limiting, general cleanup (Rod Vagg)

• Covered with feat!: gateway: fix rate limiting, better stateful handling #12327 in the sorted changelog.

[c9ea42b] - add datacap balance to circ supply (Jennifer Wang)

• Covered with fix: api: add datacap balance to circ supply #12348 in the sorted changelog.

[e75c5ff] - fix test (aarshkshah1992)
[724099c] - Apply suggestions from code review (Aarsh Shah)
[c99a66e] - works for sender and recipient (aarshkshah1992)
[b16f2ef] - update ChangeLog (aarshkshah1992)
[c0b694c] - lotus send should accept ETH addresses (aarshkshah1992)

• Covered with feat!: gateway: fix rate limiting, better stateful handling #12327 in the sorted changelog.

I get 404s on these PR numbers??

[7ea0e6c] - chore: docs: fix typos (fsgerse) #12198
[5d0db70] - fix: chain: remove unnecessary error check (fsgerse) #12195

Since we have now moved to building the changelog incrementally, I'm not sure what value the "sorted changelog" actually brings in value. At least from a time consumption perspective, just adding a link of the diff to previous release will save a lot of time: v1.28.2...release/v1.29.0.

[rvagg]

Thanks for digging!

Backported to other releases:

Looking at a few of these, I think this is as a result of squash merging or squashing on a merge-back, so we lose the individual commits. One of the reasons we should avoid squashing big batches of commits I think.

How should we handle Lotus-Miner related PRs/Commits in the changelog for Lotus Node only releases:

I think .. going forward, when we branch from master we're going to be capturing them by default on the .0 release, but then as long as we're only doing patch releases we should probably just hold them back unless there's some compelling reason to do so. If we have a high enough cadence of semver-minor releases then they'll make it out quick enough.

Stacked PRs / Closed PRs pulled into other PRs

Bugs in matching the commit to the PR. I know GitHub does a decent job of mapping commit to PR these days so I wonder if there's a new API I could build in to branch-diff to do this properly. It currently uses a fall-back best-guess method to figure it out if you don't have a PR-URL: metadata in the commit, which we don't do here.

I get 404s on these PR numbers??

User was deleted and so were their PRs! PR is properly referenced by GitHub: 7ea0e6c but it links to a 404. But as does user: https://github.com/fsgerse - one of the downsides of accepting these tiny PRs that look like someone just fishing for airdrop cred. But the commits themselves look OK from this user.

[rvagg]

👍 for 65e5fba