Create eosio-debug-build Pipeline

[kj4ezj]

Change Description

From AUTO-363, Blockchain asked that we publish an Ubuntu package produced by a debug build in the eosio pipeline.

This pull request modifies the CI code to support the eosio-debug-build pipeline, which builds debug builds for each commit. For example:

• Build 12 -- eos:develop
• Build 13 -- eos:release/2.1.x
• Build 14 -- eos:release/2.0.x
• Build 15 -- eos:release/1.8.x

The eosio-debug-build pipeline also allows engineers to define a build type by declaring DCMAKE_BUILD_TYPE in the environment. For example, I performed a RelWithDebInfo build in build 16:

DCMAKE_BUILD_TYPE='RelWithDebInfo'

This variable can also be declared in the eosio, eosio-build-unpinned, or eosio-lrt pipelines. However, we do not recommend running tests against non-optimized (debug) builds.

Background

During startup, nodeos performs a number of checks to protect itself from accidentally corrupting the state files, such as verifying the state files were not created with a newer version of nodeos, checking the genesis.json or snapshot.bin chain ID, and comparing the library versions leveraged to write the state files to those in the running copy of nodeos. That last check can make it time consuming for Blockchain to debug when replaying real chains.

For example, during an intense late-night debugging video call, we had stood up a cloud instance, loaded hundreds of gigabytes of chain data onto the cloud instance, then ran a release build of nodeos obtained from CI. We stopped it at a particular block just before our bug and wanted to step through with a debugger. Unfortunately, these state files are unlikely to be accepted by a debugging build of nodeos built on the cloud instance or our local machine, because the build environment and thus libraries are different. So we had to rebuild nodeos twice on the same machine, once for a release build and once for a debug build, in order to switch back and forth between a release (optimized) and debug (non-optimized) build of nodeos while replaying this problematic chain segment.

To prevent this from being an issue in the future, this pull request introduces Debug and RelWithDebInfo build and packaging steps for the latest supported Ubuntu operating system. Since the CI system uses a deterministic build environment with consistent library versions, this will allow Blockchain engineers to pull packages with the level of debug tooling and optimization necessary for their work, and to switch between them at-will without risk of corrupting their state files or failing the startup checks.

See Also

• Pull Request 9940 -- eos:develop
• Pull Request 9943 -- eos:release/2.1.x
• Pull Request 9944 -- eos:release/2.0.x
• Pull Request 9945 -- eos:release/1.8.x

Change Type

Select ONE:

• Documentation
• Stability bug fix
• Other
• Other - special case

Testing Changes

Select ANY that apply:

• New Tests
• Existing Tests
• Test Framework
• CI System
• Other

Consensus Changes

• Consensus Changes

API Changes

• API Changes

Documentation Additions

• Documentation Additions

[spoonincode]

I don't have access to AUTO-363 so I can't see the reasoning for this. But does it make any sense to do a RelWithDebInfo instead?

[kj4ezj]

I don't have access to AUTO-363 so I can't see the reasoning for this.

I always copy the JIRA ticket description to my pull request body so that community contributors have visibility into my changes, so the ticket just says the same thing. The link is for traceability.

I have also updated the pull request description to go into more detail about the background and context of the problem being fixed.

But does it make any sense to do a RelWithDebInfo instead?

That's up to you guys, whatever would be most useful for you trying to debug errors. I did some reading on the differences and it looks like debug binaries contain a bit more tooling than release with debugging info binaries. I don't want you to end up in a situation where the binary provided is insufficient for what you are trying to do.

However, from my reading, it looks like there are situations where bugs appear in release and release with debugging info binaries, but not debugging binaries. So should we just build all three?