Solidity compilation / Rust ABI generation isn't incremental

[tri-joe]

If I run nix-shell --run cape-test-geth, I see

Ensuring node dependencies are installed
Lockfile is up-to-date, resolution step is skipped
Exporting all vars in .env file
nix-pre-commit-hooks: hooks up to date
Compile contracts and generate ABI artifacts
Compiling contracts ...
Solidity 0.8.10 is not fully supported yet. You can still use Hardhat, but some features, like stack traces, might not work correctly.

Learn more at https://hardhat.org/reference/solidity-support

Nothing to compile
No need to generate any newer typings.
artifacts/contracts/CAPE.sol/CAPE.json -> artifacts/contracts/CAPE.sol/CAPE
artifacts/contracts/Greeter.sol/Greeter.json -> artifacts/contracts/Greeter.sol/Greeter
artifacts/contracts/interfaces/IPlonkVerifier.sol/IPlonkVerifier.json -> artifacts/contracts/interfaces/IPlonkVerifier.sol/IPlonkVerifier
artifacts/contracts/libraries/AccumulatingArray.sol/AccumulatingArray.json -> artifacts/contracts/libraries/AccumulatingArray.sol/AccumulatingArray
artifacts/contracts/libraries/BN254.sol/BN254.json -> artifacts/contracts/libraries/BN254.sol/BN254
artifacts/contracts/libraries/RescueLib.sol/RescueLib.json -> artifacts/contracts/libraries/RescueLib.sol/RescueLib
artifacts/contracts/mocks/TestAccumulatingArray.sol/TestAccumulatingArray.json -> artifacts/contracts/mocks/TestAccumulatingArray.sol/TestAccumulatingArray
artifacts/contracts/mocks/TestBN254.sol/TestBN254.json -> artifacts/contracts/mocks/TestBN254.sol/TestBN254
artifacts/contracts/mocks/TestCAPE.sol/TestCAPE.json -> artifacts/contracts/mocks/TestCAPE.sol/TestCAPE
artifacts/contracts/mocks/TestCapeTypes.sol/TestCapeTypes.json -> artifacts/contracts/mocks/TestCapeTypes.sol/TestCapeTypes
artifacts/contracts/mocks/TestRecordsMerkleTree.sol/TestRecordsMerkleTree.json -> artifacts/contracts/mocks/TestRecordsMerkleTree.sol/TestRecordsMerkleTree
artifacts/contracts/mocks/TestRescueNonOptimized.sol/TestRescueNonOptimized.json -> artifacts/contracts/mocks/TestRescueNonOptimized.sol/TestRescueNonOptimized
artifacts/contracts/mocks/TestRescue.sol/TestRescue.json -> artifacts/contracts/mocks/TestRescue.sol/TestRescue
artifacts/contracts/mocks/TestRootStore.sol/TestRootStore.json -> artifacts/contracts/mocks/TestRootStore.sol/TestRootStore
artifacts/contracts/mocks/TestTranscript.sol/TestTranscript.json -> artifacts/contracts/mocks/TestTranscript.sol/TestTranscript
artifacts/contracts/RecordsMerkleTree.sol/RecordsMerkleTree.json -> artifacts/contracts/RecordsMerkleTree.sol/RecordsMerkleTree
artifacts/contracts/RescueNonOptimized.sol/RescueNonOptimized.json -> artifacts/contracts/RescueNonOptimized.sol/RescueNonOptimized
artifacts/contracts/RootStore.sol/RootStore.json -> artifacts/contracts/RootStore.sol/RootStore
artifacts/contracts/SimpleToken.sol/SimpleToken.json -> artifacts/contracts/SimpleToken.sol/SimpleToken
artifacts/contracts/verifier/PlonkVerifier.sol/PlonkVerifier.json -> artifacts/contracts/verifier/PlonkVerifier.sol/PlonkVerifier
artifacts/contracts/verifier/Transcript.sol/Transcript.json -> artifacts/contracts/verifier/Transcript.sol/Transcript
artifacts/hardhat/console.sol/console.json -> artifacts/hardhat/console.sol/console
artifacts/@openzeppelin/contracts/token/ERC20/ERC20.sol/ERC20.json -> artifacts/@openzeppelin/contracts/token/ERC20/ERC20.sol/ERC20
artifacts/@openzeppelin/contracts/token/ERC20/extensions/IERC20Metadata.sol/IERC20Metadata.json -> artifacts/@openzeppelin/contracts/token/ERC20/extensions/IERC20Metadata.sol/IERC20Metadata
artifacts/@openzeppelin/contracts/token/ERC20/IERC20.sol/IERC20.json -> artifacts/@openzeppelin/contracts/token/ERC20/IERC20.sol/IERC20
artifacts/@openzeppelin/contracts/utils/Context.sol/Context.json -> artifacts/@openzeppelin/contracts/utils/Context.sol/Context
artifacts/solidity-bytes-utils/contracts/BytesLib.sol/BytesLib.json -> artifacts/solidity-bytes-utils/contracts/BytesLib.sol/BytesLib
Finished extracting ABIs
Starting geth node

every time regardless of whether any solidity code has changed. This takes a few seconds every time (which is pretty annoying if I'm fixing a bunch of compilation errors/warning and want to check that I got them all), and if I try to just use cargo test or cargo clippy I sometimes get compilation errors because it doesn't have ABI files, or get test failures because there isn't a geth instance running.

[tri-joe]

This is mostly solvable by running hivemind as mentioned in README.md, but there is still a race condition where cargo doesn't know that the ABI is out of date -- if I run cargo test before hivemind has finished rebuilding, it will use the previous build result.

[sveitser]

I know the current situation isn't ideal. It would be best to get rid of that build-abi script.

• We should be able to use the hardhat json directly but this wasn't working before and it looks like it's still not fixed
  • Error deploying contract with hardhat artifact: "invalid type: map, expected valid abi spec file" gakonst/ethers-rs#683
• It would also be possible to use ethers-solc to compile instead of hardhat. This is also a relatively recent ethers-rs feature and wasn't working very well / lacking some features like library linking a month ago. At least the library linking has been added in the meantime.

If we do have to implement a manual solution we could check the timestamps. This should be easy but I vaguely remember the straightforward implementation I tried for this some time ago also did not work well in some cases.

[sveitser]

Another related note. We could consider moving back to having a build script that writes the bindings out to file. This would ensure that

• the bindings are up to date whenever cargo runs.
• completion works in all IDEs (it does also work now but is somewhat tricky to configure). Not what this issue is about but nice to have.

The main reason why we removed the build script is because we needed the de-duplication and it was only available via the abigen macro until

• feat(abigen): add MultiAbigen to generate multiple contract bindings gakonst/ethers-rs#724

was merged.

[sveitser]

• Error deploying contract with hardhat artifact: "invalid type: map, expected valid abi spec file" gakonst/ethers-rs#683

Was closed so I will try using the hardhat artifacts directly again.

[sveitser]

The solidity structs are not generated if we use the hardhat json directly. Reported upstream in gakonst/ethers-rs#683 (comment)