Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logging and tracing to shelley integration tests #1754

Merged
merged 5 commits into from
Jun 19, 2020

Conversation

rvl
Copy link
Contributor

@rvl rvl commented Jun 13, 2020

Overview

To help debug integration tests cluster issues.

  • Add debug tracing around cluster startup and shutdown.
  • Allow increasing the minimum log severity for integration tests with an environment variable.
  • Make sure all cluster nodes have exactly the same start time. This prevents these errors:
    [blue:cardano.node.IpSubscription:Error:46] [2020-06-15 15:34:20.86 UTC] [String "Failed to start all required subscriptions",String "[127.0.0.1:3263,127.0.0.1:32865]",String "WithIPList SubscriptionTrace",String "LocalAddresses {laIpv4 = Just 0.0.0.0:0, laIpv6 = Just [::]:0, laUnix = Nothing}"]
    
  • Re-enable stake pools in test cluster.

Comments

To do later:

@rvl rvl self-assigned this Jun 13, 2020
Copy link
Contributor

@paweljakubas paweljakubas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM modulo redundant imports

Copy link
Member

@KtorZ KtorZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@rvl
Copy link
Contributor Author

rvl commented Jun 15, 2020

Not finished yet!

@rvl rvl force-pushed the rvl/integration-tests-tracing branch 4 times, most recently from 337c299 to 5803d9b Compare June 15, 2020 17:05
@rvl rvl marked this pull request as ready for review June 15, 2020 17:05
@rvl rvl requested review from KtorZ and paweljakubas June 15, 2020 17:06
@rvl
Copy link
Contributor Author

rvl commented Jun 15, 2020

Now it's ready for review...

bors try

iohk-bors bot added a commit that referenced this pull request Jun 15, 2020
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jun 15, 2020

try

Build failed

@rvl rvl force-pushed the rvl/integration-tests-tracing branch 3 times, most recently from 8849f8c to 2901e16 Compare June 17, 2020 10:07
@rvl
Copy link
Contributor Author

rvl commented Jun 17, 2020

bors r+

iohk-bors bot added a commit that referenced this pull request Jun 17, 2020
1754: Add logging and tracing to shelley integration tests r=rvl a=rvl

### Overview

To help debug integration tests cluster issues.

- [x] Add debug tracing around cluster startup and shutdown.
- [x] Allow increasing the minimum log severity for integration tests with an environment variable.
- [x] Make sure all cluster nodes have exactly the same start time. This prevents these errors:
   ```
   [blue:cardano.node.IpSubscription:Error:46] [2020-06-15 15:34:20.86 UTC] [String "Failed to start all required subscriptions",String "[127.0.0.1:3263,127.0.0.1:32865]",String "WithIPList SubscriptionTrace",String "LocalAddresses {laIpv4 = Just 0.0.0.0:0, laIpv6 = Just [::]:0, laUnix = Nothing}"]
   ```
- [x] Re-enable stake pools in test cluster.

# Comments

To do later:
- Output debug log files for each cardano-node process to the temporary directory — requires [CAD-1247](https://jira.iohk.io/browse/CAD-1247)
- Different hostnames in cardano-node logs — requires IntersectMBO/cardano-node#1278


Co-authored-by: Rodney Lorrimar <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jun 17, 2020

Build failed

@rvl
Copy link
Contributor Author

rvl commented Jun 18, 2020

The integration tests are flaky at the moment, due to the problem which #1768 will fix.

Copy link
Member

@KtorZ KtorZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@KtorZ KtorZ force-pushed the rvl/integration-tests-tracing branch from afcff17 to 6f31485 Compare June 19, 2020 11:29
@rvl
Copy link
Contributor Author

rvl commented Jun 19, 2020

bors r+
please

iohk-bors bot added a commit that referenced this pull request Jun 19, 2020
1754: Add logging and tracing to shelley integration tests r=rvl a=rvl

### Overview

To help debug integration tests cluster issues.

- [x] Add debug tracing around cluster startup and shutdown.
- [x] Allow increasing the minimum log severity for integration tests with an environment variable.
- [x] Make sure all cluster nodes have exactly the same start time. This prevents these errors:
   ```
   [blue:cardano.node.IpSubscription:Error:46] [2020-06-15 15:34:20.86 UTC] [String "Failed to start all required subscriptions",String "[127.0.0.1:3263,127.0.0.1:32865]",String "WithIPList SubscriptionTrace",String "LocalAddresses {laIpv4 = Just 0.0.0.0:0, laIpv6 = Just [::]:0, laUnix = Nothing}"]
   ```
- [x] Re-enable stake pools in test cluster.

# Comments

To do later:
- Output debug log files for each cardano-node process to the temporary directory — requires [CAD-1247](https://jira.iohk.io/browse/CAD-1247)
- Different hostnames in cardano-node logs — requires IntersectMBO/cardano-node#1278


Co-authored-by: Rodney Lorrimar <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jun 19, 2020

Build failed

@rvl
Copy link
Contributor Author

rvl commented Jun 19, 2020

bors r+

@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jun 19, 2020

@iohk-bors iohk-bors bot merged commit 47651ad into master Jun 19, 2020
@iohk-bors iohk-bors bot deleted the rvl/integration-tests-tracing branch June 19, 2020 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants