Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intermittent failure when connecting to Pact Daemon #11

Closed
serinth opened this issue Feb 9, 2017 · 8 comments
Closed

Intermittent failure when connecting to Pact Daemon #11

serinth opened this issue Feb 9, 2017 · 8 comments
Assignees
Labels
bug Indicates an unexpected problem or unintended behavior

Comments

@serinth
Copy link

serinth commented Feb 9, 2017

It looks like sometimes we get test failures when connecting to the Pact Daemon/Mock service it provides. This is inconsistent when running the tests. It can fail a bunch of times and then pass.
This can cause CI to fail.

See below for sample output.
I've replaced the username with .

=== RUN   TestRule_Exists
2017/02/09 15:48:10 Error on Verify: Post http://localhost:61841/interactions: dial tcp [::1]:61841: getsockopt: connection refused
exit status 1
FAIL    consumer    1.074s

<USER>-rMBP:consumer <USER>$ go test -v
=== RUN   TestRule_Exists
--- PASS: TestRule_Exists (0.44s)
=== RUN   TestRule_DoesNotExists
--- PASS: TestRule_DoesNotExists (0.01s)
PASS
ok      consumer    0.983s
@mefellows mefellows self-assigned this Feb 9, 2017
@mefellows
Copy link
Member

This on a mac?

@serinth
Copy link
Author

serinth commented Feb 9, 2017

Yes, running on Mac OS X El Capitan.
Verified that this happened with a few others as well.

@mefellows
Copy link
Member

Could you please try the following and record the output during a failed run:

  • start daemon with DEBUG and verbose logging: pact-go daemon -l DEBUG -v
  • start the client with DEBUG logging e.g. dsl.Pact{LogLevel: "DEBUG"}

(please include all of the successful ones before hand also)

@mefellows mefellows added the bug Indicates an unexpected problem or unintended behavior label Feb 12, 2017
@mefellows
Copy link
Member

OK got a random repro this morning. I'm guessing it's a race condition with the Mock Service. It reports up but the port not yet available.

Should be an easy enough fix in daemon.StartServer to wait for the port to become available.

@mefellows
Copy link
Member

Have been able to reliably repro under load (using stress)

Here's a run:

matt ~/go/src/github.com/pact-foundation/pact-go λ go run examples/consumer.go
2017/02/21 19:02:27 [DEBUG] pact setup logging
2017/02/21 19:02:27 [DEBUG] pact setup
2017/02/21 19:02:27 [DEBUG] client: starting a server
2017/02/21 19:02:27 [DEBUG] creating an HTTP client
2017/02/21 19:02:27 [DEBUG] waiting for port 6666 to become available
2017/02/21 19:02:28 [DEBUG] waiting for port 52458 to become available
2017/02/21 19:02:29 [ERROR] Expected server to start < 1s. Timed out waiting for Mock Server to
			start on port 52458 - are you sure it's running?
2017/02/21 19:02:29 [DEBUG] pact add interaction
2017/02/21 19:02:29 [DEBUG] pact setup logging
2017/02/21 19:02:29 [DEBUG] pact setup
2017/02/21 19:02:29 [DEBUG] pact add interaction
2017/02/21 19:02:29 [DEBUG] pact setup logging
2017/02/21 19:02:29 [DEBUG] pact setup
2017/02/21 19:02:29 [DEBUG] pact verify
2017/02/21 19:02:29 [DEBUG] mock service add interaction
2017/02/21 19:02:29 Error on Verify: Post http://localhost:52458/interactions: dial tcp [::1]:52458: getsockopt: connection refused
exit status 1

The timeout is just too small (1s) and additionally, the test should fail earlier (as you can see from the log). I'll bump this to 10 temporarily, and consider making this configurable (it's unlikely to be part of an end-state API so hesitant to add it in at this point).

mefellows added a commit that referenced this issue Feb 22, 2017
fix: increased timeout to reduce likelihood of intermittent failure #11
@mefellows
Copy link
Member

mefellows commented Feb 22, 2017

I've pushed a package that I've tested against a reliable repro locally that I can no longer reproduce. It's a fairly simple workaround and seems to do the trick.

Could you please check it out? (v0.0.3)

@mefellows
Copy link
Member

Feedback from @serinth via direct email seems to confirm this is now fixed.

@mefellows
Copy link
Member

Update, just waiting on a release before closing this off fully.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Indicates an unexpected problem or unintended behavior
Projects
None yet
Development

No branches or pull requests

2 participants