Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test and remove warnings of undefined behaviour #375

Merged
merged 14 commits into from
Feb 6, 2019

Conversation

eborden
Copy link
Collaborator

@eborden eborden commented Feb 2, 2019

It is no longer true that sending or receiving data from a closed socket
will result in undefined behaviour. Version 3.0.0.0 defined this
behaviour as an exception by introducing the -1 sigil.

This commit adds tests to ensure this behaviour remains stable and
removes all documentation elluding to undefined behaviour.

It is no longer true that sending or receiving data from a closed socket
will result in undefined behaviour. Version 3.0.0.0 defined this
behaviour as an exception by introducing the `-1` sigil.

This commit adds tests to ensure this behaviour remains stable and
removes all documentation elluding to undefined behaviour.
@eborden eborden self-assigned this Feb 2, 2019
@eborden eborden requested a review from kazu-yamamoto February 2, 2019 22:56
@@ -111,6 +176,13 @@ spec = do

client sock = send sock testMsg
tcpTest client server

it "returns empty string at EOF" $ do
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an additional test to cover the removal of EOF errors.

This commit breaks up the large `SimpleSpec` into its public module
constituents. This provides a more granular and easier to manage
overview of `network`'s test coverage. More test coverage has also been
added to `Network.Socket.ByteString.Lazy` as it was mostly uncovered.
`unsafeInterleaveIO` means that closed socket failures are not thrown
from `getContents`.
@kazu-yamamoto
Copy link
Collaborator

Though this PR looks excellent, I got this error on Mac (Mojave):

Network.Socket.ByteString.Lazy
  send
    works well
    throws when closed
  sendAll
    works well
    throws when closed
  getContents
spec: Network.Socket.shutdown: invalid argument (Socket is not connected)
spec: thread blocked indefinitely in an MVar operation
Test suite spec: FAIL
Test suite logged to: dist/test/network-3.0.1.0-spec.log
0 of 1 test suites (0 of 1 test cases) passed.

@kazu-yamamoto
Copy link
Collaborator

At least, GHC 8.0.2 and GHC 8.6.3 got this error.

@kazu-yamamoto
Copy link
Collaborator

This PR works well on Linux.

@eborden
Copy link
Collaborator Author

eborden commented Feb 4, 2019

Sounds like this found a Mac bug. We could pending the test for now and file an issue.

@kazu-yamamoto
Copy link
Collaborator

shutdown of macOS sucks: https://bugs.python.org/issue6774

@kazu-yamamoto
Copy link
Collaborator

My current opinion:

  • Adding "On macOS, shutdown does not work well" or something to the doc AND
  • Disable this part of tests on macOS

@llelf
Copy link
Member

llelf commented Feb 4, 2019

spec: thread blocked indefinitely in an MVar operation

what operation?

@eborden
Copy link
Collaborator Author

eborden commented Feb 4, 2019

@llelf getContents runs via lazy IO and swallows exceptions. So on Mac it fails to shutdown and continues to loop indefinitely. As a result ghc reports a locked MVar from the test helpers.

@eborden
Copy link
Collaborator Author

eborden commented Feb 4, 2019

@kazu-yamamoto There is some discussion in that python thread about whether to document OS specific behaviour. They err on the side of not documenting it. Their reason being that opening the door to documenting OS specific quirks could lead to an unending list of quirks. I generally agree.

My opinion would be:

  • Adding "On macOS, shutdown does not work well" or something to the doc AND
  • Disable this part of tests on macOS
  • Open an issue as this behavior is likely fixable with a properly placed catch

@eborden
Copy link
Collaborator Author

eborden commented Feb 5, 2019

@kazu-yamamoto Does this look good on your mojave machine?

sendAll sock lazyTestMsg `shouldThrow` anyException
tcpTest client server

#if !defined(darwin_HOST_OS)
Copy link

@vdukhovni vdukhovni Feb 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, this #ifdef is not needed. The solution is to fix a bug in getContents which does not handle I/O errors when calling shutdown(SHUT_RD) after EOF. While many platforms don't return errors from that call, some do. The getContents implementation needs to either drop the shutdown call or handle errors (perhaps just some I/O errors and not others, but see: https://ghc.haskell.org/trac/ghc/ticket/14730#comment:2)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likely also good to link to the original commit with the Shutdown added. Thanks for digging that up @vdukhovni
0d0b990#commitcomment-32203878

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a catchall after shutdown, which I believe to be acceptable. @kazu-yamamoto and @vdukhovni I'm happy to hear reason why we should just remove shutdown instead.

Shutdown can fail on darwin. To avoid bugs in `getContents` we catch
exceptions at shutdown and return content as if the shutdown was
successful.

Another strategy is to avoid calling `shutdown` all together, as this
action is not required.
`onException` would also capture async errors and is too broad. Use
`catchIOError`.
@@ -66,7 +66,7 @@ getContents s = loop
sbs <- N.recv s defaultChunkSize
if S.null sbs
then do
shutdown s ShutdownReceive `onException` return ()
shutdown s ShutdownReceive `catchIOError` const (return ())

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is better, thanks! For example:

ghci> timeout 100000 $ catch (threadDelay 200000) (\e -> print (e :: SomeException))
<<timeout>>
Just ()

vs.

timeout 100000 $ catchIOError (threadDelay 200000) (\e -> print e)
Nothing

Copy link
Collaborator

@kazu-yamamoto kazu-yamamoto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now LGTM

kazu-yamamoto added a commit to kazu-yamamoto/network that referenced this pull request Feb 6, 2019
@kazu-yamamoto kazu-yamamoto merged commit 286f559 into master Feb 6, 2019
@kazu-yamamoto
Copy link
Collaborator

Merged. Thank you for your efforts!

@kazu-yamamoto kazu-yamamoto deleted the eborden/test-closed-socket-behaviour branch February 6, 2019 00:58
@eborden eborden mentioned this pull request Feb 7, 2019
2 tasks
kazu-yamamoto added a commit to kazu-yamamoto/network that referenced this pull request Apr 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants