fatal error: concurrent map writes panic with inmemory storage

[nutbunnies]

We are having frequent crashes when running high volumes thru mailhog. Looks to be due to mutation of the inmemory map not being sync'd.

2016/11/04 04:50:03 [SMTP 172.17.0.1:54289] Session ended
fatal error: concurrent map writes
goroutine 58782 [running]:
runtime.throw(0x9d6600, 0x15)
        /usr/local/go/src/runtime/panic.go:530 +0x90 fp=0xc845aeb8d8 sp=0xc845aeb8c0
runtime.mapassign1(0x809780, 0xc84d3a41e0, 0xc845aeb9d8, 0xc845aeb9c8)
        /usr/local/go/src/runtime/hashmap.go:445 +0xb1 fp=0xc845aeb980 sp=0xc845aeb8d8
github.com/mailhog/storage.(*InMemory).Store(0xc8200c4f20, 0xc82d382420, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/mailhog/storage/memory.go:26 +0xda fp=0xc845aeb9f0 sp=0xc845aeb980
github.com/mailhog/MailHog-Server/smtp.(*Session).acceptMessage(0xc845e50000, 0xc842d2c0a0, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/mailhog/MailHog-Server/smtp/session.go:104 +0x19b fp=0xc845aeba80 sp=0xc845aeb9f0
github.com/mailhog/MailHog-Server/smtp.(*Session).(github.com/mailhog/MailHog-Server/smtp.acceptMessage)-fm(0xc842d2c0a0, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/mailhog/MailHog-Server/smtp/session.go:53 +0x46 fp=0xc845aebab8 sp=0xc845aeba80
github.com/mailhog/smtp.(*Protocol).ProcessData(0xc840ed7220, 0xc820188a30, 0x1, 0x0)
        /go/src/github.com/mailhog/smtp/protocol.go:186 +0x540 fp=0xc845aebc28 sp=0xc845aebab8
github.com/mailhog/smtp.(*Protocol).Parse(0xc840ed7220, 0xc820188a30, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/mailhog/smtp/protocol.go:160 +0x25e fp=0xc845aebcb0 sp=0xc845aebc28
github.com/mailhog/MailHog-Server/smtp.(*Session).Read(0xc845e50000, 0xc85e80f201)
        /go/src/github.com/mailhog/MailHog-Server/smtp/session.go:139 +0x607 fp=0xc845aebe30 sp=0xc845aebcb0
github.com/mailhog/MailHog-Server/smtp.Accept(0xc83f07a430, 0x10, 0x7f5be6ff0858, 0xc85f2e9708, 0x7f5be702c8c0, 0xc8200c4f20, 0xc820066240, 0x98e720, 0xf, 0x0, ...)
        /go/src/github.com/mailhog/MailHog-Server/smtp/session.go:61 +0x7db fp=0xc845aebf48 sp=0xc845aebe30
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc845aebf50 sp=0xc845aebf48
created by github.com/mailhog/MailHog-Server/smtp.Listen
        /go/src/github.com/mailhog/MailHog-Server/smtp/smtp.go:41 +0x590
goroutine 1 [chan receive, 3659 minutes]:
main.main()
        /go/src/github.com/mailhog/MailHog/main.go:98 +0x8e5
goroutine 5 [IO wait, 18 minutes]:
net.runtime_pollWait(0x7f5be702d808, 0x72, 0x0)
        /usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc8201c4ca0, 0x72, 0x0, 0x0)
        /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8201c4ca0, 0x0, 0x0)
        /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).accept(0xc8201c4c40, 0x0, 0x7f5be702d988, 0xc823031ca0)
        /usr/local/go/src/net/fd_unix.go:426 +0x27c
net.(*TCPListener).AcceptTCP(0xc820124208, 0x451960, 0x0, 0x0)
        /usr/local/go/src/net/tcpsock_posix.go:254 +0x4d
net/http.tcpKeepAliveListener.Accept(0xc820124208, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/http/server.go:2427 +0x41
net/http.(*Server).Serve(0xc82012bf80, 0x7f5be6ff0598, 0xc820124208, 0x0, 0x0)
        /usr/local/go/src/net/http/server.go:2117 +0x129
net/http.(*Server).ListenAndServe(0xc82012bf80, 0x0, 0x0)
        /usr/local/go/src/net/http/server.go:2098 +0x136
net/http.ListenAndServe(0x9825a0, 0xc, 0x7f5be6fe81f8, 0xc82019b3c0, 0x0, 0x0)
        /usr/local/go/src/net/http/server.go:2195 +0x98
github.com/mailhog/http.Listen(0x9825a0, 0xc, 0xa64548, 0xc820066480, 0xa64858)
        /go/src/github.com/mailhog/http/server.go:108 +0x255
created by main.main
        /go/src/github.com/mailhog/MailHog/main.go:82 +0x87b

[ian-kent]

closing, fixed by @nutbunnies - mailhog/storage#4

[hdpe]

This issue isn't properly resolved: although mutex lock/unlock is applied around all the map write operations, it hasn't been around the read operations.

Saw the failure:

fatal error: concurrent map read and map write
goroutine 1802438 [running]:
runtime.throw(0x94a63d, 0x21)
/usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc0004bb888 sp=0xc0004bb858 pc=0x42f8b2
runtime.mapaccess2_faststr(0x8a54e0, 0xc000093950, 0xc00013c940, 0x3c, 0x2, 0x2)
/usr/local/go/src/runtime/map_faststr.go:116 +0x48f fp=0xc0004bb8f8 sp=0xc0004bb888 pc=0x41494f
github.com/mailhog/MailHog/vendor/github.com/mailhog/storage.(*InMemory).Load(0xc000093980, 0xc00013c940, 0x3c, 0xc0000ec800, 0xc00014a088, 0xc0001040a0)

(trace from my fork).

Thanks