Redis Engine has concurrency bug

[banks]

At high volume I start to see redis connections drop. I've not worked out why but it's not very surprising in general.

When they do I see a lot of errors like redigo: connection closed.

The only way you can get that as far As I can see from redigo code is if you tried to issue some command to a closed pool connection (and logged the err so additional conn.Close() doesn't count).

I suspect this happens:

1. initializePubSub is in subscribe loop
2. redis error occurs and it return, closing e.psc (actually not it closes it twice - once explicitly and once in defer but that shouldn't matter as second is a no-op)
3. before checkConnectionStatus manages to run and reconnect e.psc, new client subscribes and subscribe() call blindly calls e.psc.Subscribe on the closed conn.

Note that this doesn't even require multiple processors to be triggered (could happen with GOMAXPROCS=1 due to checkConnectionStatus sleeping for a second before it reconnects).