System.InvalidOperationException: Collection was modified; enumeration operation may not execute

[vitaliishandra]

We are facing this error from time to time when we switched to binary serialization.
We use the 1.0.0.rc2 version of lib and JanusGraph 0.6.2.
What can be the root cause of this issue?
It was reproduced on different queries. Here is the example of one:

Stack trace:

System.InvalidOperationException: Collection was modified; enumeration operation may not execute. at System.Collections.Generic.Dictionary2.KeyCollection.Enumerator.MoveNext()
at Gremlin.Net.Structure.IO.GraphBinary.TypeSerializerRegistry.GetSerializerFor(Type valueType)
at Gremlin.Net.Structure.IO.GraphBinary.GraphBinaryWriter.WriteAsync(Object value, Stream stream, CancellationToken cancellationToken)
at Gremlin.Net.Structure.IO.GraphBinary.Types.BytecodeSerializer.WriteArgumentsAsync(Object[] arguments, Stream stream, GraphBinaryWriter writer, CancellationToken cancellationToken)
at Gremlin.Net.Structure.IO.GraphBinary.Types.BytecodeSerializer.WriteInstructionsAsync(IReadOnlyCollection1 instructions, Stream stream, GraphBinaryWriter writer, CancellationToken cancellationToken) at Gremlin.Net.Structure.IO.GraphBinary.Types.BytecodeSerializer.WriteValueAsync(Bytecode value, Stream stream, GraphBinaryWriter writer, CancellationToken cancellationToken) at Gremlin.Net.Structure.IO.GraphBinary.Types.SimpleTypeSerializer1.WriteValueAsync(Object value, Stream stream, GraphBinaryWriter writer, Boolean nullable, CancellationToken cancellationToken)
at Gremlin.Net.Structure.IO.GraphBinary.Types.SimpleTypeSerializer1.WriteAsync(Object value, Stream stream, GraphBinaryWriter writer, CancellationToken cancellationToken) at Gremlin.Net.Structure.IO.GraphBinary.GraphBinaryWriter.WriteAsync(Object value, Stream stream, CancellationToken cancellationToken) at Gremlin.Net.Structure.IO.GraphBinary.Types.MapSerializer2.WriteValueAsync(IDictionary2 value, Stream stream, GraphBinaryWriter writer, CancellationToken cancellationToken) at Gremlin.Net.Structure.IO.GraphBinary.Types.SimpleTypeSerializer1.WriteValueAsync(Object value, Stream stream, GraphBinaryWriter writer, Boolean nullable, CancellationToken cancellationToken)
at Gremlin.Net.Structure.IO.GraphBinary.GraphBinaryWriter.WriteValueAsync(Object value, Stream stream, Boolean nullable, CancellationToken cancellationToken)
at Gremlin.Net.Structure.IO.GraphBinary.RequestMessageSerializer.WriteValueAsync(RequestMessage requestMessage, MemoryStream stream, GraphBinaryWriter writer, CancellationToken cancellationToken)
at Gremlin.Net.Structure.IO.GraphBinary.GraphBinaryMessageSerializer.SerializeMessageAsync(RequestMessage requestMessage, CancellationToken cancellationToken)
at Gremlin.Net.Driver.Connection.SendMessageAsync(RequestMessage message, CancellationToken cancellationToken)
at Gremlin.Net.Driver.Connection.SendMessagesFromQueueAsync()
at Gremlin.Net.Driver.ProxyConnection.SubmitAsync[T](RequestMessage requestMessage, CancellationToken cancellationToken)
at Gremlin.Net.Driver.GremlinClient.SubmitAsync[T](RequestMessage requestMessage, CancellationToken cancellationToken)
at Gremlin.Net.Driver.Remote.DriverRemoteConnection.SubmitBytecodeAsync(Guid requestid, Bytecode bytecode, CancellationToken cancellationToken)
at Gremlin.Net.Driver.Remote.DriverRemoteConnection.SubmitAsync[S,E](Bytecode bytecode, CancellationToken cancellationToken)
at Gremlin.Net.Process.Remote.RemoteStrategy.ApplyAsync[S,E](ITraversal2 traversal, CancellationToken cancellationToken) at Gremlin.Net.Process.Traversal.DefaultTraversal2.ApplyStrategiesAsync(CancellationToken cancellationToken)
at Gremlin.Net.Process.Traversal.DefaultTraversal2.Promise[TReturn](Func2 callback, CancellationToken cancellationToken)
at BrainStorm.Abyss.Cqrs.Commands.Users.Create.CreateUserGraphCommandHandler.HandleAsync(CreateUserCommand command, CancellationToken cancellationToken) in /app/BrainStorm.Abyss/src/BrainStorm.Abyss.Cqrs.Commands/Users/Create/CreateUserGraphCommandHandler.cs:line 22
at BrainStorm.Abyss.Cqrs.Commands.CommandHandlerBase2.Handle(TRequest request, CancellationToken cancellationToken) in /app/BrainStorm.Abyss/src/BrainStorm.Abyss.Cqrs.Commands/CommandHandlerBase.cs:line 12

[FlorianHockmann]

When you say "from time to time", I guess you cannot reproduce this in a reliable way?

It would be very helpful to have a simpler traversal to reproduce this or at least know which type causes this exception.

[vitaliishandra]

Yes, it is randomly reproduced.
It is reproduced on different queries, one of query I added to the example.
It just creates the vertex if it doesn't exist and add some properties:

[FlorianHockmann]

I created an issue for this in the TinkerPop issue tracker as this seems to be a problem in Gremlin.Net which is the core component of JanusGraph.Net: https://issues.apache.org/jira/browse/TINKERPOP-2976

I also tried to reproduce this, but failed. My guess is that this is caused by two threads concurrently working on the dictionary which leads to this exception. That would also explain why it wasn't caught before and why it's not easy to reproduce.

This PR should fix this problem: apache/tinkerpop#2176
Note however that it will take some time until it's available via JanusGraph.Net as TinkerPop is currently preparing a release and this PR will therefore probably land in the next release after that.

[FlorianHockmann]

Closing this as this should be fixed by the update of Gremlin.Net to 3.7.2: #182 which will be part of the next release of JanusGraph.Net.

You can alternatively already use JanusGraph.Net together with Gremlin.Net 3.7.2 if you manually add Gremlin.Net as a dependency to your project, in addition to JanusGraph.Net. There were no breaking changes between Gremlin.Net 3.7.0 which JanusGraph.Net 1.0.0 uses by default and 3.7.2.