Cannot read property 'messageStream' of undefined #47
Comments
|
Does the server output have anything else before the error? Like a recap of the script submitted or maybe a warning? |
|
No, I don't think so but I have include some messages before the warning, which are typical:- |
|
Ok so all in all it seems like the issue is somewhere between the Is this random or always on that file? On our end we should try to download that file and run tests as well (this file right? : https://github.com/ternjs/acorn/blob/master/AUTHORS ). Which version of gremlin-server are you using? In any event this might be the good time to take care of #20. Thoughts @jbmusso ? |
|
Yes, I agree - I think it gets caught up on a character sequence in the file. But I think the opProcessor error is a symptom of the serializer not being able to deserialize the message (PooledUnsafeDirectByteBuf). I am going to do some more testing because it's not too hard to find other files that cause the problem.
|
|
Thanks for sending a detailed issue, I'll try to have a look at it this On Tuesday, 12 April 2016, David Crouch [email protected] wrote:
Jean-Baptiste |
|
For kicks I tried reading the raw contents of the file with this:- ...and got this error message below indicating that the message type is not supported. I thought I had read somewhere that Gremlin client had been modified recently to support binary data. Is this not correct? Am I doing something wrong? Do I need to explicitly set the Mime type to something other than the default? BTW, the "runAs" in the script below is a custom extension, which essentially wraps standard Gremlin. |
|
I'm not sure I really understand what you mean but to clarify : gremlin-javascript sends the messages in binary but the payload should still be in JSON. This is a little different from supporting a binary payload. Here the server is receiving your javascript binary map and is treating it like a LinkedHashMap which it seems to choke on. You could base64 it to a string instead. |
|
I read the thread carefully, and I don't have a single clue of what could be going wrong. Bleh. The error indicates that when handling a server message, the client fails to find the corresponding pending command (= sent query) for that message. The client looks into a dictionary of pending commands ( |
|
I added the following log messages to GremlinClient:- I ran essentially the same test, which generated the following log. I have included more log than necessary so you can see some successful communication requests before the failure. The important one is that it successfully uploads ".travis.yml" and then later fails on "AUTHORS". And here is the trace from the Gremlin Server log (same as before). The server is having trouble deserializing the message:- |
|
I ran some more tests and was able to reproduce the problem in other files. Most of the time when a file upload failed I re-ran the same test to confirm it would fail on the same file (when I remembered :-)). The failures were somewhat random on when they would occur i.e. 20 files, 50 files, 70 files into the upload. I captured the log and saved it in these Gists:- https://gist.github.com/davidcrouch/75d455ecaa07051635f78785fb0ee715 |
|
This might be a little tedious but can you try this against |
|
Unfortunately migrating to 3.1.1 is not so simple since we have custom traversal strategies that support what we submit on the client e.g. authentication and authorization. However, I'm wondering if this problem can easily be reproduced inside a standard/clean install using a quick and dirty utility? Maybe its a test you could try? I'm basically walking through a NodeJS node_modules modules directories, reading the contents of each file found inside modules, creating a new vertex and uploading the file contents into a vertex property. It seems from the tests I've run that something in the payload is causing problems for the Gremlin Server to deserialize, and hence an empty message is returned (requestId = null) causing the Gremlin client to fail. |
|
I set this up locally and get the same result on I'm guessing there's a problem with gremlin-javascript binary serialization. I noticed that your failing gists all contain some special chars. |
|
FYI this breaks : const Gremlin = require('gremlin');
const client = Gremlin.createClient();
client.execute('ö'); |
|
What about if you try this? |
|
Same result. I think I'm narrowing down on this issue. If I understand correctly gremlin-server and the binary serialization of the javascript driver are currently expecting As a demonstration of this, the following fixes the issue ( var serializedMessage = this.options.accept + JSON.stringify(message);
serializedMessage = unescape(encodeURIComponent(serializedMessage));I haven't really fully tested this in browser but a manual nodejs test works. @jbmusso it might be wise to use some dependency here as they will probably cover more cases than we can imagine (though I know the thought of one more dependency is a pain for browser implementations). Or we can fully test the fix above and just roll with that, but I believe there are going to be some issues with the above as per : http://stackoverflow.com/questions/19752988/javascript-charat-breaking-multibyte-character-string and I found some other edge cases with |
|
Yes, the fix appears to be working for me. I've tested it on 3 of the failed Gists and it now successfully completes for those. I'm now in the process of running a large test (~2000 files) and will let you know how it goes. |
|
Bummer :). Thanks for investigating on this. I was about to add a comment as another user reported issues with v2.3.1 when sending accentuated characters. I'm not aware of any lightweight dependency that would make it easy to support this both in Node.js and browser environments, so I'll have to investigate. |
|
We can always implement the above fix in the meantime. I've run the tests in both node and browser and the above works. It might not cover 100% of all cases but it'll be heaps better than currently. I'll make a PR on the off chance it's worth merging. |
|
FYI, I ran a test for uploading the contents of about 4000 files and it ran with no problems. The files were taken from modules typically found in the NodeJS node_modules directory. |
|
I appreciate your help. The fix (+ test) has been published on npm as v2.3.2. |
UPDATE: The problem I'm seeing may be related to Issue #29 and possibly addressed in PR #31 (or at least provide a better/safer foundation for supporting what I'm trying to do).
I've created a NodeJs utility for uploading multiple files to a Neo4j database via Tinkerpop 3 (one request per file). Essentially, a vertex with a "Content" property is created that represents the file.
I've seen the error sometimes appear after uploading 800+ files but on this occasion it was after just 30 files i.e. doesn't appear to be overloading buffer maximums. I was able to remove all the other files and reproduce it with the one text file in question. The files are almost all ".js" text files so they are not large.
I read the contents of the file with:-
The Gremlin script adds a vertex with a 'Content' property containing the contents of the file. When I submit the script using the gremlin client "execute" method I receive the following:-
I don't believe the problem is with the script since it can successfully complete for 1000s of transactions before seeing this error.
Here is my yaml file:-
On the server I'm seeing the error below. The "PooledUnsafeDirectByteBuf" tells me that the message is being submitted by the client in the wrong format:-
I tried to attach the file causing the problem but the Github Issue page gave an error saying "we don't support that file type". This provides a hint that perhaps the mime type is defaulting to the wrong value causing the server to fail.
The file is the "AUTHORS" file (no suffix) from the NPM "Acorn" node module and simply contains:-
The text was updated successfully, but these errors were encountered: