-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Getting this error: Exception in thread "main" java.lang.UnsupportedOperationException: Serialized ATN data element out of range #840
Comments
likely classpath issue. remove any prior from class path
|
I've already tryed with this, but same result: #!/bin/zsh And I'm sure I'm using the right antlr jar, if I add my debugs and rebuild the jar I see them appearing: nsets: 29 These are the debugs I've added to ATNSerializer.java:
I've seen #76 that seem very similar to my problem but is now fixed; the file attacched for issue 76 (https://gist.github.com/sharwell/3955844#file-example-g4) succeeds in my env; so seems like really I don't have any classpath issue. Can you please try the file attached to this issue (https://gist.github.com/agiovacchini/52cc6dc9210a65dfc659#file-example-g4) in your env and see if it succeeds? If I make it smaller by cutting part of the "ingr:" entryes it succeeds in my env but not with the full list, maybe I've hitten some other limitation in data size in JDK/ANTLR? |
have you completely regenerated all your grammars? are you sure 4.5.1 is built correctly? |
Yes, I have regenerated everything, before running again I did this: This is the content of bild.log: [03/20/15 09:39:20 ./bild.py:42 bilder.py:124] platform=darwin But I've tryed building myself after using with the same error the prebuilded jar downloaded from the ANTLR site. Does my grammar work in your environment? |
with 4.5 i get error and with 4.5.1. weird!
I'll check at work. |
The ATN for the grammar (most likely the number of states and/or the number of transitions) is larger than the current representation is capable of encoding. |
Thank you for support. Is there some kind of workaround to this problem? |
@agiovacchini There's not that I know of right now. The serialization logic could be rewritten to use compressed integers like the ones used in ECMA-335 (bytecode for .NET), but it wouldn't be a small undertaking. It's arguably a good idea in the long run though. |
Ok, so the only option is to have a smaller grammar, right? |
I've fixed the same issue caused such a grammer in my .g4 file: I've modified it as below and then compiled successfully: |
In my case I have many different literal strings to use as key in grammar but maybe I've found a workaround done this way:
It takes a bit of work to start using but then allows for many more possible keys given the less space used by one. |
Hiya. I guess I'll close and I'll likely not fix this. thanks for documenting though!! |
I had the same issue with the grammar of language I was writing. One thing you can do to avoid this is to have one token type for several operators with the same precedence (e.g., relational operators) instead of separate tokens. It seems to help. It would be nice if this was fixed eventually... |
I am seeing this error, I guess it still exists. mpPath = C:\Users\ADAM~1.SMI\AppData\Local\Temp/jhades eleting temporary directory |
yep, size limitation. try to simplify your lexer or parser. |
shoudn't antlr produce some kind user-friendly message in this case? |
yup. |
I had a similar problem, so I began commenting out lines in my grammar until the error went away. |
@DocPlaid you can also use |
Sorry to raise this question again! I am trying to build a large dictionary of about 200,000 words (which use simple regular expressions to cover for edge cases. Unfortunately, I am running into the mentioned issue as well. From what I understand, this is due to the reason that the number of internal ATN states is limited to `// create lexer CommonTokenStream tokens = new CommonTokenStream(lexEngine); // parse text This is an example of how the rules look like (but with a lot more elements) `lexer grammar TestLexer; T_0 : LETTER_A FILL_TOKEN? FILL_TOKEN? FILL_TOKEN? LETTER_B FILL_TOKEN? FILL_TOKEN? FILL_TOKEN? LETTER_F FILL_TOKEN? FILL_TOKEN? FILL_TOKEN? '1000' FILL_TOKEN? FILL_TOKEN? FILL_TOKEN? LETTER_C ; SKIP_RULE : . -> skip ;` Right now, I am generating N Lexer grammars and execute them one by one (in threads), but this takes several seconds to complete. Was hoping for a solution, which can process the input text within miliseconds. Is there anything I am missing? Thank you very much in advance! |
unfortunately, Java can only do strings for static data; no way to make a static array of ints that just becomes available as a global or field. So, we had to encode in strings, using 16bit chars as ints :( |
@mullekay Excuse me, I need to build a large dictionary of about 200,000+ words and get this error: Exception in thread "main" java.lang.UnsupportedOperationException: Serialized ATN data element out of range , did you find any solution?I'll appreciate it very much if you lend me a hand. |
…er.MAX_VALUE) fix antlr#840, fix antlr#1863, fix antlr#2732, fix antlr#3338
Reopening as we are now going to allow very large ATN. |
Fixed by #3591 |
I'm using last version pulled out from git (4.5.1) and built under OSx Yosemite, tryed both with jdk6 and jdk7 but with my file, which is: https://gist.github.com/agiovacchini/52cc6dc9210a65dfc659#file-example-g4 I get this error:
Exception in thread "main" java.lang.UnsupportedOperationException: Serialized ATN data element out of range
at org.antlr.v4.runtime.atn.ATNSerializer.serialize(ATNSerializer.java:384)
at org.antlr.v4.runtime.atn.ATNSerializer.getSerialized(ATNSerializer.java:563)
at org.antlr.v4.codegen.model.SerializedATN.(SerializedATN.java:46)
at org.antlr.v4.codegen.model.Recognizer.(Recognizer.java:87)
at org.antlr.v4.codegen.model.Lexer.(Lexer.java:51)
at org.antlr.v4.codegen.OutputModelController.lexer(OutputModelController.java:176)
at org.antlr.v4.codegen.OutputModelController.buildLexerOutputModel(OutputModelController.java:129)
at org.antlr.v4.codegen.CodeGenerator.generateLexer(CodeGenerator.java:142)
at org.antlr.v4.codegen.CodeGenPipeline.process(CodeGenPipeline.java:73)
at org.antlr.v4.Tool.processNonCombinedGrammar(Tool.java:428)
at org.antlr.v4.Tool.process(Tool.java:378)
at org.antlr.v4.Tool.processGrammarsOnCommandLine(Tool.java:345)
at org.antlr.v4.Tool.main(Tool.java:192)
The launch script is like this:
!/bin/zsh
export CLASSPATH=".:./antlr4/dist/antlr4-4.5.1-complete.jar"
alias antlr4='java -jar ./antlr4/dist/antlr4-4.5.1-complete.jar'
alias grun='java org.antlr.v4.runtime.misc.TestRig'
antlr4 example.g4
The text was updated successfully, but these errors were encountered: