ace-grammar

Transform a JSON grammar into an ACE syntax-highlight parser

A simple and light-weight (~ 57kB minified, ~ 19kB zipped) ACE add-on

to generate syntax-highlight parsers (ace modes) from a grammar specification in JSON format.

see also:

• Abacus advanced Combinatorics and Algebraic Number Theory Symbolic Computation library for JavaScript, Python
• Plot.js simple and small library which can plot graphs of functions and various simple charts and can render to Canvas, SVG and plain HTML
• HAAR.js image feature detection based on Haar Cascades in JavaScript (Viola-Jones-Lienhart et al Algorithm)
• HAARPHP image feature detection based on Haar Cascades in PHP (Viola-Jones-Lienhart et al Algorithm)
• FILTER.js video and image processing and computer vision Library in pure JavaScript (browser and node)
• Xpresion a simple and flexible eXpression parser engine (with custom functions and variables support), based on GrammarTemplate, for PHP, JavaScript, Python
• Regex Analyzer/Composer Regular Expression Analyzer and Composer for PHP, JavaScript, Python
• GrammarTemplate grammar-based templating for PHP, JavaScript, Python
• codemirror-grammar transform a formal grammar in JSON format into a syntax-highlight parser for CodeMirror editor
• ace-grammar transform a formal grammar in JSON format into a syntax-highlight parser for ACE editor
• prism-grammar transform a formal grammar in JSON format into a syntax-highlighter for Prism code highlighter
• highlightjs-grammar transform a formal grammar in JSON format into a syntax-highlight mode for Highlight.js code highlighter
• syntaxhighlighter-grammar transform a formal grammar in JSON format to a highlight brush for SyntaxHighlighter code highlighter
• SortingAlgorithms implementations of Sorting Algorithms in JavaScript
• PatternMatchingAlgorithms implementations of Pattern Matching Algorithms in JavaScript

Note: The invariant codebase for all the *-grammar add-ons resides at editor-grammar repository (used as a git submodule)

Contents

• Live Playground Example
• Todo
• Features
• How To use
• API Reference
• Grammar Reference
• Other Examples

Todo

Code Indentation is ACE default, see Modularity and Future Directions

• handle arbitrary, user-defined, code (de-)indentation in the grammar specification (e.g via indent action tokens)
• handle arbitrary, user-defined, code matching (e.g brackets, tags, etc..) in the grammar specification (e.g via match action tokens) [DONE]

• handle arbitrary, user-defined, local/global/scoped relations in the grammar specification (e.g via scope action tokens) [DONE]
• and so on..
• enable grammar add-on to pre-compile a grammar specification directly into mode source code, so it can be used without the add-on as standalone mode [TODO, maybe]

Features

• A Grammar can extend other Grammars (so arbitrary variations and dialects can be handled more easily)
• Grammar includes: Style Model , Lex Model and Syntax Model (optional), plus a couple of settings (see examples)
• Grammar specification can be minimal, defaults will be used (see example grammars)
• Grammar.Syntax Model can enable highlight in a more context-specific way, plus detect possible syntax errors and display appropriate error messages (see below)
• Grammar.Syntax Model can contain recursive references (see /test/grammar-js-recursion.html)
• Grammar.Syntax Model can be (fully) specificed using PEG-like notation or BNF-like notation (NEW feature)
• Grammar.Syntax Model implements positive / negative lookahead tokens (analogous to PEG and-/not- predicates) (NEW feature)
• Grammar.Syntax Model can include external (sub-)grammars so that new multiplexed / mixed grammars are created easily and intuitively (see test examples) (NEW feature)
• Grammar can define action tokens to perform complex context-specific parsing functionality, including associated tag matching and duplicate identifiers (see for example xml.grammar example) (NEW feature)
• Generated highlight modes can support toggle comments and keyword autocompletion functionality if defined in the grammar
• Context-sensitive autocompletion extracted directly from the grammar specification (NEW feature)
• Dynamic (Context-sensitive) autocompletion from typed user actions like code/token/symbols (NEW feature)
• Generated highlight modes can support lint-like syntax-annotation functionality generated from the grammar
• Generated highlight modes can support custom, user-defined, code folding functionality from the grammar fold model (NEW feature)
• Generated highlight modes can support custom, user-defined, code token matching functionality from the grammar match model (NEW feature)
• Generated parsers are optimized for speed and size
• Can generate a syntax-highlight parser from a grammar interactively and on-the-fly ( see example, http://foo123.github.io/examples/ace-grammar )
• see also Modularity and Future Directions

How to use:

See working examples under /test folder.

An example for XML:

// 1. a partial xml grammar in simple JSON format
var xml_grammar = {
    
// prefix ID for regular expressions, represented as strings, used in the grammar
"RegExpID"                          : "RE::",

"Extra"                     : {
    
    "fold"                  : "xml"
    
},
    
// Style model
"Style"                             : {
    
     "declaration"                  : "keyword"
    ,"doctype"                      : "string"
    ,"meta"                         : "string"
    ,"comment"                      : "comment"
    ,"cdata"                        : "string"
    ,"atom"                         : "string"
    ,"tag"                          : "keyword.tag"
    ,"attribute"                    : "variable"
    ,"number"                       : "constant.numeric"
    ,"string"                       : "string"
    ,"error"                        : "invalid"

},

// Lexical model
"Lex"                               : {
     
     "comment:comment"              : ["<!--", "-->"]
    ,"declaration:block"            : ["<?xml", "?>"]
    ,"doctype:block"                : ["RE::/<!doctype\\b/i", ">"]
    ,"meta:block"                   : ["RE::/<\\?[_a-zA-Z][\\w\\._\\-]*/", "?>"]
    ,"cdata:block"                  : ["<![CDATA[", "]]>"]
    ,"open_tag"                     : "RE::/<((?:[_a-zA-Z][_a-zA-Z0-9\\-]*:)?[_a-zA-Z][_a-zA-Z0-9\\-]*)\\b/"
    ,"close_tag"                    : "RE::/<\\/((?:[_a-zA-Z][_a-zA-Z0-9\\-]*:)?[_a-zA-Z][_a-zA-Z0-9\\-]*)>/"
    ,"attribute"                    : "RE::/[_a-zA-Z][_a-zA-Z0-9\\-]*/"
    ,"string:line-block"            : [["\""], ["'"]]
    ,"number"                       : ["RE::/[0-9]\\d*/", "RE::/#[0-9a-fA-F]+/"]
    ,"atom"                         : ["RE::/&#x[a-fA-F\\d]+;/", "RE::/&#[\\d]+;/", "RE::/&[a-zA-Z][a-zA-Z0-9]*;/"]
    ,"text"                         : "RE::/[^<&]+/"
    
    // actions
    ,"@tag:action"                  : {"context":true}
    ,"tag@:action"                  : {"context":false}
    ,"@unique_id:action"            : {"unique":["xml", "$1"],"msg":"Duplicate id value \"$0\"","mode":"hash"}
    ,"@unique_att:action"           : {"unique":["att", "$0"],"msg":"Duplicate attribute \"$0\"","mode":"hash","in-context":true}
    ,"@tag_opened:action"           : {"push":"<$1>","ci":true}
    ,"@tag_closed:action"           : {"pop":"<$1>","ci":true,"msg":"Tags \"$0\" and \"$1\" do not match"}
    ,"@tag_autoclosed:action"       : {"pop":null}
    ,"@autocomplete:action"         : {"define":["autocomplete","$1"],"msg":false,"autocomplete":true,"mode":"hash"}
    ,"@out_of_place:error"          : "\"$2$3\" can only be at the beginning of XML document"
    
},
    
// Syntax model (optional)
"Syntax"                            : {
     
     "tag_att"                      : "'id'.attribute @unique_att '=' string @unique_id | attribute @unique_att '=' (string | number)"
    ,"start_tag"                    : "open_tag.tag @tag @autocomplete @tag_opened tag_att* ('>'.tag | '/>'.tag @tag_autoclosed) tag@"
    ,"end_tag"                      : "close_tag.tag @autocomplete @tag_closed"
    ,"xml"                          : "(^^1 declaration? doctype?) (declaration.error @out_of_place | doctype.error @out_of_place | comment | meta | cdata | start_tag | end_tag | atom | text)*"
    
},
    
// what to parse and in what order
"Parser"                            : [ ["xml"] ]

};

// 2. parse the grammar into an ACE syntax-highlight mode
var xml_mode = AceGrammar.getMode( xml_grammar );

// enable user-defined code folding in the specification (new feature)
xml_mode.supportCodeFolding = true;

// enable syntax lint-like validation in the grammar
xml_mode.supportGrammarAnnotations = true;

// enable user-defined autocompletion (if defined)
xml_mode.supportAutoCompletion = true;
xml_mode.autocompleter.options = {prefixMatch:true, caseInsensitiveMatch:false};
// or for context-sensitive autocompletion, extracted from the grammar
xml_mode.autocompleter.options = {prefixMatch:true, caseInsensitiveMatch:false, inContext:true};
// or for dynamic (context-sensitive) autocompletion, extracted from user actions
xml_mode.autocompleter.options = {prefixMatch:true, caseInsensitiveMatch:false, inContext:true|false, dynamic:true};

// 3. use it with ACE
var editor = ace.edit("editor");
editor.setValue( document.getElementById("code").value, -1 );
editor.getSession().setMode( xml_mode );
editor.getSession().setFoldStyle("markbegin"); // code-folding
editor.getSession().setOptions({useWorker: true}); // lint-like annotations through workers

Result:

Other Examples: