help on how to use ocaml-treesitter

[AbhinavDutta]

$ ./gumtree  parse -g ocaml-treesitter  demo.ml
Error while running client 'parse'.
com.github.gumtreediff.gen.SyntaxException
	at com.github.gumtreediff.gen.ExternalProcessTreeGenerator.readStandardOutput(ExternalProcessTreeGenerator.java:43)
	at com.github.gumtreediff.gen.treesitter.AbstractTreeSitterGenerator.generate(AbstractTreeSitterGenerator.java:37)
	at com.github.gumtreediff.gen.TreeGenerator.generateTree(TreeGenerator.java:41)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.reader(TreeGenerator.java:119)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:90)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:100)
	at com.github.gumtreediff.gen.TreeGenerators.getTree(TreeGenerators.java:74)
	at com.github.gumtreediff.client.ParseClient.getTreeContext(ParseClient.java:63)
	at com.github.gumtreediff.client.ParseClient.run(ParseClient.java:54)
	at com.github.gumtreediff.client.Run.startClient(Run.java:94)
	at com.github.gumtreediff.client.Run.main(Run.java:128)

demo.ml looks like this :

print_string "Hello world!\n"

Am I using the treesitter correctly ? (I have basically installed https://github.com/GumTreeDiff/tree-sitter-parser and added tree-sitter-parser.py in PATH) . I feel I am not using treesitter properly because I tried to parse PHP files with php-treesitter and faced similar syntaxException but was able to parse the same with php-antlr ! I am guessing maybe some additional configuration needs to be done to use tree-sitter , maybe? Any help would be appreciated !

[jrfaller]

Hi!

This is what I get :

> gumtree parse test.ml

compilation_unit [0,30]
    expression_item [0,29]
        application_expression [0,29]
            value_path [0,12]
                value_name: print_string [0,12]
            string [13,29]
                ": " [13,14]
                string_content [14,28]
                    escape_sequence: \n [26,28]
                ": " [28,29]

You can try out tree sitter parser directly like this :

> tree-sitter-parser.py test.ml ocaml

<?xml version="1.0" ?>
<tree type="compilation_unit" pos="0" length="30">
	<tree type="expression_item" pos="0" length="29">
		<tree type="application_expression" pos="0" length="29">
			<tree type="value_path" pos="0" length="12">
				<tree type="value_name" pos="0" length="12" label="print_string"/>
			</tree>
			<tree type="string" pos="13" length="16">
				<tree type="&quot;" pos="13" length="1" label="&quot;"/>
				<tree type="string_content" pos="14" length="14">
					<tree type="escape_sequence" pos="26" length="2" label="\n"/>
				</tree>
				<tree type="&quot;" pos="28" length="1" label="&quot;"/>
			</tree>
		</tree>
	</tree>
</tree>

What do you get?

Cheers.

[AbhinavDutta]

gumtree parse demo.ml
Error while running client 'parse'.
com.github.gumtreediff.gen.SyntaxException
	at com.github.gumtreediff.gen.ExternalProcessTreeGenerator.readStandardOutput(ExternalProcessTreeGenerator.java:43)
	at com.github.gumtreediff.gen.treesitter.AbstractTreeSitterGenerator.generate(AbstractTreeSitterGenerator.java:37)
	at com.github.gumtreediff.gen.TreeGenerator.generateTree(TreeGenerator.java:41)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.reader(TreeGenerator.java:119)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:90)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:100)
	at com.github.gumtreediff.gen.TreeGenerators.getTree(TreeGenerators.java:58)
	at com.github.gumtreediff.gen.TreeGenerators.getTree(TreeGenerators.java:70)
	at com.github.gumtreediff.client.ParseClient.getTreeContext(ParseClient.java:63)
	at com.github.gumtreediff.client.ParseClient.run(ParseClient.java:54)
	at com.github.gumtreediff.client.Run.startClient(Run.java:94)
	at com.github.gumtreediff.client.Run.main(Run.java:128)

tree-sitter-parser.py test.ml ocaml
Traceback (most recent call last):
  File "/home/abhinav/anaconda3/lib/python3.8/site-packages/tree_sitter/tree-sitter-parser/tree-sitter-parser.py", line 17, in <module>
    Language.build_library(
  File "/home/abhinav/anaconda3/lib/python3.8/site-packages/tree_sitter/__init__.py", line 41, in build_library
    source_mtimes = [path.getmtime(__file__)] + [
  File "/home/abhinav/anaconda3/lib/python3.8/site-packages/tree_sitter/__init__.py", line 42, in <listcomp>
    path.getmtime(path_) for path_ in source_paths
  File "/home/abhinav/anaconda3/lib/python3.8/genericpath.py", line 55, in getmtime
    return os.stat(filename).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '/home/abhinav/anaconda3/lib/python3.8/site-packages/tree_sitter/tree-sitter-parser/tree-sitter-c/src/parser.c'

[AbhinavDutta]

Hi, I just did not clone the tree-sitter repository recursively. It works fine now (just like your outputs!). Thanks for the help!

[jrfaller]

Beware that the Ocaml parser has not been battle tested and there might be some issues. For instance on your example I do not see the value of the string literal... I think it won't hurt to start including ocaml in the rule files (https://raw.githubusercontent.com/GumTreeDiff/tree-sitter-parser/main/rules.yml) in order to have a clean AST for code diff.

[jrfaller]

I think a good way to start is:

ocaml:
  flattened:
    - string
  aliased: []
  ignored: []

[AbhinavDutta]

Yeah. I noticed that too. Is there any way (articles, docs, etc) of understanding how to write the rule file? (I don't really understand what it is that your additional rules mean )