Incorrect parsing of comments after preprocessor directives

[amachronic]

#define FOO 1 /* this comment is not parsed as a comment */
#define BAR   /* although this comment is */

I found this while trying emacs-tree-sitter. Its debug output:

translation_unit:
  preproc_def:
    identifier:
    preproc_arg:
  preproc_def:
    identifier:
    comment:

The root of the problem seems to be preproc_arg:

preproc_arg: $ => token(prec(-1, repeat1(/.|\\\r?\n/)))

The C preprocessor interprets both // and /* */ style comments and replaces them with a single whitespace according to C99. It seems like token might the root cause of the issue by preventing tree-sitter from applying extras (which normally handles comments).

This would also be the cause of #55 and #44, also this issue in tree-sitter-cpp.

[siraben]

I ran into this as well.

#ifndef PI_TIMER_H
#define PI_TIMER_H

// System Timer defined in BCM2837 section 12 (page 172)
// For pi < 2 base = 0x7E003000 For pi 2&3 base = 0x3F003000

#define SYSTEM_TIMER_BASE 0x3F003000      // Physical address (processor view)
#define CS_REG (SYSTEM_TIMER_BASE + 0x0)  // Control Status register

translation_unit:
  preproc_ifdef:
    identifier:
    preproc_def:
      identifier:
    comment:
    comment:
    preproc_def:
      identifier:
      preproc_arg:
    preproc_def:
      identifier:
      preproc_arg: