With CST parser newline is ignored/skipped after comment

[tiithansen]

Describe the bug

Newline which should exist after comment is not serialized and start of array element is serialized directly after comment. This bug does not occur when there is indent before array element. - name: a #Comment a 3 instead of - name: a #Comment a 3.

Input:

version: 1.0.0
#
# Test
services: # Comment 1
# Comment 2
- name: a #Comment a 3
  extraLabels:
    k8s-app: some-app
  # Comment 4
  prop: 1

Output:

version: 1.0.0
#
# Test
services: # Comment 1
# Comment 2- name: a #Comment a 3
  extraLabels:
    k8s-app: some-app
  # Comment 4
  prop: 1

JSON dump of services map key where it can be seen that last element of sep is not newline.

"start": [
  {
    "type": "comment",
    "offset": 15,
    "indent": 0,
    "source": "#"
  },
  {
    "type": "newline",
    "offset": 16,
    "indent": 0,
    "source": "\n"
  },
  {
    "type": "comment",
    "offset": 17,
    "indent": 0,
    "source": "# Test"
  },
  {
    "type": "newline",
    "offset": 23,
    "indent": 0,
    "source": "\n"
  }
],
"key": {
  "type": "scalar",
  "offset": 24,
  "indent": 0,
  "source": "services"
},
"sep": [
  {
    "type": "map-value-ind",
    "offset": 32,
    "indent": 0,
    "source": ":"
  },
  {
    "type": "space",
    "offset": 33,
    "indent": 0,
    "source": " "
  },
  {
    "type": "comment",
    "offset": 34,
    "indent": 0,
    "source": "# Comment 1"
  },
  {
    "type": "newline",
    "offset": 45,
    "indent": 0,
    "source": "\n"
  },
  {
    "type": "comment",
    "offset": 46,
    "indent": 0,
    "source": "# Comment 2"
  }
]

Expected behaviour
Should keep exact same output.

Versions (please complete the following information):

• Environment: Node v20.5.0
• yaml: 2.3.4

Additional context

Method used to load:

constructor(file: string) {
    const contents = fs.readFileSync(file).toString()

    let foundDocument = undefined
    for (const token of new Parser().parse(contents)) {
      this.content.push(token)
      if (token.type === 'document') {
        foundDocument = token
      }
    }

    if (foundDocument) {
      this.document = foundDocument
    } else {
      console.error('Document is empty')
      process.exit(1)
    }
  }

Method to save

private stringifyContent(input: Array<CST.Token>) {
    let string = ''
    for (const idx in input) {
      string += CST.stringify(input[idx])
    }
    return string
  }

[eemeli]

Yup, confirming that's a bug. This looks like the minimal case for triggering this:

let src = 'a:\n#\n- b'
let [doc] = Array.from(new YAML.Parser().parse(src))
YAML.CST.stringify(doc)
// 'a:\n#- b'

[eemeli]

The newline was getting dropped because the - sequence indicator's treatment as whitespace wasn't covered by a detector for comments on empty values in block mappings, because their assignment needs to be adjusted during the parse.