Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3: preserve comments lines and indents after unmarshal -> marshal #709

Open
Unix4ever opened this issue Mar 9, 2021 · 3 comments
Open

Comments

@Unix4ever
Copy link

Unix4ever commented Mar 9, 2021

I wonder if it is possible to preserve the same positions for all decoded comments, when you unmarshal into yaml.Node and then marshal that yaml.Node back to yaml.

So for example:

 block1:
    # the comment
    map:
        key1: a
        key2: b
    
    # foot comment
block2:
    hi: there

Turns into:

 block1:
    # the comment
    map:
        key1: a
        key2: b
        # foot comment
block2:
    hi: there

Where foot comment changes it's indent and becomes foot comment of key2 (?).

Here's the code snippet:

package main

import (
	"log"
	"strings"
	
	yaml "gopkg.in/yaml.v3"
)

func main() {
	var node yaml.Node
	
	data := []byte(strings.TrimSpace(`
block1:
    # the comment
    map:
        key1: a
        key2: b
    
    # foot comment
block2:
    hi: there

`))

	log.Printf("INPUT:\n %s", data)

	if err := yaml.Unmarshal(data, &node); err != nil {
		log.Fatalf("Unmarshalling failed %s", err)
	}
	
	results, err := yaml.Marshal(node.Content[0])
	if err != nil {
		log.Fatalf("Marshalling failed %s", err)
	}
	
	log.Printf("RESULT:\n %s", results)
}

https://play.golang.org/p/BtzerXV8ofI
It looks like the comment is attached to the last element in the map, instead of attaching it to the map.

Do you think it's a bug in the library? Or that's intended behavior? Is there any workaround for that?

Unix4ever added a commit to Unix4ever/yaml that referenced this issue Mar 12, 2021
Removed the function that was adding the comments to the last key from
the map and replaced that with the separate comments queue.

Unfold that comment queue each time we get the end of a flow, a document and
a block.
Append comments to the end token if their indents are >= than the first
comment indent (this is safe as we only add comments to that queue if we
detect that their indent is < than the last key indent).

Additionally fixed the bug in the emitter, related to skipping
foot_comments for the mapping end events.

Fixes: go-yaml#709

Signed-off-by: Artem Chernyshev <[email protected]>
Unix4ever added a commit to Unix4ever/yaml that referenced this issue Mar 15, 2021
Removed the function that was adding the comments to the last key from
the map and replaced that with the separate comments queue.

Unfold that comment queue each time we get the end of a flow, a document and
a block.
Append comments to the end token if their indents are >= than the first
comment indent (this is safe as we only add comments to that queue if we
detect that their indent is < than the last key indent).

Additionally fixed the bug in the emitter, related to skipping
foot_comments for the mapping end events.

Fixes: go-yaml#709

Signed-off-by: Artem Chernyshev <[email protected]>
Unix4ever added a commit to Unix4ever/yaml that referenced this issue Mar 15, 2021
Removed the function that was adding the comments to the last key from
the map and replaced that with the separate comments queue.

Unfold that comment queue each time we get the end of a flow, a document and
a block.
Append comments to the end token if their indents are >= than the first
comment indent (this is safe as we only add comments to that queue if we
detect that their indent is < than the last key indent).

Additionally fixed the bug in the emitter, related to skipping
foot_comments for the mapping end events.

Fixes: go-yaml#709

Signed-off-by: Artem Chernyshev <[email protected]>
Unix4ever added a commit to Unix4ever/yaml that referenced this issue Mar 15, 2021
Removed the function that was adding the comments to the last key from
the map and replaced that with the separate comments queue.

Unfold that comment queue each time we get the end of a flow, a document and
a block.
Append comments to the end token if their indents are >= than the first
comment indent (this is safe as we only add comments to that queue if we
detect that their indent is < than the last key indent).

Additionally fixed the bug in the emitter, related to skipping
foot_comments for the mapping end events.

Fixes: go-yaml#709

Signed-off-by: Artem Chernyshev <[email protected]>
Unix4ever added a commit to Unix4ever/yaml that referenced this issue Mar 15, 2021
Removed the function that was adding the comments to the last key from
the map and replaced that with the separate comments queue.

Unfold that comment queue each time we get the end of a flow, a document and
a block.
Append comments to the end token if their indents are >= than the first
comment indent (this is safe as we only add comments to that queue if we
detect that their indent is < than the last key indent).

Additionally fixed the bug in the emitter, related to skipping
foot_comments for the mapping end events.

Fixes: go-yaml#709

Signed-off-by: Artem Chernyshev <[email protected]>
@niemeyer
Copy link
Contributor

On a first sight this seems to be a bug. It should remain as a foot comment of the key, otherwise it'll keep walking down the hierarchy on future encode/decode cycles. I'll have a look at this on the next update.

Thanks for the report.

@wburningham
Copy link

@niemeyer how big of a lift is it to fix this bug? I also have seen this issue.

@rafaela-soares
Copy link

Is there any update about this issue? I am facing a similar problem.

Original file content:

---
# this playbook creates users and projects in openstack,
# it's used for a "local authentication" mode, not for SSO

- name: Playbook to Create users and Projects in Openstack
  hosts: localhost
  collections:
    - openstack.cloud
  tasks:
    - name: 'Ensure Projects are as defined'
      include: subroutines/openstack_per_project_actions.yaml

    - name: 'Create Users in Openstack'
      openstack.cloud.identity_user:
        state: present
        name: "{{ add_user.name }}"
        password: "{{ all_openstack_default_pass }}"
        email: "{{ add_user.email }}"
        # kics-scan ignore-line
        update_password: on_create
        default_project: "{{ add_user.orgunit }}"
        domain: default
      loop: "{{ users_present }}"
      loop_control:
        loop_var: add_user

After after unmarshal -> marshal:

# this playbook creates users and projects in openstack,

# it's used for a "local authentication" mode, not for SSO
- name: Playbook to Create users and Projects in Openstack
  hosts: localhost
  collections:
    - openstack.cloud
  tasks:
    - name: 'Ensure Projects are as defined'
      include: subroutines/openstack_per_project_actions.yaml
    - name: 'Create Users in Openstack'
      openstack.cloud.identity_user:
        state: present
        name: "{{ add_user.name }}"
        password: "{{ all_openstack_default_pass }}"
        email: "{{ add_user.email }}"
        # kics-scan ignore-line

        update_password: on_create
        default_project: "{{ add_user.orgunit }}"
        domain: default
      loop: "{{ users_present }}"
      loop_control:
        loop_var: add_user

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants