Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: adding an option to control style of strings #170

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,49 @@ parses this metadata, re-applies the tags and styles, and discards the extra pai

yq does not support passing YAML comments into the JSON representation used by jq, or roundtripping such comments.

Forcing string styles using the ``-z`` (``--yaml-string_styles``) option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The ``-z`` option will assume that strings might start with some style information.
This option is only useful when using YAML-output (using ``-y`` or ``-Y``)

To control the string style, the string itself has to be prepanded by additional information.
Valid control strings are:

* ``__yq_style_'__``: uses single quotes ``'``
* ``__yq_style_"__``: uses double quotes ``"``
* ``__yq_style_>__`` uses ``>``, ``>`` or ``>-``, (depending on if the text has trailing break lines)
* ``__yq_style_|__`` uses ``|``, ``|+`` or ``|-``, (depending on if the text has trailing break lines)

Assume you have some input file ``input.yaml``::

field1: "I am a\nmultiline string"

Example usage::

yq -y -z '.field1 |= "__yq_style_|__" + .' input.yaml

This will output::

field1: |
I am a
multiline string

The usage can be simplified by adding the function ``style`` to ``~/.jq`` and/or to your scripts::

# remove existing styles
def style: if test("^__yq_style_.__") then .[14:] | style else . end;

# set a style
def style(s):
if s | length == 1 then "__yq_style_" + s + "__" + (. | style) # remove previous styles
else error("Only valid symbols are \", ', |, >, _")
end;


This allows to simpify the above example to::

yq -y -z '.field1 |= style("|")' input.yaml

XML support
-----------
``yq`` also supports XML. The ``yq`` package installs an executable, ``xq``, which
Expand Down
2 changes: 2 additions & 0 deletions yq/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,7 @@ def yq(
expand_aliases=True,
max_expansion_factor=1024,
yaml_output_grammar_version="1.1",
yaml_string_styles=False,
jq_args=frozenset(),
exit_func=None,
):
Expand Down Expand Up @@ -261,6 +262,7 @@ def yq(
use_annotations=use_annotations,
indentless=indentless_lists,
grammar_version=yaml_output_grammar_version,
use_string_styles=yaml_string_styles
)
yaml.dump_all(
decode_docs(jq_out, json_decoder),
Expand Down
30 changes: 29 additions & 1 deletion yq/dumper.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,27 @@ def ignore_aliases(self, data):
yaml_item_annotation_re = re.compile(r"^__yq_(?P<type>tag|style)_(?P<key>\d+)_(?P<value>.+)__$")


def get_dumper(use_annotations=False, indentless=False, grammar_version="1.1"):
def extractStringStyle(v):
if v.value.startswith("__yq_style_|__"):
v.value = v.value[14:]
v.style = '|'
elif v.value.startswith("__yq_style_>__"):
v.value = v.value[14:]
v.style = '>'
elif v.value.startswith("__yq_style_'__"):
v.value = v.value[14:]
v.style = '\''
elif v.value.startswith("__yq_style_\"__"):
v.value = v.value[14:]
v.style = '"'
elif v.value.startswith("__yq_style____"):
v.value = v.value[14:]
v.style = None

return v


def get_dumper(use_annotations=False, indentless=False, grammar_version="1.1", use_string_styles=False):
# if not (use_annotations or indentless):
# return default_dumper

Expand Down Expand Up @@ -55,6 +75,7 @@ def represent_dict(dumper, data):
v.flow_style = True
if hashed_key in custom_tags:
v.tag = custom_tags[hashed_key]

return mapping

def represent_list(dumper, data):
Expand All @@ -79,10 +100,17 @@ def represent_list(dumper, data):
v.flow_style = True
if str(i) in custom_tags:
v.tag = custom_tags[str(i)]

return sequence

def represent_str(dumper, data):
scalar = dumper.represent_scalar("tag:yaml.org,2002:str", value=data)
return extractStringStyle(scalar)

dumper = OrderedIndentlessDumper if indentless else OrderedDumper
dumper.add_representer(dict, represent_dict)
dumper.add_representer(list, represent_list)
if use_string_styles:
dumper.add_representer(str, represent_str)
set_yaml_grammar(dumper, grammar_version=grammar_version)
return dumper
6 changes: 6 additions & 0 deletions yq/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,12 @@ def get_parser(program_name, description):
parser.add_argument(
"--yaml-output-grammar-version", "--yml-out-ver", choices=["1.1", "1.2"], default="1.1", help=grammar_help
)
parser.add_argument(
"--yaml-string-styles",
"--yml-string-styles",
"-z",
action="store_true",
help="Allows special strings to control style of strings (only valid in combination with -y or -Y)")
parser.add_argument("--width", "-w", type=int, help=width_help)
parser.add_argument("--indentless-lists", "--indentless", action="store_true", help=indentless_help)
parser.add_argument("--explicit-start", action="store_true", help=explicit_start_help)
Expand Down