Initialise debug_cb{,_data} in jq_init(); Populate captures also for zero-width matches; In the manual, define examples in the "examples" field not in "example"

[emanuele6]

Only examples defined in the "examples" field are generated, if they are defined in "example", they are just ignored.

Ref: #2547 (comment)

Screenshots from https://jqlang.github.io/jq/manual:

[emanuele6]

A few of the manonig.test tests that were not being generated fail/leak

[emanuele6]

I need to use + instead of * as a consequece of #2641

EDIT: actually, since the input is "123abc456def", and even if it were "123abc456def7", it should have output "ZabcZdefZ", not "ZabcZdefZnull"; this should not be necessary.
It seems to be a bug in _match_impl/3, if the pattern is able to match the empty string, it always matches the empty string at the end of the input:

$ ./jq -n '"1" | _match_impl("1*"; "g"; false)'
[
  {
    "offset": 0,
    "length": 1,
    "string": "1",
    "captures": []
  },
  {
    "offset": 1,
    "length": 0,
    "string": "",
    "captures": []
  }
]

---

The reason why this didn't work is that #2677 introduced a breaking change: If a pattern that is able to match the empty string has a match that goes up to the end of string, both the match and the empty string at the end of the string are matched (like for javascript's .replaceAll(), perl's s///g, and python's re.sub()) instead of matching only the match and not the empty string at the end that was jq's old behaviour (similar to vim's and sed's s///g, awk's gsub()).

The reson why "123foo" | sub("[^a-z]*(?<x>[a-z]+)"; "Z\(.x)"; "g") outputs "ZfooZnull" instead of "ZfooZ" is that match/2 currently does not populate the captures array for zero length matches, so for the empty string at the end of "123foo", "Z\(.x)" runs with an empty object as input.
I have now fixed this problem in one the commits in this PR; the captures array is also populated for empty matches.

[pkoppstein]

Hopefully someone can tweak things so that both "example:" and "examples:" are recognized.
That would make things much easier, not least for those who might only want to give one example :-)

[emanuele6]

Hopefully someone can tweak things so that both "example:" and "examples:" are recognized.
That would make things much easier, not least for those who might only want to give one example :-)

I don't see the point of doing that.

[pkoppstein]

I don't see the point of doing that

Oh.

1. If there's only one example, then the heading should not be "examples:". For one thing, it's misleading; for another, it's just wrong.
2. If "examples:" is required, then writers who are not aware of the requirement will undoubtedly (as they have done) write "example:" when there is only one example, thus introducing grief, possibly for many people.
3. If "examples:" is required, then writers who are aware of the requirement and who want to avoid being misleading will write more than one example, even if one example suffices. This leads to bloat and may well be annoying to more than just the aforesaid writer.

[emanuele6]

@pkoppstein

But that is not an heading, it's a yaml file, and that part is the "examples" key of a yaml object.

It would be like having a JSON file in which you can use either {"queries":["foo","bar"]} or {"query":["foo","bar"]} which do exactly the same thing.

I don' think that is good, it just makes the formatted data harder to use for users. e.g. if a user wants to extract the examples from that file, they can do:

$ fq '.sections[].entries[]?.examples | arrays[]' docs/content/manual/manual.yml
{
  "input": "\"Hello, world!\"",
  "output": [
    "\"Hello, world!\""
  ],
  "program": "."
}
{
  "input": "12345678909876543212345",
  "output": [
...

If there are aliases for examples, now the user has to use:

$ fq '.sections[].entries[] | first(.["examples", "example", "Examples"] | arrays)[]' docs/content/manual/manual.yml

And you have to make sure that the precedence order you use for the query is the same that other programs that use that file use (the website generator, the man.test generator, etc).

Those types of things in structured data are the opposite of user friendly imho.

[pkoppstein]

But that is not an heading, it's a yaml file

Well, I see your point, but the manual is primarily for humans (and hopefully for enjoyment and not just enlightenment). YAML is just a tool. Or as Lewis Carrol wrote:

'The question is,' said Humpty Dumpty, 'which is to be master — that's all.

[itchyny]

If some CI (maybe manpage?) fails against unknown keys with descriptive error messages, that would be okay.

[nicowilliams]

LGTM!

[nicowilliams]

Thanks!