Skip to content

Commit

Permalink
minor fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
biojppm committed Jun 8, 2024
1 parent cb2692a commit 686298f
Show file tree
Hide file tree
Showing 13 changed files with 119 additions and 108 deletions.
94 changes: 46 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -432,37 +432,6 @@ CHECK(loc.col == 4u);
## Using ryml in your project
### Package managers
ryml is available in most package managers (thanks to all the
contributors!) and linux distributions. But please be aware: those
packages are maintained downstream of this repository, so if you have
issues with the package, file a report with the respective maintainer.
Here's a quick roundup (not maintained):
* Package managers:
* [conan](https://conan.io/center/recipes/rapidyaml)
* [vcpkg](https://vcpkg.io/en/packages.html): `vcpkg install ryml`
* [PyPI](https://pypi.org/project/rapidyaml/)
* Linux distributions:
* Arch Linux/Manjaro:
* [rapidyaml (aarch64)](https://archlinuxarm.org/packages/aarch64/rapidyaml)
* [rapidyaml-git (AUR)](https://aur.archlinux.org/packages/rapidyaml-git/)
* [python-rapidyaml-git (AUR)](https://aur.archlinux.org/packages/python-rapidyaml-git/)
* [Fedora Linux](https://getfedora.org/)/[EPEL](https://docs.fedoraproject.org/en-US/epel/):
* `dnf install rapidyaml-devel`
* `dnf install python3-rapidyaml`
* [Gentoo](https://packages.gentoo.org/packages/dev-cpp/rapidyaml)
* [OpenSuse](https://build.openbuildservice.org/package/show/Emulators/rapidyaml)
* [Slackbuilds](https://slackbuilds.org/repository/15.0/libraries/rapidyaml/)
* [AltLinux](https://packages.altlinux.org/en/sisyphus/srpms/rapidyaml/3006055151670528141)
Although package managers are very useful for quickly getting up to
speed, the advised way is still to bring ryml as a submodule of your
project, building both together. This makes it easy to track any
upstream changes in ryml. Also, ryml is small and quick to build, so
there's not much of a cost for building it with your project.
### Single header file
ryml is provided chiefly as a cmake library project, but it can also
be used as a single header file, and there is a [tool to
Expand Down Expand Up @@ -531,6 +500,38 @@ If you omit `--recursive`, after cloning you
will have to do `git submodule update --init --recursive`
to ensure ryml's submodules are checked out.

### Package managers

ryml is available in most package managers (thanks to all the
contributors!) and linux distributions. But please be aware: those
packages are maintained downstream of this repository, so if you have
issues with the package, file a report with the respective maintainer.

Here's a quick roundup (not maintained):
* Package managers:
* [conan](https://conan.io/center/recipes/rapidyaml)
* [vcpkg](https://vcpkg.io/en/packages.html): `vcpkg install ryml`
* [PyPI](https://pypi.org/project/rapidyaml/)
* Linux distributions:
* Arch Linux/Manjaro:
* [rapidyaml (aarch64)](https://archlinuxarm.org/packages/aarch64/rapidyaml)
* [rapidyaml-git (AUR)](https://aur.archlinux.org/packages/rapidyaml-git/)
* [python-rapidyaml-git (AUR)](https://aur.archlinux.org/packages/python-rapidyaml-git/)
* [Fedora Linux](https://getfedora.org/)/[EPEL](https://docs.fedoraproject.org/en-US/epel/):
* `dnf install rapidyaml-devel`
* `dnf install python3-rapidyaml`
* [Gentoo](https://packages.gentoo.org/packages/dev-cpp/rapidyaml)
* [OpenSuse](https://build.openbuildservice.org/package/show/Emulators/rapidyaml)
* [Slackbuilds](https://slackbuilds.org/repository/15.0/libraries/rapidyaml/)
* [AltLinux](https://packages.altlinux.org/en/sisyphus/srpms/rapidyaml/3006055151670528141)

Although package managers are very useful for quickly getting up to
speed, the advised way is still to bring ryml as a submodule of your
project, building both together. This makes it easy to track any
upstream changes in ryml. Also, ryml is small and quick to build, so
there's not much of a cost for building it with your project.


### Quickstart samples

These samples show different ways of getting ryml into your application. All the
Expand All @@ -555,6 +556,7 @@ more about each sample:
| [`fetch_content`](./samples/fetch_content) | **yes** | [`CMakeLists.txt`](./samples/fetch_content/CMakeLists.txt) | [`run.sh`](./samples/fetch_content/run.sh) |
| [`find_package`](./samples/find_package) | **no**<br>needs prior install or package | [`CMakeLists.txt`](./samples/find_package/CMakeLists.txt) | [`run.sh`](./samples/find_package/run.sh) |


### CMake build settings for ryml
The following cmake variables can be used to control the build behavior of
ryml:
Expand Down Expand Up @@ -726,20 +728,16 @@ See also [the roadmap](./ROADMAP.md) for a list of future work.

### Known limitations

ryml deliberately makes no effort to follow the standard in the
ryml deliberately makes no effort to follow the YAML standard in the
following situations:

* ryml's tree does NOT accept containers are as mapping keys: keys
must be scalars. HOWEVER, this is a limitation only of the tree. The
event-based parser engine DOES parse container keys. The parser
engine is the result of a recent refactor and its usage is meant to
be used by other programming languages to create their native
data-structures. This engine is fully tested and fully conformant
(other than the general error permissiveness noted below). But
because it is recent, it is still undocumented, and it requires some
API cleanup before being ready for isolated use. Please get in touch
if you are interested in integrating the event-based parser engine
without the standalone `ryml::parse_*()`
* ryml's tree does NOT accept containers as map keys: keys stored in
the tree must always be scalars. HOWEVER, this is a limitation only
of the final tree. The event-based parse engine DOES parse container
keys, as it is is meant to be used by other programming languages to
create their native data-structures, and it is fully tested and
fully conformant (other than the general error permissiveness noted
below).
* Tab characters after `:` and `-` are not accepted tokens, unless
ryml is compiled with the macro `RYML_WITH_TAB_TOKENS`. This
requirement exists because checking for tabs introduces branching
Expand Down Expand Up @@ -774,11 +772,11 @@ following situations:
If you do run into trouble and would like to investigate conformance
of your YAML code, **beware** of existing online YAML linters, many of
which are not fully conformant. Instead, try using
[https://play.yaml.io](https://play.yaml.io), an amazing tool which
lets you dynamically input your YAML and continuously see the results
from all the existing parsers (kudos to @ingydotnet and the people
from the YAML test suite). And of course, if you detect anything wrong
with ryml, please [open an
[https://play.yaml.io](https://play.yaml.io), an amazingly useful tool
which lets you dynamically input your YAML and continuously see the
results from all the existing parsers (kudos to @ingydotnet and the
people from the YAML test suite). And of course, if you detect
anything wrong with ryml, please [open an
issue](https://github.com/biojppm/rapidyaml/issues) so that we can
improve.

Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@

Moved! See ryml's [Kanban board on github](https://github.com/biojppm/rapidyaml/projects/1).
Moved! See rapidyaml's [Kanban board on github](https://github.com/users/biojppm/projects/1/views/1).
4 changes: 2 additions & 2 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,10 @@ ryml is written in C++11, and compiles cleanly with:

* Visual Studio 2015 and later

* clang++ 3.9 and later

* g++ 4.8 and later

* clang++ 3.9 and later

* Intel Compiler

.. note::
Expand Down
2 changes: 1 addition & 1 deletion doc/sphinx_quicklinks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Quick links

* `Pull Requests <https://github.com/biojppm/rapidyaml/pull>`_

* `Kanban board <https://github.com/biojppm/rapidyaml/projects/1>`_
* `Kanban board <https://github.com/users/biojppm/projects/1/views/1>`_

* Latest release: `0.6.0 <https://github.com/biojppm/rapidyaml/releases/tag/v0.6.0>`_

Expand Down
20 changes: 8 additions & 12 deletions doc/sphinx_yaml_standard.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ welcome.
linters**, many of which are not fully conformant; instead, try
using `https://play.yaml.io
<https://play.yaml.io/main/parser?input=IyBFZGl0IE1lIQoKJVlBTUwgMS4yCi0tLQpmb286IEhlbGxvLCBZQU1MIQpiYXI6IFsxMjMsIHRydWVdCmJhejoKLSBvbmUKLSB0d28KLSBudWxsCg==>`__,
an amazing tool which lets you dynamically input your YAML and
an amazingly useful tool which lets you dynamically input your YAML and
continuously see the results from all the existing parsers (kudos
to @ingydotnet and the people from the YAML test suite). And of
course, if you detect anything wrong with ryml, please `open an
Expand All @@ -31,17 +31,13 @@ Deliberate deviations
ryml deliberately makes no effort to follow the standard in the
following situations:

- ryml's tree does NOT accept containers are as mapping keys: keys
must be scalars. HOWEVER, this is a limitation only of the tree. The
event-based parser engine DOES parse container keys. The parser
engine is the result of a recent refactor and its usage is meant to
be used by other programming languages to create their native
data-structures. This engine is fully tested and fully conformant
(other than the general error permissiveness noted below). But
because it is recent, it is still undocumented, and it requires some
API cleanup before being ready for isolated use. Please get in touch
if you are interested in integrating the event-based parser engine
without the standalone `ryml::parse_*()`
- ryml's tree does NOT accept containers as map keys: keys stored in
the tree must always be scalars. HOWEVER, this is a limitation only
of the final tree. The event-based parse engine DOES parse container
keys, as it is is meant to be used by other programming languages to
create their native data-structures, and it is fully tested and
fully conformant (other than the general error permissiveness noted
below).
- Tab characters after ``:`` and ``-`` are not accepted tokens, unless
ryml is compiled with the macro ``RYML_WITH_TAB_TOKENS``. This
requirement exists because checking for tabs introduces branching
Expand Down
10 changes: 5 additions & 5 deletions src/c4/yml/detail/stack.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,11 @@ class stack

public:

T m_buf[size_t(N)];
T * m_stack;
id_type m_size;
id_type m_capacity;
Callbacks m_callbacks;
T m_buf[size_t(N)];
T *C4_RESTRICT m_stack;
id_type m_size;
id_type m_capacity;
Callbacks m_callbacks;

public:

Expand Down
2 changes: 1 addition & 1 deletion src/c4/yml/event_handler_stack.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ struct EventHandlerStack
state *C4_RESTRICT m_curr; ///< current stack level: top of the stack. cached here for easier access.
state *C4_RESTRICT m_parent; ///< parent of the current stack level.
pfn_relocate_arena m_relocate_arena; ///< callback when the arena gets relocated
void *C4_RESTRICT m_relocate_arena_data;
void * m_relocate_arena_data;

protected:

Expand Down
26 changes: 26 additions & 0 deletions test/test_parse_engine_6_qmrk.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,32 @@ ENGINE_TEST(Qmrk3,
___(ps.end_stream());
}

ENGINE_TEST(Qmrk4_0,
("[?baz:,]",
"[{?baz: }]"),
"+STR\n"
"+DOC\n"
"+SEQ []\n"
"+MAP {}\n"
"=VAL :?baz\n"
"=VAL :\n"
"-MAP\n"
"-SEQ\n"
"-DOC\n"
"-STR\n")
{
___(ps.begin_stream());
___(ps.begin_doc());
___(ps.begin_seq_val_flow());
___(ps.begin_map_val_flow());
___(ps.set_key_scalar_plain("?baz"));
___(ps.set_val_scalar_plain({}));
___(ps.end_map());
___(ps.end_seq());
___(ps.end_doc());
___(ps.end_stream());
}

ENGINE_TEST(Qmrk4,
("[ ? an explicit key, ? foo,? bar,?baz:,?bat]",
"[{an explicit key: },{foo: },{bar: },{?baz: },?bat]"),
Expand Down
2 changes: 1 addition & 1 deletion test/test_suite/test_suite_event_handler.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ void append_escaped(extra::string *es, csubstr val)
}
}
// flush the rest
this->append(val.sub(prev));
es->append(val.sub(prev));
#undef _c4flush_use_instead
}

Expand Down
24 changes: 11 additions & 13 deletions test/test_suite/test_suite_event_handler.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,6 @@
#ifndef _C4_YML_EVENT_HANDLER_STACK_HPP_
#include "c4/yml/event_handler_stack.hpp"
#endif
#ifndef _C4_YML_STD_STRING_HPP_
#include "c4/yml/std/string.hpp"
#endif
#ifndef _C4_YML_DETAIL_PRINT_HPP_
#include "c4/yml/detail/print.hpp"
#endif
Expand Down Expand Up @@ -100,6 +97,7 @@ struct EventHandlerYamlStd : public EventHandlerStack<EventHandlerYamlStd, Event
m_has_yaml_directive = false;
for(TagDirective &td : m_tag_directives)
td = {};
m_val_buffers.clear();
m_val_buffers.resize((size_t)m_stack.size());
m_arena.clear();
m_arena.reserve(1024);
Expand Down Expand Up @@ -650,20 +648,20 @@ struct EventHandlerYamlStd : public EventHandlerStack<EventHandlerYamlStd, Event

EventSink& _buf_() noexcept
{
_RYML_CB_ASSERT(m_stack.m_callbacks, (size_t)m_curr->level < m_val_buffers.size());
return m_val_buffers[(size_t)m_curr->level];
_RYML_CB_ASSERT(m_stack.m_callbacks, m_curr->level < m_val_buffers.size());
return m_val_buffers[m_curr->level];
}

EventSink& _buf_(id_type level) noexcept
{
_RYML_CB_ASSERT(m_stack.m_callbacks, (size_t)level < m_val_buffers.size());
return m_val_buffers[(size_t)level];
_RYML_CB_ASSERT(m_stack.m_callbacks, level < m_val_buffers.size());
return m_val_buffers[level];
}

EventSink const& _buf_(id_type level) const noexcept
{
_RYML_CB_ASSERT(m_stack.m_callbacks, (size_t)level < m_val_buffers.size());
return m_val_buffers[(size_t)level];
_RYML_CB_ASSERT(m_stack.m_callbacks, level < m_val_buffers.size());
return m_val_buffers[level];
}

static void _buf_flush_to_(EventSink &C4_RESTRICT src, EventSink &C4_RESTRICT dst) noexcept
Expand All @@ -686,8 +684,8 @@ struct EventHandlerYamlStd : public EventHandlerStack<EventHandlerYamlStd, Event

void _buf_ensure_(id_type size_needed) noexcept
{
if((size_t)size_needed > m_val_buffers.size())
m_val_buffers.resize((size_t)size_needed);
if(size_needed > m_val_buffers.size())
m_val_buffers.resize(size_needed);
}

C4_ALWAYS_INLINE void _send_(csubstr s) noexcept { _buf_().append(s); }
Expand All @@ -699,7 +697,7 @@ struct EventHandlerYamlStd : public EventHandlerStack<EventHandlerYamlStd, Event
_send_key_props_();
_send_(' ');
_send_(scalar_type_code);
_buf_().append_escaped(scalar);
append_escaped(&_buf_(), scalar);
_send_('\n');
}
void _send_val_scalar_(csubstr scalar, char scalar_type_code)
Expand All @@ -708,7 +706,7 @@ struct EventHandlerYamlStd : public EventHandlerStack<EventHandlerYamlStd, Event
_send_val_props_();
_send_(' ');
_send_(scalar_type_code);
_buf_().append_escaped(scalar);
append_escaped(&_buf_(), scalar);
_send_('\n');
}

Expand Down
13 changes: 13 additions & 0 deletions test/test_tag_property.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1041,6 +1041,19 @@ TEST(tags, EHF6)
}
}

TEST(tags, fuzzcrash0)
{
Tree tree;
ExpectError::do_check(&tree, [&]{
parse_in_arena("%TAG !! " "\n"
"})" "\n"
"" "\n"
"!!<" "\n"
,
&tree);
});
}


//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
Expand Down
Loading

0 comments on commit 686298f

Please sign in to comment.