Skip to content

Latest commit

 

History

History
502 lines (383 loc) · 15.2 KB

README.md

File metadata and controls

502 lines (383 loc) · 15.2 KB
Raw
  • infos = Information about the yamlcpp plugin is in keys below
  • infos/author = René Schwaiger [email protected]
  • infos/licence = BSD
  • infos/needs = base64 directoryvalue
  • infos/provides = storage/yaml
  • infos/recommends =
  • infos/placements = getstorage setstorage
  • infos/status = maintained unittest preview unfinished concept discouraged
  • infos/metadata =
  • infos/description = This storage plugin reads and writes data in the YAML format

YAML CPP

Introduction

The YAML CPP plugin reads and writes configuration data via the yaml-cpp library.

Usage

You can mount this plugin via kdb mount:

sudo kdb mount config.yaml /tests/yamlcpp yamlcpp

. To unmount the plugin use kdb umount:

sudo kdb umount /tests/yamlcpp

. The following examples show how you can store and retrieve data via yamlcpp.

# Mount yamlcpp plugin to cascading namespace `/tests/yamlcpp`
sudo kdb mount config.yaml /tests/yamlcpp yamlcpp

# Manually add a mapping to the database
echo "🔑 : 🐳"               > `kdb file /tests/yamlcpp`
# Retrieve the value of the manually added key
kdb get /tests/yamlcpp/🔑
#> 🐳

# Manually add syntactically incorrect data
echo "some key: @some  value" >> `kdb file /tests/yamlcpp`
kdb get "/tests/yamlcpp/some key"
# STDERR: .*yaml-cpp: error at line 2, column 11: unknown token.*
# ERROR: 10
# RET: 5

# Overwrite incorrect data
echo "🔑: value" >  `kdb file /tests/yamlcpp`

# Add some values via `kdb set`
kdb set /tests/yamlcpp 🎵
kdb set /tests/yamlcpp/fleetwood mac
kdb set /tests/yamlcpp/the chain

# Retrieve the new values
kdb get /tests/yamlcpp
#> 🎵
kdb get /tests/yamlcpp/the
#> chain
kdb get /tests/yamlcpp/fleetwood
#> mac

# Undo modifications
kdb rm -r /tests/yamlcpp
sudo kdb umount /tests/yamlcpp

Arrays

YAML CPP provides support for Elektra’s array data type.

# Mount yamlcpp plugin to `user/tests/yamlcpp`
sudo kdb mount config.yaml user/tests/yamlcpp yamlcpp

# Manually add an array to the database
echo 'sunny:'       >  `kdb file user/tests/yamlcpp`
echo '  - Charlie'  >> `kdb file user/tests/yamlcpp`
echo '  - Dee'      >> `kdb file user/tests/yamlcpp`

# List the array entries
kdb ls user/tests/yamlcpp
#> user/tests/yamlcpp/sunny
#> user/tests/yamlcpp/sunny/#0
#> user/tests/yamlcpp/sunny/#1

# Read an array entry
kdb get user/tests/yamlcpp/sunny/#1
#> Dee

# You can retrieve the last index of an array by reading the metakey `array`
kdb getmeta user/tests/yamlcpp/sunny array
# 1

# Extend the array
kdb set user/tests/yamlcpp/sunny/#2 Dennis
kdb set user/tests/yamlcpp/sunny/#3 Frank
kdb set user/tests/yamlcpp/sunny/#4 Mac

# The plugin supports empty array fields
kdb set user/tests/yamlcpp/sunny/#_10 'The Waitress'
kdb getmeta user/tests/yamlcpp/sunny array
#> #_10
kdb get user/tests/yamlcpp/sunny/#_9
# RET: 11

# Retrieve the last array entry
kdb get user/tests/yamlcpp/sunny/$(kdb getmeta user/tests/yamlcpp/sunny array)
#> The Waitress

# The plugin also supports empty arrays (arrays without any elements)
kdb setmeta user/tests/yamlcpp/empty array ''
kdb export user/tests/yamlcpp/empty yamlcpp
#> []

# For arrays with at least one value we do not need to set the type `array`
kdb set user/tests/yamlcpp/movies
kdb set user/tests/yamlcpp/movies/#0 'A Silent Voice'
kdb getmeta user/tests/yamlcpp/movies array
#> #0
kdb export user/tests/yamlcpp/movies yamlcpp
#> - A Silent Voice

# Undo modifications to the key database
kdb rm -r user/tests/yamlcpp
sudo kdb umount user/tests/yamlcpp

Nested Arrays

The plugin also supports nested arrays.

# Mount yamlcpp plugin to `user/tests/yamlcpp`
sudo kdb mount config.yaml user/tests/yamlcpp yamlcpp

# Add some key value pairs
kdb set user/tests/yamlcpp/key value
kdb set user/tests/yamlcpp/array/#0 scalar
kdb set user/tests/yamlcpp/array/#1/key value
kdb set user/tests/yamlcpp/array/#1/🔑 🙈

kdb ls user/tests/yamlcpp
#> user/tests/yamlcpp/array
#> user/tests/yamlcpp/array/#0
#> user/tests/yamlcpp/array/#1/key
#> user/tests/yamlcpp/array/#1/🔑
#> user/tests/yamlcpp/key

# Retrieve part of an array value
kdb get user/tests/yamlcpp/array/#1/key
#> value

# Since an array saves a list of values, an array parent
# - which represent the array - does not store a value!
echo "user/tests/yamlcpp/array: “`kdb get user/tests/yamlcpp/array`”"
#> user/tests/yamlcpp/array: “”

# Remove part of an array value
kdb rm user/tests/yamlcpp/array/#1/key

kdb ls user/tests/yamlcpp
#> user/tests/yamlcpp/array
#> user/tests/yamlcpp/array/#0
#> user/tests/yamlcpp/array/#1/🔑
#> user/tests/yamlcpp/key

# The plugin stores array keys using YAML sequences.
# Since yaml-cpp stores keys in arbitrary order -
# either `key` or `array` could be the “first” key -
# we remove `key` before we retrieve the data. This way
# we make sure that the output below will always look
# the same.
kdb rm user/tests/yamlcpp/key
kdb file user/tests/yamlcpp | xargs cat
#> array:
#>   - scalar
#>   - 🔑: 🙈

# Undo modifications to the key database
kdb rm -r user/tests/yamlcpp
sudo kdb umount user/tests/yamlcpp

Sparse Arrays

Since Elektra allows “holes” in a key set, YAML CPP has to support small key sets that describe relatively complex data.

# Mount yamlcpp plugin
sudo kdb mount config.yaml user/tests/yamlcpp yamlcpp

kdb set user/tests/yamlcpp/#0/map/#1/#0 value
kdb file user/tests/yamlcpp | xargs cat
#> - map:
#>     - ~
#>     -
#>       - value

# The plugin adds the missing array parents to the key set
kdb ls user/tests/yamlcpp
#> user/tests/yamlcpp
#> user/tests/yamlcpp/#0/map
#> user/tests/yamlcpp/#0/map/#0
#> user/tests/yamlcpp/#0/map/#1
#> user/tests/yamlcpp/#0/map/#1/#0

# Undo modifications to the key database
kdb rm -r user/tests/yamlcpp
sudo kdb umount user/tests/yamlcpp

Metadata

The plugin supports metadata. The example below shows how a basic Key including some metadata, looks inside the YAML configuration file:

key without metadata: value
key with metadata: !elektra/meta
  - value2
  - metakey: metavalue
    empty metakey:
    another metakey: another metavalue

. As we can see above the value containing metadata is marked by the tag handle !elektra/meta. The data type contains a list with two elements. The first element of this list specifies the value of the key, while the second element contains a map saving the metadata for the key. The data above represents the following key set in Elektra if we mount the file directly to the namespace user:

Name Value Metaname Metavalue
user/key without metadata value1 — —
user/key with metadata value2 metakey metavalue
empty metakey —
another metakey another metavalue

. The example below shows how we can read and write metadata using the yamlcpp plugin via kdb.

# Mount yamlcpp plugin to `user/tests/yamlcpp`
sudo kdb mount config.yaml user/tests/yamlcpp yamlcpp

# Manually add a key including metadata to the database
echo "🔑: !elektra/meta [🦄, {comment: Unicorn}]" >  `kdb file user/tests/yamlcpp`
kdb lsmeta user/tests/yamlcpp/🔑
#> comment
kdb getmeta user/tests/yamlcpp/🔑 comment
#> Unicorn

# Add a new key and add some metadata to the new key
kdb set user/tests/yamlcpp/brand new
kdb setmeta user/tests/yamlcpp/brand comment "The Devil And God Are Raging Inside Me"
kdb setmeta user/tests/yamlcpp/brand rationale "Because I Love It"

# Retrieve metadata
kdb lsmeta user/tests/yamlcpp/brand
#> comment
#> rationale
kdb getmeta user/tests/yamlcpp/brand rationale
#> Because I Love It

# Undo modifications to the key database
kdb rm -r user/tests/yamlcpp
sudo kdb umount user/tests/yamlcpp

We can also invoke additional plugins that use metadata like type.

sudo kdb mount config.yaml user/tests/yamlcpp yamlcpp type
kdb set user/tests/yamlcpp/typetest/number 21
kdb setmeta user/tests/yamlcpp/typetest/number check/type short

kdb set user/tests/yamlcpp/typetest/number "One"
# RET: 5
# STDERR: .*error in the type plugin.*
# ERROR: 52

kdb get user/tests/yamlcpp/typetest/number
#> 21

# Undo modifications to the key database
kdb rm -r user/tests/yamlcpp
sudo kdb umount user/tests/yamlcpp

Binary Data

YAML CPP also supports base64 encoded data via the Base64 plugin.

# Mount YAML CPP plugin at `user/tests/binary`
sudo kdb mount test.yaml user/tests/binary yamlcpp
# Manually add binary data
echo 'bin: !!binary aGk=' > `kdb file user/tests/binary`

# Base 64 decodes the data `aGk=` to `hi` and stores the value in binary form.
# The command `kdb get` prints the data as hexadecimal byte values.
kdb get user/tests/binary/bin
#> \x68\x69

# Add a string value to the database
kdb set user/tests/binary/text mate
# Base 64 does not modify textual values
kdb get user/tests/binary/text
#> mate

# The Base 64 plugin re-encodes binary data before YAML CPP stores the key set. Hence the
# configuration file contains the value `aGk=` even after YAML CPP wrote a new configuration.
grep -q 'bin: !.* aGk=' `kdb file user/tests/binary`
# RET: 0

# Undo modifications to the database
kdb rm -r user/tests/binary
sudo kdb umount user/tests/binary

Null & Empty

Sometimes you only want to save a key without a value (null key) or a key with an empty value. The commands below show that YAML CPP supports this scenario properly.

# Mount YAML CPP plugin at `user/tests/yamlcpp`
sudo kdb mount test.yaml user/tests/yamlcpp yamlcpp

# Check if the plugin saves null keys correctly
kdb set user/tests/yamlcpp/null
kdb set user/tests/yamlcpp/null/level1/level2
kdb setmeta user/tests/yamlcpp/null/level1/level2 comment 'Null key'

kdb ls user/tests/yamlcpp/null
#> user/tests/yamlcpp/null
#> user/tests/yamlcpp/null/level1/level2
kdb get -v user/tests/yamlcpp/null | grep -q 'The key is null.'
kdb get -v user/tests/yamlcpp/null/level1/level2 | grep -q 'The key is null.'

# Check if the plugin saves empty keys correctly
kdb set user/tests/yamlcpp/empty ""
kdb set user/tests/yamlcpp/empty/level1/level2

kdb ls user/tests/yamlcpp/empty
#> user/tests/yamlcpp/empty
#> user/tests/yamlcpp/empty/level1/level2
kdb get -v user/tests/yamlcpp/empty | grep -vq 'The key is null.'

# Undo modifications to the database
kdb rm -r user/tests/yamlcpp
sudo kdb umount user/tests/yamlcpp

Binary Values

Elektra saves binary data as either 0 or 1. The YAML CPP plugin supports this design decision by converting between YAML’s and Elektra’s boolean type.

# Mount YAML CPP plugin at `user/tests/yamlcpp`
sudo kdb mount config.yaml user/tests/yamlcpp yamlcpp
# Manually add boolean key
echo 'truth: true' > `kdb file user/tests/yamlcpp`

kdb get user/tests/yamlcpp/truth
#> 1

# Add another boolean value
kdb set user/tests/yamlcpp/success 0
kdb get user/tests/yamlcpp/success
#> 0

# Undo modifications to the database
kdb rm -r user/tests/yamlcpp
sudo kdb umount user/tests/yamlcpp

Dependencies

This plugin requires yaml-cpp. On a Debian based OS the package for the library is called libyaml-cpp-dev. On macOS you can install the package yaml-cpp via HomeBrew.

Limitations

Leaf Values

One of the limitations of this plugin is, that it only supports values inside leaf nodes. Let us look at an example to show what that means. The YAML file below:

root:
  subtree: 🍂
  below root: leaf
level 1:
  level 2:
    level 3: 🍁

stores all of the values (🍂, leaf and 🍁) in the leaves of the mapping. The drawing below makes this situation a little bit clearer.

Tree

The key set that this plugin creates using the data above looks like this (assuming we mount the plugin to user/tests/yamlcpp):

Name Value
user/tests/yamlcpp/level
user/tests/yamlcpp/level 1/level 2
user/tests/yamlcpp/level 1/level 2/level 3 🍁
user/tests/yamlcpp/root
user/tests/yamlcpp/root/below root leaf
user/tests/yamlcpp/root/subtree 🍂

. Now why is this plugin unable to store values outside leaf nodes? For example, why can we not store a value inside user/tests/yamlcpp/level 1/level 2? To answer this question we need to look at the YAML representation:

level 1:
  level 2:
    level 3: 🍁

. In a naive approach we might just try to add a value e.g. 🙈 right next to level 2:

level 1:
  level 2: 🙈
    level 3:  🍁

. This however would be not correct, since then the YAML node level 2 would contain both a scalar value (🙈) and a mapping ({ level 3: 🍁 }). We could solve this dilemma using a list:

level 1:
  level 2:
    - 🙈
    - level 3: 🍁

. However, if we use this approach we are not able to support Elektra’s array type properly.

Directory Values

To overcome the limitation described above, the YAML CPP plugin requires the Directory Value plugin. This plugin converts the value of a non-leaf node to a leaf node with the name ___dirdata. For example, let us assume we have the following key set:

directory      = Directory Data
directory/file = Leaf Data

. The Directory Value plugin will convert the key set in the set (write) direction to

directory            =
directory/___dirdata = Directory Data
directory/file       = Leaf Data

. Consequently the YAML plugin will store the key set as

directory: ___dirdata = Directory Data
  file       = Leaf Data

. A user of the YAML plugin will not notice this feature unless he edits the configuration file by hand, as the following example shows:

# Mount YAML CPP plugin at `user/tests/yamlcpp`
sudo kdb mount test.yaml user/tests/yamlcpp yamlcpp

kdb set user/tests/yamlcpp/directory 'Directory Data'
kdb setmeta user/tests/yamlcpp/directory comment 'Directory Metadata'
kdb set user/tests/yamlcpp/directory/file 'Leaf Data'

kdb ls user/tests/yamlcpp/directory
#> user/tests/yamlcpp/directory
#> user/tests/yamlcpp/directory/file

kdb get user/tests/yamlcpp/directory
#> Directory Data
kdb getmeta user/tests/yamlcpp/directory comment
#> Directory Metadata
kdb get user/tests/yamlcpp/directory/file
#> Leaf Data

# Undo modifications to the database
kdb rm -r user/tests/yamlcpp
sudo kdb umount user/tests/yamlcpp

.

Other Limitations

  • Adding and removing keys does remove comments inside the configuration file

  • The plugin currently lacks proper type support for scalars.

  • If Elektra uses YAML CPP as default storage plugin, multiple tests of the test suite fail. However, if you mount YAML CPP at /:

    kdb mount default.yaml / yamlcpp

    all tests should work correctly. The problem here is that Elektra does not load additional required plugins (infos/needs) for a default storage plugin.