You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The KVS internally represents all values as an opaque byte sequence. The server side does not interpret them in any way, however the API side has a convention about NULL terminating strings that complicates append support proposed in #1193.
flux_kvs_txn_put() stores a C string into the KVS with its terminating NULL. This allows flux_kvs_lookup_get() to directly return value without the need re-copy to a size + 1 buffer to add a NULL to it, or so went the logic at the time (now somewhat outdated).
The JSON API functions use the same convention so that a JSON value written with flux_kvs_txn_pack() can be read out in encoded form with flux_kvs_lookup_get().
This complicates append support. If string values are concatenated by repeatedly calling flux_kvs_txn_put(FLUX_KVS_APPEND), there is no mechanism for the server to remove the NULL terminators at the end of each blob.
Let's stop storing the NULL. Looks like we can just change treeobj_decode_val() so that when it de-base64's a value, it tacks on an extra NULL byte to the decode buffer without including it in the returned length. Then the value can be used direclty as a string if desired. When we call treeobj_create_val() to encode a string, we pass the string length, not string length + 1.
The text was updated successfully, but these errors were encountered:
garlick
added a commit
to garlick/flux-core
that referenced
this issue
Oct 27, 2017
Problem: the reasons no longer exist for storing
string/json values in the KVS with their terminating
NULL byte, and this complicates the implementation
of atomic append.
Skip storing the NULL byte, and change lookup_get()
et al to rely on the fact that treeobj_decode_val()
pads the space after the returned data with a NULL byte.
Use json_loadb() instead of _loads() when decoding
a JSON value.
Update unit and sharness tests.
Fixflux-framework#1261
garlick
added a commit
to garlick/flux-core
that referenced
this issue
Oct 27, 2017
Problem: the reasons no longer exist for storing
string/json values in the KVS with their terminating
NULL byte, and this complicates the implementation
of atomic append.
Skip storing the NULL byte, and change lookup_get()
et al to rely on the fact that treeobj_decode_val()
pads the space after the returned data with a NULL byte.
Use json_loadb() instead of _loads() when decoding
a JSON value.
Update unit and sharness tests.
Fixflux-framework#1261
The KVS internally represents all values as an opaque byte sequence. The server side does not interpret them in any way, however the API side has a convention about NULL terminating strings that complicates append support proposed in #1193.
flux_kvs_txn_put()
stores a C string into the KVS with its terminating NULL. This allowsflux_kvs_lookup_get()
to directly return value without the need re-copy to a size + 1 buffer to add a NULL to it, or so went the logic at the time (now somewhat outdated).The JSON API functions use the same convention so that a JSON value written with
flux_kvs_txn_pack()
can be read out in encoded form withflux_kvs_lookup_get()
.This complicates append support. If string values are concatenated by repeatedly calling
flux_kvs_txn_put(FLUX_KVS_APPEND)
, there is no mechanism for the server to remove the NULL terminators at the end of each blob.Let's stop storing the NULL. Looks like we can just change
treeobj_decode_val()
so that when it de-base64's a value, it tacks on an extra NULL byte to the decode buffer without including it in the returned length. Then the value can be used direclty as a string if desired. When we calltreeobj_create_val()
to encode a string, we pass the string length, not string length + 1.The text was updated successfully, but these errors were encountered: