KVS: Support valref pointing to zero length blob objects #1237
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1237 +/- ##
==========================================
+ Coverage 78.13% 78.15% +0.02%
==========================================
Files 154 154
Lines 28680 28702 +22
==========================================
+ Hits 22408 22431 +23
+ Misses 6272 6271 -1
|
|
I'm going to be travelling tomorrow so I may not get to review this for a bit. One quick question though, do the |
|
Are you referring to the fact I have a "NONE" data type? I suppose this is mostly for internal reasons, to signify the user has not yet assigned "JSON" or "RAW" data, so the cache entry doesn't know. I suppose that could be masked/hidden internally. Thought, is "type" perhaps the wrong word to use? |
|
It occurred to me last night that instead of creating
probably better than adding two new functions. Well, I'll let the review happen on what's there for now. I can change this later. Also, "NONE" maybe should be changed to "UNKNOWN" ... TBD |
|
I was just thinking that one creation function could set the type to none/unknown, and it could get its type assigned when it becomes valid. Maybe that doesn't fit - I didn't look too closely. Sorry, getting packed up today so I'll be offline most of today, but I'll try to check in later. |
|
I like this idea enough I"ll go ahead and change it. |
|
As for your question, "it could get its type assigned when it becomes valid". Basically assigning a type (or none) is an option for the caller. In some cases, assigning the type ahead of time is effectively a flag saying, "Hey, this is what I'm expecting". I use this "flag" so that content-loading is aware what to expect. |
|
Red-did the |
|
Looks like I hit a #1169 and a write error. re-run builds. |
|
This looks fine, just needs a rebase. It does seem like the cache interface could be simplified, but maybe not a topic for this PR. I'll open an issue to discuss it. |
cache_entry_create() now takes a cache_data_type_t parameter, allowing to user to create a cache entry without any data attached to it, but at the same time indicate the type of data expected for the cache entry to store. This is an alternate to passing NULL to the function cache_entry_create_json ().
Cache entry creation API has slightly different logic now. Instead of creating an "empty" entry by setting inputs to NULL (i.e. cache_entry_create_json(NULL) or cache_entry_create_raw(NULL, 0)) these create functions now expect correct input. Callers wishing to create empty entries without data shall call cache_entry_create(). As a side effect, this now means that cache_entry_create_raw() now is not ambiguous about setting zero length data. Setting cache_entry_create_raw(NULL, 0) creates a cache entry with a zero length of data. Where as before, it was ambiguous if it was an empty entry or an entry with zero length of data. Update callers of these functions. Update/add unit tests.
Refactor cache_entry_get_raw to return data buffer and length as in/out parameters, returning integer to indicate success/failure. This is so the data buffer, if NULL, can be returned to the user.
Add unit tests appropriately.
Alter usage of cache_entry_set_json(). If user wishes to clear data, the user should use cache_entry_clear_data(). A NULL input to cache_entry_set_json() will now result in an error. Update code accordingly for change.
In the internal cache API, add a data_valid flag which indicates if the user has set data or not. This flag is used instead of checking data == NULL for validity, as checking the data pointer for NULL would not allow zero length content-blobs to be stored in the cache. A major fallout of from this addition is the change to the cache_entry_set_raw() function. It was previously unclear if passing NULL & length 0 to cache_entry_set_raw() indicated if you were setting a zero length data or trying to clear prior data. Now it is made clear that this is setting zero length data and clearing data is through cache_entry_clear_data(). In addition, calling cache_entry_set_raw() with zero length data can result in wait_runqueue() being run on hp->waitlist_valid. Added new unit tests for coverage.
|
just re-pushed w/ a rebase |
|
restarted some builds, hit a write error, #1169, and one that timed out for some reason. travis seems really slow right now |
This series of patches is to support valref treeobjects pointing to zero-length blobref data.
This seems like a lot of code for a pretty dumb corner case, but it came down to updating the KVS cache API to not assume non-NULL data pointer means valid and NULL data pointer is invalid. A flag was placed into the KVS cache entry to handle this, but it beget a number of API style changes.
For starters,
cache_entry_get_raw ()had to be modified so that a NULL data buffer returned wasn't automatically assumed to be error. This is an obvious refactoring that was needed. The rest are more subtle.Under the old API:
cache_entry_create_json (NULL)- creates an empty/invalid cache entrycache_entry_create_raw (NULL, 0)- creates a valid cache entry???cache_entry_set_json (hp, NULL)- clears json from the cache entrycache_entry_set_raw (hp, NULL, 0)- should do ...???So for consistency,
cache_entry_create_json()andcache_entry_set_json()now only take valid input (i.e. non-NULL json objects). The raw equivalents can take NULL & 0, b/c that's justa zero length blobref.
New functions
cache_entry_create_json_empty()andcache_entry_create_raw_empty()were created to create "empty" entries.In order to clear data from an entry,
cache_entry_clear_data()was added. Thecache_entry_set_json()andcache_entry_set_raw()functions can no longer clear data from an entry by passing in NULL.As a consequence, it's worth noting that
cache_entry_set_raw()will also callwait_runqueue()when a zero-byte length buffer is set into the entry. This was not done before.Debate on implementation:
Tiny part of me dislikes the function name "empty". Can discuss alternates. I considered using a flag type variable to say "I have data", but that seemed even uglier and more confusing.
I also considered removing the "empty" functions and supporting a
cache_entry_set_type()function, to say these cache entries only support json or raw (i.e. user would callcache_entry_create(), thencache_entry_set_type()). But that's two function calls instead of one. I nixed that idea.Also, for one error case I choose errno EBADE. Right one??