Skip to content

Commit

Permalink
content-files: call open() with O_TRUNC
Browse files Browse the repository at this point in the history
Problem: In content-files, most files are created using a hash
of the data.  So in most cases the filenames are unique.  However,
the kvs-checkpoint.put rpc will typically write to the same file
over and over again.  In the event the file already exists, content-files
did not clear the data in a file before writing.  This can lead to
corrupted data if the user wrote fewer bytes to the file than before.

Solution: Call open() with O_TRUNC to truncate the file before writing
new data.  Add tests increasing / decreasing checkpoint data size.
  • Loading branch information
chu11 committed Feb 28, 2022
1 parent efe367a commit 267dcbc
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 1 deletion.
2 changes: 1 addition & 1 deletion src/modules/content-files/filedb.c
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ int filedb_put (const char *dbpath,
*errstr = "key name too long for internal buffer";
return -1;
}
if ((fd = open (path, O_WRONLY | O_CREAT, 0666)) < 0)
if ((fd = open (path, O_WRONLY | O_CREAT | O_TRUNC, 0666)) < 0)
return -1;
if (write_all (fd, data, size) < 0) {
ERRNO_SAFE_WRAP (close, fd);
Expand Down
20 changes: 20 additions & 0 deletions t/t0018-content-files.t
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,26 @@ test_expect_success HAVE_JQ 'kvs-checkpoint.get foo still returns rootref baz' '
test_cmp rootref3.exp rootref3.out
'

test_expect_success HAVE_JQ 'kvs-checkpoint.put updates foo rooref with longer rootref' '
kvs_checkpoint_put foo abcdefghijklmnopqrstuvwxyz
'

test_expect_success HAVE_JQ 'kvs-checkpoint.get foo returned rootref with longer rootref' '
echo abcdefghijklmnopqrstuvwxyz >rootref3.exp &&
kvs_checkpoint_get foo | jq -r .value | jq -r .rootref >rootref3.out &&
test_cmp rootref3.exp rootref3.out
'

test_expect_success HAVE_JQ 'kvs-checkpoint.put updates foo rooref to shorter rootref' '
kvs_checkpoint_put foo foobar
'

test_expect_success HAVE_JQ 'kvs-checkpoint.get foo returned rootref with shorter rootref' '
echo foobar >rootref4.exp &&
kvs_checkpoint_get foo | jq -r .value | jq -r .rootref >rootref4.out &&
test_cmp rootref4.exp rootref4.out
'

test_expect_success 'load with invalid blobref fails' '
test_must_fail backing_load notblobref 2>notblobref.err &&
grep "invalid blobref" notblobref.err
Expand Down

0 comments on commit 267dcbc

Please sign in to comment.