Skip to content

Commit

Permalink
Fix reading of gz file size for endian issue. Fixes #39
Browse files Browse the repository at this point in the history
  • Loading branch information
coolbutuseless committed Apr 12, 2024
1 parent 850127e commit f18c086
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 6 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Package: yyjsonr
Type: Package
Title: Fast 'JSON', 'NDJSON' and 'GeoJSON' Parser and Generator
Version: 0.1.20
Version: 0.1.20.9000
Authors@R: c(
person("Mike", "Cheng", role = c("aut", "cre", 'cph'),
email = "[email protected]"),
Expand Down
3 changes: 3 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@

# yyjsonr 0.1.20.9000 2024-04-12

* Get size of gzipped file in an endian-neutral way. Issue #39

# yyjsonr 0.1.20 2024-04-10

Expand Down
12 changes: 7 additions & 5 deletions src/R-yyjson-parse.c
Original file line number Diff line number Diff line change
Expand Up @@ -1953,19 +1953,21 @@ SEXP parse_from_gzfile_(SEXP filename_, SEXP parse_opts_) {
// Read tail end of .gz file to get length.
// If uncompressed length > 4GB this method will fail as there are
// only 4-bytes reserved for the field!
// Stored as a little-endian 32bit int. Reading this byte-by-byte
// to avoid an endianness issue on 32bit powerpc (Issue #39)
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
FILE *fp = fopen(filename, "rb");
if (fp == NULL) {
error("couldn't open file: %s", filename);
}

fseek(fp, -4, SEEK_END);
int32_t uncompressed_len;
size_t nbytes = fread(&uncompressed_len, 1, 4, fp);
int32_t uncompressed_len = 0;
uncompressed_len += fgetc(fp);
uncompressed_len += fgetc(fp) << 8;
uncompressed_len += fgetc(fp) << 16;
uncompressed_len += fgetc(fp) << 24;
fclose(fp);
if (nbytes != 4) {
error("Couldn't read size from end of file: %s", filename);
}

//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Allocate a buffer to hold the uncompressed file.
Expand Down

0 comments on commit f18c086

Please sign in to comment.