Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increase the input block size for bgzip. #1768

Merged
merged 2 commits into from
Nov 18, 2024
Merged

Increase the input block size for bgzip. #1768

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
186d21b
Increase the input block size for bgzip.
jkbonfield Apr 9, 2024
d1be3c2
Make use of posix_memalign for hfile buffer.
jkbonfield Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion configure.ac
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,7 @@ HTS_HIDE_DYNAMIC_SYMBOLS

dnl FIXME This pulls in dozens of standard header checks
AC_FUNC_MMAP
AC_CHECK_FUNCS([gmtime_r fsync drand48 srand48_deterministic getauxval elf_aux_info])
AC_CHECK_FUNCS([gmtime_r fsync drand48 srand48_deterministic getauxval elf_aux_info posix_memalign])

# Darwin has a dubious fdatasync() symbol, but no declaration in <unistd.h>
AC_CHECK_DECL([fdatasync(int)], [AC_CHECK_FUNCS(fdatasync)])
Expand Down
19 changes: 16 additions & 3 deletions hfile.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,12 +107,20 @@ hFILE *hfile_init(size_t struct_size, const char *mode, size_t capacity)
hFILE *fp = (hFILE *) malloc(struct_size);
if (fp == NULL) goto error;

if (capacity == 0) capacity = 32768;
const int maxcap = 128*1024;

if (capacity == 0) capacity = maxcap;
// FIXME For now, clamp input buffer sizes so mpileup doesn't eat memory
if (strchr(mode, 'r') && capacity > 32768) capacity = 32768;
if (strchr(mode, 'r') && capacity > maxcap) capacity = maxcap;

#ifdef HAVE_POSIX_MEMALIGN
fp->buffer = NULL;
if (posix_memalign((void **)&fp->buffer, 256, capacity) < 0)
daviesrob marked this conversation as resolved.
Show resolved Hide resolved
goto error;
#else
fp->buffer = (char *) malloc(capacity);
if (fp->buffer == NULL) goto error;
#endif

fp->begin = fp->end = fp->buffer;
fp->limit = &fp->buffer[capacity];
Expand Down Expand Up @@ -629,7 +637,12 @@ static size_t blksize(int fd)
#ifdef HAVE_STRUCT_STAT_ST_BLKSIZE
struct stat sbuf;
if (fstat(fd, &sbuf) != 0) return 0;
return sbuf.st_blksize;

// Pipes/FIFOs on linux return 4Kb here often, but it's much too small
// for performant I/O.
return S_ISFIFO(sbuf.st_mode)
? 128*1024
: sbuf.st_blksize;
#else
return 0;
#endif
Expand Down