Skip to content

Commit

Permalink
shell: set output limit for single user instances
Browse files Browse the repository at this point in the history
Problem: The KVS has a size limit of INT_MAX for when returning kvs
values.  This limit can be exceeded by a job's standard output because
it is continually appended and the total size is not yet tracked by
the KVS.  When reading the output later, such as via `flux job attach`,
this can lead to EOVERFLOW errors.

Solution: For a single user instance, default to a maximum standard
output of 1G instead of "unlimited".  1G should provide a practical
maximum for most users and encourage them to send standard output
to a file if they want to save excess standard output.  If desired,
the value can still be overwritten via the "output.limit" setting.

Fixes flux-framework#6256
  • Loading branch information
chu11 committed Sep 6, 2024
1 parent a75a4d9 commit 49e734e
Showing 1 changed file with 4 additions and 3 deletions.
7 changes: 4 additions & 3 deletions src/shell/output.c
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@
#include "builtins.h"
#include "log.h"

#define MULTIUSER_OUTPUT_LIMIT "10M"
#define SINGLEUSER_OUTPUT_LIMIT "1G"
#define MULTIUSER_OUTPUT_LIMIT "10M"

enum {
FLUX_OUTPUT_TYPE_TERM = 1,
Expand Down Expand Up @@ -1183,11 +1184,11 @@ static int get_output_limit (struct shell_output *out)
json_t *val = NULL;
uint64_t size;

/* Set default to unlimited (0) for single-user instances,
/* For single-user instances, cap at reasonable size limit.
* O/w use the default multiuser output limit:
*/
if (out->shell->broker_owner == getuid())
out->kvs_limit_string = "0";
out->kvs_limit_string = SINGLEUSER_OUTPUT_LIMIT;
else
out->kvs_limit_string = MULTIUSER_OUTPUT_LIMIT;

Expand Down

0 comments on commit 49e734e

Please sign in to comment.