workdir is not TMPDIR

[gaponenko]

Hello,

jobsub_submit uses the current working directory to place temporary
files. This introduces unnecessary restriction that the submission
directory has to be writable and has to have sufficient free space.
Also, the files are routinely left over when the submission process is
interrupted, another annoyance. Please use mkstemp() or equivalent to
make temporary files end up in TMPDIR or /tmp.

Andrei

[marcmengel]

We tried that, but while experienced UNIX/Linux users are aware of setting TMPDIR and/or XDG_CACHE_HOME, many users are not, and the /tmp on many systems is quite frequently full or near full, causing launch errors.

So most of the submit files are placed in $XDG_CACHE_HOME, which defaults to $HOME/.cache, and we tried
putting tarfiles in TMPDIR defaulting to /tmp, but as mentioned this got lots of complaints with /tmp being
full, then we tried putting the tarfiile directory (for tardir: particularly) next to the directory being tar-ed up, but
many users were tar-ing up directories they didn't own, so they couldn't write there; then we finally settled on
the current directory, which is where the old jobsub scripts have put the tarfiles (for tardir: at least) for many years, and thus is backwards compatible, and many users seem to be more comfortable changing directories to submit than remembering to set TMPDIR.

Besides which the tarfile conversion is usually quite quick; I suspect yours isn't because you are using dropbox: on a /pnfs/.../persistent file .

[kutschke]

@marcmengel i want to understand your reply to Andrei - since RCDS, I normally send tarfiles that are located on /exp/mu2e/data. I believe that's prefered over /pnfs, right? In the old days we had to use /pnfs but those days are long past. Right?

[kutschke]

XDG_CACHE_HOME is not defined on our machines. Is it used for anything else? Depending on the answer, it might make sense for us to define it in our /cvmfs/mu2e.opensciencegrid.org/init.sh which is scheduled to replace /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh as soon as we are up and running on AL9. This will cover all use cases except people who want to use bare jobsub_submit outside of the Mu2e environment.

[shreyb]

In response to your last question, @kutschke: In implementing this, we followed the XDG Base Directory Specification, so that if $XDG_CACHE_HOME is not defined, then $HOME/.cache is used for the submit files.