Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify a temp directory for the temporary files #178

Closed
luboslives opened this issue Aug 11, 2020 · 9 comments
Closed

Specify a temp directory for the temporary files #178

luboslives opened this issue Aug 11, 2020 · 9 comments

Comments

@luboslives
Copy link

Is there any way, using the CLI, to have the temporary files be created in an external temp directory?

I've tried some directory watching programs and they always pick up the temp files because they keep the same file extensions as the original images, so I have them kicking off the image_optim process unnecessarily every time they pick up new temp files.

Ideally I'd like an option like --tempdir /my/temp/dir, if it can't currently be achieved any other way.

@toy
Copy link
Owner

toy commented Aug 16, 2020

Probably you are talking about a temporary file that is created to atomically replace original. Processing related temp files are created in default/system temporary directory.
For atomic operation source and destination must be on the same device, that is why temporary file is created in the same directory.

From code I can see that for the case of default temporary directory residing on the same device as destination there can be an optimisation of not creating this last temporary file. Then default temporary directory can be changed using TMPDIR environment variable.
Also probably changing temporary file extension to something like .tmp could help.

@luboslives
Copy link
Author

luboslives commented Aug 19, 2020

@toy For some reason, image_optim will not use my temp directories (both on my Mac and on a VPS). I know this because I run it through a file watcher, and it shows me the temporary files being created in the same directory as the originals, and because I run it alongside ImageMagick to create webp duplicates. ImageMagick will obey the $TMPDIR I set right before running both commands – it will show it being used in the printout in verbose mode.

I thought maybe I had stumbled onto something with this restriction set by Ruby, but setting chmod +t mytempdir did not change the behaviour.

I don't know anything about Ruby but it seems you have tmpdir & temp_dir logic inside /spec/image_optim/bin_resolver_spec.rb. As far as I can tell it should be pulling the value from the environment?

@toy
Copy link
Owner

toy commented Aug 22, 2020

Sounds weird, can you try to run following shell command:

ruby -r tempfile -e 'puts Tempfile.open("test"){ |f| f.path }; puts Dir.tmpdir; puts Etc.systmpdir'

Those methods are used in generation of temporary files for image_optim, so if the command shows usage of temp directory for first two, then the files you see through file watcher are those that are created for atomic replacement.

@luboslives
Copy link
Author

The printout looks correct... By default I get

/tmp/test20200823-1288-hztmcb
/tmp
/tmp

When I update the TMPDIR I get:

/root/testdir/tmp/test20200823-1295-cvkbgv
/root/testdir/tmp
/tmp

So maybe it is those replacement files I'm seeing... is there any way to give them a different file extension like .tmp? Thanks for your help.

toy added a commit that referenced this issue Aug 24, 2020
…lacement if temporary directory is on same device as destination, part of solution for #178
toy added a commit that referenced this issue Aug 24, 2020
…d for atomic replacement, part of solution for #178
@toy
Copy link
Owner

toy commented Aug 24, 2020

I've added few changes and an entry to readme about changing temporary directory, would be great if you can try version from less-tempfiles branch.

@luboslives
Copy link
Author

@toy I grabbed it and tried it out, but it looks like the bin itself is still an old version (so I get the same results). Do you or I have to build/make it? I'm not familiar with how that would work with Ruby 😊

@toy
Copy link
Owner

toy commented Aug 26, 2020

Sure :)
If it is ok to replace installed gem:

cd path-to-checkout-of-image_optim
gem build image_optim.gemspec
gem install image_optim-*.gem
image_optim ARGUMENTS_HERE

Afterwards you can run gem uninstall image_optim to remove it.

Or just running directly without installing:

cd path-to-checkout-of-image_optim
ruby -I lib bin/image_optim ARGUMENTS_HERE
# or without cd
ruby -I path-to-checkout-of-image_optim/lib path-to-checkout-of-image_optim/bin/image_optim ARGUMENTS_HERE

@luboslives
Copy link
Author

It seems to be working correctly! Nothing strange to report – no temp files being created in the watched/original directory.

@toy
Copy link
Owner

toy commented Aug 27, 2020

Great, I'll merge it then

@toy toy closed this as completed in 285affa Aug 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants