Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Subprocess memory error when trying to save large Docker images #69

Closed
nishakm opened this issue Jul 18, 2018 · 1 comment
Closed

Subprocess memory error when trying to save large Docker images #69

nishakm opened this issue Jul 18, 2018 · 1 comment
Assignees
@nishakm
Labels
bug Something went wrong
Milestone
Release 0.2.0

Comments

@nishakm
Copy link
Contributor

nishakm commented Jul 18, 2018

Reproducible on: golang:1.9.4
golang's image size is 735MB.

docker_command uses popen rather than check_output or check_call so the stderr can be captured and recorded for the logger. However, when trying to save a large image this does not work as it uses buffered memory.
Possible solutions:

  1. subprocess.Popen can flush the buffer after it gets filled
  2. subprocess.check_output can be used without shell=True and redirect stderr to a variable instead of the tty's stderr
  3. Ignore the pipe to tty's std error and return some generic message for a failure to 'docker save'
@nishakm nishakm added the bug Something went wrong label Jul 18, 2018
@nishakm nishakm added this to the Release 0.2.0 milestone Jul 19, 2018
@nishakm
Copy link
Contributor Author

nishakm commented Aug 21, 2018

Going with using subprocess.check_output for only docker save command.

@nishakm nishakm self-assigned this Sep 10, 2018
rnjudge pushed a commit to rnjudge/tern that referenced this issue Jun 5, 2020
Used subprocess check_output for docker save
14e3064 
The docker save command is what creates the tar of the container
image. For larger images the pipe buffer is too small. So using
subprocess.check_output to get the result of docker save.

- Moved logic to check for sudo use in is_sudo function
- Added function docker_command_check for use of
subprocess.check_output to fork processes
- Used docker_command_check in the extract_image_metadata function

Resolves tern-tools#69

Signed-off-by: Nisha K <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nishakm nishakm

Labels
bug Something went wrong
Projects
None yet
Milestone
Release 0.2.0
Development

No branches or pull requests
1 participant
@nishakm