Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use docker to run containers started from Jupyter Notebook #3098

Closed
fjpa121197 opened this issue Feb 16, 2023 · 0 comments · Fixed by #3099
Closed

Unable to use docker to run containers started from Jupyter Notebook #3098

fjpa121197 opened this issue Feb 16, 2023 · 0 comments · Fixed by #3099

Comments

@fjpa121197
Copy link

fjpa121197 commented Feb 16, 2023

Hello,

I'm currently following this tutorial: https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-pipelines/tabular/local-mode/sagemaker-pipelines-local-mode.ipynb

I'm getting the following error from trying to run it (it uses docker in the background to run contaiiners). It executes everything when the following command is run:

execution = pipeline.start()

I get the following error:

Creating q0r36pja78-algo-1-ywafn ... 
Creating q0r36pja78-algo-1-ywafn ... done
Attaching to q0r36pja78-algo-1-ywafn
Traceback (most recent call last):
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\Scripts\docker-compose.exe\__main__.py", line 7, in <module>
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\compose\cli\main.py", line 81, in main
    command_func()
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\compose\cli\main.py", line 203, in perform_command
    handler(command, command_options)
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\compose\metrics\decorator.py", line 18, in wrapper
    result = fn(*args, **kwargs)
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\compose\cli\main.py", line 1216, in up
    cascade_starter = log_printer.run()
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\compose\cli\log_printer.py", line 88, in run
    for line in consume_queue(queue, self.cascade_stop):
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\compose\cli\log_printer.py", line 250, in consume_queue
    raise item.exc
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\compose\cli\log_printer.py", line 162, in tail_container_logs
    for item in build_log_generator(container, log_args):
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\compose\utils.py", line 50, in split_buffer
    for data in stream_as_text(stream):
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\compose\utils.py", line 26, in stream_as_text
    for data in stream:
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\docker\types\daemon.py", line 32, in __next__
    return next(self._stream)
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\docker\api\client.py", line 418, in <genexpr>
    gen = (data for (_, data) in gen)
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\docker\utils\socket.py", line 95, in <genexpr>
    return ((STDOUT, frame) for frame in frames_iter_tty(socket))
  File "C:\Users\franc\anaconda3\envs\sm-pipelines-modelbuild\lib\site-packages\docker\utils\socket.py", line 128, in frames_iter_tty
    if len(result) == 0:
TypeError: object of type 'int' has no len()
Pipeline step 'AbaloneProcess' FAILED. Failure message is: RuntimeError: Failed to run: ['docker-compose', '-f', 'C:\\Users\\franc\\AppData\\Local\\Temp\\tmpga4umz96\\docker-compose.yaml', 'up', '--build', '--abort-on-container-exit']
Pipeline execution c0a11456-aec5-48ec-adde-4ee45085efa8 FAILED because step 'AbaloneProcess' failed.

Version of the modules:
Python 3.8.16
docker 6.0.1
docker-compose 1.29.2
docker desktop 4.16.3

Thanks

milas added a commit to milas/docker-py that referenced this issue Feb 16, 2023
Need to return data, not size. By returning an empty
string, EOF will be detected properly since `len()`
will be `0`.

Fixes docker#3098.

Signed-off-by: Milas Bowman <[email protected]>
@milas milas self-assigned this Feb 16, 2023
milas added a commit that referenced this issue Feb 22, 2023
Need to return data, not size. By returning an empty
string, EOF will be detected properly since `len()`
will be `0`.

Fixes #3098.

Signed-off-by: Milas Bowman <[email protected]>
felixfontein added a commit to felixfontein/community.docker that referenced this issue May 10, 2023
Need to return data, not size. By returning an empty
string, EOF will be detected properly since `len()`
will be `0`.

Fixes docker/docker-py#3098.

Cherry-picked from docker/docker-py@f846232

Co-authored-by: Milas Bowman <[email protected]>
felixfontein added a commit to ansible-collections/community.docker that referenced this issue May 20, 2023
* socket: fix for errors on pipe close in Windows (docker/docker-py#3099)

Need to return data, not size. By returning an empty
string, EOF will be detected properly since `len()`
will be `0`.

Fixes docker/docker-py#3098.

Cherry-picked from docker/docker-py@f846232

Co-authored-by: Milas Bowman <[email protected]>

* socket: use poll() instead of select() except on Windows (docker/docker-py#2865)

Fixes docker/docker-py#2278, which was originally addressed in docker/docker-py#2279, but was not
properly merged. Additionally it did not address the problem
of poll not existing on Windows. This patch falls back on the
more limited select method if host system is Windows.

Cherry-picked from docker/docker-py@a02ba74

Co-authored-by: Tyler Westland <[email protected]>

* api: respect timeouts on Windows named pipes (docker/docker-py#3112)

Cherry-picked from docker/docker-py@9cadad0

Co-authored-by: Imogen <[email protected]>

* Add URL to changelog.

* api: avoid socket timeouts when executing commands (docker/docker-py#3125)

Only listen to read events when polling a socket in order
to avoid incorrectly trying to read from a socket that is
not actually ready.

Cherry-picked from docker/docker-py@c5e582c

Co-authored-by: Loïc Leyendecker <[email protected]>

---------

Co-authored-by: Milas Bowman <[email protected]>
Co-authored-by: Tyler Westland <[email protected]>
Co-authored-by: Imogen <[email protected]>
Co-authored-by: Loïc Leyendecker <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants