Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make data explorer non-blocking with docker compose #731

Merged
merged 4 commits into from
Jan 2, 2024

Conversation

PhilippeMoussalli
Copy link
Contributor

Small PR to avoid showing the localhost url before the image is build

@RobbeSneyders
Copy link
Member

The docker run call is blocking, so with this change the url will only be shown after you stop the explorer again. I think the only way to improve on the current situation is by making the docker run command non-blocking, which would probably be an improvement for usage in the notebooks anyway. We might want to use docker compose then instead of docker so the explorer is only started once and easy to bring down.

@PhilippeMoussalli
Copy link
Contributor Author

We might want to use docker compose then instead of docker so the explorer is only started once and easy to bring down.

Makes sense indeed, it's always a bit clunky to stop it from the notebook. I'll update the PR

@PhilippeMoussalli
Copy link
Contributor Author

We might want to use docker compose then instead of docker so the explorer is only started once and easy to bring down.

Makes sense indeed, it's always a bit clunky to stop it from the notebook. I'll update the PR

Updated the PR, the explorer now uses docker compose to launch the service, slight change to the cli command:

starting the service:

fondant explore start -b <base_path>

Stopping the service:

fondant explore stop

@RobbeSneyders RobbeSneyders changed the title Fix logs ordering Make data explorer non-blocking with docker compose Dec 20, 2023
Copy link
Member

@RobbeSneyders RobbeSneyders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @PhilippeMoussalli!

I had it a bit different in my mind, thinking about a docker compose file which we include in the Fondant package. But this approach indeed makes more sense so the compose file can be generated with different options 👍

The approach and code looks good to me, but I still want to test it. Will probably do so tomorrow!

Copy link
Member

@RobbeSneyders RobbeSneyders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just tested @PhilippeMoussalli. Works well, I would just catch errors with the docker command, see my comment below.

"--detach",
]

subprocess.call(cmd, stdout=subprocess.PIPE) # nosec
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use check_call here so we don't print the messages below if this fails (eg. when the docker daemon is not running).

try:
subprocess.check_call(cmd, stdout=subprocess.PIPE) # nosec
except subprocess.CalledProcessError as e:
raise OSError(e.returncode)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you choose an OSError here? I would probably have gone for a SystemExit exception myself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might make more sense here indeed, updated

Copy link
Member

@RobbeSneyders RobbeSneyders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PhilippeMoussalli PhilippeMoussalli merged commit 2c51c07 into main Jan 2, 2024
6 checks passed
@PhilippeMoussalli PhilippeMoussalli deleted the fix-explorer-logs-order branch January 2, 2024 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants