Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make data exchange workspace with cluster work again / improve it #16

Open
mspringma opened this issue Jun 4, 2016 · 1 comment
Open

Comments

@mspringma
Copy link
Contributor

Right now, sync'ing local files from workspace with cluster fails.

The good news: Mesosphere did improve default security setup on DC/OS 1.7 compared to prior releases.
The bad news: You have to access any node in the cluster via a proxy - in particular also for SSH.

For interactive SSH sessions, that's no big deal as "dcos node ssh --master-proxy --leader" as used in ssh-into-dcos-mater.sh is already available and in use.

But for the scripts using rsync, some reworking would be needed to make them work again (infrastructure tests relying on that currently fail). It may be possible to solve this issue - but probably we should switch to a different solution altogether, e.g. using S3 - which could also help providing non-volatile storage for big inout and result data.

@mspringma
Copy link
Contributor Author

Workaround for non-critical data is described in https://github.com/Zuehlke/hackzurich-sensordataanalysis/blob/master/S3ForSparkSubmit.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant