Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use directory as output for dvc run when using hdfs as target #1275

Closed
amjadsaadeh opened this issue Oct 30, 2018 · 4 comments
Closed

Use directory as output for dvc run when using hdfs as target #1275

amjadsaadeh opened this issue Oct 30, 2018 · 4 comments

Comments

@amjadsaadeh
Copy link

Hello,

I would like to track directories on hdfs for my pipeline. E.g. I run:

dvc run -d ./images -o hdfs://<hostname>:<port>/image_data hdfs dfs -put ./images /image_data

but currently this fails because hadoop cli cannot calculate the checksum of a directory.
amjadsaadeh added a commit to amjadsaadeh/dvc that referenced this issue Oct 30, 2018
@amjadsaadeh
hdfs remote: add support for directories as output for dvc run
79aae45 
Fixes: iterative#1275
@amjadsaadeh amjadsaadeh mentioned this issue Oct 30, 2018
Closed
@efiop
Copy link
Contributor

efiop commented Nov 24, 2018

Duplicate of #1083 .

@efiop efiop closed this as completed Nov 24, 2018
@efiop efiop reopened this Nov 24, 2018
@ghost
Copy link

ghost commented Feb 4, 2019

@efiop, why did you reopened the issue? Was it a mistake?

@efiop efiop mentioned this issue Feb 4, 2019
Closed
@efiop
Copy link
Contributor

efiop commented Feb 4, 2019

@MrOutis Yep, sorry. Closed #1083 instead. Thanks for the heads up! :)

@efiop
Copy link
Contributor

efiop commented Feb 22, 2019

Moving to #1654

@efiop efiop closed this as completed Feb 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@amjadsaadeh @efiop