Rewrite download/upload #169
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chStaiger
requested changes
May 28, 2024
| msg = f'Cannot write to {local_path}.' | ||
| if not ignore_err: | ||
| raise PermissionError(msg) from error | ||
| warnings.warn(msg) |
There was a problem hiding this comment.
We will also need to catch:
except irods.exception.CAT_NO_ACCESS_PERMISSION as error:
as in the _put method
There was a problem hiding this comment.
I think this is done now.
|
The upload and download have a dry_run. At the moment the dryrun will fail if overwrite is set to false. I think we can always carry out the dry run since we are not potentially overwriting data. |
I kind of agree, but then I'm also not sure what you would return? A special object that signifies that it can't be done? Then also how do we communicate why it can't be done? |
Co-authored-by: chStaiger <[email protected]>
Co-authored-by: chStaiger <[email protected]>
chStaiger
added a commit
that referenced
this pull request
May 30, 2024
API: - Move data objects and collections, aka rename - Improve connection checks - Rework the API - Bugfix in search - Examples in docstrings CLI: - Adding mkcoll and tree Update tutorials * Adding mkdir to CLI (#137) * Adding mkdir to CLI * pylint * Update ibridges/__main__.py Co-authored-by: qubixes <[email protected]> * Update ibridges/__main__.py Co-authored-by: qubixes <[email protected]> * Update ibridges/__main__.py Co-authored-by: qubixes <[email protected]> * Rename to mkcoll --------- Co-authored-by: qubixes <[email protected]> * Move data objects and collections (#136) * Adding moveing data objects and collections * Adding test for renaming * Renaming and adjusting tests. * Create dependabot.yml (#143) * Check for host and port before checking network (#139) * Check for host and port before checking network * Small changes --------- Co-authored-by: Staiger, Christine <[email protected]> * Update sphinx requirement from <7.0.0 to <8.0.0 (#145) * Adding IrodsPath doc * Small changes and index * Turning network_check into class method for GUI (#146) * Turning network_check into class method for GUI * Ruff * Install doc (#147) * Create install.rst * Adding install instructions * Update docs/source/install.rst Co-authored-by: qubixes <[email protected]> * Update docs/source/install.rst Co-authored-by: qubixes <[email protected]> --------- Co-authored-by: qubixes <[email protected]> * Fix docstring formatting error (#140) * Fix docstring formatting error * Add ruff to CI and fix small linting errors * Add sync documentation. (#148) * Add sync documentation. * add index * Update docs/source/sync.rst Co-authored-by: qubixes <[email protected]> * FOrmatting and small changes * Ruff --------- Co-authored-by: qubixes <[email protected]> Co-authored-by: Staiger, Christine <[email protected]> * Ipath doc (#153) * Update ipath with other documentation (#152) * Turning network_check into class method for GUI (#146) * Turning network_check into class method for GUI * Ruff * Install doc (#147) * Create install.rst * Adding install instructions * Update docs/source/install.rst Co-authored-by: qubixes <[email protected]> * Update docs/source/install.rst Co-authored-by: qubixes <[email protected]> --------- Co-authored-by: qubixes <[email protected]> * Fix docstring formatting error (#140) * Fix docstring formatting error * Add ruff to CI and fix small linting errors * Add sync documentation. (#148) * Add sync documentation. * add index * Update docs/source/sync.rst Co-authored-by: qubixes <[email protected]> * FOrmatting and small changes * Ruff --------- Co-authored-by: qubixes <[email protected]> Co-authored-by: Staiger, Christine <[email protected]> --------- Co-authored-by: qubixes <[email protected]> Co-authored-by: Staiger, Christine <[email protected]> * Formatting and cross referencing. --------- Co-authored-by: qubixes <[email protected]> Co-authored-by: Staiger, Christine <[email protected]> * Update ipath.rst * Update ipath.rst * Move functions to utils and add walk to IrodsPath (#151) * Add missing search for data_name (#161) * Add missing search for data_name * Rework API documentation (#156) * Rework API documentation * TRying to add file to resolve conflict * Rremove added file * Adding to edit * Delete file again. --------- Co-authored-by: chstaiger <[email protected]> Co-authored-by: chStaiger <[email protected]> * Bugfix in search (#162) * Don't use assert for control flow (#166) * Don't use assert for control flow * Fix __main__.py Only throw exception when needed. * Update ibridges/path.py Co-authored-by: chStaiger <[email protected]> --------- Co-authored-by: chStaiger <[email protected]> * Update iRODS path tutorial. (#165) * Update iRODS path tutorial. * Refer to tutorials for iRODS paths in RTD * Some more hints on paths in cli doc * Adjustong README * Remove old tutorial * Small edits * Update docs/source/cli.rst Co-authored-by: Terrell Russell <[email protected]> --------- Co-authored-by: Terrell Russell <[email protected]> * Sync update (#158) Rewrite of the sync_data function so that it uses the walk functionality of the IrodsPath instead of duplicated effort. Performance should be roughly the same. This update also fixes some issues with the walk functionality of the IrodsPath and makes it much faster. * Add tree for the iBridges CLI (#159) * Add tree for the iBridges CLI * Add options to tree - Ascii - depth - show-max * Make pylint happy * Add type annotations * Fix spelling errors in the README (#171) * Rewrite download/upload (#169) The main goal is code reduction and using the similarities of the download/upload and sync functionality. As a direct benefit, the upload and download functions now also have a progress bar. Some other notable changes: - `--dry-run` for upload and download - IrodsPath.absolute_path becomes IrodsPath.absolute * Cleanup tutorials. Create a First steps tutorial for the impatient. (#174) * Cleanup tutorials. Create a First steps tutorial for the impatient. * Review First steps * Update Irods-paths tutorial * Update working-with-data * Remove duplicate file --------- Co-authored-by: Raoul Schram <[email protected]> * update metadata * Fix behavior of parent property of IrodsPath (#175) * Fix behavior of parent property of IrodsPath * Fix the fix * Fix tests for new behavior * Fix tutorials (#177) * Spellchecking tutorials * Turn None time into empty string in listing tickets. (#178) * Add examples (#180) Add examples to many of the mostly used docstrings. Also contains a few tiny fixes. * Ruff --------- Co-authored-by: qubixes <[email protected]> Co-authored-by: Dawa Ometto <[email protected]> Co-authored-by: Staiger, Christine <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Terrell Russell <[email protected]> Co-authored-by: Raoul Schram <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The main goal is code reduction and using the similarities of the download/upload and sync functionality. As a direct benefit, the upload and download functions now also have a progress bar.
TODO:
Also fixes #168 , fixes #172