Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GDrive improvements and code cleanup #3294

Merged
merged 8 commits into from
Feb 11, 2020
Merged

GDrive improvements and code cleanup #3294

merged 8 commits into from
Feb 11, 2020

Conversation

shcheklein
Copy link
Member

@shcheklein shcheklein commented Feb 10, 2020

It's a major rewrite now. It was needed since it was very hard an unproductive fixing certain issues w/o addressing some underlying problems with code.

  • Performance of remote id resolving improved. It does not hit API now for "remote path". Affects pull - one API call per file less!
  • Performance of the first run should be improved. We now cache path->id as we create them (case when we push a lot first time we create a remote).
  • Number of retries increased. The previous was not enough with default GDrive limits.
  • Does not create remote path anymore (see the Stop creating a path to the DVC remote root if it does not exist checkbox) in Google Drive support further enhancements #2865
  • Code cleanup.

  • ❗ Have you followed the guidelines in the Contributing to DVC list?

  • 📖 Check this box if this PR does not require documentation updates, or if it does and you have created a separate PR in dvc.org with such updates (or at least opened an issue about it in that repo). Please link below to your PR (or issue) in the dvc.org repo.

  • ❌ Have you checked DeepSource, CodeClimate, and other sanity checks below? We consider their findings recommendatory and don't expect everything to be addressed. Please review them carefully and fix those that actually improve code or fix bugs.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

@shcheklein shcheklein changed the title [WIP] Gdrive code cleanup [WIP] GDrive improvements and code cleanup Feb 10, 2020
@codecov
Copy link

codecov bot commented Feb 11, 2020

Codecov Report

❗ No coverage uploaded for pull request base (master@2aff0a0). Click here to learn what that means.
The diff coverage is 96.42%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master    #3294   +/-   ##
=========================================
  Coverage          ?   92.83%           
=========================================
  Files             ?      139           
  Lines             ?     8541           
  Branches          ?        0           
=========================================
  Hits              ?     7929           
  Misses            ?      612           
  Partials          ?        0
Impacted Files Coverage Δ
dvc/command/imp.py 86.36% <ø> (ø)
dvc/exceptions.py 97.27% <ø> (ø)
dvc/command/metrics.py 96.82% <ø> (ø)
dvc/command/get.py 91.89% <100%> (ø)
dvc/output/local.py 98.27% <100%> (ø)
dvc/repo/metrics/diff.py 95.38% <100%> (ø)
dvc/repo/diff.py 100% <100%> (ø)
dvc/repo/reproduce.py 98.66% <100%> (ø)
dvc/command/diff.py 92.85% <100%> (ø)
dvc/repo/__init__.py 96.76% <88.88%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2aff0a0...85fef88. Read the comment docs.

@shcheklein
Copy link
Member Author

@efiop I suggest we review codeclimate and deepsource settings a bit and make the more useful/less noisy for us. E.g. number of arguments per function - if we think that 4 is to aggressive let's increase it a bit?

@shcheklein shcheklein requested review from maxhora and efiop February 11, 2020 03:21
@shcheklein shcheklein changed the title [WIP] GDrive improvements and code cleanup GDrive improvements and code cleanup Feb 11, 2020
Copy link
Contributor

@maxhora maxhora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great to me! 👍

@efiop efiop merged commit 5e98a70 into master Feb 11, 2020
@delete-merged-branch delete-merged-branch bot deleted the gdrive-cleanup branch February 11, 2020 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants