Weekly Digest (12 July, 2020 - 19 July, 2020) #4238

weekly-digest · 2020-07-19T04:26:07Z

Here's the Weekly Digest for iterative/dvc:

ISSUES

Last week 40 issues were created.
Of these, 25 issues have been closed and 15 issues are still open.

OPEN ISSUES

💚 #4235 safe Dvcfile.dump(), by skshetry
💚 #4234 Problems with dvc push, by LarryThermo
💚 #4233 repro: update status message for dvc-added files, by sarthakforwet
💚 #4232 dvc move fails when cache dir is read-only, by florianspecker
💚 #4230 hdfs tests failing, by skshetry
💚 #4229 Special dir warning #3436, by aerubanov
💚 #4224 S3: Use key_id and key_secret directly, by farizrahman4u
💚 #4223 Make dvc pull --run-cache allow targets for a granular pull process, by maricatovictor
💚 #4220 bump azure-storage-blob version to 12.3.2, by dtrifiro
💚 #4218 dvc gc remove , by HaddocktheHorrible
💚 #4213 Allow pipeline to create pipeline, by MatthieuBizien
💚 #4211 metrics diff: accept multiple revisions (like plots diff), by jorgeorpinel
💚 #4210 Back-fill metrics, by jonilaserson
💚 #4206 Daemon fails when dvc is called via a wrapper script., by danfischetti
💚 #4194 TLDR for DVC [https://github.com/tldr-pages/tldr], by PuneethaPai

CLOSED ISSUES

LIKED ISSUE

👍 #4199 Experiments proof of concept, by pmrowla
It received 👍 x3, 😄 x0, 🎉 x4 and ❤️ x3.

NOISY ISSUE

🔈 #4205 dvc commit fails with files imported via dvc import, by Cortysus
It received 11 comments.

PULL REQUESTS

Last week, 23 pull requests were created, updated or merged.

UPDATED PULL REQUEST

Last week, 5 pull requests were updated.
💛 #4229 Special dir warning #3436, by aerubanov
💛 #4224 S3: Use key_id and key_secret directly, by farizrahman4u
💛 #4220 bump azure-storage-blob version to 12.3.2, by dtrifiro
💛 #4114 version: redesign output format for the dvc version command #3499, by sahilbhosale63
💛 #3585 get/import: --rev BRANCH: shallow clone, by casperdcl

MERGED PULL REQUEST

Last week, 18 pull requests were merged.
💜 #4222 snap: deploy: fix major channels not being pushed, by casperdcl
💜 #4221 dvc: get rid of CleanTree, by efiop
💜 #4219 Bump networkx, by faucct
💜 #4216 dvc: get rid of WorkingTree, by efiop
💜 #4215 tree: remove duplicated code, by efiop
💜 #4214 dvc: remove tree-specific wrappers from cache, by efiop
💜 #4212 dvc: split remotes from trees, by efiop
💜 #4209 RepoDependency: never changed, by pared
💜 #4208 deps/outs: get rid of remotes in favor of trees, by efiop
💜 #4204 list: support tracked file path as a target, by efiop
💜 #4201 tests: http: fix typo, by efiop
💜 #4200 tests: remove real SSH, by efiop
💜 #4199 Experiments proof of concept, by pmrowla
💜 #4196 remove: remove dvc.yaml and dvc.lock if they are empty, by nik123
💜 #4192 pull: target message, by dmpetrov
💜 #4190 config: use repo.tree, by efiop
💜 #4125 LocalRemoteTree: use repo tree as work_tree with local outputs, by pared
💜 #4120 Optimize ignore performance, by karajan1001

COMMITS

Last week there were 22 commits.
🛠️ [Experiments proof of concept (#4199) * experiments: add initial experiments.show

only includes metrics + params (no code/data information)
experiments: basic dvc experiments show functionality
experiments: clone and run repro inside clone workspace
repro: add -e/--experiment option
experiments: hash experiments to identify duplicates
experiments: update show
experiments: checkout experiment after running repro
experiments: add dvc experiments checkout
experiments: add simple dvc experiments diff command
just shows combined output from metrics diff and plots diff for
now
add very simple tests
add some useful output messages
experiments: suppress help messages while feature is in development
experiment related commands will be accessible but hidden from the
default command help messages
add simple experiments command tests
experiments: disable feature by default
experiments only enabled in test environment (DVC_TEST) or when
core.experiments config option is true
use fetch on checkout missing revs](a56c8bb) by pmrowla
🛠️ snap: deploy: fix major channels not being pushed (#4222) by casperdcl
🛠️ dvc: bump to 1.1.11 by efiop
🛠️ output: local: use repo tree if possible (#4125) by pared
🛠️ RepoDependency: never changed (#4209) by pared
🛠️ [dvc: get rid of CleanTree (dvc: get rid of CleanTree #4221) * dvcignore: move file/dir/parent checks into DvcIgnoreFilter
dvc: get rid of CleanTree

CleanTree is a very awkward wrapper that spreads the context across
the codebase and results in unexpected behaviour when using it.

This PR starts moving dvcignore-related logic into the trees themselves
(it makes a lot of sense, kinda like state) so they could deal with
it however they like.

There are at least two temporary ugly parts about this PR:

dvcignore is used by individual trees and not packed into
tree/base.py yet;
dvcignore_root argument. This one is caused by the dvcignore trying
to collect everything topdown starting from the certain root dir. What
it should do instead is for a certain path that is being checked look up
the tree through the parents until it finds repo root (.dvc dir) and
then stop. That would handle subrepos as well. At the same time we need
to leverage existing dvcignore trie structure to cache those results.](625b208) by efiop
🛠️ Bump networkx (#4219) To avoid conflict with https://github.com/aws-cloudformation/cfn-python-lint by faucct
🛠️ [dvc: get rid of WorkingTree (dvc: get rid of WorkingTree #4216) Working tree is really just a regular local tree and should be
used by outputs when trying to compute a hash for themselves.
We didn't use it previously because local tree was embeded into
the local remote class.

Related to #4050](33abc40) by efiop
🛠️ tree: remove duplicated code (#4215) Leftover from remote/tree separation. by efiop
🛠️ dvc: remove tree-specific wrappers from cache (#4214) This helps combat the remote/tree/cache confusion. by efiop
🛠️ [dvc: split remotes from trees (#4212) * dvc: remove tree-specific wrappers from remote

This helps combat the remote/tree/cache confusion.

dvc: split remotes from trees

Related to #4050](76228a2) by efiop
🛠️ [deps/outs: get rid of remotes in favor of trees (#4208) Outputs and dependencies don't really care about the remotes, as they
don't do any pushing or pulling. What they really need are the trees,
so let's use them directly.

Part of #4050](bbb1c59) by efiop
🛠️ [Optimize ignore performance (#4120) * Remove Duplicate ignore Match

Continue Optimize Dvcignore

fix #3869

Add a new test with multi ignore files
Solve merging two dvc files
Solve Code Climate
For Windows
Complete addition of patterns. Add one test
Systematic test
Change request
Change request
Seperate path sepcification math
Rename and add comment
rename change_dirname to private
Update dvc/pathspec_math.py

list comprehension

Co-authored-by: Alexander Schepanovski [email protected]

Change request
Update dvc/ignore.py

Co-authored-by: karajan1001 [email protected]
Co-authored-by: Alexander Schepanovski [email protected]
Co-authored-by: Ruslan Kuprieiev [email protected]](58f6534) by karajan1001
🛠️ [list: support tracked file path as a target (#4204) * list: support tracked file path as a target

Fixes #4202

erepo: add str to improve error readablity

Related to #4202](66def03) by efiop
🛠️ tests: http: fix typo (#4201) Fixes #4182 by efiop
🛠️ [tests: remove real SSH (#4200) * tests: remove real SSH

No longer needed, all tests are using mocked ssh already.

travis: don't setup ssh](4c9245a) by efiop
🛠️ readme: remove python versions note by efiop
🛠️ [remove: remove dvc.yaml and dvc.lock if they are empty (remove: remove dvc.yaml and dvc.lock if they are empty #4196) * remove: remove dvc.yaml, dvc.lock if they are empty
tests: remove: simplified dvc.yaml in test_remove_stage_removes_dvcfiles_if_no_stages_left
tests: remove: minor fixes
tests: remove: simplified test_remove_stage_removes_dvcfiles_if_no_stages_left
remove: independent removal of dvc.lock and dvc.yaml](1c9588e) by nik123
🛠️ dvc: bump to 1.1.10 by efiop
🛠️ [pull: target message (pull: target message #4192) * pull: target message
update fetch/status/etc help

Co-authored-by: Ruslan Kuprieiev [email protected]](59409be) by dmpetrov
🛠️ dvc: bump to 1.1.9 by efiop
🛠️ config: use repo.tree (#4190) Fixes #4188 by efiop

CONTRIBUTORS

Last week there were 8 contributors.
👤 pmrowla
👤 casperdcl
👤 efiop
👤 pared
👤 faucct
👤 karajan1001
👤 nik123
👤 dmpetrov

STARGAZERS

Last week there were 86 stagazers.
⭐ MarkTuddenham
⭐ craigrshenton
⭐ jvmncs
⭐ fxghqc
⭐ sp1thas
⭐ programminghumanity
⭐ knightelvis
⭐ Duchadian
⭐ craig-shenton
⭐ itsmeale
⭐ litteZe
⭐ zhangmx
⭐ TeisNP
⭐ apethani21
⭐ Randy-Soper
⭐ Natumsol
⭐ yanqinghao
⭐ sollde
⭐ xiguagua
⭐ drunkwretch
⭐ lfeakm
⭐ xhj
⭐ kebohiki
⭐ zxytim
⭐ yanyaoer
⭐ Anguliachao
⭐ sumtumn
⭐ buzai
⭐ robscc
⭐ ilias-ant
⭐ andreaschandra
⭐ lsanotes
⭐ galaxy001
⭐ DN-Web64
⭐ vagmcs
⭐ marcosgama
⭐ aankasman
⭐ dsatecnologia
⭐ syan4826
⭐ jeffhsu3
⭐ shan224
⭐ JoshuaTao
⭐ khalidperwira
⭐ kangvcar
⭐ uhhc
⭐ Svtter
⭐ maxmilia
⭐ philMarius
⭐ Emrys-Hong
⭐ spiros
⭐ shkey
⭐ dwelcaslu
⭐ monz
⭐ mfhbam
⭐ lucmos
⭐ vasulakkaraju
⭐ gad2103
⭐ Jamesits
⭐ gcgbarbosa
⭐ krzemienski
⭐ luxiaolei
⭐ moealmaw
⭐ Clement25
⭐ rbreunev
⭐ IvanYue
⭐ DACUS1995
⭐ BrunoGomesCoelho
⭐ marskar
⭐ almereyda
⭐ lucasvazq
⭐ robert35h
⭐ gabriellovate
⭐ scientistkev
⭐ JaeyeongYang
⭐ SolsticeDante
⭐ devender-yadav
⭐ victorgrubio
⭐ benwtrent
⭐ sabralod
⭐ wtyree
⭐ Hyun-June-Choi
⭐ vishweshs4
⭐ orrshilon
⭐ chapmanjacobd
⭐ Nazar96
⭐ abdulshawakri
You all are the stars! 🌟

RELEASES

Last week there were 3 releases.
🚀 1.1.11 1.1.11
🚀 1.1.10 1.1.10
🚀 1.1.9 1.1.9

That's all for last week, please 👀 Watch and ⭐ Star the repository iterative/dvc to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

The text was updated successfully, but these errors were encountered:

weekly-digest bot added the weekly-digest label Jul 19, 2020

efiop closed this as completed Jul 19, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Weekly Digest (12 July, 2020 - 19 July, 2020) #4238

Weekly Digest (12 July, 2020 - 19 July, 2020) #4238

weekly-digest bot commented Jul 19, 2020

Weekly Digest (12 July, 2020 - 19 July, 2020) #4238

Weekly Digest (12 July, 2020 - 19 July, 2020) #4238

Comments

weekly-digest bot commented Jul 19, 2020

ISSUES

OPEN ISSUES

CLOSED ISSUES

LIKED ISSUE

NOISY ISSUE

PULL REQUESTS

UPDATED PULL REQUEST

MERGED PULL REQUEST

COMMITS

CONTRIBUTORS

STARGAZERS

RELEASES