Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gittargets: data version control for the targets package #486

Closed
11 of 24 tasks
wlandau opened this issue Nov 17, 2021 · 64 comments
Closed
11 of 24 tasks

gittargets: data version control for the targets package #486

wlandau opened this issue Nov 17, 2021 · 64 comments
Assignees

Comments

@wlandau
Copy link
Member

wlandau commented Nov 17, 2021

Date accepted: 2022-01-12
Submitting Author Name: Will Landau
Submitting Author Github Handle: @wlandau
Repository: https://github.com/wlandau/gittargets
Version submitted: 0.0.0.9000
Submission type: Standard
Editor: @adamhsparks
Reviewers: @smwindecker, @mdneuzerling

Due date for @smwindecker: 2021-12-20

Due date for @mdneuzerling: 2021-12-28
Archive: TBD
Version accepted: TBD
Language: en

  • Paste the full DESCRIPTION file inside a code block below:
Package: gittargets
Title: Data Version Control for the Targets Package
Description: Version control systems such as Git help researchers
  track changes and history in data science projects,
  and the 'targets' package (2021, <doi:10.21105/joss.02959>)
  minimizes the computational cost of keeping the latest results
  reproducible and up to date. The 'gittargets' package
  combines these two capabilities. The 'targets' data store
  becomes a version control repository and stays synchronized
  with the Git repository of the source code.
  Users can switch commits and branches
  without invalidating the 'targets' pipeline.
Version: 0.0.0.9000
License: MIT + file LICENSE
URL: https://wlandau.github.io/gittargets/, https://github.com/wlandau/gittargets
BugReports: https://github.com/wlandau/gittargets/issues
Authors@R: c(
  person(
    given = c("William", "Michael"),
    family = "Landau",
    role = c("aut", "cre"),
    email = "[email protected]",
    comment = c(ORCID = "0000-0003-1878-3253")
  ),
  person(
    family = "Eli Lilly and Company",
    role = "cph"
  ))
Depends:
  R (>= 3.5.0)
Imports:
  cli (>= 3.0.0),
  data.table (>= 1.12.8),
  gert (>= 1.0.0),
  processx (>= 3.0.0),
  stats,
  targets (>= 0.8.1.9000),
  tibble (>= 3.0.0),
  utils,
  uuid (>= 1.0.0)
Suggests:
  knitr (>= 1.30),
  markdown (>= 1.1),
  rmarkdown (>= 2.4),
  testthat (>= 3.0.0)
Remotes:
  ropensci/targets
SystemRequirements: Git (>= 2.0.0)
Encoding: UTF-8
Language: en-US
VignetteBuilder: knitr
Config/testthat/edition: 3
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.2

Scope

  • Please indicate which category or categories from our package fit policies this package falls under: (Please check an appropriate box below. If you are unsure, we suggest you make a pre-submission inquiry.):

    • data retrieval
    • data extraction
    • data munging
    • data deposition
    • workflow automation
    • version control
    • citation management and bibliometrics
    • scientific software wrappers
    • field and lab reproducibility tools
    • database software bindings
    • geospatial data
    • text analysis
  • Explain how and why the package falls under these categories (briefly, 1-2 sentences):

Version control systems such as Git help researchers track changes and history in data science projects, and the targets package minimizes the computational cost of keeping the latest results reproducible and up to date. The gittargets package combines these two capabilities. The targets data store becomes a version control repository and stays synchronized with the Git repository of the source code. Users can switch commits and branches without invalidating the targets pipeline.

  • Who is the target audience and what are scientific applications of this package?

gittargets is for people who use targets for reproducible analysis pipelines and Git for tracking the code files of those pipelines.

Packages gh, gert, git2r, and git2rdata all work with Git from R, but in a way that is more general, less opinionated, and less targets-specific than gittargets. gittargets establishes a separate repository for the targets data that is linked to the project's code repository. This allows the code and data to synchronize without affecting the efficiency or the size of the code repository.

N/A

  • If you made a pre-submission inquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

N/A

Technical checks

Confirm each of the following by checking the box.

This package:

Publication options

  • Do you intend for this package to go on CRAN?

  • Do you intend for this package to go on Bioconductor?

  • Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:

MEE Options
  • [n/a] The package is novel and will be of interest to the broad readership of the journal.
  • [n/a] The manuscript describing the package is no longer than 3000 words.
  • [n/a] You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
  • (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no guarantee that your manuscript will be within MEE scope.)
  • (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
  • (Please do not submit your package separately to Methods in Ecology and Evolution)

Code of conduct

@ropensci-review-bot
Copy link
Collaborator

Thanks for submitting to rOpenSci, our editors and @ropensci-review-bot will reply soon. Type @ropensci-review-bot help for help.

@ropensci-review-bot
Copy link
Collaborator

🚀

Error: Issue template has no 'repourl'

👋

@ropensci-review-bot
Copy link
Collaborator

Missing values: author1, repourl

@wlandau
Copy link
Member Author

wlandau commented Nov 17, 2021

Sorry about that, should be fixed now.

@ldecicco-USGS
Copy link

@ropensci-review-bot check package

@ropensci-review-bot
Copy link
Collaborator

Thanks, about to send the query.

@ropensci-review-bot
Copy link
Collaborator

🚀

Editor check started

👋

@ropensci-review-bot
Copy link
Collaborator

Checks for gittargets (v0.0.0.9000)

git hash: b6eadc4b

  • ✔️ Package name is available
  • ✔️ has a 'CITATION' file.
  • ✔️ has a 'codemeta.json' file.
  • ✔️ has a 'contributing' file.
  • ✔️ uses 'roxygen2'.
  • ✔️ 'DESCRIPTION' has a URL field.
  • ✔️ 'DESCRIPTION' has a BugReports field.
  • ✔️ Package has at least one HTML vignette
  • ✔️ All functions have examples.
  • ✔️ Package has continuous integration checks.
  • ✖️ Package coverage is 13.3% (should be at least 75%).
  • ✔️ R CMD check found no errors.
  • ✔️ R CMD check found no warnings.

Important: All failing checks above must be addressed prior to proceeding

Package License: MIT + file LICENSE


1. Statistical Properties

This package features some noteworthy statistical properties which may need to be clarified by a handling editor prior to progressing.

Details of statistical properties (click to open)

The package has:

  • code in R (100% in 15 files) and
  • 1 authors
  • 1 vignette
  • no internal data file
  • 9 imported packages
  • 9 exported functions (median 23 lines of code)
  • 94 non-exported functions in R (median 6 lines of code)

Statistical properties of package structure as distributional percentiles in relation to all current CRAN packages
The following terminology is used:

  • loc = "Lines of Code"
  • fn = "function"
  • exp/not_exp = exported / not exported

The final measure (fn_call_network_size) is the total number of calls between functions (in R), or more abstract relationships between code objects in other languages. Values are flagged as "noteworthy" when they lie in the upper or lower 5th percentile.

measure value percentile noteworthy
files_R 15 70.6
files_vignettes 0 0.0 TRUE
files_tests 17 94.1
loc_R 663 53.5
loc_tests 417 68.5
num_vignettes 1 60.7
n_fns_r 103 71.7
n_fns_r_exported 9 38.9
n_fns_r_not_exported 94 76.9
n_fns_per_file_r 4 51.7
num_params_per_fn 4 54.3
loc_per_fn_r 7 23.4
loc_per_fn_r_exp 23 54.1
loc_per_fn_r_not_exp 6 29.1
rel_whitespace_R 6 24.3
rel_whitespace_tests 7 76.1
doclines_per_fn_exp 62 74.1
doclines_per_fn_not_exp 0 0.0 TRUE
fn_call_network_size 123 80.4

1a. Network visualisation

Click to see the interactive network visualisation of calls between objects in package


2. goodpractice and other checks

Details of goodpractice and other checks (click to open)

3a. Continuous Integration Badges

github
github

GitHub Workflow Results

name conclusion sha date
check success b6eadc 2021-11-17
cover success b6eadc 2021-11-17
lint success b6eadc 2021-11-17
pkgdown success b6eadc 2021-11-17

3b. goodpractice results

R CMD check with rcmdcheck

rcmdcheck found no errors, warnings, or notes

Test coverage with covr

Package coverage: 13.35

The following files are not completely covered by tests:

file coverage
R/tar_git_checkout.R 0%
R/tar_git_init.R 0%
R/tar_git_log.R 0%
R/tar_git_ok.R 0%
R/tar_git_snapshot.R 0%
R/tar_git_status_code.R 0%
R/tar_git_status_data.R 0%
R/tar_git_status_targets.R 0%
R/tar_git_status.R 0%
R/utils_assert.R 0%
R/utils_git.R 30.77%

Cyclocomplexity with cyclocomp

No functions have cyclocomplexity >= 15

Static code analyses with lintr

lintr found no issues with this package!


Package Versions

package version
pkgstats 0.0.2.72
pkgcheck 0.0.2.107


Editor-in-Chief Instructions:

Processing may not proceed until the items marked with ✖️ have been resolved.

@wlandau
Copy link
Member Author

wlandau commented Nov 17, 2021

gittargets requires command line Git in order for git-lfs to work (related: r-lib/gert#160). The majority of tests will automatically be skipped if tar_git_ok() returns FALSE. Is there still a path forward?

@ldecicco-USGS
Copy link

Yeah, that makes sense. I'll get started on getting this moving.

@mpadge
Copy link
Member

mpadge commented Nov 17, 2021

And I'll check out why coverage might have failed tomorrow, although git is definitely installed. @wlandau Nothing you need to worry about, thanks.


Edit: For the record, @wlandau helped debug there, and the bot checks now give coverage of 100% 🚀 💯

@wlandau
Copy link
Member Author

wlandau commented Nov 17, 2021

Thank you both!

Related: I have not been able to correctly set up Git on the Windows GitHub Actions for gittargets. I tried a bunch of different approaches, but I still get errors claiming I need to configure git config --global user.name and git config --global user.email. The other runners appear to work just fine though, including the one that checks coverage.

@ldecicco-USGS
Copy link

@ropensci-review-bot assign @adamhsparksas editor

@ldecicco-USGS
Copy link

@ropensci-review-bot assign @adamhsparks as editor

@ropensci-review-bot
Copy link
Collaborator

Assigned! @adamhsparks is now the editor

@adamhsparks
Copy link
Member

@ropensci-review-bot check package

@ropensci-review-bot
Copy link
Collaborator

Thanks, about to send the query.

@ropensci-review-bot
Copy link
Collaborator

🚀

Editor check started

👋

@ropensci-review-bot
Copy link
Collaborator

Checks for gittargets (v0.0.0.9000)

git hash: 5c046f1d

  • ✔️ Package name is available
  • ✔️ has a 'CITATION' file.
  • ✔️ has a 'codemeta.json' file.
  • ✔️ has a 'contributing' file.
  • ✔️ uses 'roxygen2'.
  • ✔️ 'DESCRIPTION' has a URL field.
  • ✔️ 'DESCRIPTION' has a BugReports field.
  • ✔️ Package has at least one HTML vignette
  • ✔️ All functions have examples.
  • ✔️ Package has continuous integration checks.
  • ✔️ Package coverage is 100%.
  • ✔️ R CMD check found no errors.
  • ✔️ R CMD check found no warnings.

Package License: MIT + file LICENSE


1. Statistical Properties

This package features some noteworthy statistical properties which may need to be clarified by a handling editor prior to progressing.

Details of statistical properties (click to open)

The package has:

  • code in R (100% in 15 files) and
  • 1 authors
  • 1 vignette
  • no internal data file
  • 9 imported packages
  • 9 exported functions (median 23 lines of code)
  • 96 non-exported functions in R (median 6 lines of code)

Statistical properties of package structure as distributional percentiles in relation to all current CRAN packages
The following terminology is used:

  • loc = "Lines of Code"
  • fn = "function"
  • exp/not_exp = exported / not exported

The final measure (fn_call_network_size) is the total number of calls between functions (in R), or more abstract relationships between code objects in other languages. Values are flagged as "noteworthy" when they lie in the upper or lower 5th percentile.

measure value percentile noteworthy
files_R 15 70.6
files_vignettes 1 64.8
files_tests 17 94.1
loc_R 669 53.8
loc_vignettes 143 62.1
loc_tests 503 72.2
num_vignettes 1 60.7
n_fns_r 105 72.3
n_fns_r_exported 9 38.9
n_fns_r_not_exported 96 77.3
n_fns_per_file_r 4 52.4
num_params_per_fn 4 54.3
loc_per_fn_r 7 23.4
loc_per_fn_r_exp 23 54.1
loc_per_fn_r_not_exp 6 23.5
rel_whitespace_R 6 24.9
rel_whitespace_vignettes 43 80.0
rel_whitespace_tests 7 78.4
doclines_per_fn_exp 61 73.3
doclines_per_fn_not_exp 0 0.0 TRUE
fn_call_network_size 132 81.5

1a. Network visualisation

Click to see the interactive network visualisation of calls between objects in package


2. goodpractice and other checks

Details of goodpractice and other checks (click to open)

3a. Continuous Integration Badges

github
github

GitHub Workflow Results

name conclusion sha date
check success 5c046f 2021-11-18
cover success 5c046f 2021-11-18
cover-ubuntu success 5c046f 2021-11-18
lint success 5c046f 2021-11-18
pkgdown success 5c046f 2021-11-18

3b. goodpractice results

R CMD check with rcmdcheck

rcmdcheck found no errors, warnings, or notes

Test coverage with covr

Package coverage: 100

Cyclocomplexity with cyclocomp

No functions have cyclocomplexity >= 15

Static code analyses with lintr

lintr found no issues with this package!


Package Versions

package version
pkgstats 0.0.3.51
pkgcheck 0.0.2.130


Editor-in-Chief Instructions:

This package is in top shape and may be passed on to a handling editor

@adamhsparks
Copy link
Member

adamhsparks commented Nov 25, 2021

Hi @wlandau, sorry for the delays. I'll endeavour to complete my checks today and start looking for reviewers so we can at least have some assigned before Christmas.

@adamhsparks
Copy link
Member

adamhsparks commented Nov 25, 2021

Editor checks:

  • Documentation: The package has sufficient documentation available online (README, pkgdown docs) to allow for an assessment of functionality and scope without installing the package. In particular,
    • Is the case for the package well made?
    • Is the reference index page clear (grouped by topic if necessary)?
    • Are vignettes readable, sufficiently detailed and not just perfunctory?
  • Fit: The package meets criteria for fit and overlap.
  • Installation instructions: Are installation instructions clear enough for human users?
  • Tests: If the package has some interactivity / HTTP / plot production etc. are the tests using state-of-the-art tooling?
  • Contributing information: Is the documentation for contribution clear enough e.g. tokens for tests, playgrounds?
  • License: The package has a CRAN or OSI accepted license.
  • Project management: Are the issue and PR trackers in a good shape, e.g. are there outstanding bugs, is it clear when feature requests are meant to be tackled?

Editor comments

Thanks for this submission, Will. There's a few of us that were excited to see this one submitted for review. Everything looks good from here with my editor checks. I don't have any major suggestions so I'll start looking for reviewers now.


@wlandau
Copy link
Member Author

wlandau commented Nov 27, 2021

Thanks, @adamhsparks! I am excited too.

@ropensci-review-bot
Copy link
Collaborator

Logged review for mdneuzerling (hours: 5)

@adamhsparks
Copy link
Member

@ropensci-review-bot submit review time 7

@ropensci-review-bot
Copy link
Collaborator

Logged review for smwindecker (hours: 7)

@adamhsparks
Copy link
Member

@ropensci-review-bot approve gittargets

@ropensci-review-bot
Copy link
Collaborator

Approved! Thanks @wlandau for submitting and @smwindecker, @mdneuzerling for your reviews! 😁

To-dos:

  • Transfer the repo to rOpenSci's "ropensci" GitHub organization under "Settings" in your repo. I have invited you to a team that should allow you to do so.
  • After transfer write a comment @ropensci-review-bot finalize transfer of <package-name> where <package-name> is the repo/package name. This will give you admin access back.
  • Fix all links to the GitHub repo to point to the repo under the ropensci organization.
  • Delete your current code of conduct file if you had one since rOpenSci's default one will apply, see https://devguide.ropensci.org/collaboration.html#coc-file
  • If you already had a pkgdown website and are ok relying only on rOpenSci central docs building and branding,
    • deactivate the automatic deployment you might have set up
    • remove styling tweaks from your pkgdown config but keep that config file
    • replace the whole current pkgdown website with a redirecting page
    • replace your package docs URL with https://docs.ropensci.org/package_name
    • In addition, in your DESCRIPTION file, include the docs link in the URL field alongside the link to the GitHub repository, e.g.: URL: https://docs.ropensci.org/foobar (website) https://github.com/ropensci/foobar
  • Fix any links in badges for CI and coverage to point to the new repository URL.
  • Increment the package version to reflect the changes you made during review. In NEWS.md, add a heading for the new version and one bullet for each user-facing change, and each developer-facing change that you think is relevant.
  • We're starting to roll out software metadata files to all rOpenSci packages via the Codemeta initiative, see https://docs.ropensci.org/codemetar/for how to include it in your package, after installing the package - should be easy as running codemetar::write_codemeta() in the root of your package.
  • You can add this installation method to your package README install.packages("<package-name>", repos = "https://ropensci.r-universe.dev") thanks to R-universe.

Should you want to acknowledge your reviewers in your package DESCRIPTION, you can do so by making them "rev"-type contributors in the Authors@R field (with their consent).

Welcome aboard! We'd love to host a post about your package - either a short introduction to it with an example for a technical audience or a longer post with some narrative about its development or something you learned, and an example of its use for a broader readership. If you are interested, consult the blog guide, and tag @stefaniebutland in your reply. She will get in touch about timing and can answer any questions.

We maintain an online book with our best practice and tips, this chapter starts the 3d section that's about guidance for after onboarding (with advice on releases, package marketing, GitHub grooming); the guide also feature CRAN gotchas. Please tell us what could be improved.

Last but not least, you can volunteer as a reviewer via filling a short form.

@adamhsparks
Copy link
Member

@mdneuzerling, I noted after reading your approval that you still hadn't ticked the "Statement of need" in your checklist. I'm assuming that this has been met since you've approved the package revisions in your most recent post regarding updates to the README. If this is correct, could you please just tic that box to complete the review for us? 🙏

@mdneuzerling
Copy link

@adamhsparks Sorry about that! Done.

@adamhsparks
Copy link
Member

Well done, @wlandau! Thanks for this new package submission to rOpenSci. I'm guessing you know the drill now. If you need assistance with anything, please let me know.

@wlandau
Copy link
Member Author

wlandau commented Jan 12, 2022

Amazing! Thanks @adamhsparks for facilitating the process and thanks @smwindecker and @mdneuzerling for the reviews! gittargets is in much better shape now, especially regarding documentation.

@adamhsparks, would you be willing to grant me admin permissions at https://github.com/ropensci/gittargets? The rOpenSci transfer process still removes that on my end.

@wlandau
Copy link
Member Author

wlandau commented Jan 12, 2022

Ah, sorry I didn't see this:

After transfer write a comment @ropensci-review-bot finalize transfer of where is the repo/package name. This will give you admin access back.

@wlandau
Copy link
Member Author

wlandau commented Jan 12, 2022

@ropensci-review-bot finalize transfer gittargets

@wlandau
Copy link
Member Author

wlandau commented Jan 12, 2022

@ropensci-review-bot finalize transfer

@wlandau
Copy link
Member Author

wlandau commented Jan 12, 2022

@ropensci-review-bot finalize transfer of gittargets

@ropensci-review-bot
Copy link
Collaborator

Transfer completed. The gittargets team is now owner of the repository

@wlandau
Copy link
Member Author

wlandau commented Jan 12, 2022

I still do not have admin permissions. Did I need to add myself to the gittargets team in advance?

@wlandau
Copy link
Member Author

wlandau commented Jan 12, 2022

Other than that, I believe all the to-dos are now complete.

@maelle
Copy link
Member

maelle commented Jan 13, 2022

@wlandau I've now added you to that team. You hadn't received any invitation to it beforehands, correct? Thanks for the bug catching. 🙏

@wlandau
Copy link
Member Author

wlandau commented Jan 13, 2022

Thanks, @maelle! I did not receive an invitation before, but I do have admin permissions now.

@wlandau
Copy link
Member Author

wlandau commented Jan 18, 2022

Very minor: on my end, it looks like the rOpenSci README badge still says "Under Review". Are these badges still supported?

Screen Shot 2022-01-18 at 3 03 39 PM

@mpadge
Copy link
Member

mpadge commented Jan 18, 2022

Sorry @wlandau, our badge server went to sleep a month ago without anybody realising - I've re-enabled it, so it should update soon.

@wlandau
Copy link
Member Author

wlandau commented Jan 19, 2022

Thanks Mark! Looks to be fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants