Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closes #164 Closes #228 Closes #234 xportr format messaging #243

Merged
merged 18 commits into from
Feb 29, 2024

Conversation

sophie-gem
Copy link
Collaborator

@sophie-gem sophie-gem commented Feb 18, 2024

Thank you for your Pull Request!

We have developed a Pull Request template to aid you and our reviewers. Completing the below tasks helps to ensure our reviewers can maximize their time on your code as well as making sure the xportr codebase remains robust and consistent.

The scope of {xportr}

{xportr}'s scope is to enable R users to write out submission compliant xpt files that can be delivered to a Health Authority or to downstream validation software programs. We see labels, lengths, types, ordering and formats from a dataset specification object (SDTM and ADaM) as being our primary focus. We also see messaging and warnings to users around applying information from the specification file as a primary focus. Please make sure your Pull Request meets this scope of {xportr}. If your Pull Request moves beyond this scope, please get in touch with the {xportr} team on slack or create an issue to discuss.

Please check off each task box as an acknowledgment that you completed the task. This checklist is part of the Github Action workflows and the Pull Request will not be merged into the main branch until you have checked off each task.

Changes Description

Following checks were added to the format function:

  • If variable ends DT, DTM, TM then a warning is shown if no format associated.
  • If a variable is character then a warning is shown if there is no $ prefix.
  • If a variable is character then a warning is shown if length > 31 (excluding $).
  • If a variable is numeric then a warning is shown if there is a $ prefix.
  • If a variable is numeric then a warning is shown if length > 32.
  • Message is shown if format is not part of a 'standard' set of formats expected for an ADaM dataset.

Relevant information added to the function documentation.

Tests for each of the checks above added to the function testthat file.

Task List

  • The spirit of xportr is met in your Pull Request
  • Place Closes #<insert_issue_number> into the beginning of your Pull Request Title (Use Edit button in top-right if you need to update)
  • Summary of changes filled out in the above Changes Description. Can be removed or left blank if changes are minor/self-explanatory.
  • Code is formatted according to the tidyverse style guide. Use styler package and functions to style files accordingly.
  • New functions or arguments follow established convention found in the Wiki.
  • Updated relevant unit tests or have written new unit tests. See our Wiki for conventions used in this package.
  • Creation/updated relevant roxygen headers and examples. See our Wiki for conventions used in this package.
  • Run devtools::document() so all .Rd files in the man folder and the NAMESPACE file in the project root are updated appropriately
  • Run pkgdown::build_site() and check that all affected examples are displayed correctly and that all new/updated functions occur on the "Reference" page.
  • Update NEWS.md if the changes pertain to a user-facing function (i.e. it has an @export tag) or documentation aimed at users (rather than developers)
  • The NEWS.md entry should go under the # xportr development version section. Don't worry about updating the version because it will be auto-updated using the vbump.yaml CI.
  • Address any updates needed for vignettes and/or templates.
  • Link the issue Development Panel so that it closes after successful merging.
  • The developer is responsible for fixing merge conflicts not the Reviewer.
  • Pat yourself on the back for a job well done! Much love to your accomplishment!

@sophie-gem sophie-gem linked an issue Feb 18, 2024 that may be closed by this pull request
@sophie-gem
Copy link
Collaborator Author

Due to the series of checks added to the format function - it has the impact of increasing the cyclomatic complexity such that it produces a [cyclocomp_lintr] function. Any advice on how I can avoid this would be greatly appreciated!

@sophie-gem
Copy link
Collaborator Author

I've run pkgdown::build_reference() to check the function documentation has updated correctly. However, when I try to run pkgdown::build_site() I get the following error...

image

Anyone seen this before? It may just be a case that I need to delete my local git repo and re-start again.

@sophie-gem sophie-gem marked this pull request as ready for review February 18, 2024 16:06
@sophie-gem sophie-gem requested a review from bms63 February 18, 2024 16:07
@sophie-gem
Copy link
Collaborator Author

@bms63 I have added you as reviewer - but if there is someone else better placed to do this - please let me know.

@EeethB
Copy link
Collaborator

EeethB commented Feb 18, 2024

I've run pkgdown::build_reference() to check the function documentation has updated correctly. However, when I try to run pkgdown::build_site() I get the following error...

image

Anyone seen this before? It may just be a case that I need to delete my local git repo and re-start again.

It looks like there's an error when it tries to install. What does devtools::install() do when you run it?

@sophie-gem
Copy link
Collaborator Author

I've run pkgdown::build_reference() to check the function documentation has updated correctly. However, when I try to run pkgdown::build_site() I get the following error...
image
Anyone seen this before? It may just be a case that I need to delete my local git repo and re-start again.

It looks like there's an error when it tries to install. What does devtools::install() do when you run it?

devtools::install() seems to run fine with no errors...

NEWS.md Outdated Show resolved Hide resolved
R/format.R Outdated Show resolved Hide resolved
R/format.R Show resolved Hide resolved
R/format.R Show resolved Hide resolved
tests/testthat/test-format.R Outdated Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you create a few tests that show xportr_format() working with xport_type() and xportr_write().

unless you spot these tests are already being checked in other functions' test. Just want to make sure everything is playing nice together.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course, will put these in place.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where would these tests be best placed, @bms63? If it's testing the interaction between varying functions - should a new script be created?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is fine to leave them in the test file for xportr_format(). My take on this is that we are focusing on testing the interactions of xportr_format with the other functions so it is a natural build up of tests...i.e., where we look at single use of the function and its arguments, possible bad input scenarios, writing out the xpt and then working well with others.

But it might be a good practice to document in our testing wiki on this testing strategy, e.g.

  • Single focus tests on specific function and it arguments
  • Possible data corruption scenarios and bad inputs
  • Single focus tests Writing out xpt with specific function
  • Tests showing interaction with other xportr functions

Any thoughts @atorus-research/xportr-development-team.

R/format.R Outdated
Comment on lines 14 to 16
#' 1) If the variable has a suffix of `DT`, `DTM`, `TM` (indicating a
#' numeric date/time variable) then a warning will be shown if there is
#' no format associated with it.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The warning only fires when the user sets verbose = "warn" correct?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that is not the case currently - will update so that this becomes the case.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bms63 - Would this also be the case for the 'message' also?

Copy link
Collaborator Author

@sophie-gem sophie-gem Feb 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOTE TO SELF - need to update (as a result of this change...)

  • Function documentation
  • Tests to include verbose option
  • Additional test for a change in verbose option

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Halfway through the tests - will continue tomorrow evening)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bms63 - Would this also be the case for the 'message' also?

If the users sets to 'nonethen nothing should come out of this function. The other functions are set up like this. It might be really good to make a test that usesxportr_metadata()andxportr()just to make sure these wrapper functions still work withxportr_format()` update

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is already a test in test-xportr.R which tests that xportr() works - is this sufficient do you think, or would you like another? (It passes with the updates to xportr_format() still).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if a similar test already exists then no worries.

maybe we should just have a test-xportr-fcns-interactions.R rather than my other proposed suggestion

Copy link
Collaborator

@bms63 bms63 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really amazing work @sophie-gem !!! I was able to build the website. Not sure if you were able to solve your issues.
image

@sophie-gem
Copy link
Collaborator Author

Really amazing work @sophie-gem !!! I was able to build the website. Not sure if you were able to solve your issues.

Many thanks for your detailed review @bms63 - really appreciate the feedback! Will start implementing now. :)

Update NEWS.md according to changes requested.

Co-authored-by: Ben Straub <[email protected]>
@sophie-gem sophie-gem changed the title Closes #164 xportr format messaging@main Closes #164 Closes #228 Closes #234 xportr format messaging Feb 25, 2024
@bms63
Copy link
Collaborator

bms63 commented Feb 26, 2024

Due to the series of checks added to the format function - it has the impact of increasing the cyclomatic complexity such that it produces a [cyclocomp_lintr] function. Any advice on how I can avoid this would be greatly appreciated!

You can boast the cyclomatic check up to 25 and this will pass. Thanks for making an issue around this!!

Copy link
Collaborator

@averissimo averissimo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job with this!!

Mostly minor comments with 1 exception on excluding the $

R/format.R Outdated Show resolved Hide resolved
R/format.R Outdated Show resolved Hide resolved
}

# if the variable is numeric
if (class(.df[[i]])[1] == "numeric") {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason why you're comparing the class?

The same might be achieved using base R (this will return a single TRUE/FALSE)

Suggested change
if (class(.df[[i]])[1] == "numeric") {
if (is.numeric(.df[[i]])) {

Or via the {checkmate} library

Suggested change
if (class(.df[[i]])[1] == "numeric") {
if (checkmate::test_numeric(.df[[i]])) {

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was using the class attribute as that is what is being used in the xportr_type() function - was trying to keep consistent.

R/format.R Outdated Show resolved Hide resolved
R/format.R Show resolved Hide resolved
R/format.R Outdated Show resolved Hide resolved
Copy link

github-actions bot commented Feb 29, 2024

Code Coverage

Package Line Rate Health
xportr 100%
Summary 100% (777 / 778)

@sophie-gem
Copy link
Collaborator Author

Due to the series of checks added to the format function - it has the impact of increasing the cyclomatic complexity such that it produces a [cyclocomp_lintr] function. Any advice on how I can avoid this would be greatly appreciated!

You can boast the cyclomatic check up to 25 and this will pass. Thanks for making an issue around this!!

I'm not entirely sure I understand? How would I go about doing this?

@sophie-gem
Copy link
Collaborator Author

@bms63 @averissimo , I'm getting slightly concerned that we are approaching the date that you'd like to release and am running out of capacity to get this completed...does anyone else have the bandwidth to help finish this off? The only things I believe are left to do are:

  • interaction tests between the functions in xportr.
  • increase the cyclomatic check in order so it passes (for the time being).

Other than that I believe all other changes requested have been made.

Many thanks both for your detailed review - really appreciated!

@@ -1,5 +1,5 @@
linters: linters_with_defaults(
cyclocomp_linter(complexity_limit = 18),
cyclocomp_linter(complexity_limit = 25),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @sophie-gem this is where we can adjust complexity lintr. Just FYI most packages have a .lintr file to adjust things like this

Copy link
Collaborator

@bms63 bms63 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll make a issue for 0.5 to have a bit of rethink of the interaction tests. We have really good board coverage (100%!) but I think now we can start to deepen the coverage as well!! :) Thanks again for seeing this one through to the end!

@bms63 bms63 merged commit 05177c6 into main Feb 29, 2024
14 of 15 checks passed
@bms63 bms63 deleted the 164_xportr_format_messaging@main branch February 29, 2024 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants