Skip to content

Commit

Permalink
[SPARK-12168][SPARKR] Add automated tests for conflicted function in R
Browse files Browse the repository at this point in the history
Currently this is reported when loading the SparkR package in R (probably would add is.nan)
```
Loading required package: methods

Attaching package: ‘SparkR’

The following objects are masked from ‘package:stats’:

    cov, filter, lag, na.omit, predict, sd, var

The following objects are masked from ‘package:base’:

    colnames, colnames<-, intersect, rank, rbind, sample, subset,
    summary, table, transform
```

Adding this test adds an automated way to track changes to masked method.
Also, the second part of this test check for those functions that would not be accessible without namespace/package prefix.

Incidentally, this might point to how we would fix those inaccessible functions in base or stats.
Looking for feedback for adding this test.

Author: felixcheung <[email protected]>

Closes #10171 from felixcheung/rmaskedtest.
  • Loading branch information
felixcheung authored and shivaram committed Jan 20, 2016
1 parent 3e84ef0 commit 37fefa6
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 1 deletion.
2 changes: 1 addition & 1 deletion R/pkg/NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -271,10 +271,10 @@ export("as.DataFrame",
"createExternalTable",
"dropTempTable",
"jsonFile",
"read.json",
"loadDF",
"parquetFile",
"read.df",
"read.json",
"read.parquet",
"read.text",
"sql",
Expand Down
23 changes: 23 additions & 0 deletions R/pkg/inst/tests/testthat/test_context.R
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,29 @@

context("test functions in sparkR.R")

test_that("Check masked functions", {
# Check that we are not masking any new function from base, stats, testthat unexpectedly
masked <- conflicts(detail = TRUE)$`package:SparkR`
expect_true("describe" %in% masked) # only when with testthat..
func <- lapply(masked, function(x) { capture.output(showMethods(x))[[1]] })
funcSparkROrEmpty <- grepl("\\(package SparkR\\)$|^$", func)
maskedBySparkR <- masked[funcSparkROrEmpty]
expect_equal(length(maskedBySparkR), 18)
expect_equal(sort(maskedBySparkR), sort(c("describe", "cov", "filter", "lag", "na.omit",
"predict", "sd", "var", "colnames", "colnames<-",
"intersect", "rank", "rbind", "sample", "subset",
"summary", "table", "transform")))
# above are those reported as masked when `library(SparkR)`
# note that many of these methods are still callable without base:: or stats:: prefix
# there should be a test for each of these, except followings, which are currently "broken"
funcHasAny <- unlist(lapply(masked, function(x) {
any(grepl("=\"ANY\"", capture.output(showMethods(x)[-1])))
}))
maskedCompletely <- masked[!funcHasAny]
expect_equal(length(maskedCompletely), 4)
expect_equal(sort(maskedCompletely), sort(c("cov", "filter", "sample", "table")))
})

test_that("repeatedly starting and stopping SparkR", {
for (i in 1:4) {
sc <- sparkR.init()
Expand Down

0 comments on commit 37fefa6

Please sign in to comment.