Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a v2 example_package() #258

Merged
merged 34 commits into from
Aug 29, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
c4ec0dc
folder move
sannegovaert Aug 27, 2024
36900c3
update path
sannegovaert Aug 27, 2024
c6a2d36
document()
sannegovaert Aug 27, 2024
ab26085
precompile vignette
sannegovaert Aug 27, 2024
f86a6fb
update path
sannegovaert Aug 27, 2024
4a48d26
Merge branch 'main' into v2-example_package
sannegovaert Aug 27, 2024
85e97f4
add param version
sannegovaert Aug 27, 2024
f8b3168
update path
sannegovaert Aug 27, 2024
8865e29
Create test-example_package.R
sannegovaert Aug 27, 2024
d8eef96
add error class
sannegovaert Aug 27, 2024
bbce506
copy v1 example data packaga
sannegovaert Aug 27, 2024
678ed47
add https://datapackage.org/overview/changelog/#packageschema-new
sannegovaert Aug 27, 2024
f829191
reorder to match https://datapackage.org/standard/data-package/
sannegovaert Aug 27, 2024
5344d0f
https://datapackage.org/overview/changelog/#resourcetype-new
sannegovaert Aug 27, 2024
0c0fdf5
add version (https://datapackage.org/overview/changelog/#packageversi…
sannegovaert Aug 27, 2024
55b32cb
Version should be a string value "1.0"
sannegovaert Aug 27, 2024
423a367
add resources.$schema
sannegovaert Aug 27, 2024
7575c54
fix error
sannegovaert Aug 27, 2024
0f193e4
primaryKey and foreignKeys should be an array of strings
sannegovaert Aug 28, 2024
a62d028
add version 2.0
sannegovaert Aug 28, 2024
5dd7ac6
add tests on versions
sannegovaert Aug 28, 2024
c40ddb3
document()
sannegovaert Aug 28, 2024
e6f1ff0
Add version to v1 and order image and created the same
peterdesmet Aug 29, 2024
fce05a0
Add $schema to schema and dialect
peterdesmet Aug 29, 2024
2b1e012
Change missing values to one with labels
peterdesmet Aug 29, 2024
2401644
Update created date
peterdesmet Aug 29, 2024
2993338
Change enum to categories
peterdesmet Aug 29, 2024
b06ff8c
Update doc and example
peterdesmet Aug 29, 2024
7e44709
Update doc and example
peterdesmet Aug 29, 2024
8c7a66a
Rather than returning error, default to 1.0
peterdesmet Aug 29, 2024
d25aaf9
Merge branch 'v2-example_package' of https://github.com/frictionlessd…
peterdesmet Aug 29, 2024
24bb578
Update example_package.R
peterdesmet Aug 29, 2024
ae72c51
Fix test
peterdesmet Aug 29, 2024
07f2002
Update NEWS.md
peterdesmet Aug 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions R/add_resource.R
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,10 @@
#' )
#'
#' # Replace the resource "observations" with a file-based resource (2 TSV files)
#' path_1 <- system.file("extdata", "observations_1.tsv", package = "frictionless")
#' path_2 <- system.file("extdata", "observations_2.tsv", package = "frictionless")
#' path_1 <-
#' system.file("extdata", "v1", "observations_1.tsv", package = "frictionless")
#' path_2 <-
#' system.file("extdata", "v1", "observations_2.tsv", package = "frictionless")
#' package <- add_resource(
#' package,
#' resource_name = "observations",
Expand Down
16 changes: 14 additions & 2 deletions R/example_package.R
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,24 @@
#' `"path": ["observations_1.tsv", "observations_2.tsv"]`.
#' 3. `media`: inline data stored in `data`.
#'
#' @param version Version of the [Data Package](
#' https://datapackage.org/blog/2024-06-26-v2-release/) standard that this
#' example Data Package uses. Currently only version 1 is supported.
#' @return A Data Package object, see [create_package()].
#' @family sample data
#' @export
#' @examples
#' example_package()
example_package <- function() {
path <- system.file("extdata", "datapackage.json", package = "frictionless")
example_package <- function(version = 1) {
peterdesmet marked this conversation as resolved.
Show resolved Hide resolved
if (!version %in% c(1, "1", 1.0, "1.0")) {
cli::cli_abort(
"Currently only version 1 (`version = 1`) is supported.",
class = "frictionless_error_example_package_version_invalid"
)
} else {
path <- system.file(
"extdata", "v1", "datapackage.json", package = "frictionless"
)
}
read_package(path)
}
2 changes: 1 addition & 1 deletion R/read_package.R
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
#' @examples
#' # Read a datapackage.json file
#' package <- read_package(
#' system.file("extdata", "datapackage.json", package = "frictionless")
#' system.file("extdata", "v1", "datapackage.json", package = "frictionless")
#' )
#'
#' package
Expand Down
2 changes: 1 addition & 1 deletion R/read_resource.R
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
#' @examples
#' # Read a datapackage.json file
#' package <- read_package(
#' system.file("extdata", "datapackage.json", package = "frictionless")
#' system.file("extdata", "v1", "datapackage.json", package = "frictionless")
#' )
#'
#' package
Expand Down
2 changes: 1 addition & 1 deletion R/write_package.R
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
#' @examples
#' # Load the example Data Package from disk
#' package <- read_package(
#' system.file("extdata", "datapackage.json", package = "frictionless")
#' system.file("extdata", "v1", "datapackage.json", package = "frictionless")
#' )
#'
#' package
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
6 changes: 4 additions & 2 deletions man/add_resource.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion man/example_package.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/read_package.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/read_resource.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/write_package.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions tests/testthat/test-add_resource.R
Original file line number Diff line number Diff line change
Expand Up @@ -338,7 +338,7 @@ test_that("add_resource() can add resource from local, relative, absolute,

# Absolute (doesn't throw unsafe error)
absolute_path <- system.file(
"extdata", "deployments.csv", package = "frictionless" # Will start with /
"extdata", "v1", "deployments.csv", package = "frictionless" # Will start with /
)
p <- add_resource(p, "absolute", absolute_path, schema)
expect_identical(p$resources[[6]]$path, absolute_path)
Expand Down Expand Up @@ -377,7 +377,7 @@ test_that("add_resource() can add resource from CSV file with other delimiter,

test_that("add_resource() sets correct properties for CSV resources", {
p <- create_package()
path <- system.file("extdata", "deployments.csv", package = "frictionless")
path <- system.file("extdata", "v1", "deployments.csv", package = "frictionless")

# Encoding UTF-8 (0.8), ISO-8859-1 (0.59), ISO-8859-2 (0.26)
p <- add_resource(p, "deployments", path)
Expand Down
14 changes: 14 additions & 0 deletions tests/testthat/test-example_package.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
test_that("example_package() returns error on invalid version", {
expect_error(
example_package("2"),
class = "frictionless_error_example_package_version_invalid"
)
expect_error(
example_package(2),
class = "frictionless_error_example_package_version_invalid"
)
expect_error(
example_package(2.0),
class = "frictionless_error_example_package_version_invalid"
)
})
6 changes: 3 additions & 3 deletions tests/testthat/test-read_package.R
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
test_that("read_package() returns a valid Data Package reading from path", {
# Load example package and a valid minimal one
p_path <- system.file("extdata", "datapackage.json", package = "frictionless")
p_path <- system.file("extdata", "v1", "datapackage.json", package = "frictionless")
minimal_path <- test_path("data/valid_minimal.json")
p_local <- read_package(p_path)
p_minimal <- read_package(minimal_path)
Expand Down Expand Up @@ -67,7 +67,7 @@ test_that("read_package() returns error on missing file and properties", {
# Not a json file
expect_error(
read_package(
system.file("extdata", "deployments.csv", package = "frictionless")
system.file("extdata", "v1", "deployments.csv", package = "frictionless")
),
regexp = "lexical error: invalid char in json text.",
fixed = FALSE
Expand Down Expand Up @@ -119,7 +119,7 @@ test_that("read_package() allows YAML descriptor", {
})

test_that("read_package() converts JSON null to NULL", {
p_path <- system.file("extdata", "datapackage.json", package = "frictionless")
p_path <- system.file("extdata", "v1", "datapackage.json", package = "frictionless")
p <- read_package(p_path)
# { "image": null } is read as NULL (use chuck() to force error if missing)
expect_null(purrr::chuck(p, "image"))
Expand Down
4 changes: 2 additions & 2 deletions tests/testthat/test-read_resource.R
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ test_that("read_resource() returns error on invalid resource", {
# Add valid path
p_invalid$resources[[1]]$path <- "deployments.csv"
p_invalid$directory <- dirname(
system.file("extdata", "datapackage.json", package = "frictionless")
system.file("extdata", "v1", "datapackage.json", package = "frictionless")
)

# Not a tabular-data-resource
Expand Down Expand Up @@ -288,7 +288,7 @@ test_that("read_resource() can read local files", {
resource <- read_resource(p, "deployments")

p_local <- read_package(
system.file("extdata", "datapackage.json", package = "frictionless")
system.file("extdata", "v1", "datapackage.json", package = "frictionless")
)
expect_identical(read_resource(p_local, "deployments"), resource)
})
Expand Down
3 changes: 2 additions & 1 deletion tests/testthat/test-write_package.R
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ test_that("write_package() returns error if Data Package has no resource(s)", {
})

test_that("write_package() writes unaltered datapackage.json as is", {
p_file <- system.file("extdata", "datapackage.json", package = "frictionless")
p_file <-
system.file("extdata", "v1", "datapackage.json", package = "frictionless")
json_original <- readr::read_lines(p_file) # Will use line endings of system
p <- read_package(p_file)
dir <- file.path(tempdir(), "package")
Expand Down
2 changes: 1 addition & 1 deletion vignettes/data-package.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Frictionless supports reading, manipulating and writing packages. Much of its fu

```{r}
library(frictionless)
file <- system.file("extdata", "datapackage.json", package = "frictionless")
file <- system.file("extdata", "v1", "datapackage.json", package = "frictionless")
package <- read_package(file)
```

Expand Down
10 changes: 5 additions & 5 deletions vignettes/data-resource.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ remove_resource(package, "deployments")
add_resource(package, "iris", data = iris)

# Replace a resource with one where data is stored in a tabular file
path <- system.file("extdata", "deployments.csv", package = "frictionless")
path <- system.file("extdata", "v1", "deployments.csv", package = "frictionless")
add_resource(package, "deployments", data = path, replace = TRUE)
```

Expand Down Expand Up @@ -105,7 +105,7 @@ read_resource(package, "observations")
`add_resource()` sets `path` to the path(s) provided in `data`:

```{r}
path <- system.file("extdata", "deployments.csv", package = "frictionless")
path <- system.file("extdata", "v1", "deployments.csv", package = "frictionless")
add_resource(package, "deployments", data = path, replace = TRUE)
```

Expand Down Expand Up @@ -176,7 +176,7 @@ delim | format
any other value | `"csv"`

```{r}
path <- system.file("extdata", "observations_1.tsv", package = "frictionless")
path <- system.file("extdata", "v1", "observations_1.tsv", package = "frictionless")
package <- add_resource(package, "observations", data = path, delim = "\t", replace = TRUE)
package$resources[[2]]$format
```
Expand All @@ -194,7 +194,7 @@ delim | mediatype
any other value | `"text/csv"`

```{r}
path <- system.file("extdata", "observations_1.tsv", package = "frictionless")
path <- system.file("extdata", "v1", "observations_1.tsv", package = "frictionless")
package <- add_resource(package, "observations", data = path, delim = "\t", replace = TRUE)
package$resources[[2]]$mediatype
```
Expand All @@ -208,7 +208,7 @@ package$resources[[2]]$mediatype
`add_resource()` guesses the `encoding` (using `readr::guess_encoding()`) when data are provided as file. It leaves the `encoding` undefined when data are provided as a data frame. `write_package()` sets it to `"utf-8"` when writing to disk.

```{r}
path <- system.file("extdata", "deployments.csv", package = "frictionless")
path <- system.file("extdata", "v1", "deployments.csv", package = "frictionless")
package <- add_resource(package, "deployments", data = path, delim = ",", replace = TRUE)
package$resources[[2]]$encoding
```
Expand Down
6 changes: 3 additions & 3 deletions vignettes/frictionless.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ You can also read data from a local (e.g. downloaded) Data Package. In fact, the

``` r
local_package <- read_package(
system.file("extdata", "datapackage.json", package = "frictionless")
system.file("extdata", "v1", "datapackage.json", package = "frictionless")
)

local_package
Expand Down Expand Up @@ -327,8 +327,8 @@ If you already have your data stored as CSV or TSV files and you want to include

``` r
# Two TSV files with the same structure
path_1 <- system.file("extdata", "observations_1.tsv", package = "frictionless")
path_2 <- system.file("extdata", "observations_2.tsv", package = "frictionless")
path_1 <- system.file("extdata", "v1", "observations_1.tsv", package = "frictionless")
path_2 <- system.file("extdata", "v1", "observations_2.tsv", package = "frictionless")

# Add both TSV files as a single resource
my_package <-
Expand Down
6 changes: 3 additions & 3 deletions vignettes/frictionless.Rmd.orig
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ You can also read data from a local (e.g. downloaded) Data Package. In fact, the

```{r}
local_package <- read_package(
system.file("extdata", "datapackage.json", package = "frictionless")
system.file("extdata", "v1", "datapackage.json", package = "frictionless")
)

local_package
Expand Down Expand Up @@ -171,8 +171,8 @@ If you already have your data stored as CSV or TSV files and you want to include

```{r}
# Two TSV files with the same structure
path_1 <- system.file("extdata", "observations_1.tsv", package = "frictionless")
path_2 <- system.file("extdata", "observations_2.tsv", package = "frictionless")
path_1 <- system.file("extdata", "v1", "observations_1.tsv", package = "frictionless")
path_2 <- system.file("extdata", "v1", "observations_2.tsv", package = "frictionless")

# Add both TSV files as a single resource
my_package <-
Expand Down
2 changes: 1 addition & 1 deletion vignettes/table-dialect.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Frictionless does not support direct manipulation of the dialect. `add_resource(
library(frictionless)
package <- example_package()

path <- system.file("extdata", "observations_1.tsv", package = "frictionless")
path <- system.file("extdata", "v1", "observations_1.tsv", package = "frictionless")
package <- add_resource(package, "observations", data = path, delim = "\t", replace = TRUE)
package$resources[[2]]$dialect$delimiter
```
Expand Down
Loading