Skip to content

Commit

Permalink
VIGNETTES: Add example on 'arrow' for non-exportable objects
Browse files Browse the repository at this point in the history
  • Loading branch information
HenrikBengtsson committed Feb 29, 2024
1 parent 9526bd6 commit 6c041c2
Show file tree
Hide file tree
Showing 2 changed files with 34 additions and 1 deletion.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Package: future
Version: 1.33.1-9004
Version: 1.33.1-9005
Title: Unified Parallel and Distributed Processing in R for Everyone
Imports:
digest,
Expand Down
33 changes: 33 additions & 0 deletions vignettes/future-4-non-exportable-objects.md.rsp
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ _If you identify other cases, please consider [reporting](https://github.com/Hen

Package | Examples of non-exportable types or classes
:---------------|:-------------------------------------------
**arrow** | Table (`externalptr`)
**base** | connection (`externalptr`)
**cpp11** | E.g. functions created by `cpp_source()`
**DBI** | DBIConnection (`externalptr`)
Expand Down Expand Up @@ -144,6 +145,37 @@ However, there are objects with _external pointer_:s that can be exported, e.g.
Below are some examples of packages who produce non-exportable objects with _external pointer_:s.


#### Package: arrow

The **[arrow]** package provides efficient in-memory storage of arrays
and tables. How, these objects cannot be transferred as-is to a
parallel workers.

```r
library(arrow)

library(future)
plan(multisession)

data <- as_arrow_table(iris)

f <- future(dim(data))
v <- value(f)
#> Error: Invalid <Table>, external pointer to null
```

This error takes place on the parallel worker. We can use:

```r
options(future.globals.onReference = "error")
f <- future(dim(data))
#> Error: Detected a non-exportable reference ('externalptr') in one of the globals
#> ('data' of class 'Table') used in the future expression
```

to help us detect the problem sooner.


#### Package: cpp11

Another example is **[cpp11]**, which allows us to easily create R functions that are implemented in C++, e.g.
Expand Down Expand Up @@ -987,6 +1019,7 @@ str(e)


[data.table]: https://cran.r-project.org/package=data.table
[arrow]: https://cran.r-project.org/package=arrow
[cpp11]: https://cran.r-project.org/package=cpp11
[DBI]: https://cran.r-project.org/package=DBI
[inline]: https://cran.r-project.org/package=inline
Expand Down

0 comments on commit 6c041c2

Please sign in to comment.