Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional nanoarrow refactoring #682

Merged
merged 6 commits into from
Mar 29, 2024

Conversation

eddelbuettel
Copy link
Contributor

Building on #663 (and its SC 41534), additional clean up of the Arrow import/export code and decrufting is done here.

All interface changes are at the mostly internal function level and concern the switch from operating on an explicit two-element list of Arrow Array and Schema external pointers to an (nanoarrow-special) ArrowArrray external pointer which tugs the Schema external pointer into the tag field of itself (ie the Array pointer) making the function single argument (nicer with C API of R we have to adhere to) and a SEXP itself (as an external pointer is a possible SEXP instance within the scope of the SEXP union type). Cleverly the SEXP is actually classed as an S3 object by nanoarrow so after eg creating quickstart_sparse we can do this:

> library(tiledb) 
TileDB R 0.25.0.3 with TileDB Embedded 2.22.0 on Ubuntu 23.10.  
See https://tiledb.com for more information about TileDB. 
> library(nanoarrow)   
> uri <- "quickstart_sparse" 
> arr <- tiledb_array(uri, "READ") 
> qry <- tiledb_query(arr, "READ") 
> values <- integer(8)       # R shortcut to reserver a buffer
> tiledb_query_set_buffer(qry, "a", values) 
> tiledb_query_submit(qry) 
> tiledb_query_finalize(qry) 
>
> # we now get a nanoarrow S3 object which has a print method
> na <- tiledb_query_export_buffer(qry, "a") 
> na 
<nanoarrow_array int32[4]>
 $ length    : int 4  
 $ null_count: int 0 
 $ offset    : int 0     
 $ buffers   :List of 2 
  ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` 
  ..$ :<nanoarrow_buffer data<int32>[4][16 b]> `1 2 3 2`
 $ dictionary: NULL 
 $ children  : list()
>
> # and arrow converters
> arrow::arrow_array(na)
Array
<int32>
[
  1,
  2,
  3,
  2
]
> 

Copy link

This pull request has been linked to Shortcut Story #44187: Extend nanoarrow refactoring.

@eddelbuettel eddelbuettel merged commit d796865 into master Mar 29, 2024
1 check passed
@eddelbuettel eddelbuettel deleted the de/sc-44187/more_nanoarrow_refactoring branch March 29, 2024 16:20
@eddelbuettel eddelbuettel mentioned this pull request Apr 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants