NetCDF driver missing from GDAL in OSX binary packages

[edzer]

See https://cran.r-project.org/web/checks/check_results_stars.html : package stars now errors on reading NetCDF files on all OSX platforms. It reads raster data through GDAL, which is provided by sf. I just checked on an OSX machine, and the NetCDF driver in GDAL is missing (e.g. in both sf and terra). This used to not be the case. Other R packages (ncdf4, RNetCDF) work fine, so NetCDF seems to work on OSX. I looked into the recipes file for GDAL here and wondered whether netcdf should be listed in Depends: ?

@s-u could you pls take a look at this, when you get some time?
Cc: @rhijmans @rsbivand

[s-u]

@edzer yes, you're right, gdal is missing netcdf in Depends:. It happened to work when the binaries were built over existing runs where netcdf was already installed, but now that things are built automatically on a clean VM the dependencies matter. I'll try to rebuild and install new binaries this week.

[s-u]

BTW: please file this in https://github.com/R-macos/recipes so it can be addressed.

[rsbivand]

The sf patch for the development version of Rtools42 is at https://www.r-project.org/nosvn/winutf8/ucrt3/patches/CRAN/sf.diff, for terra same directory. These libraries have extended recently to cover

• Szip compression of HDF5 files (-lsz is already in the macOS GDAL recipe, but I think from szip given the Depends line in https://github.com/R-macos/recipes/blob/master/recipes/gdal rather than libaec, see Issues with reading HDF5 files using terra::rast() rspatial/terra#686; HDF5 build) https://github.com/R-macos/recipes/blob/master/recipes/hdf5 should use aec https://svn.r-project.org/R-dev-web/trunk/WindowsBuilds/winutf8/ucrt3/toolchain_libs/mxe/src/hdf5.mk,
• BLOSC compression of ZARR files,
• Geospatial PDF,
• KEA,
  and others that users had motivated.

[edzer]

I added a PR to R-macos/recipes: R-macos/recipes#33

[s-u]

@edzer, @rsbivand can you, please, check latest binaries for arm64? (use https://mac.R-project.org as CRAN mirror) The latest recipes should have the drivers that were requested. If there is anything else you need, please supply details with libraries are needed.

[edzer]

Thanks, @s-u - I'm away from a Mac for a few days, but noticed that the new libraries for arm64 seem to fail to build sf: https://cran.r-project.org/web/checks/check_results_sf.html while succeeding on x86_64.

[s-u]

I think that is an old sync, I don't see any issue on my end:

> available.packages(rep="https://mac.R-project.org")["sf",1:2]
Package Version 
   "sf" "1.0-9" 

$ tail big-sur-arm64/results/4.2/sf.Rcheck/00check.log 
  ‘sf5.Rmd’ using ‘UTF-8’... [0s/0s] OK
  ‘sf6.Rmd’ using ‘UTF-8’... [0s/0s] OK
  ‘sf7.Rmd’ using ‘UTF-8’... [0s/0s] OK
* checking re-building of vignette outputs ... [15s/18s] OK
* checking PDF version of manual ... OK
* DONE
Status: 1 NOTE
* using check arguments '--no-clean-on-error '

* elapsed time (check, wall clock): 4:02

[edzer]

Libraries for arm work, as seen here, for x86_64 both platforms still fail.

[s-u]

@edzer Thanks for the confirmation. I have not updated the x86 build machines yet - will do now.

[edzer]

Looks like this now works for both r-release-macos flavors - thanks!

[s-u]

Thanks, @edzer, now that both arm64 and x86_64 are both on Big Sur I hope the issues will diminish. Also please let me know if/when you want me to update GDAL - I was holding back since there are often issues with the latest releases and package compatibility so I won't try to update to any non-patch version unless I get ok from you.

[rsbivand]

@s-u We have r-spatial/sf#2156 needing to write OpenFileGDB out, this was added to GDAL in 3.6. If you are ready (with cmake), GDAL is at 3.6.4 now (you are right, 3.6.0 was withdrawn and 3.6.1 didn't last long), and 3.7 is expected soon.