Fix UnicodeDecodeError with shapefiles for plot and plot3d

[seisman]

Description of proposed changes

Fixes #1616

Reminders

• Run make format and make check to make sure the code follows the style guide.
• Add tests for new features or tests that would have caught the bug that you're fixing.
• Add new public functions/methods/classes to doc/api/index.rst.
• Write detailed docstrings for all functions/methods.
• If wrapping a new module, open a 'Wrap new GMT module' issue and submit reasonably-sized PRs.
• If adding new functionality, add an example to docstrings or tutorials.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash commands are:

• /format: automatically format and lint the code
• /test-gmt-dev: run full tests on the latest GMT development version

[seisman]

Do I need to add a test for #1616? If yes, is there a shapefile that I can use?

[maxrjones]

Do I need to add a test for #1616? If yes, is there a shapefile that I can use?

I think it would be good to have either a test or a gallery example.

One option would be to use the same data as the roads gallery example with gdf.to_file. Another option would be to convert RidgeTest.gmt to a shapefile (possibly we could cache the shapefile to speed up the new PyGMT test and https://github.com/GenericMappingTools/gmt/blob/master/test/psxy/categorical.sh):

test_data=`gmt which -Gl @RidgeTest.gmt`
ogr2ogr -f "ESRI Shapefile" RidgeTest.shp $test_data

[seisman]

One option would be to use the same data as the roads gallery example with gdf.to_file. Another option would be to convert RidgeTest.gmt to a shapefile (possibly we could cache the shapefile to speed up the new PyGMT test and GenericMappingTools/gmt@master/test/psxy/categorical.sh):

test_data=`gmt which -Gl @RidgeTest.gmt`
ogr2ogr -f "ESRI Shapefile" RidgeTest.shp $test_data

Sounds a good idea.

[seisman]

One option would be to use the same data as the roads gallery example with gdf.to_file. Another option would be to convert RidgeTest.gmt to a shapefile (possibly we could cache the shapefile to speed up the new PyGMT test and GenericMappingTools/gmt@master/test/psxy/categorical.sh):

test_data=`gmt which -Gl @RidgeTest.gmt`
ogr2ogr -f "ESRI Shapefile" RidgeTest.shp $test_data

Sounds a good idea.

Unfortunately it won't work, because the ogr2ogr command produces four files (.shp, .shx, .dbf and .prj), and the GMT cache mechanism cannot download 4 files using @RidgeTest.shp.

[weiji14]

One option would be to use the same data as the roads gallery example with gdf.to_file. Another option would be to convert RidgeTest.gmt to a shapefile (possibly we could cache the shapefile to speed up the new PyGMT test and GenericMappingTools/gmt@master/test/psxy/categorical.sh):

test_data=`gmt which -Gl @RidgeTest.gmt`
ogr2ogr -f "ESRI Shapefile" RidgeTest.shp $test_data

Sounds a good idea.

Unfortunately it won't work, because the ogr2ogr command produces four files (.shp, .shx, .dbf and .prj), and the GMT cache mechanism cannot download 4 files using @RidgeTest.shp.

You could do gmt which -Gl @RidgeTest.shp @RidgeTest.shx @RidgeTest.dbf @RidgeTest.prj 🙂

[weiji14]

There's also some sample shapefile data at https://github.com/intake/intake_geopandas/tree/0.4.0/tests/data/stations you can just download and use.

[willschlitzer]

One option would be to use the same data as the roads gallery example with gdf.to_file. Another option would be to convert RidgeTest.gmt to a shapefile (possibly we could cache the shapefile to speed up the new PyGMT test and GenericMappingTools/gmt@master/test/psxy/categorical.sh):

test_data=`gmt which -Gl @RidgeTest.gmt`
ogr2ogr -f "ESRI Shapefile" RidgeTest.shp $test_data

Sounds a good idea.

Unfortunately it won't work, because the ogr2ogr command produces four files (.shp, .shx, .dbf and .prj), and the GMT cache mechanism cannot download 4 files using @RidgeTest.shp.

Could the .shp file be added to pygmt/tests/data as a workaround to have a cached file?

[github-actions]

Summary of changed images

This is an auto-generated report of images that have changed on the DVC remote

Status	Path
added	pygmt/tests/baseline/test_plot_shapefile.png

Image diff(s)

Added images

• pygmt/tests/baseline/test_plot_shapefile.png

Modified images

Path	Old	New

Report last updated at commit 3712edc

[weiji14]

Still not sure if storing the shapefile (shp/shx/dbf/prj) in the git history is a good idea. I'm thinking whether we should put it on dvc, or maybe add some custom code to pygmt.helpers.testing.download_test_data() to download the shapefile using urllib or plain pygmt.which.

from pygmt import which

for ext in ["shp", "shx", "dbf", "prj"]:
    which(
        fname=f"https://github.com/intake/intake_geopandas/raw/0.4.0/tests/data/stations/stations.{ext}",
        download=True,
    )

[seisman]

Another option would be to convert RidgeTest.gmt to a shapefile (possibly we could cache the shapefile to speed up the new PyGMT test and https://github.com/GenericMappingTools/gmt/blob/master/test/psxy/categorical.sh):

test_data=`gmt which -Gl @RidgeTest.gmt`
ogr2ogr -f "ESRI Shapefile" RidgeTest.shp $test_data

I'm considering the option from @meghanrjones again. I think it makes more sense to cache the shapefile in the GMT server, because they can also be used by GMT and GMT.jl.

[seisman]

Opened a PR in GenericMappingTools/gmtserver-admin#142.

[weiji14]

Thanks @seisman! Looks nice and clean now.

[seisman]

/format

[seisman]

It seems the new test crashes.

[maxrjones]

It seems the new test crashes.

I do not see the new RidgeTest.shp files in the data downloaded from the cache. You may need to rerun the cache data workflow.

[seisman]

It seems the new test crashes.

I do not see the new RidgeTest.shp files in the data downloaded from the cache. You may need to rerun the cache data workflow.

You're right, there is a scheduled run of the cache data workflow yesterday.