[BUG] Error with specific shapefile

[juliohm]

Attached is a MWE with a specific shape file that is successfully loaded by GDAL: https://drive.google.com/drive/folders/1EerBvkuS8h3SX20nbqjxq__FhiUaL3yO?usp=sharing

The same file produces an error with Shapefile.jl:

ERROR: ArgumentError: invalid Array dimensions
Stacktrace:
  [1] Array
    @ ./boot.jl:448 [inlined]
  [2] Array
    @ ./baseext.jl:12 [inlined]
  [3] read(io::IOStream, #unused#::Type{Shapefile.Polygon})
    @ Shapefile ~/.julia/packages/Shapefile/kNT0Q/src/Shapefile.jl:223
  [4] read(io::IOStream, #unused#::Type{Shapefile.Handle})
    @ Shapefile ~/.julia/packages/Shapefile/kNT0Q/src/Shapefile.jl:351
  [5] #1
    @ ~/.julia/packages/Shapefile/kNT0Q/src/Shapefile.jl:139 [inlined]
  [6] open(f::Shapefile.var"#1#2", args::String; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base ./io.jl:330
  [7] open
    @ ./io.jl:328 [inlined]
  [8] Handle
    @ ~/.julia/packages/Shapefile/kNT0Q/src/Shapefile.jl:138 [inlined]
  [9] Shapefile.Table(path::String)
    @ Shapefile ~/.julia/packages/Shapefile/kNT0Q/src/table.jl:35
 [10] top-level scope
    @ REPL[5]:1

I am using this dataset for the first time, so the problem may be in the data itself. Appreciate if you can take a look.

[visr]

Hmm GDAL/shapelib has been dealing with bad shapefiles in the wild for a long time, and manages to read anything that somewhat resembles a shapefile. For this package we just read the spec and tried to implement that. So if GDAL can read it, it doesn't mean it is necessarily a bug.

If I export your file from QGIS (or do a gdal_translate), I get a shapefile that this package can read without problem, suggesting this may be the case here.

[juliohm]

You mean that GDAL does extra effort to read shape files that are not compliant with the spec? From your comment I understood that there is no bug in Shapefile.jl because it implements the spec, and that I should probably use GDAL or QGIS or any other software to clean the "messy" file before coming back to Shapefile.jl. Does that sound reasonable?

[visr]

Yes exactly, I think that is likely the case. Unfortunately there doesn't seem to be a shapefile validator that can tell us what's wrong with it.

[meggart]

I got a bit curious what it is that breaks the reading of the file, it looks like the shapefile does include some additional data between the records and since we currently do not use .shx files to retrieve the positions at which records start, the reading fails. #52 would implement a way to read the shapefile, given there is a valid index file, at least I could parse the file provided by @juliohm

[juliohm]

That is amazing. I was about to close the issue happy that we could at least convert the file with external tools but being able to load it directly is even better. Thank you @meggart for the quick fix!

What do you think @visr of the PR opened? Is it ok to merge and trigger a patch release?

[visr]

Closed by #52