Arrow read/write: add GDAL specific field metadata

[rouault]

CC @paleolimbot

[coveralls]

coverage: 67.852% (+0.004%) from 67.848% when pulling 6eeffa5 on rouault:arrow_field_metadata into c981407 on OSGeo:master.

[paleolimbot]

I think what you might want here is regular field metadata rather than an extension type? Regular field metadata might look like:

• Key: GDAL:OGR:alternative_name (with associated value)
• Key: GDAL:OGR:comment (with associated value)
• Key: GDAL:OGR:default (with associated value)
• Key: GDAL:OGR:subtype (with associated value)
• Key: GDAL:OGR:width (with associated value)

Rather than

• Key: ARROW:extension:name = gdal.field_metadata
• Key: ARROW:extension:metadata = {"alternative_name": ... }

An extension type might perhaps be better suited to a GDAL type that could not be represented losslessly by an Arrow type and had to be serialized to a binary column in some way and given an extension name + metadata.

I think what you are trying to represent here is true "field metadata": the Arrow types are accurate and the values can be interpreted safely according to the type of the array...there's just some extra information for consumers who want it.

Arrow C++ silently drops unknown extension types fairly aggressively but I don't think that behaviour is guaranteed and might be different in another implementation (I might be alone but I personally wish it would warn or error by default in those cases to prevent misinterpreting true extension types).

[rouault]

@paleolimbot Thanks for the hints!

[paleolimbot]

Looks great!