New MiraMonVector driver #9688
New MiraMonVector driver #9688
Conversation
rouault
force-pushed
the
miramon_clean_history
branch
from
21b7707 to
5443cfe
Compare
|
I've worked together with @AbelPau over the past weeks to put the driver into shape. You can find the details of the review in AbelPau#12 . The branch in its state of ~ one week ago was submitted to coverity-scan and issues were fixed .I've also run it through ossfuzz locally on my PC with the undefined and address santizers, on the new miramon dedicated fuzzer, and it now runs for a couple days without issues. So from my perspective, we are in a state acceptable for upstreaming. |
rouault
force-pushed
the
miramon_clean_history
branch
from
5443cfe to
fad106a
Compare
|
|
||
| In MiraMon the concepts of OGRMultiPoints and OGRMultiLineStrings are not supported, | ||
| but the driver translates a multipoint into N points and a multistring into N arcs. | ||
| So, when reading a MiraMon file of type *.pol*, the corresponding |
There was a problem hiding this comment.
This is a little confusing to me. How does the handling of multipolygons relate to the limitation on multipoint/linestrings?
There was a problem hiding this comment.
A multipolygon has a list of stringline indices (in the *.pol file) and when the vertices of a multipolygon (or a polygon) are dumped the coordinates are extracted from the corresponding *.arc file using that list of indices (and other details). You can consult how it's organized a multipolygon (or a polygon) more precisely here: https://www.miramon.cat/new_note/eng/notes/[MiraMon_structured_vectors_file_format.pdf](https://www.miramon.cat/new_note/eng/notes/MiraMon_structured_vectors_file_format.pdf)
But the concept of having a a group of connected lines it's not present in MiraMon, for the moment.
Co-authored-by: Nyall Dawson <[email protected]>
rouault
force-pushed
the
miramon_clean_history
branch
from
f30137f to
4b71cd2
Compare
…eaded in reverse mode
|
was there a particular reason to disable that driver on little-endian ? this makes packaging on OpenBSD/powerpc64 fail because share/gdal/MM_m_idofic.csv isn't installed. I can work that around by disabling the driver but just wondering.. |
…ndian the driver was added in OSGeo/gdal#9688 but is disabled on !BE, thus MM_m_idofic.csv isn't installed and packaging fails on at least powerpc64
You meant on big-endian. if I remember well, it would build fine, but it lacks appropriate byte-swapping, causing read errors on reading, and generating invalid files on writing on big-endian hosts, so it is a better service for potential users of those platform to just disable it. |
|
Hi,
this was the reason that we discussed when the process of creating the driver.
There is any I could do to improve the driver in this aspect?
+}
+
+int MM_WriteNRecordsMMBD_XPFile(struct MMAdmDatabase *MMAdmDB)
+{
+ GUInt32 nRecords;
+ if (!MMAdmDB->pMMBDXP || !MMAdmDB->pFExtDBF)
+ return 0;
+
+ // Updating number of features in features table
+ fseek_function(MMAdmDB->pFExtDBF, MM_FIRST_OFFSET_to_N_RECORDS, SEEK_SET);
+
+ if (MMAdmDB->pMMBDXP->nRecords > UINT32_MAX)
+ {
+ MMAdmDB->pMMBDXP->dbf_version = MM_MARCA_VERSIO_1_DBF_ESTESA;
+
+ if (fwrite_function(&MMAdmDB->pMMBDXP->nRecords, 4, 1,
ok, I see you are writing directly integers to a file without taking into account potential big-endian order. Making the driver compliant with a big-endian host is likely to be a significant and painful amount of work, so the best course of action would be to not build the driver on big endian hosts. Otherwise you'll get test errors when the pull request will be done against OSGeo/gdal which has 2 extra CI configuration using Travis-CI, one including IBM s390x which is big-endian
So apply the following change to ogr/ogrsf_frmts/CMakeLists.txt:
diff --git a/ogr/ogrsf_frmts/CMakeLists.txt b/ogr/ogrsf_frmts/CMakeLists.txt
index cd608d1f53..f11285935d 100644
…--- a/ogr/ogrsf_frmts/CMakeLists.txt
+++ b/ogr/ogrsf_frmts/CMakeLists.txt
@@ -47,7 +47,9 @@ ogr_optional_driver(vdv "VDV-451/VDV-452/INTREST Data Format")
ogr_optional_driver(flatgeobuf FlatGeobuf)
ogr_optional_driver(mapml MapML)
ogr_optional_driver(jsonfg JSONFG)
-ogr_optional_driver(miramon "MiraMonVector")
+if( NOT WORDS_BIGENDIAN )
+ ogr_optional_driver(miramon "MiraMonVector")
+endif()
De: Even Rouault ***@***.***>
Enviado el: dissabte, 16 de novembre de 2024 12:49
Para: OSGeo/gdal ***@***.***>
CC: Abel Pau ***@***.***>; Mention ***@***.***>
Asunto: Re: [OSGeo/gdal] New MiraMonVector driver (PR #9688)
was there a particular reason to disable that driver on little-endian ?
You meant on big-endian. if I remember well, it would build fine, but it lacks appropriate byte-swapping, causing read errors on reading, and generating invalid files on writing on big-endian hosts, so it is a better service for potential users of those platform to just disable it.
—
Reply to this email directly, view it on GitHub<#9688 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AWDNBTAMG6JD6N4PKIWMQVT2A4WLBAVCNFSM6AAAAABR4UC3V2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBQGUZTIMZXGY>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
GDAL has a number of CPL_LSBxxxx macros in cpl_port.h to do conditional byte-swapping . The issue is that Travis-CI is now defunct, so we no longer have any big-endian CI configuration to check our compatiblity with big-endian hosts. So to test, you'd likely have to use locally a Docker image of a Linux distribution (for example ubuntu, but also works with s390x/alpine:edge), with qemu-based emulation (which will be painfully slow, including compiling. Expect 10 times slower than bare-metal execution): $ docker pull --platform linux/s390x s390x/ubuntu:24.04 But honestly, that would be a very bad use of your time. To the best of my knowledge, all big-endian hosts (powerpc or IBM s390x or some ARM chips) are now super legacy architectures that almost nobody actually use (or if they still use them, that should only be to run their legacy apps). I'm still wondering why some open source Unix distributions lose their time (and ours!) trying to still support them. |
I have to agree with that statement, it was just out of curiosity. Since the shipped files list changes, it failed to package on powerpc64, so i tried to understand why.
powerpc64 which isn't that legacy at all.. but it falls in the 'exotic' category :) |
|
OK, thanks for your explanations. So, if anybody really needs this feature we'll let it as it is. |
What does this PR do?
This driver allows to create and open MiraMon polygon, stringline and point vectors.
What are related issues/pull requests?
There are not related issues or pull requests.
Tasklist
Environment
Provide environment details, if relevant: