Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Versioning the uMap internal format #1635

Open
yohanboniface opened this issue Feb 20, 2024 · 2 comments
Open

Versioning the uMap internal format #1635

yohanboniface opened this issue Feb 20, 2024 · 2 comments
Milestone

Comments

@yohanboniface
Copy link
Member

uMap internally uses geojson to store data. While the structure is stable, the keys change from time to time. Some are added, some are removed, some are renamed, some may change their type.
So the uMap client does a lot of "duck typing" to remain backward compatible (is there a key X so let's use it, otherwise is there a Y then use that one, etc.).
To clean this kind of code, we discussed the opportunity to version the format, and have a migration layer able to pass from a version to another. So the client would always deal with the latest format, and pass it to some helper when it is given an old version.
This version would also be used for full umap backups.

Some notes:

  • we certainly cannot get totally rid of backward compatibility code in the JS client, give some key can be used in URL query string, so stored out of our control
  • while the uMap backup is a single full file, internally the data is split:
  • the umap full backup is a "collection of FeatureCollection", but this case is not handled by the GeoJSON spec AFAIK, so we "invented" a type: "umap", with a layers: [list of FeatureCollection] key; while trying to version our format, let's try to see if there is a more GeoJSON standard way of storing a list of FeatureCollection
@yohanboniface yohanboniface added this to the 3.x milestone Feb 20, 2024
@yohanboniface
Copy link
Member Author

cf #1335
cf #174

@yohanboniface
Copy link
Member Author

the umap full backup is a "collection of FeatureCollection", but this case is not handled by the GeoJSON spec AFAIK, so we "invented" a type: "umap", with a layers: [list of FeatureCollection] key; while trying to version our format, let's try to see if there is a more GeoJSON standard way of storing a list of FeatureCollection

@sgillies hey! :) To you have any advice on this specific point ?
Basically, here is how our "umap" export looks like:

{
    "type": "umap",
    "properties": {"some map": "metadata"},
    "geometry": {
        "type": "Point",
        "coordinates": ["the", "map", "default", "center"]
    },
    "layers": [
        {
            "type": "FeatureCollection",
            "_umap_options": {"some layer": "metadata"},
            "features": [{
                "properties": {"some": "props"},
                "geometry": {"some": "geom"}
            }]
        },
        {
            "type": "FeatureCollection",
            "_umap_options": {"some layer": "metadata"},
            "features": [{
                "properties": {"some": "props"},
                "geometry": {"some": "geom"}
            }]
        },
    ]
}

As you see, this mimics a sort of "Collection of collections" type as it could exist in GeoJSON, but it does not AFAIK.

My goal would be to maximise the portability of those data. Eg. someone wants to reimport uMap export in QGis, or the other way around, and retrieve as much as possible of the original structure: data are split into layers, maybe some styling are kept, etc.

Thanks in advance for your input on this :)

yohanboniface added a commit that referenced this issue Aug 27, 2024
The main goal is to clarify the use of options/settings/properties…

- `settings` should be reserved to Django
- `options` should be reserved to pure Leaflet context
- `properties` used in the geojson context, for user data

Main changes:
- `DataLayer.settings` is renamed to `DataLayer.metadata`
- `DataLayer.geojson` is renamed to `Datalayer.data`
- `DataLayer.metadata` are not saved anymore in the geojson data file
- `Map.settings` is renamed to `Map.metadata`
- `Map.metadata` is now a flat key/value object, not a geojson Feature anymore
- `Map.zoom` is now populated and reused

Note: U.Map is still inheriting from Leaflet Map, so it mixes
`options` and `metadata`.

This changes is very intrusive, it changes

- the DB schema
- the way we store data in the FS (now without metadata)
- the way we backup map data

fix #1636
prepares #1335
prepares #1635
prepares #175
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

1 participant