Skip to content
This repository has been archived by the owner on Nov 4, 2024. It is now read-only.

Commit

Permalink
Merge pull request #4599 from minvws/release/2.65.0
Browse files Browse the repository at this point in the history
  • Loading branch information
Jorrik-Klijnsma-Work authored Jan 6, 2023
2 parents 1185253 + 7383b11 commit d04d5ad
Show file tree
Hide file tree
Showing 15 changed files with 140 additions and 221 deletions.
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,15 @@ ARG ARG_NEXT_PUBLIC_COMMIT_ID
ARG ARG_NEXT_PUBLIC_PHASE="production"
ARG ARG_NEXT_PUBLIC_HOT_RELOAD_LOKALIZE=0
ARG ARG_API_URL
ARG ARG_NEXT_HOST_URL_STG

ENV NEXT_PUBLIC_SANITY_DATASET=$ARG_NEXT_PUBLIC_SANITY_DATASET
ENV NEXT_PUBLIC_SANITY_PROJECT_ID=$ARG_NEXT_PUBLIC_SANITY_PROJECT_ID
ENV NEXT_PUBLIC_COMMIT_ID=$ARG_NEXT_PUBLIC_COMMIT_ID
ENV NEXT_PUBLIC_PHASE=$ARG_NEXT_PUBLIC_PHASE
ENV NEXT_PUBLIC_HOT_RELOAD_LOKALIZE=ARG_NEXT_PUBLIC_HOT_RELOAD_LOKALIZE
ENV API_URL=$ARG_API_URL
ENV NEXT_HOST_URL_STG=$ARG_NEXT_HOST_URL_STG

# Layer that always gets executed
FROM builder
Expand Down
98 changes: 41 additions & 57 deletions docs/choropleth-topojson.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,19 @@ This section describes how to generate this data with the correct projection app

### Importing the source data

To create the data files we need, we will be using `cbsgebiedsindelingen_2022_v1.gpkg` as the
data source. This package can be downloaded from: [www.pdok.nl/downloads/-/article/cbs-gebiedsindelingen](https://www.pdok.nl/downloads/-/article/cbs-gebiedsindelingen). Download the XML file and find the link to the latest `*.gpkg` file in it. The URL looks like this: https://geodata.nationaalgeoregister.nl/cbsgebiedsindelingen/extract/cbsgebiedsindelingen_2022_v1.gpkg.
To create the data files we need, we will be using `cbsgebiedsindelingen_2023.gpkg` as the
data source. This package can be downloaded from: [www.pdok.nl/downloads/-/article/cbs-gebiedsindelingen](https://www.pdok.nl/downloads/-/article/cbs-gebiedsindelingen). Download the XML file and find the link to the latest `*.gpkg` file in it. The URL looks like this: https://geodata.nationaalgeoregister.nl/cbsgebiedsindelingen/extract/cbsgebiedsindelingen_2023_v1.gpkg.

ATTENTION: At the time of writing, the `cbsgebiedsindelingen_2022_v1.gpkg` file is the latest version.
> ⚠️ **ATTENTION**: At the time of writing, the `cbsgebiedsindelingen_2023.gpkg` file is the latest version.
It is of course very likely that by the time new data needs to be generated that a newer file is available.
Pay attention to download the very latest version.
Pay attention to download the very latest version. If there may be a case that the latest CBS data is still not available and some manual work needs to be done, use the current TopoJson file and proceed to [Generate TopoJson](#generate-topojson).

After downloading, import the package into QGIS (the easiest way of doing this is by simply dragging the package into
the main QGIS window) and select the following layers to be added:

- `cbs_gemeente_2022_gegeneraliseerd`
- `cbs_veiligheidsregio_2021_gegeneraliseerd` (2022 not available at the moment of this writing)
- `cbs_landsdeel_2022_gegeneraliseerd`

NOTE: The year indicator will differ when dealing with newer data. Find the layer with the most recent year.
- `gemeente_gegeneraliseerd_2023`
- `veiligheidsregio_gegeneraliseerd_2023`
- `landsdeel_gegeneraliseerd_2023`

Import with the CRS:`EPSG:28992 - Amersfoort / RD New - Projected` projection which should the default while importing.

Expand All @@ -40,7 +38,7 @@ Import with the CRS:`EPSG:28992 - Amersfoort / RD New - Projected` projection wh

To clean up the data we have to perform the following steps:

Create the municipalities data file (**cbs_gemeente_2022_gegeneraliseerd**):
Create the municipalities data file (**gemeente_gegeneraliseerd_2023**):

1. Right-click on the **layer > Properties > Fields**
2. Rename (click the pencil to active edit mode):
Expand All @@ -49,72 +47,70 @@ Create the municipalities data file (**cbs_gemeente_2022_gegeneraliseerd**):
4. Right-click on the **layer > Export > Save Features As..**
5. Use the following settings:
- Format: `GeoJSON`
- File Name: `cbs_gemeente_2022_gegeneraliseerd.geojson`
- CRS: `EPSG:28992 - Amersfoort / RD New - Projected`
- File Name: Click on the three dots next to the file name on Step 5 of creating data files and choose a directory and save it as following: `gemeente_gegeneraliseerd_2023.geojson`
- CRS: `Project CRS: EPSG:28992 - Amersfoort / RD New`
- Open **"Select fields to export..."**
- Select: `code`
- Deselect all and only select: `code`
6. Export by clicking **"Ok"**

Create the safety regions data file (**cbs_veiligheidsregio_2021_gegeneraliseerd**):
Create the safety regions data file (**veiligheidsregio_gegeneraliseerd_2023**):

1. Right-click on the **layer > Properties > Source fields**
1. Right-click on the **layer > Properties > fields**
2. Rename (click the pencil to active edit mode):
- `statcode` to `code`
3. **Apply > Ok**
4. Right-click on the **layer > Export > Save Features As..**
5. Use the following settings:
- Format: `GeoJSON`
- File Name: `cbs_veiligheidsregio_2021_gegeneraliseerd.geojson`
- CRS: `EPSG:28992 - Amersfoort / RD New - Projected`
- File Name: Click on the three dots next to the file name on Step 5 of creating data files and choose a directory and save it as following: `veiligheidsregio_gegeneraliseerd_2023.geojson`
- CRS: `Project CRS: EPSG:28992 - Amersfoort / RD New`
- Open **"Select fields to export..."**
- Select: `code`
- Deselect all and only select: `code`
6. Export by clicking **"Ok"**

Create the Netherlands data file (**cbs_landsdeel_2022_gegeneraliseerd**):
Create the Netherlands data file (**landsdeel_gegeneraliseerd_2023**):

1. Select the layer and go to **Vector > Geoprocessing Tools > Dissolve > Run**, this will merge the different areas
2. Select the new layer and right-click on the **layer > Export > Save Features As..**
3. Use the following settings:
- Format: `GeoJSON`
- File Name: `cbs_landsdeel_2022_gegeneraliseerd.geojson`
- CRS: `EPSG:28992 - Amersfoort / RD New - Projected`
- File Name: Click on the three dots next to the file name on Step 5 of creating data files and choose a directory and save it as following: `landsdeel_gegeneraliseerd_2023.geojson`
- CRS: `Project CRS: EPSG:28992 - Amersfoort / RD New`
- Open **"Select fields to export..."**
- Deselect all
4. Export by clicking **"Ok"**

To make sure the coordinate system is correct we have to convert the exported files to lat and lon coordinates:

1. Create a new project and add the three files:
- `cbs_landsdeel_2022_gegeneraliseerd.geojson`
- `cbs_veiligheidsregio_2021_gegeneraliseerd.geojson`
- `cbs_gemeente_2022_gegeneraliseerd.geojson`
- `landsdeel_gegeneraliseerd_2023.geojson`
- `veiligheidsregio_gegeneraliseerd_2023.geojson`
- `gemeente_gegeneraliseerd_2023.geojson`
2. For each layer:
- Select the new layer and right-click on the **layer > Export > Save Features As..**
- Format: `GeoJSON`
- File Name: `cbs_veiligheidsregio_2021_gegeneraliseerd_WGS84.geojson`
- CRS: `EPSG:4326 - WGS 84 - Geographic`
- File Name: `veiligheidsregio_gegeneraliseerd_2023_WGS84.geojson`
- CRS: `(Default CRS:) EPSG:4326 - WGS 84`
- Export by clicking **"Ok"**
* Make sure the output files contain longitude and latitude coordinates by opening the file and manually checking this, otherwise the next step will not work.
* Make sure the output files contain longitude and latitude coordinates by opening the file and manually checking this by hovering over the map and looking for `coordinate` on the bottom toolbar, otherwise the next step will not work.

### Generate TopoJson:

1. Upload the three different (WGS84) files to: [mapshaper.org](https://mapshaper.org)
2. Rename the layers:
- Change: `cbs_gemeente_2022_gegeneraliseerd` to `gm_features`
- Change: `cbs_landsdeel_2022_gegeneraliseerd` to `nl_features`
- Change: `cbs_veiligheidsregio_2021_gegeneraliseerd` to `vr_features`
3. Export to TopoJSON > `nl-vr-gm-high-detail.topo.json`;

> Note: It's possible that the map looks a little streched on MapShaper. This is most likely not an issue and it will display correctly once it's integrated in the Dashboard again.
> ⚠️ **ATTENTION**: To make custom changes on the TopoJson using mapshaper, please take a look at [this link](https://handsondataviz.org/mapshaper.html) for tips and tricks.
> It's possible that the map looks a little streched on MapShaper. This is most likely not an issue and it will display correctly once it's integrated in the Dashboard again.
Simplifying:
(NB: this needs to be separate step; when doing this in the step above the output will not be correct**!!**):
1. Upload the three different `*_WGS84.geojson` files to: [mapshaper.org](https://mapshaper.org)
2. Rename the layers by clicking on the name in the dropdown:
- Change: `gemeente_gegeneraliseerd_2023_WGS84` to `gm_features`
- Change: `landsdeel_gegeneraliseerd_2023_WGS84` to `nl_features`
- Change: `veiligheidsregio_gegeneraliseerd_2023_WGS84` to `vr_features`
3. Export to TopoJSON > `nl-vr-gm-high-detail.topo.json`;

1. Upload the output `nl-vr-gm-high-detail.topo.json` to a new instance of [mapshaper.org](https://mapshaper.org/)
2. Do for each layer:
- Select the layer and open the console; simplify using the following command: $ -simplify 27.5%
3. When all the layers are simplified export to TopoJSON > `nl-vr-gm.topo.json`
4. Add the new data file to the project at `packages/app/pages/api/topo-json`.
4. Add the new data file to the project at `packages/app/src/pages/api/topo-json`.
5. Update the file `topology.ts`, if necessary. (located at `packages/app/src/pages/api/choropleth`)

### Example of the topology.ts file
Expand All @@ -128,25 +124,13 @@ export type CodedGeoProperties = {
code: string;
};

export type CodedGeoJSON = FeatureCollection<
MultiPolygon | Polygon,
CodedGeoProperties
>;

export const nlGeo = topojson.feature(
nlTopology,
nlTopology.objects.nl_features
) as CodedGeoJSON;

export const vrGeo = topojson.feature(
nlTopology,
nlTopology.objects.vr_features
) as CodedGeoJSON;

export const gmGeo = topojson.feature(
nlTopology,
nlTopology.objects.gm_features
) as CodedGeoJSON;
export type CodedGeoJSON = FeatureCollection<MultiPolygon | Polygon, CodedGeoProperties>;

export const nlGeo = topojson.feature(nlTopology, nlTopology.objects.nl_features) as CodedGeoJSON;

export const vrGeo = topojson.feature(nlTopology, nlTopology.objects.vr_features) as CodedGeoJSON;

export const gmGeo = topojson.feature(nlTopology, nlTopology.objects.gm_features) as CodedGeoJSON;
```

Note that the map data is stored as TopoJson. This is because of the size optimizations that TopoJson provides,
Expand Down
3 changes: 2 additions & 1 deletion packages/app/.env.local.example
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ DISABLE_SITEMAP=true

NEXT_PUBLIC_SANITY_DATASET=development
NEXT_PUBLIC_SANITY_PROJECT_ID=5mog5ask
NEXT_HOST_URL_STG=
SANITY_API_TOKEN=
SANITY_PREVIEW_SECRET=

# set content of version string
NEXT_PUBLIC_COMMIT_ID=n/a
NEXT_PUBLIC_COMMIT_ID=n/a
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
"properties": {
"values": {
"type": "array",
"minItems": 1,
"maxItems": 2,
"items": {
"$ref": "#/definitions/value"
Expand Down
16 changes: 10 additions & 6 deletions packages/app/schema/gm_collection/__index.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,36 +29,40 @@
},
"hospital_nice": {
"type": "array",
"maxItems": 344,
"minItems": 342,
"maxItems": 342,
"items": {
"$ref": "hospital_nice.json"
}
},
"hospital_nice_choropleth": {
"type": "array",
"maxItems": 344,
"minItems": 342,
"maxItems": 342,
"items": {
"$ref": "hospital_nice.json"
}
},
"tested_overall": {
"type": "array",
"maxItems": 344,
"minItems": 342,
"maxItems": 342,
"items": {
"$ref": "tested_overall.json"
}
},
"sewer": {
"type": "array",
"maxItems": 344,
"minItems": 342,
"maxItems": 342,
"items": {
"$ref": "sewer.json"
}
},
"vaccine_coverage_per_age_group": {
"type": "array",
"minItems": 688,
"maxItems": 688,
"minItems": 684,
"maxItems": 684,
"items": {
"$ref": "vaccine_coverage_per_age_group.json"
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,24 +122,7 @@ export const vrBoundingBoxGmCodes: Record<string, string[]> = {
'GM1896',
'GM0193',
],
VR05: [
'GM0141',
'GM1859',
'GM0147',
'GM1774',
'GM0153',
'GM0158',
'GM0163',
'GM0164',
'GM1735',
'GM0168',
'GM0173',
'GM0175',
'GM1742',
'GM0183',
'GM1700',
'GM0189',
],
VR05: ['GM0141', 'GM1859', 'GM0147', 'GM1774', 'GM0153', 'GM0158', 'GM0163', 'GM0164', 'GM1735', 'GM0168', 'GM0173', 'GM0175', 'GM1742', 'GM0183', 'GM1700', 'GM0189'],
VR06: [
'GM0197',
'GM0141',
Expand Down Expand Up @@ -541,10 +524,9 @@ export const vrBoundingBoxGmCodes: Record<string, string[]> = {
'GM0482',
'GM0613',
'GM0489',
'GM0501',
'GM1992',
'GM0502',
'GM0503',
'GM0530',
'GM0531',
'GM0534',
'GM0537',
Expand All @@ -569,7 +551,6 @@ export const vrBoundingBoxGmCodes: Record<string, string[]> = {
'GM0627',
'GM0629',
'GM1783',
'GM0614',
'GM0637',
'GM0638',
'GM1892',
Expand Down Expand Up @@ -639,14 +620,13 @@ export const vrBoundingBoxGmCodes: Record<string, string[]> = {
'GM0613',
'GM0484',
'GM0489',
'GM0501',
'GM0502',
'GM0503',
'GM0777',
'GM1924',
'GM0513',
'GM1655',
'GM0530',
'GM1992',
'GM0531',
'GM1963',
'GM0542',
Expand All @@ -672,7 +652,6 @@ export const vrBoundingBoxGmCodes: Record<string, string[]> = {
'GM0627',
'GM0629',
'GM1783',
'GM0614',
'GM0637',
'GM0638',
'GM1892',
Expand All @@ -683,7 +662,7 @@ export const vrBoundingBoxGmCodes: Record<string, string[]> = {
'GM0613',
'GM1959',
'GM0489',
'GM0501',
'GM1992',
'GM0502',
'GM0503',
'GM0766',
Expand Down Expand Up @@ -725,25 +704,7 @@ export const vrBoundingBoxGmCodes: Record<string, string[]> = {
'GM1892',
'GM0642',
],
VR19: [
'GM0748',
'GM0654',
'GM1924',
'GM0664',
'GM0530',
'GM1963',
'GM0677',
'GM0678',
'GM0687',
'GM1695',
'GM0703',
'GM1676',
'GM1714',
'GM0715',
'GM0716',
'GM0717',
'GM0718',
],
VR19: ['GM0748', 'GM0654', 'GM1924', 'GM0664', 'GM1992', 'GM1963', 'GM0677', 'GM0678', 'GM0687', 'GM1695', 'GM0703', 'GM1676', 'GM1714', 'GM0715', 'GM0716', 'GM0717', 'GM0718'],
VR20: [
'GM0482',
'GM0613',
Expand All @@ -755,7 +716,6 @@ export const vrBoundingBoxGmCodes: Record<string, string[]> = {
'GM1728',
'GM0757',
'GM0758',
'GM0501',
'GM0502',
'GM0766',
'GM0505',
Expand All @@ -768,7 +728,7 @@ export const vrBoundingBoxGmCodes: Record<string, string[]> = {
'GM0512',
'GM1655',
'GM0523',
'GM0530',
'GM1992',
'GM0531',
'GM0797',
'GM0798',
Expand Down
Loading

0 comments on commit d04d5ad

Please sign in to comment.