Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes recommended by Advisory Group for review by Steering Group #22

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
96aa859
Update data standards page to add Gemini/INSPIRE metadata standard
padajo Mar 21, 2024
afe8507
Merge pull request #18 from icebreakerone/main
padajo Mar 21, 2024
05e4434
Update README.md - to replace previous working version
padajo Mar 21, 2024
624e1dd
Merge branch 'icebreakerone:main' into working-add-metadata-standard
padajo Mar 21, 2024
066a323
Change from using GitHub to Gitbook
padajo Mar 21, 2024
712db06
Create schema folder
elucasaiimicom Apr 18, 2024
ddfe5d3
Delete docs/schema
elucasaiimicom Apr 18, 2024
615f498
Create README.md
elucasaiimicom Apr 18, 2024
fc43213
Update README.md
elucasaiimicom Apr 18, 2024
07d5cf4
Create drinking-water-quality.md
elucasaiimicom Apr 18, 2024
1c144fd
Create domestic-consumption.md
elucasaiimicom Apr 18, 2024
1ed9e38
Update domestic-consumption.md
elucasaiimicom Apr 18, 2024
8c12690
Update drinking-water-quality.md
elucasaiimicom Apr 18, 2024
95aef1c
Create nightflow-data
elucasaiimicom Apr 18, 2024
ace8640
Rename nightflow-data to nightflow-data.md
elucasaiimicom Apr 18, 2024
51f8350
Update drinking-water-quality.md
elucasaiimicom Apr 18, 2024
1ce9589
Create raw-water-storage-levels-reservoir-levels.md
elucasaiimicom Apr 18, 2024
aca1125
Update README.md
elucasaiimicom Apr 18, 2024
e3b2589
Rename raw-water-storage-levels-reservoir-levels.md to raw-water-stor…
elucasaiimicom Apr 18, 2024
d9300a6
Update README.md
elucasaiimicom Apr 18, 2024
9426687
Update README.md
elucasaiimicom Apr 18, 2024
0f3f990
Create README.md
elucasaiimicom Apr 18, 2024
45610c1
Delete docs/governance/data-formats/README.md
elucasaiimicom Apr 18, 2024
994c39c
Create README.md
elucasaiimicom Apr 18, 2024
4c5e148
Delete docs/data-formats/README.md
elucasaiimicom Apr 18, 2024
96c0bed
Merge pull request #17 from padajo/working-add-metadata-standard
elucasaiimicom Apr 18, 2024
a410e47
Merge pull request #19 from padajo/working-github-to-gitbook
elucasaiimicom Apr 18, 2024
38bc08c
Merge pull request #20 from elucasaiimicom/working
padajo Apr 18, 2024
cb45185
Merge pull request #21 from icebreakerone/working
elucasaiimicom May 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 25 additions & 8 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,33 @@
# Stream data standards documentation

This repository contains the in progress documentation for the Stream data standards.
This directory contains:
* The current data standards for the Stream programme: [Stream Data Standards](data-standards.md)
* The current data standards glossary for the Stream programme: [Stream Data Standards Glossary](data-standards-glossary.md)
* The Licence under which the data standards are published: [Licence for the Stream data standards documentation](license.md)
* Governance documentation directory: [Governance Documentation](#governance)

It is currently being defined and developed through the Data Standards Workstream within the Stream Programme.
It has been developed as part of the Stream Data Standards Workstream within the Stream Programme.

The data standards and documentation have been developed by the Stream Data Standards Workstream and approved by the Stream Steering Group. The standards and governance information is published here for use by the Stream Programme, the partners, and data consumers.

For more information on data standards contained in this repository please contact [[email protected]](mailto:[email protected]).

## Contents

* [Problem Statement](problem-statement.md)
* [Document Repository](document-repository.md)
* [License for the Stream data standards documentation](license.md)
* [Approval process for Stream data standards](approval-process.md)
* [Updating process for Stream data standards](update-process.md)
* [Base data standards for Stream](base-standards-used-in-Stream.md)
* Data Standards
* [Stream Data Standards](data-standards.md)
* [Stream Data Standards Glossary](data-standards-glossary.md)
* Licensing
* [Licence for the Stream data standards documentation](license.md)

## Governance

The governance of the Stream data standards is managed by the Stream Data Standards Workstream. The governance documentation is contained in the following files:

* Governance Documentation
* [Problem Statement](governance/problem-statement.md)
* [Document Repository](governance/document-repository.md)
* [Approval process for Stream data standards](governance/approval-process.md)
* [Updating process for Stream data standards](governance/update-process.md)
* [Base data standards for Stream](governance/base-standards-used-in-Stream.md)
* [Implementaiton of data standards for Stream](governance/implementation-of-standards.md)
4 changes: 4 additions & 0 deletions docs/data-standards.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ The governance for Stream data standards is managed by the Stream Data Standards

## Data Standards

## Metadata standard

The standard for Metadata is the [Gemini 2.3 standard](https://knowledge-base.inspire.ec.europa.eu/index_en) which is based on the [INSPIRE Metadata Directive](https://knowledge-base.inspire.ec.europa.eu/index_en). The ESRI platform can displaying Gemini 2.3 using the INSPIRE metadata standard built into the ESRI platform. To ensure that metadata using the INSPIRE standard is Gemini 2.3 compliant, please refer to this [Gemini 2.3 metedata guidance on the ArcGIS website](https://govportal.maps.arcgis.com/home/item.html?id=ad26f71f42e24d1eaf7fb3e347a049a9).

### CSV datasets

The standard for CSV format datasets used for Stream data is described in [RFC 4180](https://datatracker.ietf.org/doc/html/rfc4180).
Expand Down
25 changes: 25 additions & 0 deletions docs/schema/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Stream data schema documentation

This directory contains:
* The current common schemas for the Stream programme

## Contents

* Schema
* [Domestic Consumption](domestic-consumption.md)
* [Drinking Water Quality](drinking-water-quality.md)
* [Nightflow Data](nightflow-data.md)
* [Reservoir Levels](raw-water-storage-reservoir-levels.md)


## Governance

The governance of the Stream data schema is managed by the Stream Data Standards Workstream. The governance documentation is contained in the following files:

* Governance Documentation
* [Problem Statement](../governance/problem-statement.md)
* [Document Repository](../governance/document-repository.md)
* [Approval process for Stream data standards](../governance/approval-process.md)
* [Updating process for Stream data standards](../governance/update-process.md)
* [Base data standards for Stream](../governance/base-standards-used-in-Stream.md)
* [Implementaiton of data standards for Stream](../governance/implementation-of-standards.md)
11 changes: 11 additions & 0 deletions docs/schema/domestic-consumption.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Domestic Consumption

| ATTRIBUTE | DATA TYPE    | DESCRIPTION    |
| :--- | :---- | :---- |
| DATA_SOURCE  | String | Company that provided the data |
| YEAR | Integer | The calendar year covered by the data |
| LSOA_CODE | String | LSOA or Data Zone converted code of the meter location |
| NUMBER_OF_METERS  | Integer | Number of meters within the LSOA |
| TOTAL_CONSUMPTION | Decimal | Total consumption within the LSOA  |
| TOTAL_CONSUMPTION_UNITS | String | Units for consumption |

15 changes: 15 additions & 0 deletions docs/schema/drinking-water-quality.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Drinking Water Quality


| ATTRIBUTE  | DATA TYPE  | DESCRIPTION |
| :--- | :---- | :---- |
| DATA SOURCE | String | The company that provided the data |
| SAMPLE_ID | String  | The identity of the sample |
| SAMPLE_DATE  | Date   | The date the sample was taken|
| DETERMINAND  | String | The determinand being measured  |
| DWI_CODE | String | The corresponding DWI code for the determinand|
| UNITS  | String | The expression of results|
| OPERATOR | String | The measurement operator for limit of detection  |
| RESULT  | Decimal | The test results |
| LSOA or DATA ZONE | String | Lower Layer Super Output Area (population weighted centroids for England and Wales) or Data Zone (population weighted centroids for Scotland) |

26 changes: 26 additions & 0 deletions docs/schema/nightflow-data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Nightflow Data


| ATTRIBUTE | DATA TYPE | DESCRIPTION |
| :--- | :---- | :---- |
| DATA_SOURCE   | String | Company that owns the DMA |
| DATE/TIME_STAMP | Date Time UTC | Date and time of measured net flow |
| DMA_ID | String | Identity of the district metered area |
| CENTROID_X | Decimal | DMA centroid X coordinate |
| CENTROID_Y | Decimal | DMA centroid Y coordinate |
| ACTUAL_MIN_NIGHT_FLOW | Float | The lowest recorded average net flow between 12 and 6am |
| MIN_NIGHT_FLOW | Float | The average flow within the 3-4 am time window |
| UNITS | String | Measurement of flow |













15 changes: 15 additions & 0 deletions docs/schema/raw-water-storage-reservoir-levels.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Raw Water Storage Levels: Reservoir Levels

| ATTRIBUTE | DATA TYPE | DESCRIPTION |
| :--- | :---- | :---- |
| DATA SOURCE | String | The company that provided the data |
|RESERVOIR_ID|String|Reservoir ID given by the water company|
|RESERVOIR_NAME|String|Name of the reservoir |
|DATE|Date|Date of measured water storage level|
|LATITUDE|Decimal|Latitude coordinates of the reservoir |
|LONGITUDE|Decimal|Longitude coordinates of the reservoir|
|CAPACITY|Decimal|Reservoir capacity|
|CAPACITY_UNITS|String|Reservoir capacity units |
|CURRENT_LEVEL|Decimal|Measurement of raw water level|
|CURRENT_LEVEL_UNITS|String|Units of measured raw water level|
|CURRENT_PERCENTAGE|Decimal|Percentage of capacity that is filled with raw water|
6 changes: 6 additions & 0 deletions docs/suggesting-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,9 @@ The [Stream data standards](data-standards.md) are a living document and are exp
3. Raise a Pull Request (PR) in this repository with the changes you have made.

The PR will be reviewed by the Stream Advisory Group responsible. If the PR is accepted, it will be included in the next update of the Stream data standards put forward for approval.

<!---
A request has been made to consider GitBook instead of GitHub for the editing feature.

Adding in a comment to provide context for discussion via Pull Request, without changing the display
--->