Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broken links #259

Merged
merged 21 commits into from
Aug 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/workflows/md-link-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: Check Markdown links

on:
pull_request:
push:
branches:
- main

jobs:
markdown-link-check:
runs-on: ubuntu-latest

steps:
- name: checkout
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Check gh-pages
if: github.ref == 'refs/heads/gh-pages'
uses: gaurav-nelson/github-action-markdown-link-check@v1
with:
use-quiet-mode: 'yes'
use-verbose-mode: 'yes'
config-file: 'mlc_config.json'
file-path: './README.md'
folder-path: '_docs/'

- name: Check main
uses: gaurav-nelson/github-action-markdown-link-check@v1
with:
use-quiet-mode: 'yes'
use-verbose-mode: 'yes'
config-file: 'mlc_config.json'
file-path: './README.md'
31 changes: 31 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
# - id: trailing-whitespace
# - id: end-of-file-fixer
- id: check-added-large-files

- repo: https://github.com/codespell-project/codespell
rev: v2.2.5
hooks:
- id: codespell
exclude: >
(?x)^(
.*\.yaml|
.*\.yml|
)$
args:
- --ignore-words-list=nd,ot,adn,redict


ci:
autofix_commit_msg: |
[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci
autofix_prs: false
autoupdate_commit_msg: '[pre-commit.ci] pre-commit autoupdate'
autoupdate_schedule: monthly
skip: []
submodules: false
30 changes: 0 additions & 30 deletions _docs/data-workflow.md

This file was deleted.

7 changes: 1 addition & 6 deletions _docs/faqs.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,9 @@ Either way works.

## Can I submit variables other than the core CTD variables?

Yes! The DAC now accepts any science variables that have a valid [CF Standard Name](http://cfconventions.org/standard-names.html). Any ancillary variables (as specified in the variable attribute `ancillary_variables`) will also be ingested into ERDDAP.
Yes! The DAC now accepts any science variables that have a valid [CF Standard Name](https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html). Any ancillary variables (as specified in the variable attribute `ancillary_variables`) will also be ingested into ERDDAP.


## What is the current procedure for sending data to the Global Telecommunications System (GTS)?

The National Data Buoy Center harvests new profile observations from the [DAC's ERDDAP server](https://gliders.ioos.us/erddap/index.html) once per hour, encodes the profiles in to a modified drifting buoy BUFR format and releases the messages to the GTS. The development of a glider specific BUFR format is currently being developed and finalized.


<!-- 2. Can I submit raw glider files to the DAC? -->

<!-- At the moment no, you must convert the raw files to netCDF following the DAC [metadata conventions](/ioosngdac/ngdac-netcdf-file-format-version-2.html). But we're in the process of developing a raw data upload tool that will enable users to -->
18 changes: 4 additions & 14 deletions _docs/glider-background-and-sampling-terminology.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,30 +8,20 @@ toc: false
#permalink: index.html
summary: This page defines glider and sampling terminology used throughout the rest of the Wiki.
---
<!--
> [Wiki](https://github.com/kerfoot/ioosngdac/wiki) ▸ **Glider Background and Sampling Terminology**

## Contents

+ [Glider Types](#glider-types)
+ [Sampling Pattern Terminology](#sampling-pattern-terminology)
+ [NetCDF File Format Description](ngdac-netcdf-file-format-version-2.html)
-->

## Glider Types
As of this writing, there are 3 major buoyancy driven glider types that are currently utilized by IOOS Regional Associations:
+ [Seaglider](http://www.apl.washington.edu/project/project.php?id=seaglider#:~:text=The%20AUV%20Seaglider%20is%20the,time%20via%20satellite%20data%20telemetry.): originally designed and built through a collaboration with the University of Washington's [Applied Physics Lab](http://www.apl.washington.edu/) and [School of Oceanography](http://www.ocean.washington.edu/). The Seaglider is now manufactured by [Hydroid](https://www.hydroid.com/seaglider)
+ [Spray](http://spray.ucsd.edu/pub/rel/info/spray_description.php): originally designed by [Scripps Institution of Oceanography](https://scripps.ucsd.edu/) and [Woods Hole Oceangraphic Institution](http://www.whoi.edu/) with funding provided by the [Office of Naval Research](http://www.onr.navy.mil/), the Spray glider is now manufactured by [Bluefin Robotics](http://www.bluefinrobotics.com/products/spray-glider/).
+ [Slocum](http://www.webbresearch.com/slocumglider.aspx): designed and built by [Teledyne Webb Research Corporation](http://www.webbresearch.com/).
+ [Seaglider](https://apl.uw.edu/project/project.php?id=seaglider): originally designed and built through a collaboration with the University of Washington's [Applied Physics Lab](https://apl.uw.edu/) and [School of Oceanography](https://www.ocean.washington.edu/).
+ [Spray](https://spray.ucsd.edu/pub/rel/info/spray_description.php): originally designed by [Scripps Institution of Oceanography](https://scripps.ucsd.edu/) and [Woods Hole Oceanographic Institution](http://www.whoi.edu/) with funding provided by the [Office of Naval Research](https://www.nre.navy.mil/), the Spray glider is now manufactured by [Bluefin Robotics](https://gdmissionsystems.com/underwater-vehicles/bluefin-robotics/).
+ [Slocum](https://www.teledynemarine.com/en-us/products/product-line/Pages/Autonomous-Underwater-Glider.aspx): designed and built by [Teledyne Webb Research Corporation](https://www.teledynemarine.com/brands/webb-research/).

## Sampling Pattern Terminology

The schematic and definitions below define the sampling terminology of a profiling glider. While all of the terms defined below are commonly used in the glider community, the 2 fundamental terms used by the **NGDAC** to organize data are the **profile** and **trajectory**. The **NGDAC** receives glider data as individual, sequentially numbered **profiles** and aggregates files from the same **trajectory** into a single data set representing the deployment.

<!-- ![Glider Sampling Patterns and Terms](https://raw.githubusercontent.com/kerfoot/ioosngdac/master/doco/glider-sampling-terminology.png) -->
![Glider Sampling Patterns and Terms](/glider-dac/glider-sampling-terminology.png)

+ **Profile**: A single vertically oriented track of a glider, either upward or downward through the water column. A profile is one-half of a **dive**. The profile is the fundamental atomic data type used by the **NGDAC**. All data submitted to the **NGDAC** is submitted as individual profiles, containing the various water column properties or sensor values. Examples of the file format description can be found as [CDL](https://github.com/kerfoot/ioosngdac/blob/master/nc/template/IOOS_Glider_NetCDF_v2.0.cdl), [NetCDF file](https://github.com/kerfoot/ioosngdac/blob/master/nc/template/IOOS_Glider_NetCDF_v2.0.nc) and [ncml](https://github.com/kerfoot/ioosngdac/blob/master/nc/template/IOOS_Glider_NetCDF_v2.0.ncml) can be found [here](https://github.com/kerfoot/ioosngdac/tree/master/nc/template).
+ **Profile**: A single vertically oriented track of a glider, either upward or downward through the water column. A profile is one-half of a **dive**. The profile is the fundamental atomic data type used by the **NGDAC**. All data submitted to the **NGDAC** is submitted as individual profiles, containing the various water column properties or sensor values. Examples of the file format description can be found as [CDL](/glider-dac/nc/template/IOOS_Glider_NetCDF_v2.0.cdl), [NetCDF file](/glider-dac/nc/template/IOOS_Glider_NetCDF_v2.0.nc) and [ncml](/glider-dac/nc/template/IOOS_Glider_NetCDF_v2.0.ncml) can be found [here](https://github.com/ioos/glider-dac/tree/gh-pages/nc/template).
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll probably have to iterate on this relative link. I'm not sure how Jekyll behaves on local vs remote builds. (It has been a while that I used it.) Regardless, after this one is merged, I should probably change these to full links instead of relative ones.

+ **Dive**: A single vertical <strong>profile</strong> to depth followed by a vertical **profile** towards the surface. A dive does not necessarily begin with or terminate with a surfacing and/or gps fix.
+ **Segment**: The set of data collected between 2 gps fixes obtained while the glider is on the surface of the water. The first gps fix is acquired prior to the beginning of a dive and the second gps fix is acquired following the completion of at least one dive. Glider **segments** always consist of at least one, and possibly many **dives**.
+ **Trajectory** or **Deployment**: A series of one or more **segments** completed by a glider between the time of deployment and the time of recovery.
Expand Down
24 changes: 5 additions & 19 deletions _docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,34 +9,20 @@ toc: false
summary: This wiki is a collection of documents and resources describing the NetCDF file specification, data provider registration and data set submission processes for contributing real-time and delayed-mode glider data sets to the U.S. IOOS National Glider Data Assembly Center (NGDAC).
---

<!--
This wiki is a collection of documents and resources describing the NetCDF file specification, data provider registration and data set submission processes for contributing real-time and delayed-mode glider data sets to the U.S. IOOS <b>N</b>ational <b>G</b>lider <b>D</b>ata <b>A</b>ssembly <b>C</b>enter (__NGDAC__).

## Wiki Contents

+ [Introduction](#introduction)
+ [Glider Background and Sampling Terminology](/ioosngdac/glider-background-and-sampling-terminology.html)
+ [NetCDF file format description](ngdac-netcdf-file-format-version-2.html)
+ [Links for Data Providers](/ioosngdac/useful-links.html)
+ [NGDAC Architecture](/ioosngdac/ngdac-architecture.html)
+ [NGDAC NetCDF File Submission Process](/ioosngdac/ngdac-netcdf-file-submission-process.html)
+ [Backup and Recovery](/ioosngdac/data-backup-recovery)
-->

## Introduction

The goals of the <b>U.S. IOOS National Glider Data Assembly Center</b>:

+ Develop a simple, fully self-describing [NetCDF](http://en.wikipedia.org/wiki/NetCDF) file specification that preserves the resolution of the original glider data sets.
+ Develop a simple, fully self-describing [NetCDF](https://docs.unidata.ucar.edu/netcdf-c/current/) file specification that preserves the resolution of the original glider data sets.
+ Provide glider operators with a simple process for registering and submitting glider data sets to a centralized storage location.
+ Provide public access to glider data sets via existing web services and standards, in a variety of well-known formats.
+ Facilitate the distribution of glider data sets on the [Global Telecommunication System](http://www.wmo.int/pages/prog/www/TEM/GTS/index_en.html).
+ Facilitate the distribution of glider data sets on the [Global Telecommunication System](https://community.wmo.int/en/activity-areas/global-telecommunication-system-gts).
+ Work with the [National Centers for Environmental Information](https://www.ncei.noaa.gov/) to create a permanent data archive.

The **NGDAC** accepts a [simple NetCDF file](/ioosngdac/ngdac-netcdf-file-format-version-2.html) containing water column measurements collected by a glider during a single profile (see glider sampling terminology [here](glider-background-and-sampling-terminology.html#sampling-pattern-terminology)). Groups of these NetCDF files, gathered during a deployment (also known as a [**trajectory**](glider-background-and-sampling-terminology.html#sampling-pattern-terminology)), are uploaded to the **NGDAC** by individual glider operators. Once they arrive at the **NGDAC**, the files are validated for compliance, aggregated into a single dataset representing the **deployment/trajectory** and distributed via [ERDDAP](http://coastwatch.pfeg.noaa.gov/erddap/information.html) and [THREDDS](http://www.unidata.ucar.edu/software/thredds/current/tds/TDS.html) end-points. The data sets served by the **NGDAC** provide access to the **trajectory/deployment** data both as time-series and on a profile-by-profile basis.
The **NGDAC** accepts a [simple NetCDF file](/glider-dac/ngdac-netcdf-file-format-version-2.html) containing water column measurements collected by a glider during a single profile (see glider sampling terminology [here](glider-background-and-sampling-terminology.html#sampling-pattern-terminology)). Groups of these NetCDF files, gathered during a deployment (also known as a [**trajectory**](glider-background-and-sampling-terminology.html#sampling-pattern-terminology)), are uploaded to the **NGDAC** by individual glider operators. Once they arrive at the **NGDAC**, the files are validated for compliance, aggregated into a single dataset representing the **deployment/trajectory** and distributed via [ERDDAP](https://coastwatch.pfeg.noaa.gov/erddap/information.html) and [THREDDS](https://www.unidata.ucar.edu/software/tds/) end-points. The data sets served by the **NGDAC** provide access to the **trajectory/deployment** data both as time-series and on a profile-by-profile basis.

Transmission of the data sets via the [Global Telecommunication System](http://www.wmo.int/pages/prog/www/TEM/GTS/index_en.html) is made possible by the [National Data Buoy Center](http://www.ndbc.noaa.gov/), which accesses the data sets from the **ERDDAP** and/or **THREDDS** end-points. After performing some internal quality checks, the profiles are encoded into [BUFR](http://en.wikipedia.org/wiki/BUFR) format and released on the [Global Telecommunication System](http://www.wmo.int/pages/prog/www/TEM/GTS/index_en.html), making them available for assimilation by regional and global scale ocean forecasting models.
Transmission of the data sets via the [Global Telecommunication System](https://community.wmo.int/en/activity-areas/global-telecommunication-system-gts) is made possible by the [National Data Buoy Center](http://www.ndbc.noaa.gov/), which accesses the data sets from the **ERDDAP** and/or **THREDDS** end-points. After performing some internal quality checks, the profiles are encoded into [BUFR](http://en.wikipedia.org/wiki/BUFR) format and released on the [Global Telecommunication System](https://community.wmo.int/en/activity-areas/global-telecommunication-system-gts), making them available for assimilation by regional and global scale ocean forecasting models.

Please read the documentation **thoroughly** before beginning the data submission process. Additional questions and information requests should be directed to:
Additional questions and information requests should be directed to:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[[email protected]](mailto:[email protected]?subject=GliderDAC%20Support)
9 changes: 0 additions & 9 deletions _docs/ngdac-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,6 @@ toc: false
summary: This page presents an in-depth description and discussion of the U.S. IOOS National Data Assembly Center architecture.
---

<!--
> [Wiki](https://github.com/kerfoot/ioosngdac/wiki) ▸ **NGDAC Architecture**

# Contents

+ [Data Assembly Center Architecture](#data-assembly-center-architecture)

-->

## Data Assembly Center Architecture

The following diagram illustrates the architecture of the <b>U.S. IOOS National Glider Data Assembly Center</b> and end-to-end data flow pathway.
Expand Down
Loading