Continue shrinking the Internal SGID

[gregbunce]

Benefit

The benefit of continuing to shrink the Internal SGID is that we are not maintaining irrelevant data, and therefore this frees up our time to work on more important projects.

In the world of UGRC, this is an ongoing project that we should make progress on each quarter. That's the nature of of our business (keeping the data relevant).... datasets change, the stewards find better ways to serve them, etc.. In turn, we need to stay current and clean up the Internal so we're not serving outdated or irrelevant data.

For this quarter I propose the data team take a look at datasets that have not had an update since 2020. We flag those datasets and then establish a path forward for them. Some may need to be deprecated, others may become a part of the SGID Index b/c the stewards are now serving their own web service (data).

Acceptance Criteria

• Document (begin documenting) what the SGID will look like over the next 3 to 5 years.
  - [ ] Create a list of datasets in the Internal SGID that have not had an update since 2020
  - [ ] Determine a path forward for those datasets (keep, deprecate, data-come-full-circle and steward has a web service, etc.)

Notes

Other things to think about:

• Regarding the website, what constitutes a data page versus simply an SGID index entry?

Risks

None

Issue Reference

• UGRC - SGID data classification (rating system) #5

[gregbunce]

Rick and I are meeting June 4th to discuss this and come up with a potential mini project for this in sprint 6

[gregbunce]

Rick and I decided to use some time in sprint 6 to create an outline on what the sgid looks like moving forward. We'll be working on that and then seeking feedback.

[rkelson]

We should create a list of feature layers that reside in AGOL only with no Internal layer. This idea is in response to removing State Fuel Sites from Internal while having it persist in AGOL and needing a strategy to backup AGOL only layers.

[rkelson]

ran this query on the SGID Index tab and it only came up with one layer, the Statewide parcel layer used in applications:
storageType IN ('AGOL', 'cloud storage') And hostedBy = 'UGRC' And arcGisOnline = 'TRUE' And itemId IS NOT NULL

refs #325

[steveoh]

Statewide parcels are a bit unique since we can recreate them from data in the internal database.

[gregbunce]

rick and i had some good discussions this quarter. as a data team, we've all also discussed this during our 'sgid: a plan for the future" meetings.

[gregbunce]

informed from the rating system project

[rkelson]

geoscience.alluvial_fans is likely a good candidate to remove from the internal database. I need to confirm but I believe this is a static dataset. I am adding this as a reminder for something to get to down the road