Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove uniquehousesenate2002 from SGID #225

Closed
29 tasks done
steveoh opened this issue Feb 27, 2023 · 31 comments
Closed
29 tasks done

Remove uniquehousesenate2002 from SGID #225

steveoh opened this issue Feb 27, 2023 · 31 comments
Assignees
@gregbunce
Labels
deprecation Removing data and or a service porter issue related to adding and removing data and services

Comments

@steveoh
Copy link
Member

steveoh commented Feb 27, 2023 •
edited by jacobdadams
Loading

Summary

The layers that went into creating this I assume were shelved with #221 and #220

Migration Guide

The migration would be to use more current data. We have a combined 2022-2032 but I am unsure how that is different than unique house and senate.

This dataset has been replaced by () which is named () in the Open SGID and () in the SGID Open Data.
The replaced data is still accessible via our shelved policy in AGOL (a link to the shelved item).

Action items

  1. Assign a person who should complete the task by replacing name with their github @name.
  2. Check [x] the box when the task is completed and add the date of completion.
  3. Strike out all items that do not apply.

Soft Delete

The purpose of the soft delete is to ensure that all of our users and applications have gracefully migrated off of the dataset. Soft deletes will remain in effect for 14 days. During this time, we will have the ability to restore the dataset to its original SGID offering(s). After these 14 days, the item is then ready for a hard delete.

Note: If this dataset is being replaced, then wait until the new data is publicly available before completing these steps:

  • Append "(Mature Support)" to the end of the item title (@DenisePeterson, completed: 2023/11/29)
  • Remove all tags other than "Deprecated" (@DenisePeterson, completed: 2023/11/29)
  • Add note to ArcGIS Online description noting when layer will be deleted and any replacement layer (@DenisePeterson, completed: 2023/11/29)
  • Unshare item from Open Data (@DenisePeterson, completed: 2023/11/29)
  • Update the SGID Index row as deprecated by pasting the related Porter issue URL in the Deprecated field (@DenisePeterson, completed: 2023/11/29)
  • Mark ArcGIS Online item as deprecated in preparation for future deletion (@DenisePeterson, completed: 2022/00/00)
    • Change Authoritative field to d in SGID.META.AGOLItems to automatically set the Deprecated AGOL flag. Allow the d to persist through one run of Auditor - currently, Auditor runs daily at 5:00am (@DenisePeterson, completed: 2023/11/29)
      • After one successful run of Auditor, remove the row from the SGID.META.AGOLItems table. This will trigger the removal of this item in Open SGID (@DenisePeterson, completed: 2022/00/00)

Hard Delete

Hard deletes are final. It is recommended to complete the soft delete process before moving on to these steps. If you decide to skip the soft delete, note that you will need to incorporate some of those steps here.

  • Manually remove data from the Internal SGID (@DenisePeterson, completed: 2023/12/11)
    • Deprecated database layers can be backup up on Google Drive > AGRC Share > Team Projects > SGID > [deprecated layers]
  • Shelve ArcGIS Online item: (@DenisePeterson, completed: 2023/12/18`)
    • Shelve the data by copying the row from SGID.META.AGOLItems to the AGOLItems_shelved table in AGOL and changing the AGOL_ITEM_ID field in SGID.META.AGOLItems to shelved or some other note.
    • Deprecated AGOL items can be backup on Google Drive > AGRC Share > Team Projects > SGID > deprecated layers.
  • Update relevant gis.utah.gov data pages (@DenisePeterson, completed: 2022/00/00)
  • Add this porter url to the Deprecated field of the Stewardship record (@DenisePeterson, completed: 2023/11/29)
  • Remove row from SGID.META.ChangeDetection (@steveoh, completed: 2023/12/11)
  • Remove data from forklift hashing and receiving (@steveoh, completed: 2023/12/11)
  • Remove row from data/hashed/changedetection.gdb/TableHashes (@steveoh, completed: 2023/12/11)
  • Update AGOL item description and name to better reflected "shelved" status on the layer. (@DenisePeterson, completed: 2022/00/00)

- [ ] Delete Google Drive data (@DenisePeterson, completed: 2022/00/00)
- [ ] Update SGID.META.AGOLItems table (@DenisePeterson, completed: 2022/00/00)
- cut and paste row to AGOLItems_shelved table if shelving (see below)
- set the AGOL_ITEM_ID field to hosted by <agency> for Farm from AGOL
- set the AGOL_ITEM_ID field to exclude from AGOL to not publish to ArcGIS Online

Shelve/Static

Choose one based on situation.

  • Move existing AGOL item to AGRC_Shelved AGOL folder (shelving an item already in AGOL) (@DenisePeterson, completed: 2022/00/00)
    - [ ] Upload to appropriate UtahAGRC/{SGID Category} folder in AGOL (for static datasets) (@DenisePeterson, completed: 2022/00/00)
    - [ ] Upload to UtahAGRC/AGRC_Shelved folder in AGOL (New shelved item not already in AGOL) (@DenisePeterson, completed: 2022/00/00)

Add record to table.

  • Add record to AGOLItems_shelved table in ArcGIS Online with shelved or static in the CATEGORY field (@DenisePeterson, completed: 2022/00/00)

🤖 Automation validation

  1. Assign yourself or someone to check the item by replacing name with their github @name.
  2. Check [x] the box and add the date of verification 2020/01/01 when the task is verified.
  3. Strike out all items that do not apply.
  • Remove data from Open SGID (@steveoh, completed: 2023/12/11`)
  • sgid-index (@steveoh on 2023/11/30)
  • Auditor sets appropriate Deprecated information (@jacobdadams on 2023/11/30)
  • Auditor sets appropriate shelved information (@jacobdadams on 2023/12/18)

Are there service dependencies

  • Are clients querying this data with the WebAPI?
  • Notify Don Jackson (at motorolasolutions.com) if the data is in the Next-Generation 911 Motorola Aware Map? (@gregbunce on 2023/03/13)
  • Does any other application depend on this data?

Notification

Group Task Assignments

  1. Check [x] the box when you have assigned all the tasks relevant to your group.
@steveoh steveoh added deprecation Removing data and or a service porter issue related to adding and removing data and services labels Feb 27, 2023
@agrc-conductor
Copy link

conductor results for tasks - 225

check status
@steveoh has completed 0 out of 3 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@eneemann has completed 0 out of 1 tasks ⛔
@gregbunce has completed 0 out of 1 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔

@agrc-conductor
Copy link

conductor results for tasks - 225

check status
@steveoh has completed 0 out of 3 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@eneemann has completed 0 out of 1 tasks ⛔
@gregbunce has completed 0 out of 1 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔

@gregbunce
Copy link
Contributor

note: I have this on my radar to begin looking at later in the week.

@steveoh steveoh added the scheduled A task that has a schedule for action (conductor will skip checks) label Mar 13, 2023
@gregbunce gregbunce self-assigned this Mar 30, 2023
@steveoh
Copy link
Member Author

steveoh commented Oct 13, 2023

🔔🔔🔔

@gregbunce gregbunce mentioned this issue Oct 17, 2023
Closed
@steveoh steveoh removed the scheduled A task that has a schedule for action (conductor will skip checks) label Oct 17, 2023
@agrc-conductor
Copy link

conductor results for tasks - 225

check status
@steveoh has completed 0 out of 3 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@gregbunce has completed 1 out of 2 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔

@agrc-conductor
Copy link

conductor results for tasks - 225

check status
@steveoh has completed 0 out of 3 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@gregbunce has completed 1 out of 2 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔

@agrc-conductor
Copy link

conductor results for tasks - 225

check status
@steveoh has completed 0 out of 3 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@gregbunce has completed 1 out of 2 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔

@agrc-conductor
Copy link

conductor results for tasks - 225

check status
@gregbunce has completed 1 out of 21 tasks ⛔
@steveoh has completed 1 out of 6 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔

@agrc-conductor
Copy link

conductor results for tasks - 225

check status
@DenisePeterson has completed 0 out of 19 tasks ⛔
@steveoh has completed 1 out of 6 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
gregbunce has completed 2 out of 2 tasks 👍
@rkelson has completed 0 out of 1 tasks ⛔

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔

@agrc-conductor
Copy link

conductor results for tasks - 225

check status
@DenisePeterson has completed 0 out of 19 tasks ⛔
@steveoh has completed 1 out of 6 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
gregbunce has completed 2 out of 2 tasks 👍
@rkelson has completed 0 out of 1 tasks ⛔

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔

@steveoh steveoh mentioned this issue Nov 30, 2023
Merged
@agrc-conductor
Copy link

conductor results for tasks - 225

check status
@DenisePeterson has completed 8 out of 20 tasks ⛔
@steveoh has completed 3 out of 6 tasks ⛔
jacobdadams has completed 1 out of 1 tasks 👍
gregbunce has completed 2 out of 2 tasks 👍
rkelson has completed 1 out of 1 tasks 👍

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid ⛔
meta table 👍
stewardship
- deprecation issue link 👍

@agrc-conductor
Copy link

conductor results for tasks - 225

check status
@DenisePeterson has completed 8 out of 20 tasks ⛔
@steveoh has completed 3 out of 6 tasks ⛔
jacobdadams has completed 1 out of 1 tasks 👍
gregbunce has completed 2 out of 2 tasks 👍
rkelson has completed 1 out of 1 tasks 👍

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid ⛔
meta table 👍
stewardship
- deprecation issue link 👍

@agrc-conductor
Copy link

conductor results for tasks - 225

check status
DenisePeterson has completed 15 out of 15 tasks 👍
steveoh has completed 7 out of 7 tasks 👍
jacobdadams has completed 1 out of 1 tasks 👍
gregbunce has completed 2 out of 2 tasks 👍
rkelson has completed 1 out of 1 tasks 👍

@agrc-conductor
Copy link

conductor results for POLITICAL.uniquehousesenate2002

check status
internal sgid 👍
meta table 👍
stewardship
- deprecation issue link 👍

@nathankota
Copy link
Contributor

Looks like everything has been checked off the list. If you agree, @DenisePeterson would you like to do the honors of closing this issue?

@steveoh
Copy link
Member Author

steveoh commented Dec 18, 2023 •
edited
Loading

Shouldn't this be in the shelved folder with updated tags like

image

@jacobdadams
Copy link
Member

It looks like there's not an entry for this item in the AGOLItems_shelved table, so Auditor doesn't know it should be shelved. @DenisePeterson, do you want to add the relevant row in that table and then let me know? I can then run auditor on it and make sure it gets moved appropriately.

And again, this was caused by the auditor verification actually needing to happen twice: once for the deprecation in the soft delete stage and again for the shelving during the hard delete.

@steveoh
Copy link
Member Author

steveoh commented Dec 18, 2023

Please update the dates when you complete the tasks @DenisePeterson so we can keep track of everything
image

@DenisePeterson
Copy link

fixed the table issue to shelve uniquehousesenate2002 and close.

@steveoh
Copy link
Member Author

steveoh commented Dec 19, 2023 •
edited
Loading

let's verify auditor and close this out when it's complete. @DenisePeterson if you could update all the dates next to your tasks as well that would be great!

@steveoh steveoh reopened this Dec 19, 2023
@jacobdadams
Copy link
Member

We did both, and I added a second checkbox for it under the validation checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gregbunce gregbunce

Labels
deprecation Removing data and or a service porter issue related to adding and removing data and services
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@steveoh @nathankota @gregbunce @jacobdadams @agrc-conductor @DenisePeterson