agrc/porter

Remove sgid.political.districtcombinationareas2012 from SGID

steveoh opened this issue · 29 comments

steveoh commented

Summary

There is a newer version and the data that created this combination were shelved.

Migration Guide

Update to the newest data. https://gis.utah.gov/data/political/political-districts/#CombinedDistricts

This dataset has been replaced by the 2022 combined areas which is named district_combination_areas_2022 in the Open SGID and utah-district-combination-areas-2022 in the SGID Open Data.
The replaced data is still accessible via our shelved policy in AGOL (a link to the shelved item).

Action items

  1. Assign a person who should complete the task by replacing name with their github @name.
  2. Check [x] the box when the task is completed and add the date of completion.
  3. Strike out all items that do not apply.

Soft Delete

The purpose of the soft delete is to ensure that all of our users and applications have gracefully migrated off of the dataset. Soft deletes will remain in effect for 14 days. During this time, we will have the ability to restore the dataset to its original SGID offering(s). After these 14 days, the item is then ready for a hard delete.

Note: If this dataset is being replaced, then wait until the new data is publicly available before completing these steps:

  • Append "(Mature Support)" to the end of the item title (@gregbunce, completed: 2023/11/29)
  • Remove all tags other than "Deprecated" (@gregbunce, completed: ``2023/11/29`)
  • Add note to ArcGIS Online description noting when layer will be deleted and any replacement layer (name, completed: 2023/11/29)
  • Unshare item from Open Data (@gregbunce, completed: 2023/11/29)
  • Update the SGID Index row as deprecated by pasting the related Porter issue URL in the Deprecated field (@gregbunce, completed: 2023/11/29)
  • Mark ArcGIS Online item as deprecated in preparation for future deletion (@gregbunce, completed: 2023/11/30)
    • Change Authoritative field to d in SGID.META.AGOLItems to automatically set the Deprecated AGOL flag. Allow the d to persist through one run of Auditor - currently, Auditor runs daily at 5:00am (@gregbunce, completed: 2023/11/29)
      • After one successful run of Auditor, remove the row from the SGID.META.AGOLItems table. This will trigger the removal of this item in Open SGID (@DenisePeterson, completed: 2023/11/30)

Hard Delete

Hard deletes are final. It is recommended to complete the soft delete process before moving on to these steps. If you decide to skip the soft delete, note that you will need to incorporate some of those steps here.

  • Manually remove data from the Internal SGID (@gregbunce, completed: 2023/12/11)
    • Deprecated database layers can be backup up on Google Drive > AGRC Share > Team Projects > SGID > [deprecated layers]
  • Shelve ArcGIS Online item: (@gregbunce, completed: 2023/12/11)
    • Shelve the data by copying the row from SGID.META.AGOLItems to the AGOLItems_shelved table in AGOL and changing the AGOL_ITEM_ID field in SGID.META.AGOLItems to shelved or some other note.
    • Deprecated AGOL items can be backup on Google Drive > AGRC Share > Team Projects > SGID > deprecated layers.
  • Update relevant gis.utah.gov data pages (@gregbunce, completed: 2023/12/11)
  • Add this porter url to the Deprecated field of the Stewardship record (@gregbunce, completed: 2023/11/29)
  • Update SGID.META.AGOLItems table (@gregbunce, completed: 2023/12/11)
    • cut and paste row to AGOLItems_shelved table if shelving (see below)
  • Remove row from SGID.META.ChangeDetection (@steveoh, completed: 2023/12/11`)
  • Remove data from forklift hashing and receiving (@steveoh, completed: 2023/12/11`)
  • Remove row from data/hashed/changedetection.gdb/TableHashes (@steveoh, completed: 2023/12/11`)
  • Update AGOL item description and name to better reflect the "shelved" status on the layer (@gregbunce, completed: 2023/12/11)
    - [ ] Delete Google Drive data (@gregbunce, completed: 2022/00/00)

Shelve/Static

Choose one based on situation.

  • Move existing AGOL item to AGRC_Shelved AGOL folder (shelving an item already in AGOL) (@gregbunce, completed: 2023/12/11)
    - [ ] Upload to appropriate UtahAGRC/{SGID Category} folder in AGOL (for static datasets) (@gregbunce, completed: 2022/00/00)
    - [ ] Upload to UtahAGRC/AGRC_Shelved folder in AGOL (New shelved item not already in AGOL) (@gregbunce, completed: 2022/00/00)

Add record to table.

  • Add record to AGOLItems_shelved table in ArcGIS Online with shelved or static in the CATEGORY field (@gregbunce, completed: 2023/12/11)

🤖 Automation validation

  1. Assign yourself or someone to check the item by replacing name with their github @name.
  2. Check [x] the box and add the date of verification 2020/01/01 when the task is verified.
  3. Strike out all items that do not apply.
  • Remove data from Open SGID (@steveoh, completed: 2023/12/11`)
  • sgid-index (@steveoh on 2023/11/30)
  • Auditor sets appropriate shelved/static/Deprecated information (@jacobdadams on 2023/11/30)

Are there service dependencies

  • Are clients querying this data with the WebAPI?
  • Notify Don Jackson (jat motorolasolutions.com) if the data is in the Next-Generation 911 Motorola Aware Map? (@gregbunce on 2023/03/13)
  • Does any other application depend on this data?

Notification

Group Task Assignments

  1. Check [x] the box when you have assigned all the tasks relevant to your group.
agrc-conductor commented

conductor results for tasks - 226

check status
@steveoh has completed 0 out of 3 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@eneemann has completed 0 out of 1 tasks ⛔
@gregbunce has completed 0 out of 1 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔
agrc-conductor commented

conductor results for tasks - 226

check status
@steveoh has completed 0 out of 3 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@eneemann has completed 0 out of 1 tasks ⛔
@gregbunce has completed 0 out of 1 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔
gregbunce commented

note: I have this on my radar to begin looking at later in the week.

steveoh commented

🔔🔔🔔

  • 👍1
agrc-conductor commented

conductor results for tasks - 226

check status
@steveoh has completed 0 out of 3 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@gregbunce has completed 1 out of 2 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔
agrc-conductor commented

conductor results for tasks - 226

check status
@steveoh has completed 0 out of 3 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@gregbunce has completed 1 out of 2 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔
agrc-conductor commented

conductor results for tasks - 226

check status
@steveoh has completed 0 out of 3 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@gregbunce has completed 1 out of 2 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔
agrc-conductor commented

conductor results for tasks - 226

check status
@gregbunce has completed 1 out of 19 tasks ⛔
@steveoh has completed 1 out of 6 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔
agrc-conductor commented

conductor results for tasks - 226

check status
@gregbunce has completed 2 out of 19 tasks ⛔
@steveoh has completed 1 out of 6 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔
agrc-conductor commented

conductor results for tasks - 226

check status
@gregbunce has completed 2 out of 19 tasks ⛔
@steveoh has completed 1 out of 6 tasks ⛔
@jacobdadams has completed 0 out of 1 tasks ⛔
@rkelson has completed 0 out of 1 tasks ⛔
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid ⛔
meta table ⛔
open sgid ⛔
open data ⛔
arcgis online ⛔
stewardship
- deprecation issue link ⛔
agrc-conductor commented

conductor results for tasks - 226

check status
@gregbunce has completed 8 out of 19 tasks ⛔
DenisePeterson has completed 1 out of 1 tasks 👍
@steveoh has completed 3 out of 6 tasks ⛔
jacobdadams has completed 1 out of 1 tasks 👍
@rkelson has completed 0 out of 1 tasks ⛔
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid ⛔
meta table 👍
stewardship
- deprecation issue link 👍
agrc-conductor commented

conductor results for tasks - 226

check status
@gregbunce has completed 8 out of 19 tasks ⛔
DenisePeterson has completed 1 out of 1 tasks 👍
@steveoh has completed 3 out of 6 tasks ⛔
jacobdadams has completed 1 out of 1 tasks 👍
@rkelson has completed 0 out of 1 tasks ⛔
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid ⛔
meta table 👍
stewardship
- deprecation issue link 👍
gregbunce commented

I believe my work is done

steveoh commented

Let's wait until next monday to see what conductor says. The layer doesn't show as shelved yet. I've asked jake to clarify his check and what auditor will be doing.

  • 👍1
jacobdadams commented

Ok, the initial Auditor validation I checked off on was when auditor caught that it had been deprecated and updated the tags, title, and deprecated note. The group and folder as set to "shelved" if it's in the AGOLItems_shelved table with a CATEGORY of shelved, otherwise it gets them via the category from the full table name (SGID.CATEGORY.table).

Thus, when we set them to deprecated in the SGID meta table, Auditor fixes the relevant bits on the item, and when the row is created in AGOLItems_shelved it actually moves it.

So, for this layer, tomorrow morning's Auditor run should actually put it on the shelf where it belongs with all the other misfit toys.

I'll be the first to admit this isn't a particularly elegant solution. I'd love to fix this with the asset tracker tool.

  • 👍1
agrc-conductor commented

conductor results for tasks - 226

check status
gregbunce has completed 16 out of 16 tasks 👍
DenisePeterson has completed 1 out of 1 tasks 👍
steveoh has completed 7 out of 7 tasks 👍
jacobdadams has completed 1 out of 1 tasks 👍
rkelson has completed 1 out of 1 tasks 👍
agrc-conductor commented

conductor results for POLITICAL.districtcombinationareas2012

check status
internal sgid 👍
meta table 👍
stewardship
- deprecation issue link 👍
nathankota commented

@gregbunce it looks like this is ready to be closed?

  • 👍1
gregbunce commented

thanks everyone.