Deploy & run load-transform in production

[jamesbursa]

Summary

Deploy the latest code to production and run the load-transform job in production.

Troubleshoot and fix any errors or issues. Verify that the database looks right.

Acceptance criteria

• Code has been run in production.
• Fixes have been made and/or follow up tickets created.

[jamesbursa]

Deployment was successful (https://github.com/HHS/simpler-grants-gov/releases/tag/2024.5.9-1) after setting up schemas #1848.

The foreign tables were created manually.

Then executed the load using:

bin/run-command.sh api prod '["flask", "--app=src.app:create_app", "data-migration", "load-transform", "--load", "--no-transform", "--no-set-current"]'

It was successful on the following tables:

• "tforecast"
• "tforecast_hist"
• "tapplicanttypes_forecast"
• "tapplicanttypes_forecast_hist"
• "tfundactcat_forecast"
• "tfundactcat_forecast_hist"
• "tfundinstr_forecast"
• "tfundinstr_forecast_hist"
• "topportunity"
• "topportunity_cfda"

Failed on tables:

• "tsynopsis" - psycopg.errors.FdwUnableToCreateExecution: error fetching result: OCIStmtFetch2 failed to fetch next result row / DETAIL: ORA-01555: snapshot too old: rollback segment number 21 with name "_SYSSMU21_2881811878$" too small

[chouinar]

Comparing the raw counts of the opportunity statuses between the legacy and our process after running fully in prod.

We have 3 fewer forecasted opportunities.
The same amount of posted.
The same amount of closed.
The same amount of archived.

Looking a bit closer:

344227: forecasted != archived
344228: forecasted != archived
337347: forecasted != archived
322537: forecasted != archived
336962: None != forecasted

The left value is grants.gov - the right value is our data

I remember looking at 336962 before the break - I wasn’t sure why it wasn’t forecasted in the legacy system. I’m guessing just some issue with the record getting loaded into their search?

https://grants.gov/search-results-detail/344227 and https://grants.gov/search-results-detail/344228 are both clearly the “has both a synopsis and forecast - but uses the forecast” case that the legacy system supports and we do not.

https://grants.gov/search-results-detail/337347 and https://grants.gov/search-results-detail/322537 are also the same case - but it’s not as obvious from the UI. The synopsis record they have that we have take precedence doesn’t display on the UI because they only have a single version of the synopsis (the legacy UI never displays the most recent version of the forecast/synopsis when the synopsis/forecast shows)