As a user, I want my API request to execute successfully even when the registry contains corrupted documents

[tloubrieu-jpl]

Checked for duplicates

Yes - I've already checked

🧑‍🔬 User Persona(s)

• user does not want to be polluted by improperly loaded products (we want to consider them as not loaded from a user perspective)
• operator want to get logs of the errors so to fix the corrupted products

💪 Motivation

...so that I can do my requests on the properly loaded products without being blocked by corrupted products

Motivated by ticket #349

📖 Additional Details

Make the api more robust by skipping the corrupted products and generating a log error but answering the user with what is not corrupted

Acceptance Criteria

Given
When I perform
Then I expect

⚙️ Engineering Details

No response

[al-niessner]

@jordanpadams @tloubrieu-jpl

We can do this because most the problems with parsing the document are local (strings when arrays are expected etc) by installing try/catch blocks. However this will mask a serious host of other errors. Are far enough along in development for such a masking? It also means that we treat all opensearch data as valid (no corrupt) because no one is going to read the log files. Just want to verify that you want to live in a bog before we build the fourth castle in it.

[jordanpadams]

@al-niessner

It also means that we treat all opensearch data as valid (no corrupt) because no one is going to read the log files.

We need a follow-on ticket in the monitoring repo to then have an alarm to trigger a notification to the dev team when an error is found in the logs.

The rationale here being we would rather give the user as much of the metadata as we can versus totally failing and completely blocking a user from building off of the API.

[miguelp1986]

@tloubrieu-jpl @gxtchen Issue #349 is referenced as being the motivation for this issue, and Gary mentioned on TestRail that this issue is tested by testing #349.

Is testing #349 sufficient for this issue or is there some other way we want to test this?