You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are opportunities within the ingestor to try/catch create then open, rather than the other way around, at zero functional change but with improved processing times.
Idea:
There are times we want to open something for write if it exists, or create it if it doesn't
Sometimes we simply do that: just to get a thing existing at a given URI, whether by creating or opening for write
Other times it's an error if the thing already exists (e.g. ingest without ingest_mode="resume")
Currently we've been doing a try-open-for-write, and catching a does-not-exist error with trying a create
All else being equal this would be as good as doing try-create, catching an already-exists error with trying an open for write
However in certain contexts, the if-exists logic in the try-open-for-write case can take unnecessarily long
This is particularly relevant for ab nihilo ingests for which the thing not existing is the normal case
In either ab nihilo ingests, or append-mode ingests, it turns out to be quite performant to do the try-create first and then fail back to try-open-for-write
In summary: we have create and open, and open can be more expensive, so it's always best to try create then open
[sc-44853]
The text was updated successfully, but these errors were encountered:
There are opportunities within the ingestor to try/catch create then open, rather than the other way around, at zero functional change but with improved processing times.
Idea:
ingest_mode="resume"
)[sc-44853]
The text was updated successfully, but these errors were encountered: