You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
First we should choose an execution environment, the candidates are:
Github Actions (leaning towards this, as I've had nothing but good experiences with it)
AWS Lambda (the existing infra, but I'm only using it because it's free...no real reason to stay)
The high level requirements of the ingestor are
dynamic schema generation from CSVs (existing python already does this, will need to be modified to work with cockroach schemas)
dynamic service ingestion (as new services are added by AWS the ingestor automatically picks them up and imports them, this already happens with the python scripts)
drop any table we are about to import
trigger an async cockroach import of a delimited data file, this has been tested in a local POC and works wonderfully
create an index on the EC2 table after it has been generated
The text was updated successfully, but these errors were encountered:
First we should choose an execution environment, the candidates are:
The high level requirements of the ingestor are
The text was updated successfully, but these errors were encountered: