feat: Cache Database Credentials #585
Conversation
runner/src/index.ts
Outdated
| grpcServer.forceShutdown(); | ||
| } | ||
| })(); | ||
| void (async function main () {})(); |
There was a problem hiding this comment.
Went ahead and cleaned up the Redis scanning here too. I can also move this to happen in another PR, if needed.
There was a problem hiding this comment.
we don't need this function anymore
| let currBlockHeight = 0; | ||
| let indexerName = `${workerContext.indexerConfig.account_id}/${workerContext.indexerConfig.function_name}`; |
There was a problem hiding this comment.
Migrated to using config passed into worker, since we no longer use storage after V2 migration.
There was a problem hiding this comment.
Can we update StreamHandler too please? I think it still assumes indexerConfig can be undefined
There was a problem hiding this comment.
Ah yes you're totally right. I'll update that!
runner/src/index.ts
Outdated
| grpcServer.forceShutdown(); | ||
| } | ||
| })(); | ||
| void (async function main () {})(); |
There was a problem hiding this comment.
we don't need this function anymore
runner/src/indexer/indexer.ts
Outdated
| export default class Indexer { | ||
| DEFAULT_HASURA_ROLE; | ||
| database_connection_parameters: any | undefined = undefined; |
There was a problem hiding this comment.
Can we avoid using any please
runner/src/indexer/indexer.ts
Outdated
| // Cache database credentials after provisioning | ||
| try { | ||
| this.database_connection_parameters = this.database_connection_parameters ?? | ||
| await this.deps.provisioner.getDatabaseConnectionParameters(functionName.split('/')[0].replace(/[.-]/g, '_')); |
There was a problem hiding this comment.
| await this.deps.provisioner.getDatabaseConnectionParameters(functionName.split('/')[0].replace(/[.-]/g, '_')); | |
| await this.deps.provisioner.getDatabaseConnectionParameters(hasuraRoleName); |
This is already defined above
| let currBlockHeight = 0; | ||
| let indexerName = `${workerContext.indexerConfig.account_id}/${workerContext.indexerConfig.function_name}`; |
There was a problem hiding this comment.
Can we update StreamHandler too please? I think it still assumes indexerConfig can be undefined
Currently, Runner fetches DB credentials every invocation of runFunction because provisioning of the user's DB occurs in Runner. As such, we couldn't be sure if we could fetch credentials for the user's DB from Hasura or not. The fetch is slow and increases the latency of any indexers which leverage context.db.
I updated the code to cache the credentials locally after the first successful fetch. This allows all future requests to succeed as these credentials will never change after they are set. The changes were aimed to be minimal as we intend to migrate provisioning out of Runner. In fact, the changes aid in the effort as the credentials fetch is done through the Provisioner class rather than separately in DmlHandler.