Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DPLT-1084 Add support for Runner infrastructure #166

Merged
merged 10 commits into from
Aug 7, 2023
Merged

DPLT-1084 Add support for Runner infrastructure #166

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
a53d343
feat: Add `Dockerfile` for runner
morgsmccauley Aug 3, 2023
7364e76
chore: Specify node version in `package.json`
morgsmccauley Aug 3, 2023
1f9d7ca
feat: Change `REGION` env to `AWS_REGION`
morgsmccauley Aug 3, 2023
9ed46eb
fix: Avoid unnecessary TS rebuild in docker
morgsmccauley Aug 6, 2023
7f85607
refactor: Use default `node-pg` env vars
morgsmccauley Aug 6, 2023
4216cbc
fix: Broken docker start command
morgsmccauley Aug 6, 2023
d614b49
fix: Run tsc build in docker file
morgsmccauley Aug 6, 2023
7487cb6
refactor: use multi-stage docker build
morgsmccauley Aug 6, 2023
7527250
fix: Move `pluralize` to prod dependencies
morgsmccauley Aug 6, 2023
6ddc549
feat: Remove `AWS_REGION` env as it is sourced automatically
morgsmccauley Aug 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions runner/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
FROM node:18.17 AS builder
WORKDIR /usr/src/app
COPY . .
RUN npm install
RUN npm run build

FROM node:18.17
WORKDIR /usr/src/app
COPY --from=builder /usr/src/app/package*.json ./
RUN npm install --omit=dev
COPY --from=builder /usr/src/app/dist ./dist
CMD [ "npm", "run", "start:docker" ]
6 changes: 4 additions & 2 deletions runner/package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion runner/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,14 @@
"version": "1.0.0",
"description": "",
"main": "index.js",
"engines": {
"node": "18.17"
},
"scripts": {
"build": "rm -rf ./dist && tsc",
"start": "npm run build && node dist/index.js",
"start:dev": "ts-node ./src/index.ts",
"start:docker": "node dist/index.js",
"test": "node --experimental-vm-modules ./node_modules/.bin/jest",
"lint": "eslint -c .eslintrc.js"
},
Expand All @@ -31,14 +35,14 @@
"eslint-plugin-prettier": "^5.0.0",
"eslint-plugin-promise": "^6.1.1",
"jest": "^29.6.2",
"pluralize": "^8.0.0",
"prettier": "^3.0.0",
"ts-jest": "^29.1.1",
"ts-node": "^10.9.1",
"typescript": "^5.1.6"
},
"dependencies": {
"@near-lake/primitives": "^0.1.0",
"pluralize": "^8.0.0",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dev dependencies are stripped from the docker build, and this was incorrectly placed there.

"aws-sdk": "^2.1402.0",
"node-fetch": "^2.6.11",
"pg": "^8.11.1",
Expand Down
11 changes: 5 additions & 6 deletions runner/src/globals.d.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,10 @@ declare namespace NodeJS {
export interface ProcessEnv {
HASURA_ENDPOINT: string
HASURA_ADMIN_SECRET: string
PG_HOST: string
PG_PORT: string
PG_ADMIN_USER: string
PG_ADMIN_PASSWORD: string
PG_ADMIN_DATABASE: string
REGION: string
PGHOST: string
PGPORT: string
PGUSER: string
PGPASSWORD: string
PGDATABASE: string
}
}
2 changes: 1 addition & 1 deletion runner/src/indexer/indexer.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ export default class Indexer {
this.network = network;
this.deps = {
fetch,
s3: new AWS.S3({ region: process.env.REGION }),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This env var has been changed to AWS_REGION, which is automatically pulled in from the environment.

s3: new AWS.S3(),
provisioner: new Provisioner(),
...deps,
};
Expand Down
10 changes: 5 additions & 5 deletions runner/src/provisioner/provisioner.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ import PgClient from '../pg-client';
const DEFAULT_PASSWORD_LENGTH = 16;

const sharedPgClient = new PgClient({
user: process.env.PG_ADMIN_USER,
password: process.env.PG_ADMIN_PASSWORD,
database: process.env.PG_ADMIN_DATABASE,
host: process.env.PG_HOST,
port: Number(process.env.PG_PORT),
user: process.env.PGUSER,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now using node-postgres & Postgres specified environment variables, these don't actually need to be defined, but I'm just doing it for explicitness.

password: process.env.PGPASSWORD,
database: process.env.PGDATABASE,
host: process.env.PGHOST,
port: Number(process.env.PGPORT),
});

export default class Provisioner {
Expand Down