Quickstart not working with esm

[akutruff]

Edit: found out that I could get it to create the database with tsx but the error was that I hadn't setup my DATABASE_URL to have a database name after the / However, ts-node still does not work.

I'm trying to setup my project, using the quickstart with the demo tables and no other modifications.

I've tried several derivations of the configuration and none seem to be working, but I'm assuming it's a configuration issue.

I'm listing all below:

Project root tsconfig.json

{
  "compilerOptions": {
    "target": "es2017",
    "lib": ["dom", "dom.iterable", "esnext"],
    "allowJs": true,
    "checkJs": true,
    "skipLibCheck": true,
    "strict": true,
    "forceConsistentCasingInFileNames": true,
    "noEmit": true,
    "esModuleInterop": true,
    "module": "esnext",
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "isolatedModules": true,
    "jsx": "preserve",
    "incremental": true,
    "noUncheckedIndexedAccess": true
  },
  "include": [".eslintrc.js", "prettier.config.cjs"]
}

The above is imported for the package with the quickstart code

packages/orchiddb/tsconfig.json

{
  "extends": "../../tsconfig.json",
  "compilerOptions": {
    "noEmit": false
  },
  "ts-node": {
    "swc": true
  }
}

package.json

Note: with-env does not matter. It's just a way to load the environment file, and I verified that the problems are the same with or without using with-env.

I put in 3 versions - without esm, with esm, and then also tried to use tsx which actually executed the code but failed later on. See below for errors.

tsx actually is able to execute the script but fails for postgres reasons but gives no helpful output.

{
  "type": "module",
  "scripts": {
    "db": "pnpm with-env ts-node src/db/dbScript.ts",
    "db:esm": "pnpm with-env node --loader ts-node/esm src/db/dbScript.ts",
    "db:tsx": "pnpm with-env tsx src/db/dbScript.ts",
    "with-env": "dotenv -e ../../.env --"
  },
  "dependencies": {
    "dotenv": "^16.1.4",
    "orchid-orm": "^1.10.5",
    "pqb": "^0.12.4",
    "pg": "^8.11.0",
    "orchid-orm-schema-to-zod": "^0.3.79"
  },
  "devDependencies": {
    "dotenv-cli": "^7.2.1",
    "rake-db": "^2.8.44",
    "orchid-orm-test-factory": "^0.3.95",
    "@swc/core": "^1.3.62",
    "@types/node": "^20.2.5",
    "ts-node": "^10.9.1",
    "tsx": "^3.12.7",
    "typescript": "^5.1.3"
  }
}

Attempts

(No ESM) pnpm db create

node ➜ /workspaces/planner/packages/orchiddb (main) $ pnpm db create

> @ db /workspaces/planner/packages/orchiddb
> pnpm  ts-node src/db/dbScript.ts "create"

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /workspaces/planner/packages/orchiddb/src/db/dbScript.ts
    at new NodeError (node:internal/errors:399:5)
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:79:11)
    at defaultGetFormat (node:internal/modules/esm/get_format:121:38)
    at defaultLoad (node:internal/modules/esm/load:81:20)
    at nextLoad (node:internal/modules/esm/loader:163:28)
    at ESMLoader.load (node:internal/modules/esm/loader:605:26)
    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:457:22)
    at new ModuleJob (node:internal/modules/esm/module_job:64:26)
    at ESMLoader.#createModuleJob (node:internal/modules/esm/loader:480:17)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:434:34) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}
 ELIFECYCLE  Command failed with exit code 1.

(ESM) pnpm db:esm create

node ➜ /workspaces/planner/packages/orchiddb (main) $ pnpm db:esm create

> @ db:esm /workspaces/planner/packages/orchiddb
> pnpm node --loader ts-node/esm src/db/dbScript.ts "create"

(node:26314) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
/workspaces/planner/node_modules/ts-node/dist-raw/node-internal-modules-esm-resolve.js:366
    throw new ERR_MODULE_NOT_FOUND(
          ^
CustomError: Cannot find module '/workspaces/planner/packages/orchiddb/src/db/config' imported from /workspaces/planner/packages/orchiddb/src/db/dbScript.ts
    at finalizeResolution (/workspaces/planner/node_modules/ts-node/dist-raw/node-internal-modules-esm-resolve.js:366:11)
    at moduleResolve (/workspaces/planner/node_modules/ts-node/dist-raw/node-internal-modules-esm-resolve.js:801:10)
    at Object.defaultResolve (/workspaces/planner/node_modules/ts-node/dist-raw/node-internal-modules-esm-resolve.js:912:11)
    at /workspaces/planner/node_modules/ts-node/src/esm.ts:218:35
    at entrypointFallback (/workspaces/planner/node_modules/ts-node/src/esm.ts:168:34)
    at /workspaces/planner/node_modules/ts-node/src/esm.ts:217:14
    at addShortCircuitFlag (/workspaces/planner/node_modules/ts-node/src/esm.ts:409:21)
    at resolve (/workspaces/planner/node_modules/ts-node/src/esm.ts:197:12)
    at nextResolve (node:internal/modules/esm/loader:163:28)
    at ESMLoader.resolve (node:internal/modules/esm/loader:838:30)
 ELIFECYCLE  Command failed with exit code 1.

(TSX) pnpm db:tsx create

Update: this actually works now. I didn't see that I needed to have a dbname after the forward slash. However, there is no clear error being output, so there's still an issue here.

DATABASE_URL=postgresql://postgres:postgres@localhost/orchid-demo:5432
DATABASE_URL=postgresql://postgres:postgres@localhost/orchid-demo-test:5432

node ➜ /workspaces/planner/packages/orchiddb (main) $ pnpm db:tsx create

> @ db:tsx /workspaces/planner/packages/orchiddb
> pnpm with-env tsx src/db/dbScript.ts "create"


> @ with-env /workspaces/planner/packages/orchiddb
> dotenv -e ../../.env -- "tsx" "src/db/dbScript.ts" "create"

/workspaces/planner/node_modules/pg-protocol/dist/parser.js:287
        const message = name === 'notice' ? new messages_1.NoticeMessage(length, messageValue) : new messages_1.DatabaseError(messageValue, length, name);
                                                                                                 ^


error: zero-length delimited identifier at or near """"
    at Parser.parseErrorMessage (/workspaces/planner/node_modules/pg-protocol/dist/parser.js:287:98)
    at Parser.handlePacket (/workspaces/planner/node_modules/pg-protocol/dist/parser.js:126:29)
    at Parser.parse (/workspaces/planner/node_modules/pg-protocol/dist/parser.js:39:38)
    at Socket.<anonymous> (/workspaces/planner/node_modules/pg-protocol/dist/index.js:11:42)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23) {
  length: 111,
  severity: 'ERROR',
  code: '42601',
  detail: undefined,
  hint: undefined,
  position: '17',
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'scan.l',
  line: '1188',
  routine: 'scanner_yyerror'
}

Node.js v18.16.0
 ELIFECYCLE  Command failed with exit code 1.
 ELIFECYCLE  Command failed with exit code 1.

[romeerez]

Need to experiment more with different tools and write more docs about ESM.

If nothing works, try to use vite-node, it worked for me. ES modules.

ES modules in node.js are the way of pain, so if it continues to be as broken, consider using commonjs.

[akutruff]

Thanks for letting me know. tsx is actually working perfectly well now. You don't get scm but it's robust against project configs. I'll checkout vite-node after I finish initial project setup.

[IlyaSemenov]

I use vite-node with esm modules and it works just fine. Moreover, I reuse the same vite config for the production build (a single ESM bundle), which is very convenient.

The problem with tsx is that it struggles with tsconfig path aliases in monorepo: privatenumber/tsx#96

[romeerez]

@akutruff what is scm?

[romeerez]

I added instructions to configure a project with vite-node.

I tried ts-node and tsx without success, so the vite-node seems to be the only straightforward way for ESM.

[KMJ-007]

I added instructions to configure a project with vite-node.

I tried ts-node and tsx without success, so the vite-node seems to be the only straightforward way for ESM.

can you guide me to the correct instructions link, the old link is broken

i am trying to setup the db with common package in mono repo, so i can share the types and validation easily between different modules, but zod types and db res are giving me unknown and [x: string]: any;

[romeerez]

@KMJ-007 for vite-node with ESM, simply follow the quickstart and choose vite-node.

But this issue is about ESM, not related to monorepos.

I don't know how to share types and validations, it highly depends on your monorepo setup and isn't easy to configure properly.

I use pnpm workspaces, defining paths to packages in tsconfig like here, and also defining paths to packages for jest in jest config.

If you export types and validations that depend on the ORM from package A and want to import them to package B, package B also should have the same version of the ORM in its dependencies.

[IlyaSemenov]

@KMJ-007 See the minimal monorepo where package db exports the ORM and types, and package web consumes it.

package B also should have the same version of the ORM in its dependencies

...or not have it in dependencies at all.

For those cases when orchid-orm is actually needed in the sibling package (e.g. I need testTransaction for tests), instead of managing the dependency of the same version, one can re-export orchid-orm with @mono/db/orchid-orm (or simply re-export the needed primitive).