Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow schema file only generation #57

Merged
merged 5 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 30 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,15 @@ This means, the option "memory" for connections is no longer available, and you

## How It Works

1. SurrealDB Schema Generator connects to your specified database
2. If you provide a surql schema file, the SurrealQL schema is written to the database
3. The generator extracts the `DEFINE` information from the connected database
4. Based on the definitions found in the database, the zod schemas are generated
1. If you provide a surql schema file:
- An in-memory SurrealDB instance is automatically created.
- The schema is loaded into this temporary instance.
- Docker is required to run the temporary instance.
2. If no schema file is provided:
- SurrealDB Schema Generator connects to your specified database.
3. The generator extracts the `DEFINE` information from the connected database (either in-memory or external).
4. Based on the definitions found in the database, the zod schemas are generated.

hammo92 marked this conversation as resolved.
Show resolved Hide resolved

Enjoy using SurrealDB Schema Generator to streamline your schema generation process for SurrealDB and zod.
It's designed to make your life easier when working with these powerful technologies.
Expand Down Expand Up @@ -99,6 +104,27 @@ Example:
}
```

## Using a Schema File
> **_NOTE:_** Docker is required to run SurrealDB in memory.

To use a schema file either provide the -f flag:
```bash
surql-gen -f ./path/to/your/schema.surql
```

or you can specify the path in the config file:
```json
{
"schemaFile": "./path/to/your/schema.surql"
}
```

## Connecting to an Existing SurrealDB Instance

To connect to an existing SurrealDB instance, simply omit the `-f` option, or omit the `schemaFile` in the config file.

In this case, you need to provide the connection information for your running instance.

## Code Generation Structure

The generated code is organized into two distinct parts for your convenience:
Expand Down
19 changes: 16 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,17 @@
"node": ">=18"
},
"description": "A small tool which generates a typescript client for SurrealDB based on the schema of a given database",
"keywords": ["typescript", "surrealdb", "client", "javascript", "zod", "orm", "database", "generator", "tool"],
"keywords": [
"typescript",
"surrealdb",
"client",
"javascript",
"zod",
"orm",
"database",
"generator",
"tool"
],
"author": {
"name": "Sebastian Wessel",
"url": "https://github.com/sebastianwessel"
Expand All @@ -21,7 +31,9 @@
"email": "[email protected]"
},
"license": "MIT",
"files": ["dist"],
"files": [
"dist"
],
"bin": {
"surql-gen": "./dist/commonjs/index.js"
},
Expand All @@ -37,8 +49,8 @@
"devDependencies": {
"@biomejs/biome": "^1.8.3",
"@types/node": "^20.14.10",
"tshy": "^3.0.2",
"jsr": "^0.12.4",
"tshy": "^3.0.2",
"tsx": "^4.15.7",
"typescript": "^5.5.2",
"vitest": "^2.0.1"
Expand All @@ -48,6 +60,7 @@
"mkdirp": "^3.0.1",
"rimraf": "^6.0.0",
"surrealdb.js": "^1.0.0-beta.9",
"testcontainers": "^10.11.0",
"zod": "^3.23.8"
},
"tshy": {
Expand Down
53 changes: 46 additions & 7 deletions src/database/db.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import { Surreal } from 'surrealdb.js'
import { GenericContainer, StartedTestContainer, Wait } from 'testcontainers'

import type { Config } from '../config/types.js'

let db: Surreal
let container: StartedTestContainer | null = null

export const getDb = () => {
if (db) {
Expand All @@ -11,10 +13,39 @@ export const getDb = () => {
throw new Error('Not connected to a database')
}

export const connectDb = async (config: Config) => {
console.log('connect to database')
export const connectDb = async (config: Config, createInstance: boolean = false) => {
if (createInstance) {
console.log('Starting temporary SurrealDB instance')
container = await new GenericContainer('surrealdb/surrealdb:latest')
.withExposedPorts(8000)
.withCommand(['start', '--user', config.username, '--pass', config.password, 'memory'])
.withWaitStrategy(Wait.forLogMessage('Started web server'))
.start()

const port = container.getMappedPort(8000)
const host = container.getHost()
config.surreal = `http://${host}:${port}`
console.log(`Temporary SurrealDB instance started at ${config.surreal}`)
}

console.log('Connecting to database')
db = new Surreal()
await db.connect(config.surreal)

let retries = 5
while (retries > 0) {
try {
await db.connect(config.surreal)
break
} catch (error) {
console.log(`Connection failed. Retrying... (${retries} attempts left)`)
retries--
if (retries === 0) {
throw error
}
await new Promise(resolve => setTimeout(resolve, 1000))
}
}

await db.use({
namespace: config.ns,
database: config.db,
Expand All @@ -23,14 +54,22 @@ export const connectDb = async (config: Config) => {
username: config.username,
password: config.password,
})
console.log('Connected to database successfully')
}

export const insertDefinitions = async (content: string) => {
const db = getDb()
const _result = await db.query(content, {})
console.log('definitions written to database')
await db.query(content, {})
console.log('Definitions written to database')
}

export const closeDb = async () => {
console.log('database closed')
}
if (db) {
await db.close()
}
if (container) {
await container.stop()
console.log('Temporary SurrealDB instance stopped')
}
console.log('Database connection closed')
}
11 changes: 0 additions & 11 deletions src/genSchema/generateZodSchemaCode.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,8 @@ describe('generateZodSchemaCode', () => {
.split(';')
.filter(x => x.trim().length)
.map(def => getDetailsFromDefinition(def, false))
console.log('fields', fields)
const generatedSchema = generateZodSchemaCode(fields, 'schema')

console.log('generatedSchema', generatedSchema)

expect(generatedSchema).toEqualIgnoringWhitespace(`
const schema = z.object({
review: z.object({
Expand All @@ -61,11 +58,8 @@ describe('generateZodSchemaCode', () => {
.split(';')
.filter(x => x.trim().length)
.map(def => getDetailsFromDefinition(def, false))
console.log('fields', fields)
const generatedSchema = generateZodSchemaCode(fields, 'schema')

console.log('generatedSchema', generatedSchema)

expect(generatedSchema).toEqualIgnoringWhitespace(`
const schema = z.object({
review: z.object({
Expand All @@ -86,10 +80,8 @@ describe('generateZodSchemaCode', () => {
.split(';')
.filter(x => x.trim().length)
.map(def => getDetailsFromDefinition(def, false))
console.log('fields', fields)
const generatedSchema = generateZodSchemaCode(fields, 'schema')

console.log('generatedSchema', generatedSchema)

expect(generatedSchema).toEqualIgnoringWhitespace(`
const schema = z.object({
Expand Down Expand Up @@ -154,11 +146,8 @@ describe('generateZodSchemaCode', () => {
.split(';')
.filter(x => x.trim().length)
.map(def => getDetailsFromDefinition(def, false))
console.log('fields', fields)
const generatedSchema = generateZodSchemaCode(fields, 'schema')

console.log('generatedSchema', generatedSchema)

expect(generatedSchema).toEqualIgnoringWhitespace(`
const schema = z.object({
name: z.string(),
Expand Down
1 change: 0 additions & 1 deletion src/genSchema/generateZodSchemaCode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ const createRegex = (key: string) => {
}

export const generateZodSchemaCode = (fields: FieldDetail[], schemaName: string): string => {
console.log('fields', fields)
// biome-ignore lint/suspicious/noExplicitAny: <explanation>
const buildSchema = (fieldMap: { [key: string]: any }, fields: FieldDetail[]) => {
for (const field of fields) {
Expand Down
65 changes: 32 additions & 33 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { resolve } from 'node:path'
import { program } from 'commander'

import { configFileSchema } from './config/configFileSchema.js'
import { closeDb, connectDb, insertDefinitions } from './database/db.js'
import { connectDb, closeDb, insertDefinitions } from './database/db.js'
import { getAllTableInfo } from './database/getAllTableInfo.js'
import { generateClientJs } from './genClient/generateClientJs.js'
import { generateTableSchema } from './genSchema/generateTableSchema.js'
Expand All @@ -14,7 +14,7 @@ import { printSorry } from './helper/printSorry.js'
const main = async () => {
program
.name('surql-gen')
.description('Generate zod schema and typescript client code from running Surreal database')
.description('Generate zod schema and typescript client code from running Surreal database or schema file')
.version('1.0.0')

program
Expand Down Expand Up @@ -63,52 +63,51 @@ const main = async () => {

const config = configFileSchema.parse({ ...options, ...fileContent })

await connectDb(config)
if (config.schemaFile) {
const schemaFilePath = resolve(__dirname, config.schemaFile)
let surQLContent: Buffer
try {
surQLContent = await readFile(schemaFilePath)
} catch (error) {
const err = error as Error & { code?: string }
if (err.code === 'ENOENT') {
console.error('')
console.error('Unable to find schema file', schemaFilePath)
console.error('Please check!')
console.error('')
process.exit(1)
} else {
console.error('')
console.error('Please have a look at your config file!')
console.error('Looks like, your configuration file is invalid.')
console.error('')
throw new Error(`Invalid configuration: ${err.message}`)
try {
if (config.schemaFile) {
await connectDb(config, true)
const schemaFilePath = resolve(__dirname, config.schemaFile)
let schemaContent: string
try {
schemaContent = await readFile(schemaFilePath, 'utf-8')
} catch (error) {
const err = error as Error & { code?: string }
if (err.code === 'ENOENT') {
console.error('')
console.error('Unable to find schema file', schemaFilePath)
console.error('Please check!')
console.error('')
process.exit(1)
} else {
throw new Error(`Error reading schema file: ${err.message}`)
}
}
}

try {
await insertDefinitions(surQLContent.toString())
} catch (error) {
printSorry(error)
process.exit(1)
try {
await insertDefinitions(schemaContent)
} catch (error) {
printSorry(error)
process.exit(1)
}
}else{
await connectDb(config)
}
}

try {
const tableInfo = await getAllTableInfo()

await generateTableSchema(resolve(__dirname, config.outputFolder), tableInfo)

if (config.generateClient) {
await generateClientJs(resolve(__dirname, config.outputFolder), Object.keys(tableInfo), 'surrealdb.js')
}

} catch (error) {
printSorry(error)
process.exit(1)
} finally {
await closeDb()
}

await closeDb()

console.log('')
console.log('')
console.log('Thanks for using my tool')
Expand All @@ -124,4 +123,4 @@ const main = async () => {
process.exit()
}

main()
main()