Generating single file for each directory in the queries directory

[abbazs]

As the project may grow complex generating a single file for each query is cumbersome to use them. And having all in one file for a large project is also cumbersome.

I want to propose the idea:
If the following is the queries folder structure:

queries
├── hero
│   ├── delete.edgeql
│   └── insert.edgeql
└── movies
    ├── delete.edgeql
    ├── insert.edgeql
    ├── select.edgeql
    └── update.edgeql

With an option to generate a python file for each directory under the queries directory, the generated file shall look like this:

from __future__ import annotations
import dataclasses
import edgedb
import uuid


@dataclasses.dataclass
class HeroDeleteResult:
    id: uuid.UUID


@dataclasses.dataclass
class HeroInsertResult:
    id: uuid.UUID
    name: str
    secret_identity: str | None

# Observe hero_ is prefixed to the query delete.edgeql
def hero_delete(
    executor: edgedb.Executor,
    *,
    id: uuid.UUID,
) -> HeroDeleteResult | None:
    return executor.query_single(
        """\
        with module default
        delete (select Hero filter .id=<uuid>$id)\
        """,
        id=id,
    )

# Observe hero_ is prefixed to the query insert.edgeql
def hero_insert(
    executor: edgedb.Executor,
    *,
    name: str,
    si: str,
) -> HeroInsertResult:
    return executor.query_single(
        """\
        with module default
        select (
            insert Hero {
                name:=<str>$name,
                secret_identity:=<str>$si,
            }
        ){
            id,
            name,
            secret_identity
        };\
        """,
        name=name,
        si=si,
    )

Automatically prefix the folder name before each method in the file generated.

I've modified the local installation of edgedb-python for my convenience of using it in a project and achieved the desired result, albeit I need to run the command for each folder under the queries folder manually.

I've modified the edgedb\codegen\generator.py and edgedb\codegen\cli.py. Added an option called --cwd-file in cli.py and then for the option modified the generator.py to get the desired result.
I see this issue can be clubbed with #410

[abbazs]

Why is the default mode (one Python file per .edgeql file) disliked here?

Consider the queries folder structure as follows:

queries
├── hero
│   ├── delete.edgeql
│   ├── insert.edgeql
│   └── select_all.edgeql
└── movies
   ├── delete.edgeql
   ├── insert.edgeql
   ├── select.edgeql
   └── update.edgeql

When I generate a python for file each edgql file the queries folder shall look like this:

queries
├── hero
│   ├── delete.edgeql
│   ├── delete_edgeql.py
│   ├── insert.edgeql
│   ├── insert_edgeql.py
│   ├── select_all.edgeql
│   └── select_all_edgeql.py
└── movies
    ├── delete.edgeql
    ├── delete_edgeql.py
    ├── insert.edgeql
    ├── insert_edgeql.py
    ├── select.edgeql
    ├── select_edgeql.py
    ├── update.edgeql
    └── update_edgeql.py

And let's say I need to use the queries/movies/delete_edgeql.py it needs to be imported as follows:

from queries.movies import delete_edgeql

And in the same location if I have to import queries/hero/delete_edgeql.py it is going to conflict.

# This is not possible, the last delete_edgeql overrides all the previous ones.
from queries.movies import delete_edgeql
from queries.hero import delete_edgeql

And it is not possible to make an import like:

# This doesn't work
from queries import movies.delete_edgeql
from queries import hero.delete_edgeql

Whereas single python file per folder in the queries folder works fine:

queries
├── hero
│   ├── delete.edgeql
│   ├── hero_edgeql.py
│   ├── insert.edgeql
│   └── select_all.edgeql
└── movies
    ├── delete.edgeql
    ├── insert.edgeql
    ├── movies_edgeql.py
    ├── select.edgeql
    └── update.edgeql

Here is how it works;

import edgedb
from queries.movies import movies_edgeql
from queries.hero import hero_edgeql

client = edgedb.create_client()

hero_edgeql.insert(client, name="Stepper", si="Eyewear")
# Object{id := UUID('48db2118-9d66-11ed-9cec-63ae220dadc2'), name := 'Stepper', secret_identity := 'Eyewear'}
movies_edgeql.insert(client,title="Golen Eye", release_year=2000)
# Object{id := UUID('6a0846cc-9d66-11ed-9cec-67c377ef462e'), title := 'Golen Eye', release_year := 2000}
movies_edgeql.update(client,id="6a0846cc-9d66-11ed-9cec-67c377ef462e",person="48db2118-9d66-11ed-9cec-63ae220dadc2")
# Object{id := UUID('6a0846cc-9d66-11ed-9cec-67c377ef462e'), title := 'Golen Eye', release_year := 2000}

Alright, what if I rename the edgeql files with the prefix of folder names as follows:

queries
├── hero
│   ├── hero_delete.edgeql
│   ├── hero_delete_edgeql.py
│   ├── hero_insert.edgeql
│   ├── hero_insert_edgeql.py
│   ├── hero_select_all.edgeql
│   └── hero_select_all_edgeql.py
└── movies
    ├── movies_delete.edgeql
    ├── movies_delete_edgeql.py
    ├── movies_insert.edgeql
    ├── movies_insert_edgeql.py
    ├── movies_select.edgeql
    ├── movies_select_edgeql.py
    ├── movies_update.edgeql
    └── movies_update_edgeql.py

This works and it is not as clean as one file per module:

import edgedb
from queries.hero import hero_insert_edgeql
client = edgedb.create_client()
hero_insert_edgeql.hero_insert(client, name="Radomos Brader", si="The real mosquito")

This also works and again it is not clean as one file per module:

from queries.hero.hero_insert_edgeql import hero_insert

Indeed the following is much cleaner:

from queries.hero import hero_edgeql
hero_edgeql.insert(client, name="Stepper", si="Eyewear")

Every language has its own advantage and while we maintain consistency across different language implementations, most will appreciate if we allow the users to play the advantage of their preferred language.

[neda-dtu]

I have created a fork, which would allow this work flow to be used. It adds an optional command line argument called edgeql_path. This is then the directory that you want to search for edgeql files within. So in your example, you could run the codegen script twice, once for hero and once for movies to make the hero_edgeql.py and movies_edgeql.py files.

I haven't done much testing, but this helped me with being able to have edgeql queries in a submodule to a parent project, which was what I needed.