Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgres schema support #12505

Closed
5 tasks
osbeker opened this issue Aug 17, 2020 · 7 comments · Fixed by #12634
Closed
5 tasks

Postgres schema support #12505

osbeker opened this issue Aug 17, 2020 · 7 comments · Fixed by #12634
Labels

Comments

@osbeker
Copy link

osbeker commented Aug 17, 2020

  • Gitea version (or commit ref): 1.12.3
  • Git version:
  • Operating system: CentOS 8
  • Database (use [x]):
    • [x ] PostgreSQL
    • MySQL
    • MSSQL
    • SQLite
  • Can you reproduce the bug at https://try.gitea.io:
    • Yes (provide example URL)
    • No
    • [x ] Not relevant
  • Log gist:
    2020/08/17 06:59:17 ...m.io/xorm/core/db.go:154:QueryContext() [I] [SQL] SELECT user_id, count(*) AS count FROM notification WHERE user_id IN (SELECT user_id FROM notification WHERE updated_unix >= $1 AND updated_unix < $2) AND status = $3 GROUP BY user_id [1597661945 1597661955 1] - 260.505µs
    2020/08/17 06:59:17 ...ource/manager_run.go:41:Run() [E] Unable to get UIDcounts: pq: relation "notification" does not exist

Description

I'm using Postgres (11) and I want to use a schema (-> I don't want my data in a public schema in my cluster and I may want to have dev and test and prod instances)
I created a schema "prod" and configured it. After generating the schema, which seems successful, I soon get a "500" in the UI.
I look into the log and see this:
...
2020/08/17 06:59:17 ...m.io/xorm/core/db.go:154:QueryContext() [I] [SQL] SELECT user_id, count(*) AS count FROM notification WHERE user_id IN (SELECT user_id FROM notification WHERE updated_unix >= $1 AND updated_unix < $2) AND status = $3 GROUP BY user_id [1597661945 1597661955 1] - 260.505µs
2020/08/17 06:59:17 ...ource/manager_run.go:41:Run() [E] Unable to get UIDcounts: pq: relation "notification" does not exist
...

Looking at the sql that was generated, I have to agree - there is not table "notification": that is because there only is a prod.notification.

I assume the problem is you do not qualify the table names properly for schema support?

Screenshots

@lunny
Copy link
Member

lunny commented Aug 17, 2020

I think we have support schema, you should set it in

[database]
SCHEMA=

@osbeker
Copy link
Author

osbeker commented Aug 17, 2020 via email

@zeripath
Copy link
Contributor

I wonder if we simply need to do "SET search_path = ?", setting.Database.Schema in models/models.go:140:func getEngine()

@zeripath
Copy link
Contributor

zeripath commented Aug 29, 2020

Actually that would probably suffice. We could prepend the search_path for maximum portability though:

db.Exec(`SELECT set_config(
    'search_path',
    ? || ',' || current_setting('search_path'),
    false
) WHERE current_setting('search_path') !~ '(^|,)'|| ? || '(,|$)';`, settting.Database.Schema, settting.Database.Schema)

In the end I've dropped the WHERE because we should just always prepend - we always want to be looking in our schema first.

zeripath added a commit to zeripath/gitea that referenced this issue Aug 29, 2020
Rather than rely on the user running the gitea server and db setting the
schema search_path correctly - if gitea is run with a schema we should
simply set the search_path to have that schema first in the path.

Fix go-gitea#12505

Signed-off-by: Andrew Thornton <[email protected]>
zeripath added a commit that referenced this issue Sep 11, 2020
Rather than rely on the user running the gitea server and db setting the
schema search_path correctly - if gitea is run with a schema we should
simply set the search_path to have that schema first in the path.

Fix #12505

Signed-off-by: Andrew Thornton <[email protected]>
@dmayle
Copy link

dmayle commented Nov 13, 2020

I've tried installing gitea 1.13.0-rc2 (from Docker Hub) and I'm still seeing problems with this. For the moment, the logs still show issues with certain queries:

2020/11/13 16:02:03 ...m.io/xorm/core/db.go:286:afterProcess() [I] [SQL] SELECT count(*) FROM "gitea"."repository" WHERE (owner_id=$1 OR (owner_id<>$2 AND ("repository".id IN (SELECT "access".repo_id FROM access WHERE "access".user_id=$3) OR "repository".id IN (SELECT "team_repo".repo_id FROM team_repo INNER JOIN team_user ON "team_user".team_id = "team_repo".team_id WHERE "team_user".uid=$4) OR ("repository".is_private=$5 AND "repository".owner_id IN (SELECT "org_user".org_id FROM org_user INNER JOIN "user" ON "user".id = "org_user".org_id WHERE "org_user".uid=$6 AND "user".type=$7 AND "user".visibility=$8)))) OR (is_private=$9 AND owner_id IN (SELECT "user".id FROM "user" WHERE "user".visibility=$10)) OR (is_private=$11 AND owner_id IN (SELECT "user".id FROM "user" WHERE "user".visibility=$12))) [2 2 2 2 false 2 1 private false public false limited] - 4.640044ms
2020/11/13 16:02:03 routers/home.go:155:RenderRepoSearch() [E] SearchRepository: Count: pq: relation "access" does not exist

and also:

2020/11/13 16:02:09 ...m.io/xorm/core/db.go:286:afterProcess() [I] [SQL] SELECT user_id, count(*) AS count FROM notification WHERE user_id IN (SELECT user_id FROM notification WHERE updated_unix >= $1 AND updated_unix < $2) AND status = $3 GROUP BY user_id [1605283317 1605283327 1] - 2.815717ms
2020/11/13 16:02:09 ...ource/manager_run.go:41:Run() [E] Unable to get UIDcounts: pq: relation "notification" does not exist

@zeripath
Copy link
Contributor

Since #12634 your search path should be being set in such a way that this does not matter.

It would be useful to ensure that all of these tables are in the gitea schema and that you have not set something in such a way that setting set_config is not helpful.

It would also be helpful to tell us the version of postgres you are using and to check if SELECT set_config('search_path', ? || ',' || current_setting('search_path'), false) works on that version of postgres.

@dmayle
Copy link

dmayle commented Nov 15, 2020

@zeripath I was also having problems because my memcached was set to use authentication, which gitea doesn't support. After updating this, the DBAs reinitialized the gitea database and realized they made a mistake in their config. We create schemas and users with a search path predefined. Once we did that, everything started working, even with 1.12.5

@go-gitea go-gitea locked and limited conversation to collaborators Nov 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants