Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to 12.8.0 #2090

Merged
merged 12 commits into from
Mar 29, 2020
Merged

Upgrade to 12.8.0 #2090

merged 12 commits into from
Mar 29, 2020

Conversation

solidnerd
Copy link
Collaborator

No description provided.

@dennisoderwald
Copy link

https://about.gitlab.com/releases/2020/02/24/gitlab-12-8-1-released/

@mankilowa
Copy link

Have anyone tried to build custom image with gitaly server version > 1.83.0?
I got this error message with 1.84.0-12.8.1:
remote: /home/git/gitaly/ruby/git-hooks/pre-receive: 4: exec: /gitaly-hooks: not found

@achretien
Copy link
Contributor

Yes I did with the same kind of fix than 5efa861

@martin-juul
Copy link

martin-juul commented Mar 6, 2020

Symlink gitlaly-hooks to / fixes it

# Fix for gitaly-hooks
echo "Linking $(command -v gitaly-hooks) to /"
ln -s "$(command -v gitaly-hooks)" /

@paolomainardi
Copy link
Contributor

hi @solidnerd are there any changes to have this merged ?

@solidnerd
Copy link
Collaborator Author

Currently with this release it isn‘t possible to run the migrations from a fresh installation.

@solidnerd
Copy link
Collaborator Author

Hey got anyone of you a clean version with 12.8.0 up & running.

@solidnerd
Copy link
Collaborator Author

Here is my current trace of the stack that doesn't come up.

Probably @splattael knows something ?

docker-compose up
Creating network "docker-gitlab_default" with the default driver
Creating volume "docker-gitlab_redis-data" with default driver
Creating volume "docker-gitlab_postgresql-data" with default driver
Creating volume "docker-gitlab_gitlab-data" with default driver
Creating docker-gitlab_postgresql_1 ... done
Creating docker-gitlab_redis_1      ... done
Creating docker-gitlab_gitlab_1     ... done
Attaching to docker-gitlab_postgresql_1, docker-gitlab_redis_1, docker-gitlab_gitlab_1
gitlab_1      | Loading /etc/docker-gitlab/runtime/env-defaults
postgresql_1  | Initializing datadir...
redis_1       | Starting redis-server...
postgresql_1  | Initializing certdir...
gitlab_1      | Initializing logdir...
postgresql_1  | Initializing logdir...
postgresql_1  | Initializing rundir...
redis_1       | 1:C 19 Mar 07:32:14.078 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1       | 1:C 19 Mar 07:32:14.078 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1       | 1:C 19 Mar 07:32:14.078 # Configuration loaded
postgresql_1  | Setting resolv.conf ACLs...
postgresql_1  | Initializing database...
redis_1       | 1:M 19 Mar 07:32:14.080 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1       | 1:M 19 Mar 07:32:14.080 # Server initialized
redis_1       | 1:M 19 Mar 07:32:14.080 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
gitlab_1      | Initializing datadir...
gitlab_1      | Generating OpenSSH host keys... RSA DSA ECDSA ED25519
gitlab_1      | Installing configuration templates...
gitlab_1      | Configuring gitlab...
postgresql_1  | Configuring hot standby...
postgresql_1  | ‣ Setting postgresql.conf parameter: wal_level = 'hot_standby'
postgresql_1  | ‣ Setting postgresql.conf parameter: max_wal_senders = '16'
postgresql_1  | ‣ Setting postgresql.conf parameter: checkpoint_segments = '8'
postgresql_1  | ‣ Setting postgresql.conf parameter: wal_keep_segments = '32'
postgresql_1  | ‣ Setting postgresql.conf parameter: hot_standby = 'on'
postgresql_1  | ‣ Setting postgresql.conf parameter: data_directory = '/var/lib/postgresql/10/main'
postgresql_1  | ‣ Setting postgresql.conf parameter: log_directory = '/var/log/postgresql'
postgresql_1  | ‣ Setting postgresql.conf parameter: log_filename = 'postgresql-10-main.log'
postgresql_1  | ‣ Setting postgresql.conf parameter: ssl = 'off'
postgresql_1  | Creating database user: gitlab
postgresql_1  | Creating database: gitlabhq_production...
postgresql_1  | ‣ Loading pg_trgm extension...
postgresql_1  | ‣ Granting access to gitlab user...
postgresql_1  | Starting PostgreSQL 10...
postgresql_1  | 2020-03-19 07:32:16.193 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
postgresql_1  | 2020-03-19 07:32:16.193 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgresql_1  | 2020-03-19 07:32:16.197 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgresql_1  | 2020-03-19 07:32:16.214 UTC [208] LOG:  database system was shut down at 2020-03-19 07:32:16 UTC
postgresql_1  | 2020-03-19 07:32:16.221 UTC [1] LOG:  database system is ready to accept connections
gitlab_1      | Configuring gitlab::database..
gitlab_1      | Configuring gitlab::redis
gitlab_1      | Configuring gitlab::secrets...
gitlab_1      | Configuring gitlab::sidekiq...
gitlab_1      | Configuring gitlab::gitaly...
gitlab_1      | Configuring gitlab::monitoring...
gitlab_1      | Configuring gitlab::gitlab-workhorse...
gitlab_1      | Configuring gitlab::unicorn...
gitlab_1      | Configuring gitlab::timezone...
gitlab_1      | Configuring gitlab::rack_attack...
gitlab_1      | Configuring gitlab::ci...
gitlab_1      | Configuring gitlab::artifacts...
gitlab_1      | Configuring gitlab::lfs...
gitlab_1      | Configuring gitlab::uploads...
gitlab_1      | Configuring gitlab::mattermost...
gitlab_1      | Configuring gitlab::project_features...
gitlab_1      | Configuring gitlab::oauth...
gitlab_1      | Configuring gitlab::ldap...
gitlab_1      | Configuring gitlab::cron_jobs...
gitlab_1      | Configuring gitlab::backups...
gitlab_1      | Configuring gitlab::backups::schedule...
gitlab_1      | Configuring gitlab::registry...
gitlab_1      | Configuring gitlab::pages...
gitlab_1      | Configuring gitlab::sentry...
gitlab_1      | Configuring gitlab-shell...
gitlab_1      | Configuring nginx...
gitlab_1      | Configuring nginx::gitlab...
gitlab_1      | Waiting for services coming up...
gitlab_1      | 2020-03-19 13:02:18,266 CRIT Supervisor running as root (no user in config file)
gitlab_1      | 2020-03-19 13:02:18,266 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
gitlab_1      | 2020-03-19 13:02:18,266 INFO Included extra file "/etc/supervisor/conf.d/gitaly.conf" during parsing
gitlab_1      | 2020-03-19 13:02:18,266 INFO Included extra file "/etc/supervisor/conf.d/gitlab-workhorse.conf" during parsing
gitlab_1      | 2020-03-19 13:02:18,266 INFO Included extra file "/etc/supervisor/conf.d/groups.conf" during parsing
gitlab_1      | 2020-03-19 13:02:18,267 INFO Included extra file "/etc/supervisor/conf.d/mail_room.conf" during parsing
gitlab_1      | 2020-03-19 13:02:18,267 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
gitlab_1      | 2020-03-19 13:02:18,267 INFO Included extra file "/etc/supervisor/conf.d/sidekiq.conf" during parsing
gitlab_1      | 2020-03-19 13:02:18,267 INFO Included extra file "/etc/supervisor/conf.d/sshd.conf" during parsing
gitlab_1      | 2020-03-19 13:02:18,267 INFO Included extra file "/etc/supervisor/conf.d/unicorn.conf" during parsing
gitlab_1      | 2020-03-19 13:02:18,274 INFO RPC interface 'supervisor' initialized
gitlab_1      | 2020-03-19 13:02:18,274 CRIT Server 'unix_http_server' running without any HTTP authentication checking
gitlab_1      | 2020-03-19 13:02:18,274 INFO supervisord started with pid 596
gitlab_1      | 2020-03-19 13:02:19,281 INFO spawned: 'gitaly' with pid 600
gitlab_1      | 2020-03-19 13:02:19,284 INFO spawned: 'gitlab-workhorse' with pid 601
gitlab_1      | 2020-03-19 13:02:19,286 INFO spawned: 'unicorn' with pid 602
gitlab_1      | 2020-03-19 13:02:19,289 INFO spawned: 'sidekiq' with pid 605
gitlab_1      | 2020-03-19 13:02:19,291 INFO spawned: 'nginx' with pid 609
gitlab_1      | 2020-03-19 13:02:19,294 INFO spawned: 'sshd' with pid 615
gitlab_1      | 2020-03-19 13:02:19,298 INFO spawned: 'cron' with pid 619
gitlab_1      | 2020-03-19 13:02:20,331 INFO success: gitaly entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_1      | 2020-03-19 13:02:20,331 INFO success: gitlab-workhorse entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_1      | 2020-03-19 13:02:20,331 INFO success: unicorn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_1      | 2020-03-19 13:02:20,332 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_1      | 2020-03-19 13:02:20,332 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_1      | 2020-03-19 13:02:20,332 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_1      | 2020-03-19 13:02:20,332 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_1      | Setting up GitLab for firstrun. Please be patient, this could take a while...
postgresql_1  | 2020-03-19 07:32:32.333 UTC [217] ERROR:  relation "feature_gates" does not exist at character 454
postgresql_1  | 2020-03-19 07:32:32.333 UTC [217] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),
postgresql_1  | 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,
postgresql_1  | 	       c.collname, col_description(a.attrelid, a.attnum) AS comment
postgresql_1  | 	  FROM pg_attribute a
postgresql_1  | 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum
postgresql_1  | 	  LEFT JOIN pg_type t ON a.atttypid = t.oid
postgresql_1  | 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation
postgresql_1  | 	 WHERE a.attrelid = '"feature_gates"'::regclass
postgresql_1  | 	   AND a.attnum > 0 AND NOT a.attisdropped
postgresql_1  | 	 ORDER BY a.attnum
postgresql_1  |
postgresql_1  | 2020-03-19 07:32:32.404 UTC [218] ERROR:  relation "feature_gates" does not exist at character 454
postgresql_1  | 2020-03-19 07:32:32.404 UTC [218] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),
postgresql_1  | 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,
postgresql_1  | 	       c.collname, col_description(a.attrelid, a.attnum) AS comment
postgresql_1  | 	  FROM pg_attribute a
postgresql_1  | 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum
postgresql_1  | 	  LEFT JOIN pg_type t ON a.atttypid = t.oid
postgresql_1  | 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation
postgresql_1  | 	 WHERE a.attrelid = '"feature_gates"'::regclass
postgresql_1  | 	   AND a.attnum > 0 AND NOT a.attisdropped
postgresql_1  | 	 ORDER BY a.attnum
postgresql_1  |
postgresql_1  | 2020-03-19 07:32:42.384 UTC [220] ERROR:  relation "feature_gates" does not exist at character 454
postgresql_1  | 2020-03-19 07:32:42.384 UTC [220] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),
postgresql_1  | 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,
postgresql_1  | 	       c.collname, col_description(a.attrelid, a.attnum) AS comment
postgresql_1  | 	  FROM pg_attribute a
postgresql_1  | 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum
postgresql_1  | 	  LEFT JOIN pg_type t ON a.atttypid = t.oid
postgresql_1  | 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation
postgresql_1  | 	 WHERE a.attrelid = '"feature_gates"'::regclass
postgresql_1  | 	   AND a.attnum > 0 AND NOT a.attisdropped
postgresql_1  | 	 ORDER BY a.attnum
postgresql_1  |
postgresql_1  | 2020-03-19 07:32:47.050 UTC [223] ERROR:  relation "features" does not exist at character 454
postgresql_1  | 2020-03-19 07:32:47.050 UTC [223] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),
postgresql_1  | 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,
postgresql_1  | 	       c.collname, col_description(a.attrelid, a.attnum) AS comment
postgresql_1  | 	  FROM pg_attribute a
postgresql_1  | 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum
postgresql_1  | 	  LEFT JOIN pg_type t ON a.atttypid = t.oid
postgresql_1  | 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation
postgresql_1  | 	 WHERE a.attrelid = '"features"'::regclass
postgresql_1  | 	   AND a.attnum > 0 AND NOT a.attisdropped
postgresql_1  | 	 ORDER BY a.attnum
postgresql_1  |
gitlab_1      | rake aborted!
gitlab_1      | ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "features" does not exist
gitlab_1      | LINE 8:  WHERE a.attrelid = '"features"'::regclass
gitlab_1      |                             ^
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:65:in `exec'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:65:in `block (2 levels) in query'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:64:in `block in query'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/abstract_adapter.rb:718:in `block (2 levels) in log'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/abstract_adapter.rb:717:in `block in log'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/abstract_adapter.rb:708:in `log'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:63:in `query'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/postgresql_adapter.rb:817:in `column_definitions'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/abstract/schema_statements.rb:114:in `columns'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/schema_cache.rb:76:in `columns'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/schema_cache.rb:82:in `columns_hash'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/model_schema.rb:490:in `load_schema!'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/attributes.rb:247:in `load_schema!'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/attribute_decorators.rb:51:in `load_schema!'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/model_schema.rb:480:in `block in load_schema'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/model_schema.rb:477:in `load_schema'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/model_schema.rb:352:in `columns_hash'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/inheritance.rb:82:in `descends_from_active_record?'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/inheritance.rb:88:in `finder_needs_type_condition?'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/core.rb:298:in `relation'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/scoping/named.rb:57:in `default_scoped'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/scoping/named.rb:45:in `all'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/querying.rb:21:in `pluck'
gitlab_1      | /home/git/gitlab/lib/feature.rb:15:in `feature_names'
gitlab_1      | /home/git/gitlab/lib/feature.rb:42:in `block in persisted_names'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:331:in `block in fetch'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:716:in `block in save_block_result_to_cache'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:685:in `block in instrument'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/notifications.rb:182:in `instrument'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:685:in `instrument'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:715:in `save_block_result_to_cache'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:331:in `fetch'
gitlab_1      | /home/git/gitlab/lib/feature.rb:41:in `persisted_names'
gitlab_1      | /home/git/gitlab/lib/feature/gitaly.rb:17:in `server_feature_flags'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client.rb:236:in `request_kwargs'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client.rb:165:in `execute'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client.rb:146:in `block in call'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client.rb:174:in `measure_timings'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client.rb:145:in `call'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client/server_service.rb:14:in `info'
gitlab_1      | /home/git/gitlab/lib/tasks/gitlab/setup.rake:10:in `block in check_gitaly_connection'
gitlab_1      | /home/git/gitlab/lib/tasks/gitlab/setup.rake:9:in `each'
gitlab_1      | /home/git/gitlab/lib/tasks/gitlab/setup.rake:9:in `check_gitaly_connection'
gitlab_1      | /home/git/gitlab/lib/tasks/gitlab/setup.rake:4:in `block (2 levels) in <top (required)>'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
gitlab_1      |
gitlab_1      | Caused by:
gitlab_1      | PG::UndefinedTable: ERROR:  relation "features" does not exist
gitlab_1      | LINE 8:  WHERE a.attrelid = '"features"'::regclass
gitlab_1      |                             ^
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:65:in `exec'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:65:in `block (2 levels) in query'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:64:in `block in query'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/abstract_adapter.rb:718:in `block (2 levels) in log'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/abstract_adapter.rb:717:in `block in log'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/abstract_adapter.rb:708:in `log'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:63:in `query'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/postgresql_adapter.rb:817:in `column_definitions'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/abstract/schema_statements.rb:114:in `columns'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/schema_cache.rb:76:in `columns'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/connection_adapters/schema_cache.rb:82:in `columns_hash'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/model_schema.rb:490:in `load_schema!'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/attributes.rb:247:in `load_schema!'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/attribute_decorators.rb:51:in `load_schema!'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/model_schema.rb:480:in `block in load_schema'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/model_schema.rb:477:in `load_schema'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/model_schema.rb:352:in `columns_hash'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/inheritance.rb:82:in `descends_from_active_record?'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/inheritance.rb:88:in `finder_needs_type_condition?'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/core.rb:298:in `relation'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/scoping/named.rb:57:in `default_scoped'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/scoping/named.rb:45:in `all'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.2/lib/active_record/querying.rb:21:in `pluck'
gitlab_1      | /home/git/gitlab/lib/feature.rb:15:in `feature_names'
gitlab_1      | /home/git/gitlab/lib/feature.rb:42:in `block in persisted_names'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:331:in `block in fetch'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:716:in `block in save_block_result_to_cache'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:685:in `block in instrument'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/notifications.rb:182:in `instrument'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:685:in `instrument'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:715:in `save_block_result_to_cache'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.2/lib/active_support/cache.rb:331:in `fetch'
gitlab_1      | /home/git/gitlab/lib/feature.rb:41:in `persisted_names'
gitlab_1      | /home/git/gitlab/lib/feature/gitaly.rb:17:in `server_feature_flags'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client.rb:236:in `request_kwargs'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client.rb:165:in `execute'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client.rb:146:in `block in call'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client.rb:174:in `measure_timings'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client.rb:145:in `call'
gitlab_1      | /home/git/gitlab/lib/gitlab/gitaly_client/server_service.rb:14:in `info'
gitlab_1      | /home/git/gitlab/lib/tasks/gitlab/setup.rake:10:in `block in check_gitaly_connection'
gitlab_1      | /home/git/gitlab/lib/tasks/gitlab/setup.rake:9:in `each'
gitlab_1      | /home/git/gitlab/lib/tasks/gitlab/setup.rake:9:in `check_gitaly_connection'
gitlab_1      | /home/git/gitlab/lib/tasks/gitlab/setup.rake:4:in `block (2 levels) in <top (required)>'
gitlab_1      | /home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
gitlab_1      | Tasks: TOP => gitlab:setup
gitlab_1      | (See full trace by running task with --trace)

@splattael
Copy link
Contributor

@solidnerd That's odd!

I am not sure if it's related but I found some ancient issues describing this problem:

https://gitlab.com/gitlab-org/gitlab-foss/-/issues/38947
https://gitlab.com/gitlab-org/gitlab-foss/-/issues/34822

Are we missing a step? Migration e.g? Just checking 😅

@Ilya-Kuchaev
Copy link

Any news here?

@splattael
Copy link
Contributor

@solidnerd Can you try this?

diff --git a/lib/feature.rb b/lib/feature.rb
index 60a5c03a839..2a89a3e48b3 100644
--- a/lib/feature.rb
+++ b/lib/feature.rb
@@ -33,6 +33,7 @@ class Feature
 
     def persisted_names
       return [] unless Gitlab::Database.exists?
+      return [] unless Feature::FlipperFeature.table_exists?
 
       Gitlab::SafeRequestStore[:flipper_persisted_names] ||=
         begin

@paolomainardi
Copy link
Contributor

@splattael i can test that, but i don't see that patch in the upstream gitlab. Can you please describe it a bit ?

@paolomainardi
Copy link
Contributor

@solidnerd
Copy link
Collaborator Author

Hey @splattael and @paolomainardi,
I can confirm that this will mitigate the problem of a fresh install. I think this error won't occur on the upgrade way. I'll start tomorrow with the release train for at least 12.8.x . I think for 12.9 a little bit more work is needed regarding the puma change.

@solidnerd
Copy link
Collaborator Author

The problem is that this bug doesn't exist on upstream or wasn't reported until now. I'll do this as well.

@paolomainardi
Copy link
Contributor

Thanks a lot @solidnerd, i'm trying if the same bug exist as well on 12.8.2. How safe it is to run this release on a existing database ?

@solidnerd
Copy link
Collaborator Author

My last test was that I test this change until 12.8.6 or 12.8.7 and the error still occurs . So what I currently see is that there isn't a report on the upstream repository. I think this won't happen on the upgrade way. But If you want to go sure wait for the release by end of the day . I'll also test the upgrade way.

@paolomainardi
Copy link
Contributor

Thanks a lot as always @solidnerd, you rock!

@splattael
Copy link
Contributor

@splattael i can test that, but i don't see that patch in the upstream gitlab. Can you please describe it a bit ?

@paolomainardi Yes, sorry for just dropping the patch without any comment.

@solidnerd and I had a quick video call (also) about this issue and found out that this bug won't happen if you follow GitLab's installation guide https://docs.gitlab.com/ee/install/README.html#installation-core-only (omnibus or from source) because they don't expect the database to exist before running rake gitlab:setup. docker-gitlab, however, runs a container with postgresql which already creates a database on start-up - so before rake gitlab:setup is run 💥

I see two options to solve this:

  1. Don't let postgresql create the database
  2. Handle this case in GitLab's code as well

We can try to upstream option 2 otherwise we have to go with option 1 🤷‍♂️

Thanks @solidnerd for testing the patch 💪

@solidnerd
Copy link
Collaborator Author

solidnerd commented Mar 28, 2020 via email

@solidnerd solidnerd mentioned this pull request Mar 29, 2020
@solidnerd solidnerd merged commit 67a5fc3 into sameersbn:master Mar 29, 2020
@solidnerd solidnerd deleted the upgrade-to-12.8.0 branch March 29, 2020 13:29
@paolomainardi
Copy link
Contributor

Thanks a lof @splattael for the very detailed explanation.

@splattael
Copy link
Contributor

@solidnerd Upgrade to 12.8.8 ((yeah, I know wrong MR :neckbeard:) went smoothly without any issues 👍 Well done and thank you again ❤️

@splattael
Copy link
Contributor

@solidnerd Before creating an upstream fix, can you comment on this https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28270?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants