Backup is not restored

[MaxOOOOON]

Self-Hosted Version

23.7.0

CPU Architecture

x86_64

Docker Version

24.0.5

Docker Compose Version

2.20.2

Steps to Reproduce

1. perform a backup using the command ./scripts/backup.sh
2. configure the same version sentry on another host: git clone repo and ./install.sh
3. copy backup.json in sentry directory
4. run ./scripts/restore.sh

Expected Result

backup restored successfully

Actual Result

ls -1 sentry_restore_log-2023-08-24_11-29-30.txt | tail -1 | xargs cat
▶ Initializing Docker Compose ...

▶ Detecting Docker platform
Detected Docker platform is linux/amd64

▶ Setting up error handling ...
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 292B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/debian:bullseye-slim
#3 DONE 1.2s

#4 [1/2] FROM docker.io/library/debian:bullseye-slim@sha256:61386e11b5256efa33823cbfafd668dd651dbce810b24a8fb7b2e32fa7f65a85
#4 DONE 0.0s

#5 [2/2] RUN set -x && apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends jq && apt-get clean && rm -rf /var/lib/apt/lists/*
#5 CACHED

#6 exporting to image
#6 exporting layers done
#6 writing image sha256:0872a3d29236f66c8d42cb80fad87390e21a073bbf5736f247a575aa86b627d3 done
#6 naming to docker.io/library/sentry-self-hosted-jq-local done
#6 DONE 0.0s

Hey, so ... we would love to automatically find out about issues with your
Sentry instance so that we can improve the product. Turns out there is an app
for that, called Sentry. Would you be willing to let us automatically send data
about your instance upstream to Sentry for development and debugging purposes?

y / yes / 1
n / no / 0

(Btw, we send this to our own self-hosted Sentry instance, not to Sentry SaaS,
so that we can be in this together.)

Here's the info we may collect:

• OS username
• IP address
• install log
• runtime errors
• performance data

Thirty (30) day retention. No marketing. Privacy policy at sentry.io/privacy.

y or n?
Understood. To avoid this prompt in the future, use one of these flags:

--report-self-hosted-issues
--no-report-self-hosted-issues

or set the REPORT_SELF_HOSTED_ISSUES environment variable:

REPORT_SELF_HOSTED_ISSUES=1 to send data
REPORT_SELF_HOSTED_ISSUES=0 to not send data

Container sentry-self-hosted-smtp-1 Created
Container sentry-self-hosted-memcached-1 Created
Container sentry-self-hosted-redis-1 Created
Container sentry-self-hosted-postgres-1 Created
Container sentry-self-hosted-clickhouse-1 Created
Container sentry-self-hosted-symbolicator-1 Created
Container sentry-self-hosted-zookeeper-1 Created
Container sentry-self-hosted-kafka-1 Created
Container sentry-self-hosted-snuba-subscription-consumer-events-1 Created
Container sentry-self-hosted-snuba-api-1 Created
Container sentry-self-hosted-snuba-outcomes-consumer-1 Created
Container sentry-self-hosted-snuba-sessions-consumer-1 Created
Container sentry-self-hosted-snuba-consumer-1 Created
Container sentry-self-hosted-snuba-replacer-1 Created
Container sentry-self-hosted-snuba-transactions-consumer-1 Created
Container sentry-self-hosted-snuba-subscription-consumer-transactions-1 Created
Container sentry-self-hosted-vroom-1 Created
Container sentry-self-hosted-redis-1 Starting
Container sentry-self-hosted-zookeeper-1 Starting
Container sentry-self-hosted-memcached-1 Starting
Container sentry-self-hosted-symbolicator-1 Starting
Container sentry-self-hosted-clickhouse-1 Starting
Container sentry-self-hosted-postgres-1 Starting
Container sentry-self-hosted-smtp-1 Starting
Container sentry-self-hosted-symbolicator-1 Started
Container sentry-self-hosted-memcached-1 Started
Container sentry-self-hosted-redis-1 Started
Container sentry-self-hosted-postgres-1 Started
Container sentry-self-hosted-smtp-1 Started
Container sentry-self-hosted-zookeeper-1 Started
Container sentry-self-hosted-zookeeper-1 Waiting
Container sentry-self-hosted-clickhouse-1 Started
Container sentry-self-hosted-zookeeper-1 Healthy
Container sentry-self-hosted-kafka-1 Starting
Container sentry-self-hosted-kafka-1 Started
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-snuba-consumer-1 Starting
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-snuba-outcomes-consumer-1 Starting
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-snuba-api-1 Starting
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-snuba-replacer-1 Starting
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-snuba-transactions-consumer-1 Starting
Container sentry-self-hosted-snuba-sessions-consumer-1 Starting
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-snuba-subscription-consumer-transactions-1 Starting
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-snuba-subscription-consumer-events-1 Starting
Container sentry-self-hosted-vroom-1 Starting
Container sentry-self-hosted-snuba-outcomes-consumer-1 Started
Container sentry-self-hosted-snuba-consumer-1 Started
Container sentry-self-hosted-snuba-replacer-1 Started
Container sentry-self-hosted-snuba-transactions-consumer-1 Started
Container sentry-self-hosted-snuba-api-1 Started
Container sentry-self-hosted-vroom-1 Started
Container sentry-self-hosted-snuba-sessions-consumer-1 Started
Container sentry-self-hosted-snuba-subscription-consumer-events-1 Started
Container sentry-self-hosted-snuba-subscription-consumer-transactions-1 Started
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

Are you restoring from a backup of the same version of Sentry?
Are you restoring onto a clean database?
If so then this IntegrityError might be our fault, you can open an issue here:
https://github.com/getsentry/sentry/issues/new/choose
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 242, in _commit
return self.connection.commit()
psycopg2.errors.ForeignKeyViolation: insert or update on table "sentry_sentryappinstallation" violates foreign key constraint "sentry_sentryappinst_api_grant_id_a30cf323_fk_sentry_ap"
DETAIL: Key (api_grant_id)=(1) is not present in table "sentry_apigrant".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/bin/sentry", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/sentry/runner/init.py", line 193, in main
func(**kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sentry/runner/decorators.py", line 29, in inner
return ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sentry/runner/commands/backup.py", line 245, in import
raise (e)
File "/usr/local/lib/python3.8/site-packages/sentry/runner/commands/backup.py", line 234, in import
obj.save()
File "/usr/local/lib/python3.8/site-packages/sentry/utils/db.py", line 48, in _exit
rv = original_exit(self, exc_type, exc_value, traceback)
File "/usr/local/lib/python3.8/site-packages/django/db/transaction.py", line 246, in exit
connection.commit()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 33, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 266, in commit
self._commit()
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 242, in _commit
return self.connection.commit()
File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 242, in _commit
return self.connection.commit()
django.db.utils.IntegrityError: insert or update on table "sentry_sentryappinstallation" violates foreign key constraint "sentry_sentryappinst_api_grant_id_a30cf323_fk_sentry_ap"
DETAIL: Key (api_grant_id)=(1) is not present in table "sentry_apigrant".

Error in scripts/_lib.sh:69.
'$dc run --rm -T web import /etc/sentry/backup.json' exited with status 1
-> ./scripts/restore.sh:main:4
--> scripts/_lib.sh:restore:69

Cleaning up...

Event ID

No response

[azaslavsky]

This is a bug in the restore script, which will hopefully be fixed in 23.10. Work progress here: getsentry/team-ospo#153

[hubertdeng123]

Believe this is fixed now, so closing