Problem while trying to backup (legacy) #2353
Comments
|
Is it possible that you already have a backup.json file at that location with restricted permissions? Or that there is a system level permission restriction? |
|
@azaslavsky yes, the reason was that the folder had the wrong permissions. What can I do to resolve it? |
|
If you go into the JSON file itself, are there any models that start with something besides `sentry.*"? Which ones in particular? |
|
@azaslavsky |
getsantry
bot
added
Waiting for: Product Owner
and removed
Waiting for: Community
labels
getsantry
bot
moved this from Waiting for: Community
to Waiting for: Product Owner
in GitHub Issues with 👀
|
Hmm, neither of those should be in there: https://github.com/getsentry/sentry/blob/5b47ccc4bcb416f7cda37e80ad09a114aa89f1a5/src/sentry/backup/helpers.py#L7 🤔. I believe this is the exact reason we put those exclusions in. If you are importing on a fresh instance, I would just manually remove these models from the JSON and try again. |
|
I've deleted. |
|
I would delete those line as well, see #2259 (comment). We're working on improving this script by 23.10.0, but that would still require a major upgrade on your end to see the benefits. |
getsantry
bot
moved this from Waiting for: Community
to Waiting for: Product Owner
in GitHub Issues with 👀
|
Ok, my mistake was in path where tar extract files. But after that I see the error in web container. |
|
I made a nice script that does not require Docker to work. Or the overkill image |
|
@williamdes Do you have script which do the restore? |
Not yet, but I think I will have to make it today/this week as the goal is to move ingest traffic to another node and perform an upgrade |
|
@williamdes I'd really appreciate if you'd paste it here |
hubertdeng123
added
Type: Support Request
and removed
Waiting for: Product Owner
labels
|
The backup script is intended to restore "low-volume" information, like orgs, teams, and projects, not "high-volume" data like events and issues. So you won't see a full restore. We are currently working on vastly improving this script and supporting full backup capabilities: getsentry/team-ospo#153 |
|
@azaslavsky Will this script work for older versions of Sentry? |
Can you explain why? |
|
Here is the backup and restore scripts, the backup one changed a bit #!/bin/bash
set -eu
BACKUP_NAME="backup_$(date --iso-8601=seconds)_"
echo "Backing up prefix: $BACKUP_NAME"
getFolderForVolume() {
docker inspect "$1" --format '{{ .Mountpoint }}'
}
backupVolume() {
VOL_DIR="$(getFolderForVolume "$1")"
echo "Backing up $1: $VOL_DIR"
# See: https://stackoverflow.com/a/76008162/5155484
tar --use-compress-program=lz4 -cf ./${BACKUP_NAME}${1}.tar.lz4 -C "$VOL_DIR" .
}
# See: docker volume ls -q
backupVolume "sentry-clickhouse"
backupVolume "sentry-data"
backupVolume "sentry-kafka"
backupVolume "sentry-redis"
backupVolume "sentry-symbolicator"
backupVolume "sentry-zookeeper"
ls -lah ./${BACKUP_NAME}*.tar.lz4restore.sh #!/bin/bash
set -eu
read -p "Enter backup name (example: backup_2023-08-27T23:23:48+02:00_): " BACKUP_NAME
echo "Restoring backup with prefix: $BACKUP_NAME"
ls -lah ./${BACKUP_NAME}*.tar.lz4
getFolderForVolume() {
# Find or create
docker inspect "$1" --format '{{ .Mountpoint }}' 2> /dev/null || echo ""
}
restoreVolume() {
VOL_DIR="$(getFolderForVolume "$1")"
if [ -d "${VOL_DIR}" ]; then
echo "Cleaning $1: $VOL_DIR"
docker volume rm "$1" 2> /dev/null 1> /dev/null
fi
docker volume create "$1" 2> /dev/null 1> /dev/null
VOL_DIR="$(getFolderForVolume "$1")"
if [ ! -d "${VOL_DIR}" ]; then
echo "The volume $1 seems broken, path: $VOL_DIR"
exit 1
fi
echo "Restoring $1: $VOL_DIR"
if [ ! -f "./${BACKUP_NAME}${1}.tar.lz4" ]; then
echo "Could not find the backup: ./${BACKUP_NAME}${1}.tar.lz4"
exit 1
fi
# See: https://stackoverflow.com/a/76008162/5155484
tar --use-compress-program=lz4 -xf ./${BACKUP_NAME}${1}.tar.lz4 -C "$VOL_DIR"
}
# See: docker volume ls -q
restoreVolume "sentry-clickhouse"
restoreVolume "sentry-data"
restoreVolume "sentry-kafka"
restoreVolume "sentry-redis"
restoreVolume "sentry-symbolicator"
restoreVolume "sentry-zookeeper" |
That's just the nature of the script - it has always been that way. We're working on improving this, as tracked by the linked issue. If you're asking about the technical reasons, it is just much more difficult to do backup/restore of high-volume models like events, of which there could be on the order of millions. Current backups are realistically limited to a few MBs, whereas included millions of events could push us into 100+GB territory. Many parts of the system (ex, our use of JSON) start to break down at that scale, necessitating a lot more work to ensure reliability. |
|
@williamdes for some reason backup script only backups sentry-clickhouse |
Yes, this is probably why they seem to say the backup script solution is not great |
|
@williamdes Hm.. for some reason I can't restore: I try to do docker compose down and docker compose up -d previously. UPD: Sorry, my mistake. I need to install lz4. |
|
@williamdes Thanks! Your scripts helps me to backup and restore all data and update from 20.11.0 to 23.8.0. |
|
Going to mark this as resolved. Thank you for all of the help @williamdes! |
|
Awesome, glad it helped |
|
getsentry/team-ospo#153 is the tracking bug for this. The end goal is a robust backup/restore script that covers all data. |
Self-Hosted Version
20.11.0
CPU Architecture
x86_64
Docker Version
24.0.5
Docker Compose Version
v2.20.2
Steps to Reproduce
Expected Result
Successfully backing up all data to backup.json file
Actual Result
Error: Could not open file /sentry-data/backup/backup.json: Permission denied while trying to backup (legacy style)
For some reason, permission denied error occured.
My goal is make a backup and use this backup for restoring on new Rocky Linux 9 server with Sentry 20.11, after that I will update it to 21.5.0 -> 21.6.3 -> 23.6.2 -> latest. Just git checkout tags/ then ./install.sh for this goal if I'm not mistaken.
If this type of error if unrealize to solve, maybe you can you explain how I should use volume back up in Full Backup style on sentry-clickhouse example? This is not clear for me.
docker run --rm --volumes-from -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar <what shoul I type there?? Is this full path of mountpoint?>
Event ID
No response
The text was updated successfully, but these errors were encountered: