Restore doesn't properly restore everything

[androidacy-user]

Self-Hosted Version

23.7.1

CPU Architecture

x86_64

Docker Version

24.0.6

Docker Compose Version

2.21.0

Steps to Reproduce

1. Create backup with ./scripts/backup.sh
2. Restore backup on same version but different host
3. Try to login
4. See "you have no organization access"

Expected Result

Restore works fine and so does login

Actual Result

• logs contain about 5-6 "no sequences found" messages but nothing else of interest, no errors, nada

Event ID

No response

[azaslavsky]

A fix for this is coming in 23.9, which should land in a few days, though it will probably require making a new backup. The basic issue (among many) is that the old backup JSON file was incomplete, so rehydrating from it is basically impossible.

You can track work at getsentry/team-ospo#153.

[androidacy-user]

A fix for this is coming in 23.9, which should land in a few days, though it will probably require making a new backup. The basic issue (among many) is that the old backup JSON file was incomplete, so rehydrating from it is basically impossible.

You can track work at getsentry/team-ospo#153.

Is there any way to fix it as it is now? We're certainly not afraid to get dirty with a bit of postgresql 😁

Also, on a slightly unrelated note, using Sentry SaaS would cost us potentially thousands based on our event volume and replays (we stopped using SaaS when our costs started hitting scary close to the quadruple digits mark). We can host it using spare capacity on our own infrastructure, which I'm sure is a shared experience among many who stick with self hosting. But, perhaps, that's a discussion for another time.

[azaslavsky]

The backup feature is currently broken. Part of the work for the linked bug will fix it for self-hosted as well, so you are not compelled to upgrade to SaaS. The work stream for fixing this feature and for supporting easier migration to SaaS is covered by the same issue - sorry about the confusion!

Is there any way to fix it as it is now? We're certainly not afraid to get dirty with a bit of postgresql 😁

You can certainly try to pg_dump your database one one machine and pg_restore it on the other, thoug there are no guarantees that this will work as expected.

[androidacy-user]

The backup feature is currently broken. Part of the work for the linked bug will fix it for self-hosted as well, so you are not compelled to upgrade to SaaS. The work stream for fixing this feature and for supporting easier migration to SaaS is covered by the same issue - sorry about the confusion!

Is there any way to fix it as it is now? We're certainly not afraid to get dirty with a bit of postgresql 😁

You can certainly try to pg_dump your database one one machine and pg_restore it on the other, thoug there are no guarantees that this will work as expected.

So, ah, let's say we don't have the old database around. All I'm really concerned about is fixing the aforementioned login issue - though more data being restored will of course be nice in the future 👍

[azaslavsky]

Sorry, just to be clear: you no longer have the original db? So the only thing remaining is the JSON backup file?

In that case, you can probably edit the JSON manually: find the user you are logging in as and set their is_staff and is_superuser flags to True. This should give you pretty broad admin privileges (you may still need some other permissions, but those are more complicated to setup via JSON hacking alone).

[androidacy-user]

Sorry, just to be clear: you no longer have the original db? So the only thing remaining is the JSON backup file?

In that case, you can probably edit the JSON manually: find the user you are logging in as and set their is_staff and is_superuser flags to True. This should give you pretty broad admin privileges (you may still need some other permissions, but those are more complicated to setup via JSON hacking alone).

See, it does recognize me as a super admin. Just not on initial login. If I go to org settings, it shows my account as the owner and won't let me change it. But when logging in, I get "no organization access" until I click "user settings" and I can go from there

[azaslavsky]

If you do want to make this user an admin with complete privileges, you can do add something like this to the end of your import:

    {
        "model": "sentry.userpermission",
        "pk":  [[INCREMENT_ABOVE_GREATEREST_SENTRY_USERPERMISSION_PK]],
        "fields": {
            "user": [[DESIRED_USER]],
            "permission": "users.admin"
        }
    },
    {
        "model": "sentry.userrole",
        "pk": [[INCREMENT_ABOVE_GREATEREST_SENTRY_USERROLE_PK]],
        "fields": {
            "date_updated": "2023-06-22T23:00:00.123Z",
            "date_added": "2023-06-22T22:54:27.960Z",
            "name": "Super Admin",
            "permissions": "['broadcasts.admin', 'users.admin', 'options.admin']"
        }
    },
    {
        "model": "sentry.userroleuser",
        "pk": [[INCREMENT_ABOVE_GREATEREST_SENTRY_USERROLEUSER_PK]],
        "fields": {
            "date_updated": "2023-06-22T23:00:00.123Z",
            "date_added": "2023-06-22T22:59:57.000Z",
            "user": [[DESIRED_USER]],
            "role": [[PK_OF_USERROLE_ABOVE]]
        }
    }

For example, if the user you want to be a super user is [email protected], the greatest sentry.userpermission pk is 2, the greatest sentry.userrole pk is 5, and the greatest sentry.userroleuser pk is 10, you would end up with:

    {
        "model": "sentry.userpermission",
        "pk": 3,
        "fields": {
            "user": "[email protected]",
            "permission": "users.admin"
        }
    },
    {
        "model": "sentry.userrole",
        "pk": 6,
        "fields": {
            "date_updated": "2023-06-22T23:00:00.123Z",
            "date_added": "2023-06-22T22:54:27.960Z",
            "name": "Super Admin",
            "permissions": "['broadcasts.admin', 'users.admin', 'options.admin']"
        }
    },
    {
        "model": "sentry.userroleuser",
        "pk": 11,
        "fields": {
            "date_updated": "2023-06-22T23:00:00.123Z",
            "date_added": "2023-06-22T22:59:57.000Z",
            "user": "[email protected]",
            "role": 6
        }
    }

Does this help?

[azaslavsky]

See, it does recognize me as a super admin. Just not on initial login. If I go to org settings, it shows my account as the owner and won't let me change it. But when logging in, I get "no organization access" until I click "user settings" and I can go from there

This is very odd. I'm not sure I have other suggestions without seeing the backup JSON, and even then, the tool is not in a great state at the moment, so I would not be surprised if something in it was corrupted. I'm sorry - we're working hard to make this a better exeperience!

[androidacy-user]

See, it does recognize me as a super admin. Just not on initial login. If I go to org settings, it shows my account as the owner and won't let me change it. But when logging in, I get "no organization access" until I click "user settings" and I can go from there

This is very odd. I'm not sure I have other suggestions without seeing the backup JSON, and even then, the tool is not in a great state at the moment, so I would not be surprised if something in it was corrupted. I'm sorry - we're working hard to make this a better exeperience!

We're probably going to have to start over anyway. Upgrading from 23.7.1 to 23.8.0 rendered most of our instance unusable. Most JavaScript required to load any page now returns 404.

[androidacy-user]

Honestly, it really feels like y'all intentionally cripple the self hosted instance just so you can say it's open source but get people to use SaaS. We've yet to have an upgrade go smoothly, and every single time we end up having to reset the image to get in an usable state.

[hubertdeng123]

Sorry to hear that. We host our own version of self-hosted Sentry for internal use and for what it's worth, it's upgraded daily without issues so far so mileage definitely varies here. As mentioned above, we're trying to improve the backup/restore process so it's much more usable in the future so please be patient with us while we improve the process.

[getsantry]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀