-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update to ownCloud 9.1.1 (with intermediate upgrades) #894
Conversation
…isables memcached and goes with apc. The upgrade fails with memcached.
I was thinking that we should perhaps stop the upgrade if the current system isn't on the expected owncloud version. Otherwise the user might lose data. |
…verwritten when going from 9.0 to 9.1
I made a mistake when going from 8.2.3 to 9.1.0. When 9.1.0 was installed the backup from 8.2.3 was overwritten with the one from 9.0. Sorry, I try to thoroughly test before opening a PR. I made the upgrade slightly more forgiving when checking to go to 9.0. Every point release on 8.2.x is now accepted if someone manually upgraded. People upgrading from versions of miab before 0.18 will run in to issues. This has always been a problem if you skipped to many version. Shall I add an upgrade path from 8.1.1 into this? That should go to 8.2.3. |
… previously installed.
Every time I sleep on it, I think of some more test cases. I have tested an upgrade from 0.17c. (I had to add another upgrade path) This now works. Since the fail2ban changes have been merged to master I tested with that. Of course that doesn't work.. Manual testing shows that normal banning via the UI works. However no log messages are created when supplying bad login information via webdav. I don't think that is fixable from our end. It was planned for 9.1 to block repeated failed login attempts from owncloud it self. That didn't make it. |
I figured out what the problem was with fail2ban and owncloud logging. Owncloud will not log failures to the log if the username is unknown. So somebody can (trivially via a timing attack) figure out your usernames. When the attacker tries to login via those usernames we will start banning:
Imho; if they target a miab, they will most likely already know your email address. @JoshData shall we stay at 8.2 for this issue? Also I found a problem with the owncloud upgrades when somebody was trying to use owncloud at the same time. Before an upgrade I stop the service, this solved the issue. I think this should be it for this PR unless somebody can come up with another test I can perform? |
Very nice work. We'll want to upgrade ownCloud anyway eventually.
|
Thank you!
I pushed a commit for this.
I ran a test where I deleted my user account and then only accessed owncloud via carddav. A new directory was created when I did that. I used the following command to delete my account, that also deleted the directory:
|
Got it. Ok. I'll give this a test when I get the time. |
Have you already considered about taking this big upgrade as a step and a chance to move from ownCloud to Nextcloud? Nextcloud is a fork of ownCloud by Frank Karlitschek (founder of ownCloud) and a big part of the main developers.
The migration process would be as easy as an upgrade to a new version. You would just need to install Nextcloud 10 with this script (which is nice 👊 ), instead of ownCloud 9.1 😉 I should add that I am a part of the Nextcloud community 😅 If you have more questions or want to talk about that, feel free to ask me here ... visit our forums (help.nextcloud.com) or talk to us at our freenode IRC #nextcloud-dev 💬 |
That would be a compelling argument. Since username failures are no longer logged. (Still wondering why that happend... I could only find something in the change log stating that there were changes to that area) However work is also been done to move to just a backend that does just card/caldav and integrate separate frontend's for that. Ultimately the direction the project goes is up to @JoshData. |
|
I was refering to the fact that login failures with an incorrect username aren't logged anymore. But good news on the brute force front. |
@Mar1u5 If we go to ownCloud 9 first, can we move to Nextcloud 10 later? |
ownCloud 9 -> Nextcloud 9 Keep in mind, that 9.1 is a Major Release as 9. Because of this confusing names we decided to call it 10 and 11 instead of 9.1 ;) Currently - our releases are compatible to provide an easy Migration. But this will/may change in the future. |
I have merged with master and tested the upgrade from 8.2.7. No issues. |
Nextcloud 10 RC1 (ownCloud 9.1) is out. I've tested it with MiaB - no issues ;) |
Nextcloud 10 is out now 💪 ...do you have any plans about deploying it with the next version? @JoshData |
This whole ownCloud/Nextcloud thing is exactly the sort of mess that reinforces why I want to move away from ownCloud entirely. So probably not. |
I am sorry to hear about your decision 😞 @JoshData ... The question is, what you want to do now: I think you will not find an solution as Nextcloud which makes it so easy for people to manage their data, calendar and contacts ;) (ok ... would be crazy if I would say something different 😅) However: For now, there is no solution which fits for you, and the 8.X contacts and calendar implementation is not longer supported. With ownCloud/Nextcloud 9.0+ the community is providing an innovative, supported and IMHO more reliable calendar and contacts. We have a PR which works (many thanks to @yodax btw 👊) and which is (obviously) better than nothing - I think you'll see that point. If I hear what you are saying about ownCloud/Nextcloud here and in some other threads I mean that it could not be more un-reliable as it currently is ... so why don't you give it a try? |
I hadn't noticed that (well spotted!), I copied the link from the ownCloud app store. But indeed so it seems. I don't like the whole ownCloud/nextCloud thing at all. There probably isn't a "right" choice here, however calendar and contacts are the main focus for us and if that is maintained by nextCloud it might be an argument for moving to nextCloud.
I haven't been able to reproduce that. I ran it manually and I checked the logs. Could it be that cron job ran during the upgrade? Did you get any more after the first one?
Agreed. I started with adding some visual separation between the different upgrades. Do you think that is enough? We could use hide_output or print an extra warning before going to 9 and 9.1. Normally the upgrades aren't that chatty. |
Note that in Nextcloud 10.0.1 we've made the first big step in improving the upgrade process - this replaces files reliably with the new ones. Now the rest of the process is as it was but we're working on that, too. I know the split from ownCloud was messy, I'm real sorry for it - if we could've fixed things at ownCloud, we would have. But if you wonder about the future of the projects, the activity graphs are clear enough: We don't want to drop ownCloud users, you should have time to upgrade - see for example this where we helped out with security: |
But it looks like the ownCloud maintainers elected not to fix it?
|
Yeah, we can't/couldn't make the ownCloud guys see the problem with it, at least from a security pov. If you're on an older version you can try to use the patch we did for Nextcloud, I suppose. A wholesale update to Nextcloud is not much more painful than a bugfix upgrade either, so that's another option. |
Merged! (The conversation about NextCloud should be happening in one of the threads about that.) |
Wow! Fantastic work @yodax @JoshData and @ponychicken 🏁 |
Thanks for the reviews and help in completing this. I just rolled the update on my production box. Upgrade went smooth and everything is working fine. |
@yodax i commented on the commit thread my installation went smooth no visible error or break but thats my output under /cloud Memcache \OC\Memcache\APC not available for local cache Is the matching PHP module installed and enabled? thats the nginx error log FastCGI sent in stderr: "PHP message: PHP Fatal error: Class 'OCA\DAV\Connector\Sabre\ExceptionLoggerPlugin' not found in /usr/local/lib/owncloud/remote.php on line 54" while reading response header from upstream |
@sqeezy80 could you post the config file /home/user-data/owncloud/config.php ? Please remove the hashes from the config before posting. Or just post this section:
Also could you show me the output of: dpkg --get-selections | grep apc |
|
The apc install looks fine. My 'memcache.local' => '\OC\Memcache\APC' is near the other memcache setting. That is very weird. Did you manually upgrade in the past?
It shouldn't be a problem that they aren't next to each other, but it is something you can try. Perhaps give it a reboot. Otherwise you could try to restore the install and try again. |
It would be interesting to see the log during the install, do you still have that? If you don't and it still isn't working you can restore the previous version; during the install the system made a backup for you.
|
i don't have the log This machine is running miab for a long time, but i did not install anything aside from mosh. i did run the backup wich worked fine and restored a working owncloud install. |
If you run the setup again it will upgrade again. If you run it without mosh you can scroll back and copy the log and paste it here. There is some personal info in there so please redact it. |
i did the reinstall, same result but here is the log
|
That looks clean. Still the same issue? box.dns.email/cloud gives an error? |
yep still the same error
and from the ngninx log |
php -v shows
|
Everything looks exactly like on my machines. Could you run:
I didn't do this, but it might be worth a try: |
result of grep -r apc *
so i changed the apcu.ini and everything is working and no i never touched the ini before might be good to check it my apcu.ini is now looking like this
it´s working now |
What cloud provider are you using? I had this in a first iteration of the PR, but it didn't seem necessary. |
i am using netcup.de, with a kvm based virtual machine like this |
|| ! grep -q $owncloud_ver /usr/local/lib/owncloud/version.php; then | ||
InstallOwncloud() { | ||
echo | ||
echo "Upgrading to ownCloud version $1" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it say "Upgrading" even when doing the initial install? :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First it says:
Installing ownCloud (contacts/calendar)...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Excellent, apologies for reading it wrong.
I'll be running it later this week...
This PR upgrades owncloud to 9.1.0.
When upgrading owncloud you can't skip a minor version, this PR will upgrade from 8 to 9 to 9.1.
After going to 9 we need to migrate the existing contacts and calendars. This has to be done when you're at version 9, otherwise the option (and storage) is removed when upgrading to version 9.1. (Yes really, you will lose data)
When upgrading to 9, memcached doesn't seem to work. I have moved to the recommend (by Owncloud) APC method of caching. (memcached doesn't work well with file locking according to owcloud). This has to be done before the migrations and occ command line upgrades, otherwise these will fail.
We can't install the contacts/calendar app from github any more. These now need to be build, to do that you need npm, etc. Which we could install, but owncloud also supply packages. So I install directly from these packages.
I have tested this by upgrading many times from 8.2.3 to 9.1.0. I have tested:
The tests have been done on iOS with z-push and on macOS with Calendar and Contacts.
Even though I tried to test this quite thoroughly, I think more people should test this on a system for which they have backups or do not care about.
What I did to test this was copy /usr/local/lib/owncloud to ~/owncloud-lib and /home/user-data/owncloud. If you then run this script:
And after the script rerun the setup/owncloud.sh from the master branch. This will restore everything to 8.2.3. Then checkout this branch and run the setup.