-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migration to fix term column length #28019
Conversation
$prefix = $options['tablePrefix']; | ||
$table = $schema->getTable("{$prefix}account_terms"); | ||
// Check column length | ||
if($table->getColumn('term')->getLength() == 255) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
===
|
||
/** @var IDBConnection $db */ | ||
$db = \OC::$server->getDatabaseConnection(); | ||
$db->executeQuery("DELETE FROM {$prefix}account_terms WHERE CHAR_LENGTH(term) = 256;"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
>= 256
just to be on the safe side ?
lib/private/User/User.php
Outdated
for($i=0; $i<count($terms); $i++) { | ||
// Ignore terms that are too long | ||
if(strlen($terms[$i]) > 255) { | ||
unset($terms[$i]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assuming that messing with the PHP array indices won't affect subsequent code
I'm saying this because once upon a time I had to call array_values()
for such array to avoid logic that iterated over indices instead of foreach
to fail...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest to use array_filter
with a callback instead
8d05555
to
c78cba7
Compare
Orcale: |
Hmm, DBAL doesnt handle |
lib/private/User/User.php
Outdated
@@ -456,6 +456,10 @@ public function getSearchTerms() { | |||
* @since 10.0.1 | |||
*/ | |||
public function setSearchTerms(array $terms) { | |||
// Check length of terms | |||
$terms = array_filter($terms, function($term) { | |||
return strlen($term) <= 255; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not cut it off instead of fully skipping
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I felt like cut off would just cause unpredictable behaviour, but then since we are searching from the beginning of these strings I guess even keeping the beginning could still be helpful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for now cut off. texts usually start with the most important information.
@tomneedham so we need to extend our query builder. Make the default "LENGTH" and have a special case in the MySQL adapter for "CHAR_LENGTH" |
When will this be updated on the main homepage as well? I'm still getting the SQLight error when downloading the install from the website. Also, I can't seem to be able to run this one when I download it. Seems that I need to compile it first? |
@FlyingPersian well first the fix must be finished and merged. Then it must be released as part of 10.0.3. And then we can update the web page and the owncloud-latest.zip link that the web installer uses to use 10.0.3. If you're in a rush you can try installing from tarball directly and apply the patch manually before running the setup. (or manually changing that 256 value to 255 instead of patching) |
@PVince81: Alright, I tried adding the following three lines to the lib/private/User/User.php file, but the error still pops up:
It basically looks like this now: http://i.imgur.com/aa1SqRY.png
|
@FlyingPersian I suggest to start over again (without what you patched) and use my simple instruction from #28007 (comment). There is only one place to change to get a successful install. The rest of the PR here is only relevant if you're going to use LDAP. |
@FlyingPersian You can also disable mysql strict mode. @PVince81 this is the best solution / advice here, it will truncate the index as necessary, then this PR will fix the column in the next release. |
With mb4 support i believe the max index length is actually 191 characters. https://serversforhackers.com/mysql-utf8-and-indexing |
8fd0bf5
to
d62bc4a
Compare
Updated to reflect research into using 4 byte character sets. 191 is the max indexable string of characters. Also updated to trim terms if too long. |
d62bc4a
to
866d39d
Compare
Please rebase since the migration itself was merged separately. |
0e616c2
to
5be3e92
Compare
@PVince81 the oc_migration fix is unrelated - but is 'technically' the same issue -> too long columns |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
@tomneedham please increase version.php then merge. This is needed to trigger new migrations else it won't check them at all. |
5be3e92
to
fc0b1a6
Compare
Set version to 10.0.2.3 because of #28182 |
fc0b1a6
to
8eaa630
Compare
kicked jenkins |
@tomneedham please backport |
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Description
On some DB setups, the max index length can be as low as 767 bytes. This leads to a max column length of 191 chars - at the moment we have 256. https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
Related Issue
#28007
How Has This Been Tested?
Screenshots (if appropriate):
Types of changes
Checklist: