Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a check to rows.Err after processing all rows #835

Merged
merged 6 commits into from
Aug 20, 2020

Conversation

ajm188
Copy link
Contributor

@ajm188 ajm188 commented Mar 31, 2020

Closes #822.

In go-sql-driver/mysql#1075, @acharis notes
that the way the go-sql driver is written, query timeout errors don't
get set in rows.Err() until after a call to rows.Next() is made.

Because this kind of error means there will be no rows in the result
set, the for rows.Next() will never enter the for loop, so we must
check the value of rows.Err() after the loop, and surface the error up
appropriately.

I then went ahead and fixed up the error checking in the two remaining
places we use db.Query, and validated I've covered all cases by
running:

$ golangci-lint run --disable-all -E rowserrcheck

In case this PR introduced Go code changes:

  • contributed code is using same conventions as original code
  • script/cibuild returns with no formatting errors, build errors or unit test errors.

Closes github#822.

In go-sql-driver/mysql#1075, @acharis notes
that the way the go-sql driver is written, query timeout errors don't
get set in `rows.Err()` until _after_ a call to `rows.Next()` is made.

Because this kind of error means there will be no rows in the result
set, the `for rows.Next()` will never enter the for loop, so we must
check the value of `rows.Err()` after the loop, and surface the error up
appropriately.
@acharis
Copy link

acharis commented Apr 7, 2020

i think we might want to do a bit more than this.
the driver uses the "deferred errors" pattern in a few places i think, and i've been spending a bit of time (when i can scrape some together) to convince myself that this same kind of error can't creep in elsewhere in gh-ost.

but definitely happy to see that someone else cares about this!

@ajm188
Copy link
Contributor Author

ajm188 commented Apr 12, 2020

Absolutely! I'm very interested in getting this sorted out :D

I think you're absolutely right we should do more, and I'm happy to try to tag-team checking the rest of gh-ost for where we're missing this with you.

I also think that this change as-is is an unequivocal win for gh-ost's data integrity, so my preference would be if we don't get the time to do a full check of gh-ost within a couple weeks that we merge this and at least plug the one hole we already know about it.

What do you think?

@acharis
Copy link

acharis commented Apr 12, 2020

with you 100%
as i find time i've been looking more into both the database/sql package and go-sql-driver/mysql
for other things that might trip folks up, as well as instances in gh-ost of those specific mistakes.
my plan was to reach out to you when i was done to coordinate our changes, but happy to coordinate efforts.

i think this PR is definitely a win and should be merged as-is as soon as possible.

@ajm188
Copy link
Contributor Author

ajm188 commented Apr 21, 2020

@acharis I think I've covered all the relevant cases here, but open to additional feedback!

Copy link
Contributor

@shlomi-noach shlomi-noach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an important fix that would prevent a data integrity (truncation) scenario. 👍

@timvaillancourt timvaillancourt merged commit c940a85 into github:master Aug 20, 2020
cenkore pushed a commit to cenkore/gh-ost that referenced this pull request Aug 31, 2020
Change-Id: Ib2e56c02ea1f98a5a47392ebbeeb6e88feb28e3c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Server query timeout lead to data loss
4 participants