Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for existing rebuilds #54

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

chrismytton
Copy link
Contributor

@chrismytton chrismytton commented Jan 10, 2017

This checks for existing jobs before re-queueing them, which should help keep the size of the rebuild queue down.

Fixes #37

Note to merger

Make sure the following have been done before merging this:

  • Change the base of this pull request to master once Add Rubocop #53 has been merged.

@tmtmtmtm tmtmtmtm changed the base branch from rubocop to master January 10, 2017 14:10
Copy link
Contributor

@tmtmtmtm tmtmtmtm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's really two distinct types things going on here, so I think it would be worth splitting the PR in two — the first for the refactoring (which looks like it could go through like this OK) and then the second for changing the behaviour (which I think also really needs a test to show the new behaviour is working).

@chrismytton
Copy link
Contributor Author

There's really two distinct types things going on here, so I think it would be worth splitting the PR in two — the first for the refactoring (which looks like it could go through like this OK) and then the second for changing the behaviour (which I think also really needs a test to show the new behaviour is working).

Yep, good shout. I've pulled out the refactoring into a separate PR - #55.

This checks for an existing job matching the rebuild arguments. If one
is found then the rebuild is skipped to stop the queue from bloating.
@chrismytton chrismytton force-pushed the 37-check-for-existing-rebuilds branch from 9179f8b to aaa3c58 Compare January 11, 2017 09:18
The Sidekiq API doesn't work with Sidekiq's testing mode and we want to
use the API to check for existing jobs, so switching the tests over to
use the API seems like the simplest option.
This class is responsible for ensuring that jobs for the same
legislature don't get queued up more than once.
@chrismytton chrismytton requested a review from tmtmtmtm January 17, 2017 11:34
Copy link
Contributor

@tmtmtmtm tmtmtmtm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little concerned that if I remove the if clause in the new add method — i.e. making it simply

    def add(country, legislature, source = nil)
      RebuilderJob.perform_async(country, legislature, source)
      "Queued rebuild for country=#{country} legislature=#{legislature} source=#{source}\n"
    end

then the tests still pass…

This definitely seems to be the right abstraction, but I think finding it at the end of this PR makes it a little harder to follow. I'm wondering if it might make more sense to do this as two steps — firstly a 'clean' refactoring that maintains the old behaviour, but with the new Queue implementation. Then once we know that's working correctly, adding the 'skip if one exists' logic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants