-
-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add SchedulerAgent, which periodically runs other agents. #459
Changes from 11 commits
c7e67a3
b4636f8
0ae0f1a
e9fa1f2
26564c5
f6d1966
56fa4cb
de5619f
a069fa6
0f66669
a479573
a2e1348
806e41d
d0d9c8e
3004170
1cea2ba
05dd52d
8065721
5306726
95a067a
06ae684
46497f0
69d2273
706ce8d
9a7f7fe
51f8c8b
53eb531
77f03da
8dc5244
99c3fc0
12f1839
38de860
786b0f3
23b0741
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
module AgentControllerConcern | ||
extend ActiveSupport::Concern | ||
|
||
included do | ||
validate :validate_control_action | ||
end | ||
|
||
def default_options | ||
{ | ||
'action' => 'run', | ||
} | ||
end | ||
|
||
def control_action | ||
options['action'].presence || 'run' | ||
end | ||
|
||
def validate_control_action | ||
case control_action | ||
when 'run', 'enable', 'disable' | ||
else | ||
errors.add(:base, 'invalid action') | ||
end | ||
end | ||
|
||
def control_targets! | ||
targets.active.each { |target| | ||
begin | ||
case control_action | ||
when 'run' | ||
log "Agent run queued for '#{target.name}'" | ||
Agent.async_check(target.id) | ||
when 'enable' | ||
log "Enabling the Agent '#{target.name}'" | ||
target.update!(disable: false) if target.disabled? | ||
when 'disable' | ||
log "Disabling the Agent '#{target.name}'" | ||
target.update!(disable: true) unless target.disabled? | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Other ideas:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Each of them sounds interesting and useful.
Lots of ideas, like it's unlimited. 😆 |
||
end | ||
rescue => e | ||
log "Failed to #{control_action} '#{target.name}': #{e.message}" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah, of course! |
||
end | ||
} | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -15,4 +15,14 @@ def scenario_links(agent) | |
def agent_show_class(agent) | ||
agent.short_type.underscore.dasherize | ||
end | ||
end | ||
|
||
def agent_schedule(agent, delimiter = ', ') | ||
return 'n/a' unless agent.can_be_scheduled? | ||
|
||
controllers = agent.controllers | ||
[ | ||
*(CGI.escape_html(agent.schedule.humanize.titleize) unless agent.schedule == 'never' && agent.controllers.count > 0), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Very minor: I'd do There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. All right. |
||
*controllers.map { |agent| link_to(agent.name, agent_path(agent)) }, | ||
].join(delimiter).html_safe | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -138,7 +138,9 @@ def agent_node(agent) | |
def agent_edge(agent, receiver) | ||
edge(agent_id[agent], | ||
agent_id[receiver], | ||
style: ('dashed' unless receiver.propagate_immediately), | ||
style: ('dashed' unless agent.can_control_other_agents? || !receiver.propagate_immediately?), | ||
label: (" #{agent.control_action}s " if agent.can_control_other_agents?), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the "s" to pluralize it? If so, maybe There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's an "s" of the third person singular present form. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't feel strongly. |
||
arrowhead: ('empty' if agent.can_control_other_agents?), | ||
color: (@disabled if agent.disabled? || receiver.disabled?)) | ||
end | ||
|
||
|
@@ -151,10 +153,17 @@ def agent_edge(agent, receiver) | |
fontsize: 10, | ||
fontname: ('Helvetica' if rich) | ||
|
||
statement 'edge', | ||
fontsize: 10, | ||
fontname: ('Helvetica' if rich) | ||
|
||
agents.each.with_index { |agent, index| | ||
agent_node(agent) | ||
|
||
agent.receivers.each { |receiver| | ||
[ | ||
*agent.receivers, | ||
*(agent.targets if agent.can_control_other_agents?) | ||
].each { |receiver| | ||
agent_edge(agent, receiver) if agents.include?(receiver) | ||
} | ||
} | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -25,13 +25,15 @@ class Agent < ActiveRecord::Base | |
|
||
EVENT_RETENTION_SCHEDULES = [["Forever", 0], ["1 day", 1], *([2, 3, 4, 5, 7, 14, 21, 30, 45, 90, 180, 365].map {|n| ["#{n} days", n] })] | ||
|
||
attr_accessible :options, :memory, :name, :type, :schedule, :disabled, :source_ids, :scenario_ids, :keep_events_for, :propagate_immediately | ||
attr_accessible :options, :memory, :name, :type, :schedule, :controller_ids, :target_ids, :disabled, :source_ids, :scenario_ids, :keep_events_for, :propagate_immediately | ||
|
||
json_serialize :options, :memory | ||
|
||
validates_presence_of :name, :user | ||
validates_inclusion_of :keep_events_for, :in => EVENT_RETENTION_SCHEDULES.map(&:last) | ||
validate :sources_are_owned | ||
validate :controllers_are_owned | ||
validate :targets_are_owned | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The word There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Or maybe... is "controllee" a word? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cantino If not, what about "members"? Should neither work, I'll use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Got it. Now, how about the table name? I named it There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm, how about control_links? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The table has been renamed. |
||
validate :scenarios_are_owned | ||
validate :validate_schedule | ||
validate :validate_options | ||
|
@@ -52,6 +54,10 @@ class Agent < ActiveRecord::Base | |
has_many :links_as_receiver, :dependent => :delete_all, :foreign_key => "receiver_id", :class_name => "Link", :inverse_of => :receiver | ||
has_many :sources, :through => :links_as_receiver, :class_name => "Agent", :inverse_of => :receivers | ||
has_many :receivers, :through => :links_as_source, :class_name => "Agent", :inverse_of => :sources | ||
has_many :chains_as_controller, dependent: :delete_all, foreign_key: 'controller_id', class_name: 'Chain', inverse_of: :controller | ||
has_many :chains_as_target, dependent: :delete_all, foreign_key: 'target_id', class_name: 'Chain', inverse_of: :target | ||
has_many :controllers, through: :chains_as_target, class_name: "Agent", inverse_of: :targets | ||
has_many :targets, through: :chains_as_controller, class_name: "Agent", inverse_of: :controllers | ||
has_many :scenario_memberships, :dependent => :destroy, :inverse_of => :agent | ||
has_many :scenarios, :through => :scenario_memberships, :inverse_of => :agents | ||
|
||
|
@@ -176,6 +182,10 @@ def can_create_events? | |
!cannot_create_events? | ||
end | ||
|
||
def can_control_other_agents? | ||
self.class.can_control_other_agents? | ||
end | ||
|
||
def log(message, options = {}) | ||
puts "Agent##{id}: #{message}" unless Rails.env.test? | ||
AgentLog.log_for_agent(self, message, options) | ||
|
@@ -218,6 +228,14 @@ def sources_are_owned | |
errors.add(:sources, "must be owned by you") unless sources.all? {|s| s.user == user } | ||
end | ||
|
||
def controllers_are_owned | ||
errors.add(:controllers, "must be owned by you") unless controllers.all? {|s| s.user == user } | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Slightly faster to do There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, I'll change it along with the existing code above. |
||
end | ||
|
||
def targets_are_owned | ||
errors.add(:targets, "must be owned by you") unless targets.all? {|s| s.user == user } | ||
end | ||
|
||
def scenarios_are_owned | ||
errors.add(:scenarios, "must be owned by you") unless scenarios.all? {|s| s.user == user } | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We could do that here too. |
||
end | ||
|
@@ -249,7 +267,7 @@ def boolify(option_value) | |
|
||
class << self | ||
def build_clone(original) | ||
new(original.slice(:type, :options, :schedule, :source_ids, :keep_events_for, :propagate_immediately)) { |clone| | ||
new(original.slice(:type, :options, :schedule, :controller_ids, :source_ids, :keep_events_for, :propagate_immediately)) { |clone| | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added! |
||
# Give it a unique name | ||
2.upto(count) do |i| | ||
name = '%s (%d)' % [original.name, i] | ||
|
@@ -290,6 +308,10 @@ def cannot_receive_events? | |
!!@cannot_receive_events | ||
end | ||
|
||
def can_control_other_agents? | ||
include? AgentControllerConcern | ||
end | ||
|
||
# Find all Agents that have received Events since the last execution of this method. Update those Agents with | ||
# their new `last_checked_event_id` and queue each of the Agents to be called with #receive using `async_receive`. | ||
# This is called by bin/schedule.rb periodically. | ||
|
@@ -399,6 +421,8 @@ def type | |
:sources, | ||
:receivers, | ||
:schedule, | ||
:controllers, | ||
:targets, | ||
:disabled, | ||
:keep_events_for, | ||
:propagate_immediately, | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,115 @@ | ||
require 'rufus-scheduler' | ||
|
||
module Agents | ||
class SchedulerAgent < Agent | ||
include AgentControllerConcern | ||
|
||
cannot_be_scheduled! | ||
cannot_receive_events! | ||
cannot_create_events! | ||
|
||
@@second_precision_enabled = ENV['ENABLE_SECOND_PRECISION_SCHEDULE'] == 'true' | ||
|
||
cattr_reader :second_precision_enabled | ||
|
||
description <<-MD % { seconds: (<<-MD_SECONDS if second_precision_enabled) } | ||
This agent periodically takes an action on target Agents according to a user-defined schedule. | ||
|
||
# Action types | ||
|
||
Set `action` to one of the action types below: | ||
|
||
* `run`: This is the default. Target Agents are run at intervals. | ||
|
||
* `disable`: Target Agents are disabled (if not) at intervals. | ||
|
||
* `enable`: Target Agents are enabled (if not) at intervals. | ||
|
||
# Targets | ||
|
||
Select Agents that you want to run periodically by this SchedulerAgent. | ||
|
||
# Schedule | ||
|
||
Set `schedule` to a schedule specification in the [cron](http://en.wikipedia.org/wiki/Cron) format. | ||
For example: | ||
|
||
* `0 22 * * 1-5`: every day of the week at 22:00 (10pm) | ||
|
||
* `*/10 8-11 * * *`: every 10 minutes from 8:00 to and not including 12:00 | ||
|
||
This variant has several extensions as explained below. | ||
|
||
## Timezones | ||
|
||
You can optionally specify a timezone (default: `#{Time.zone.name}`) after the day-of-week field. | ||
|
||
* `0 22 * * 1-5 Europe/Paris`: every day of the week when it's 22:00 in Paris | ||
|
||
* `0 22 * * 1-5 Etc/GMT+2`: every day of the week when it's 22:00 in GMT+2 | ||
|
||
%{seconds} | ||
|
||
## Last day of month | ||
|
||
`L` signifies "last day of month" in `day-of-month`. | ||
|
||
* `0 22 L * *`: every month on the last day at 22:00 | ||
|
||
## Weekday names | ||
|
||
You can use three letter names instead of numbers in the `weekdays` field. | ||
|
||
* `0 22 * * Sat,Sun`: every Saturday and Sunday, at 22:00 | ||
|
||
## Nth weekday of the month | ||
|
||
You can specify "nth weekday of the month" like this. | ||
|
||
* `0 22 * * Sun#1,Sun#2`: every first and second Sunday of the month, at 22:00 | ||
|
||
* `0 22 * * Sun#L1`: every last Sunday of the month, at 22:00 | ||
MD | ||
|
||
## Seconds | ||
|
||
You can optionally specify seconds before the minute field. | ||
|
||
* `*/30 * * * * *`: every 30 seconds | ||
|
||
MD_SECONDS | ||
|
||
def default_options | ||
super.update({ | ||
'schedule' => '0 * * * *', | ||
}) | ||
end | ||
|
||
def working? | ||
true | ||
end | ||
|
||
def check! | ||
control_targets! | ||
end | ||
|
||
def validate_options | ||
if (spec = options['schedule']).present? | ||
begin | ||
cron = Rufus::Scheduler::CronLine.new(spec) | ||
if !second_precision_enabled && cron.seconds != [0] | ||
errors.add(:base, "second precision schedule is not allowed in this service") | ||
end | ||
rescue ArgumentError | ||
errors.add(:base, "invalid schedule") | ||
end | ||
else | ||
errors.add(:base, "schedule is missing") | ||
end | ||
end | ||
|
||
before_save do | ||
self.memory.delete('scheduled_at') if self.options_changed? | ||
end | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
# A Chain connects Agents in a run chain from the `controller` to the `target`. | ||
class Chain < ActiveRecord::Base | ||
attr_accessible :controller_id, :target_id | ||
|
||
belongs_to :controller, class_name: 'Agent', inverse_of: :chains_as_controller | ||
belongs_to :target, class_name: 'Agent', inverse_of: :chains_as_target | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this to prevent DOS?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, you can still create as many agents as you want, though, I suppose only allowing an agent to run on the minute would demotivate potential abusers. Should I note that here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess? I suppose we could allow every 15 or 30 seconds. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is fine as long as you can document that nicely in the description. I cheated by omitting the seconds part completely. 😁
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, multiples of fifteen are now allowed even when the option is disabled (default).