Identify problem with or collect information on RoundRobin Automatic Task Assignments

[lomky]

Description

We are seeing unexpectedly unbalanced distribution of tasks from the round robin assignment of VLJ support tasks. We want to collect what is actually happening in order to correct this.

Data we would like to capture

• Org Task generating the assignment
• user task being assigned
• appeal
• assignee_pool
• last task being used to generate the last assignee
• the next_assignee we will use

Background/context/resources

Investigation

• HackMD notes

Technical notes

• RoundRobin code
• VLJ Support Overrides
• Invocation

[yoomlam]

what is this chart?

1 | 
2 | |||||
3 | ||||
5 | 
8 | 

Current algorithm only uses the last assignment made.
Want to know what the algorithm's state is with each restart.

Implement as a rails log

• Use a greppable string
• Consider appropriate format

Why 2?

• Straightforward work

Why 3?

• Consideration to do it thoughfully so we don't miss data

[yoomlam]

Created PR #14897 for adding logging but we won't need to merge it.
I identified 2 root causes for the RoundRobinTaskDistributor not working as expected for the Colocated Team. When a task is created and assigned to this team, it a new (duplicate) child task that is assigned to a team member (i.e., a User from the assignee_pool):

caseflow/app/models/task.rb

Lines 604 to 612 in 79907dd

	def create_and_auto_assign_child_task(options = {})
	options[:appeal_id] = appeal&.id unless options[:appeal_id]
	options[:task_id] = id unless options[:task]
	dup.tap do |child_task|
	child_task.assigned_to = assigned_to.next_assignee(**options)
	child_task.parent = self
	child_task.save!
	end
	end

because automatically_assign_org_task? returns true for the Colocated org:

caseflow/app/models/task.rb

Lines 614 to 616 in 79907dd

	def automatically_assign_org_task?
	assigned_to.is_a?(Organization) && assigned_to.automatically_assign_to_member?
	end

The root causes occur when assigning this new child task:

• If there exists an open task on the same appeal assigned to a user in the assignee_pool, then the distributor assigns the new task to that user (this is desired behavior):

  caseflow/app/models/colocated_task_distributor.rb

  Lines 10 to 22 in 79907dd

  	def next_assignee(options = {})
  	open_assignee = options[:appeal]
  	&.tasks
  	&.open
  	&.find_by(assigned_to: assignee_pool)
  	&.assigned_to
  	@state[:appeal_id] = options[:appeal]&.id
  	@state[:existing_assignee] = open_assignee&.id
  	log_state
  	open_assignee || super()
  	end

• End-user manually reassigned a (Colocated) task to user in the assignee_pool. Reassigning a task cancels the task and creates a new task assigned to the new user.

These cause the latest_task to change, which causes last_assignee to jump/skip from the expected round-robin ordering.

caseflow/app/models/round_robin_task_distributor.rb

Lines 22 to 36 in e5d5df6

	def latest_task
	# Use id as a proxy for created_at since the id field is already indexed.
	# Request .visible_in_queue_table_view to avoid TimedHoldTask or similar tasks
	latest = task_class
	.visible_in_queue_table_view
	.where(assigned_to: assignee_pool)
	.order(id: :desc)
	.first
	@state[:latest_task_id] = latest&.id
	latest
	end
	def last_assignee
	latest_task&.assigned_to
	end

These 2 causes (plus new Collocated members being added) explain the 4531 task assignments made between May 15th and Aug 5th. I could not explain 2 assignments having task IDs 1011751 and 1071485. Notes about the spreadsheet:

• purple text indicates 113 mismatches (i.e., task assignments that did not match expected round-robin ordering)
• 2 yellow highlights indicate when a user was added to the Colocated Team
• 2 pink highlights indicate the 2 unexplained assignments

This explains the "VLJ Support: Task auto-distribution discrepancy" in #14084.

See hackMD for detailed notes, but here's how to create the spreadsheet:

# Kat's code fix was made on May 12th, so start in mid-May
interval = DateTime.new(2020,5,15,0,0,0)..DateTime.new(2020,8,5,0,0,0)
assignee_pool = Colocated.singleton.non_admins.sort_by(&:id);
assignee_ids = assignee_pool.pluck(:id);
# this will make it easier to tell if assignment is out of order
pool=assignee_pool.each_with_index.map {|a,i| [a.id,i]}.to_h;
assigned_tasks = Task.visible_in_queue_table_view.where(assigned_to: assignee_ids, created_at: interval).order(:id);
require 'csv'
CSV.open( 'rr.csv', 'w' ) do |writer|
  counter=pool[assigned_tasks.first.assigned_to_id]
  pool_size=assignee_pool.size
  match_list = assigned_tasks.map{|t| 
    # does assignee match round-robin ordering
    match = (counter % pool_size) == pool[t.assigned_to_id]
    # what the next assignee should be
    counter = pool[t.assigned_to_id] + 1
    # count of tasks that were closed at (almost) the same time this task was created; captures task reassignment
    closed_count = t.appeal.tasks.order(:id).where(closed_at: (t.created_at-0.5)..(t.created_at+0.5)).count
    # are there prior tasks in this appeal assigned to the same assignee?
    prior_task = t.appeal.tasks.order(:id).where(assigned_to_id: t.assigned_to_id).where('created_at < ?', t.created_at).last
    writer << [ pool[t.assigned_to_id], match, closed_count, t.id, prior_task&.id, t.created_at ]
  };  
end

I'll create another ticket to implement a fix to account for the 2 root causes.

[hschallhorn]

Replaced with #14914