Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Balloon list may have problem on awarding the "first to solve" #2810

Open
bhscer opened this issue Nov 20, 2024 · 6 comments · May be fixed by #2812
Open

Balloon list may have problem on awarding the "first to solve" #2810

bhscer opened this issue Nov 20, 2024 · 6 comments · May be fixed by #2812
Labels
bug

Comments

@bhscer
Copy link

bhscer commented Nov 20, 2024

Description of the problem

I'm intersted in the logic of balloon.

I found that when the submission judged as Accepted, the balloon will be created immediately.

domjudge/webapp/src/Controller/API/JudgehostController.php

Line 1078 in 7c890e8

$this->balloonService->updateBalloons($contest, $submission, $judging);

But if we have two submission on the same problem (they are the eariliest submissions of the problem) , maybe we call them A and B, and the submit time of A is earlier than B, and A's judge finish time is later, the balloon of submission A will be created after submission B. But when we looking at the balloon list, we temporally found that the balloon of submission B is remindered as the first balloon.

domjudge/webapp/src/Service/BalloonService.php

Line 92 in 7c890e8

public function collectBalloonTable(Contest $contest, bool $todo = false): array

In the collectBalloonTable function, we can see that award is determined by the balloon list

The above is my understanding of the balloon logic.

I don't know whether my worry is right or not. If it is, I think it is a bug or if you can tell me how can I temporaly solve this problem (maybe we should send balloons only if there submit time is much earlier than now to avaid eariler submission is not judged yet ?)

Your environment

Steps to reproduce

Create a problem with a long time limit.
Team A submit a code correct but run very slow first.
Team B submit a code correct and run fast later.

Expected behaviour

It's obvious Team A is the first to solve.

Actual behaviour

When Team B's submission is judged as correct but Team A's submission is still judging, we can found the balloon list maked Team B is the first to solve the problem.

Any other information that you want to share?

Sorry that currently I have no machine to test it. But according to my understing, I think this bug truely happens.
@meisterT
Copy link
Member

I can indeed reproduce this by disabling judging for a language, submitting a correct solution as a team in that language, and then with another team in a language that is enabled for judging. This will be shown as first in contest.

image

image

@meisterT meisterT added the bug label Nov 21, 2024
meisterT added a commit to meisterT/domjudge that referenced this issue Nov 21, 2024
@meisterT
Balloon list: only add 'first in contest' if there is no earlier unju…
15aeae7 
…dged submission.

The 'first for problem' is already working correctly as it re-uses the
corresponding scoreboard data.

Fixes DOMjudge#2810.
@meisterT meisterT linked a pull request Nov 21, 2024 that will close this issue
Open
@meisterT
Copy link
Member

We had some more discussion about this. The attached PR doesn't fix the problems for first in contest and it is kinda hard to fix (also should be generalized in award service). Also for first in problem, after the PR, we would not display "first in problem" until we are sure that it actually is the "first in problem", but display the balloon without it. So someone could deliver the balloon without realizing it actually could be or is the first in problem. One option would be to hold back the balloon until the info is known (which could be a long time assuming that a compiler/language could be broken) and confusing even when it's not a long time.

So no perfect solution. The question is whether we really should fix it or just remove the feature as it is hard to get it right.

I went and did some code archaeology, and found that the feature was added as part of a CCS requirement which no longer exists, see #70

I am leaning towards removing first in contest and first in problem on the balloon page. If people would want that functionality, they could use the balloon printer in ICPC tools.

@bhscer
Copy link
Author

bhscer commented Nov 23, 2024

We had some more discussion about this. The attached PR doesn't fix the problems for first in contest and it is kinda hard to fix (also should be generalized in award service). Also for first in problem, after the PR, we would not display "first in problem" until we are sure that it actually is the "first in problem", but display the balloon without it. So someone could deliver the balloon without realizing it actually could be or is the first in problem. One option would be to hold back the balloon until the info is known (which could be a long time assuming that a compiler/language could be broken) and confusing even when it's not a long time.

So no perfect solution. The question is whether we really should fix it or just remove the feature as it is hard to get it right.

I went and did some code archaeology, and found that the feature was added as part of a CCS requirement which no longer exists, see #70

I am leaning towards removing first in contest and first in problem on the balloon page. If people would want that functionality, they could use the balloon printer in ICPC tools.

But I should appologize first that I misunderstanding the logic of "first for problem".
In my newest understanding, the 'first to contest' have some problem, and need fix or remove, but the 'first to solve' comes from the scoreboard, why we should remove it?

@meisterT
Copy link
Member

See this snippet:

Also for first in problem, after the PR, we would not display "first in problem" until we are sure that it actually is the "first in problem", but display the balloon without it. So someone could deliver the balloon without realizing it actually could be or is the first in problem. One option would be to hold back the balloon until the info is known (which could be a long time assuming that a compiler/language could be broken) and confusing even when it's not a long time.

Basically, the problem is either we make the info mutable (as in add "first in problem" once we are sure it is first) or hold the balloon back which is confusing and also not great (and would require to hold back all other balloons for the same problem even the ones which cannot be first to solve in problem).

@bhscer
Copy link
Author

bhscer commented Nov 23, 2024

So you means that it maybe a case that a balloon is 'first for problem' but we don't sure so we don't show this award on the balloon list?

But we can find that there are two cases:

  1. This submission is the first to submit for the problem and judge finished firsted eariest.
  2. This submission is the first to submit for the problem but judge finished later than some later submissions.

But whatever the case is, for this truely 'first for problem' submission, we update scoreboad before insert the balloon, so in balloon list we can immedately see 'first for problem'.

@bhscer
Copy link
Author

bhscer commented Nov 25, 2024 •
edited
Loading

Ok, I'm sorry that I forgot another case. If early incorrect submission judge finished later than this truely first for problem submissuon, then the scoreboard will decide first for problem later than the balloon insert.
So that means current we have to problem:

  1. 'first for problem' may not display immedately on the list
  2. 'first to contest' may be incorrect (fixed)

Maybe we should change the logic of balloon insert. After a judge X for problem A finished as correct, we should query if there are some submission earlyer than this one but not finish the judge. If such submission exist, we skip the insert process, otherwise, we can process those submissions (for problem A, submit time <= X.submit time) to insert many balloons.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Balloon list: only add 'first in contest' if there is no earlier unju… meisterT/domjudge
2 participants
@meisterT @bhscer