De-duplicate builds #7123

humitos · 2020-05-25T11:32:16Z

When a build is triggered, it could be marked as to be skipped by builders if:

there is already a build running/queued for the same commit

Multiple builds of the same commit will lead to the same results. So, we skip
it if there is one already running/queued.
there is already a build queued for the same version

When building a version without specifying the commit, the last commit is
built. In this case, we can only skip the new triggered build if there is one
build queued because both builds will just pick the same commit.

A different approach of #7031

When a build is triggered, it could be marked as to be skipped by builders if: * there is already a build running/queued for the same commit Multiple builds of the same commit will lead to the same results. So, we skip it if there is one already running/queued. * there is already a build queued for the same version When building a version without specifying the commit, the last commit is built. In this case, we can only skip the new triggered build if there is one build queued because both builds will just pick the same commit.

ericholscher

This looks like a good approach. 👍

I think we're hitting issues where we don't have enough build states, and we should probably think more about this. @agjohnson do you have thoughts via the redesign? It seems like "rebuild this commit" and "waiting for concurrency limit" and "cancelled because a commit is already being built" are mostly new states that we don't have a great way to present to users, but we should.

ericholscher · 2020-05-27T19:30:25Z

readthedocs/core/utils/__init__.py

+            commit,
+        )
+        build.error = DuplicatedBuildError.message
+        build.success = False


I wonder if we should make this nullable and set it to None here? It didn't succeed or fail, so it kinda makes sense.

It makes sense to me. However, I don't think we can change it now without changing the UI (knockout code) that renders in the frontend; which I'm not sure if we want to touch at this point.

Yea, I just worry this will break the badge display.

readthedocs/core/utils/__init__.py

agjohnson · 2020-05-28T02:59:02Z

I'd like to see build state eventually decoupled from build status (or whatever you want to call this concept). How we do this is mostly a matter of modeling, but I see state and status as two competing concepts -- at least as it will apply to our new UX.

I'd describe build state as where in the build process the build is. This allows us to show progression to the user in the form of a progress bar or in the build listing. Our steps now are fairly broad:

Queued
Cloning
Installing
Building
Uploading
Finished

I would describe status as why the build is in a particular state. This is helpful for communicating more details about state to the user, but it doesn't help describe progression. Queued is an important state description, but "Queued" vs "Queue due to build concurrency" is not helpful in describing how far along the build is. Build status would include why a build is queued, why a build was rejected, and error states like why a build failed. Currently, these are all weak concepts in our modeling/UI.

I've long wanted something like a status field on Build with an identifier that maps to a list of build status/exception classes. We would store metadata around the status/exception on builds -- for example save build.status_code = Build.STATUS_TIMEOUT -- not just store the error as a character model field or use some obtuse template logic to display copy that exists in templates only.

I'd propose we stop adding to the list of build states -- that is, describing build flow progression -- and move all reporting of why a build is in a particular state to more flexible modeling.

So, for this particular case, I'd suggest:

build.state = 'finished'
build.status_code = Build.STATUS_SKIPPED_DUPLICATE.status_code

From there, we have many more options with class attributes/etc to help describe the build in templates.

humitos · 2020-05-28T09:15:54Z

@agjohnson super interesting. I'm aligned with what you are saying here.

build.status_code = Build.STATUS_SKIPPED_DUPLICATE.status_code

We don't have a status_code currently. We have a exit_code, but I don't think it means the same. Are you suggesting to add a new field? If that's the case, should we change the UI with this change as well, or you are just thinking to use this new field in the redesign?

ericholscher · 2020-05-28T15:34:57Z

I think this makes sense generally.

I also think we need to think more about "success" in this case as well. Here, the build doesn't fail or succeed, but is mostly just a no-op. I think we have weird edge cases here already with badges, where we show failed badges more often than we should because of this issue.

agjohnson · 2020-06-02T23:59:41Z

Are you suggesting to add a new field?

Sorry, yeah my example was adding a new field for this, which would be separate from exit_code and would have special meaning internally. I'm not sure if we have a good middle ground though, and don't want to hold up this PR on a larger overhaul of error/status reporting in the UI. However, I did remove a lot of the complex template logic used to display build various state blocks from the build detail UI in anticipation of having stronger error/status reporting.

If the the simple, temporary solution is re-using the error field, then I think that is fine. We can prioritize adding some nicer error/status reporting before I get to a more mature release with the templates. I don't think we're talking about a lot of work though, but I'd probably only support better error/status modeling on the new templates

I also think we need to think more about "success" in this case as well

This is a great point as well. We might want to turn build.success from a boolean into a select field of failed, success, skipped, and maybe even unknown. I'd like to see skipped builds as a different state than builds that succeeded or failed.

humitos · 2020-06-03T16:02:32Z

I made some improvements here and did some more testing locally. It's ready for a re-review.

it's easier to test with multiple single process builders (see readthedocs/common#61 (comment))

Build output's page looks like this:

If we want to change how the error is displayed this is the code generates the Error section.

"Error" title is hardcoded
Build.error is what it's shown to the user

The Build list's page shows "Passed" and "Failed" for each build. If we want to change that this is the code that generates that

it's based on state='finished' and success
we could use status_code instead

In both cases, we could inject these Exception classes the template context to share the string/error/message to display.

I'm still not sure if we want to do these template changes in this PR or not (cc @agjohnson). We could keep everything the same and make it behaves different if build.status_code != None to keep backward compatibility.

ericholscher · 2020-06-03T17:12:42Z

readthedocs/doc_builder/exceptions.py

@@ -59,6 +59,8 @@ class BuildMaxConcurrencyError(BuildEnvironmentError):

 class DuplicatedBuildError(BuildEnvironmentError):
    message = ugettext_noop('Duplicated build.')
+    exit_code = 1
+    status_code = 0


This should be a constant probably, and something other than 0?

I made an Enum for this. I don't think it's a problem if it's 0. Why did you think it's better to have it different than 0? and what value?

Because 0 to me means "normal" status -- we want a status that represents the issue of "duplicated build", which should be assigned a specific number.

ericholscher

I think this approach seems reasonable, so I'm 👍 on shipping it, but there is definitely more work to be done here. I think fixing the modeling would be the first thing, and then we can fix the display in the new UI.

…humitos/de-duplicate-builds

humitos · 2020-06-08T11:32:53Z

Hrm... I'm not sure what happened with Travis 🤷

Django does not convert Enum automatically to its value :/

humitos requested a review from ericholscher May 25, 2020 11:32

humitos force-pushed the humitos/de-duplicate-builds branch from 3f072f5 to e76a948 Compare May 25, 2020 11:35

humitos mentioned this pull request May 25, 2020

Use 2 builders with celery concurrency at 1 #7124

Closed

humitos requested a review from a team May 27, 2020 09:44

Lint

95350aa

ericholscher reviewed May 27, 2020

View reviewed changes

humitos added 4 commits June 3, 2020 17:19

Add Build.status_code to show different user's messages

030b0a5

Exclude the build triggered itself from the query

9fcf10d

Add test cases for de-duplicate builds

141bbf5

Check Build.status_code from the task

6ce7617

humitos marked this pull request as ready for review June 3, 2020 16:02

humitos requested review from ericholscher and agjohnson June 3, 2020 16:02

ericholscher reviewed Jun 3, 2020

View reviewed changes

ericholscher approved these changes Jun 3, 2020

View reviewed changes

humitos added 4 commits June 8, 2020 12:55

Make StatusCode an Enum

2802da9

Be more permissive when checking Build.status_code

e9767f1

De-duplicate builds under a Feature flag

30dde39

Merge branch 'master' of github.com:readthedocs/readthedocs.org into …

8ed21f6

…humitos/de-duplicate-builds

humitos closed this Jun 8, 2020

humitos reopened this Jun 8, 2020

humitos added 2 commits June 8, 2020 14:07

Enable feature on tests

954e004

Use a simple constant instead of an Enum

190a628

Django does not convert Enum automatically to its value :/

humitos merged commit ca14c78 into master Jun 8, 2020

humitos deleted the humitos/de-duplicate-builds branch June 8, 2020 13:14

This was referenced Jun 8, 2020

Use CharField.choices for Build.status_code #7166

Merged

Notification on duplicated builds #7177

Closed

humitos mentioned this pull request Nov 12, 2020

Feature Request: Allow maintainers to cancel PR builds #7660

Closed

humitos mentioned this pull request Apr 13, 2022

Create better status for concurrency-limited & duplicated builds #9100

Closed

agjohnson mentioned this pull request Apr 14, 2022

Build: plan for better description of build state and build status #9110

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

De-duplicate builds #7123

De-duplicate builds #7123

humitos commented May 25, 2020 •

edited

Loading

ericholscher left a comment

ericholscher May 27, 2020

humitos May 28, 2020

ericholscher Jun 3, 2020

agjohnson commented May 28, 2020

humitos commented May 28, 2020

ericholscher commented May 28, 2020

agjohnson commented Jun 2, 2020

humitos commented Jun 3, 2020

ericholscher Jun 3, 2020

humitos Jun 8, 2020

ericholscher Jun 8, 2020

ericholscher left a comment

humitos commented Jun 8, 2020

De-duplicate builds #7123

De-duplicate builds #7123

Conversation

humitos commented May 25, 2020 • edited Loading

ericholscher left a comment

Choose a reason for hiding this comment

ericholscher May 27, 2020

Choose a reason for hiding this comment

humitos May 28, 2020

Choose a reason for hiding this comment

ericholscher Jun 3, 2020

Choose a reason for hiding this comment

agjohnson commented May 28, 2020

humitos commented May 28, 2020

ericholscher commented May 28, 2020

agjohnson commented Jun 2, 2020

humitos commented Jun 3, 2020

ericholscher Jun 3, 2020

Choose a reason for hiding this comment

humitos Jun 8, 2020

Choose a reason for hiding this comment

ericholscher Jun 8, 2020

Choose a reason for hiding this comment

ericholscher left a comment

Choose a reason for hiding this comment

humitos commented Jun 8, 2020

humitos commented May 25, 2020 •

edited

Loading