BuildStream client-side implementation of remote execution #454
Comments
BuildStream-Migration-Bot
added
important
remote execution
|
In GitLab by [Gitlab user @jmacarthur] on Jul 3, 2018, 15:10 marked this issue as related to #417 |
|
In GitLab by [Gitlab user @jmacarthur] on Jul 3, 2018, 15:10 marked this issue as related to #387 |
|
In GitLab by [Gitlab user @jmacarthur] on Jul 3, 2018, 15:10 changed the description |
|
In GitLab by [Gitlab user @jmacarthur] on Jul 3, 2018, 15:12 changed the description |
|
In GitLab by [Gitlab user @jmacarthur] on Jul 3, 2018, 15:14 changed the description |
|
In GitLab by [Gitlab user @jmacarthur] on Jul 5, 2018, 17:16 marked the task Pull resulting output directories from the artifact server. as completed |
|
In GitLab by [Gitlab user @jmacarthur] on Jul 5, 2018, 17:20 The biggest question on this at the moment is how we configure it. It made sense for the remote execution client to take the form of a Sandbox class alongside sandboxwrap and sandboxchroot. However, I don't think it makes sense to expose configuration this way. I would rather have another key in project config called 'remote-execution' at the top level alongside 'artifacts'. We could potentially move all of the 'artifact' configuration inside another 'network' key but that means even more changes to the configuration format. We then may also need some more substantial changes to the element and sandbox configuration code to pass configuration information other than the 'sandbox' key onto |
|
In GitLab by [Gitlab user @jmacarthur] on Jul 13, 2018, 11:10 Also needs updating to V2 of the remote execution API, which will require a simultaneous, similar update to BuildGrid. |
|
In GitLab by [Gitlab user @tristanvb] on Aug 7, 2018, 06:07 marked this issue as related to #412 |
|
In GitLab by [Gitlab user @tristanvb] on Aug 7, 2018, 06:07 marked this issue as related to #411 |
|
In GitLab by [Gitlab user @sstriker] on Aug 8, 2018, 17:36
We need to be able to configure the Remote Execution endpoint(s), that is, Execution Service and CAS independently of the ArtifactCache endpoint. That way BuildStream should be able to leverage a Remote Execution service even without it providing an ArtifactCache. Note that this is a similar use case as where builds are remotely executed, but there is no-one with push rights to a configured ArtifactCache. In this scenario there would still be a benefit from the Execution service (through its ActionCache). With BuildGrid, BuildFarm, BuildBarn and now Google's Remote Build Execution service, we should ensure that we can support more than a single implementation. |
|
In GitLab by [Gitlab user @jmacarthur] on Aug 9, 2018, 16:50 mentioned in merge request !626 |
|
In GitLab by [Gitlab user @jmacarthur] on Aug 20, 2018, 08:49 marked the task Add keys to the project configuration to specify the BuildGrid server as completed |
|
In GitLab by [Gitlab user @jmacarthur] on Aug 20, 2018, 08:50 Configuration options added (and raised as a proposal on the mailing list). Needs some cleaning of the commit messages. |
|
In GitLab by [Gitlab user @jmacarthur] on Aug 21, 2018, 17:47 Commits cleaned up. I've reduced the configuration options to a minimum because they were taking a long time to get right. This is now ready for review. |
|
In GitLab by [Gitlab user @jmacarthur] on Aug 24, 2018, 17:03 changed the description |
|
In GitLab by [Gitlab user @jmacarthur] on Aug 24, 2018, 17:07 Only comment so far is that we should include a configuration variable for the instance name, currently hardcoded to 'default'. This should be fairly easy to do but I don't have time to implement and test it now. The other thing missing are tests; we could do a very simple test which verifies that the configuration variables work, but other than that testing is going to require a lot of work. We need to set up an artifact cache, buildgrid and buildbox on a machine as well as Buildstream - ideally, with at least Buildgrid and Buildstream on different machines. I'll be back on the 10th September 2018. Please feel free to take over !626 while I'm on holiday if you like. |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 10:58 mentioned in commit 7e1f9bb31b609d27f6eb10728ea0c55f67b88270 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 10:58 mentioned in commit db1b357b201908be89a7516c631d98ddb25373f5 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 10:58 mentioned in commit ee47f659e5992c05282b509d287c606f3c4988a1 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 10:58 mentioned in commit 28bd46d6fe5da0777290cddfa286f1053eb28d8d |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 10:58 mentioned in commit 9bd3a6257d3a858e680a353f55ba28f7018b5850 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 10:58 mentioned in commit 32773bf704b9bed20ce35967589a532a4d5cb217 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 10:58 mentioned in commit 8a033d8516face9beb0abf704adfdab30f4c47e5 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 10:58 mentioned in commit 2a96530e71190d97ea29aac0af9611eee54a2127 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 10:58 mentioned in commit ac116aede9a4a2802fb78271989831bef2111e11 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 10:58 mentioned in commit e7b26e076dfee64073708ccea13dc5c063d6ec08 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 4, 2018, 12:09 mentioned in commit 1e543f123d69ccd64b93c93661059bcf4c7c2435 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 6, 2018, 18:19 mentioned in commit 27f99507d297fe782e0184abfaee052f8b0696e1 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 6, 2018, 18:19 mentioned in commit 9a17aeafaf794aecc9c47fb0b0c400c21e9d6d6c |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 13:47 mentioned in commit 2726b59c4862038d40473545c8fda5fa4cef68bb |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 13:47 mentioned in commit 924b872d8c3b145d30b4ccfb63fa4a458e2bd5f3 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit c0ef710 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit da53c00 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 116f80d |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit b0f4654 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 89b29b8 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 43651af |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit c73896b |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 909b3be |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 71e2b54 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 05956c7 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 50bf313 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 7306a95 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit e7659c2 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 556f2ab |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit d4c515d |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 853e0af |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 9ef5316 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 936bb93 |
|
In GitLab by [Gitlab user @martinblanchard] on Sep 7, 2018, 14:24 mentioned in commit 923d443 |
|
In GitLab by [Gitlab user @LaurenceUrhegyi] on Sep 11, 2018, 19:36 marked this issue as related to BuildGrid/buildgrid#52 |
|
In GitLab by [Gitlab user @LaurenceUrhegyi] on Sep 11, 2018, 19:38 [Gitlab user @jmacarthur] regarding testing, I have linked to a related issue over at BuildGrid, which should capture all the details of the end to end testing taking place as a cluster of machines is set up - https://gitlab.com/BuildGrid/buildgrid/issues/52 |
|
In GitLab by [Gitlab user @juergbi] on Oct 1, 2018, 17:19 mentioned in commit c0a9353aa1beffcc50ef77a5ab9eade78c86084a |
|
In GitLab by [Gitlab user @tristanvb] on Oct 3, 2018, 08:35 mentioned in commit 9568824 |
|
In GitLab by [Gitlab user @jmacarthur] on Oct 3, 2018, 09:30 marked the task Add keys to the project configuration to specify the BuildGrid server and instance as completed |
|
In GitLab by [Gitlab user @jmacarthur] on Oct 3, 2018, 09:30 closed |
See original issue on GitLab
In GitLab by [Gitlab user @jmacarthur] on Jul 3, 2018, 15:09
Background
A continuation of #417. Add the necessary client-side code for BuildStream to use remote execution.
Task description
Acceptance Criteria
Demonstrate use of remote execution to build Freedesktop-SDK and any other projects we see fit, with the same results as building locally.
The text was updated successfully, but these errors were encountered: