Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native bst support on Windows w/ WSL, remote execution only #412

Closed
BuildStream-Migration-Bot opened this issue Feb 3, 2021 · 22 comments
Closed
Labels
enhancement New feature or request

Comments

@BuildStream-Migration-Bot

See original issue on GitLab
In GitLab by [Gitlab user @sstriker] on May 30, 2018, 10:31

As outlined in https://mail.gnome.org/archives/buildstream-list/2018-May/msg00066.html:

Even when the target platform is Linux, there is a large set of developers on either OS X or Windows [laptops]. As we are considering different execution environments on workers, we can also start considering supporting BuildStream natively in these environments.

Remote execution only support is a good starting point. This issue specifically limits the scope to Windows w/ WSL installed (https://docs.microsoft.com/en-us/windows/wsl/about).

Some work may be possible to start, but completion will depend on the remote execution client landing in BuildStream https://gitlab.com/BuildStream/buildstream/issues/454

Some other relevant ML posts:

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @richardmaw-codethink] on Jun 25, 2018, 18:23

marked this issue as related to #411

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @LaurenceUrhegyi] on Jul 23, 2018, 16:58

changed the description

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @LaurenceUrhegyi] on Jul 23, 2018, 17:20

changed the description

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @knownexus] on Aug 6, 2018, 20:16

I'm going to attempt to tackle both this issue and #411 at the same time, so i will be posting this to both issues.
I've been looking into this and attempting to break down the task into smaller chunks

Unfortunately, until i have more knowledge/understanding, a lot of it is vague or guess work, but this is the breakdown i have so far

Research

  • Windows - WSL
  • Alternatives to FUSE

Prep

  • Prepare Windows Laptop
  • Enable WSL on Windows Laptop

Functionality

  • Install Buildstream
  • Find an alternative to the FUSE layer
    • This is not available on WSL
  • Find out which tests fail
  • Make tests pass, or if not possible, skip

Testing

  • (Optional) Add additional test runners to test Windows

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @edbaunton] on Aug 6, 2018, 21:28

Am I correct in thinking that both this and #411 depend on #454?

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @tristanvb] on Aug 7, 2018, 06:07

marked this issue as related to #454

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @tristanvb] on Aug 7, 2018, 06:08

[Gitlab user @edbaunton] I would certainly think that is the sensible way forward, otherwise I don't see how you would test that anything works.

In any case, marking #454 as related to both of these.

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @LaurenceUrhegyi] on Aug 7, 2018, 08:33

changed the description

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @toscalix] on Aug 7, 2018, 10:49

[Gitlab user @knownexus] from what I read about your proposal, the only plan is to consider Mac OSX -> VM -> Windows 10 -> WSL -> BuildStream. Correct?

If the above assumption is correct, from a maintenance and community support perspective, I believe we will need to consider the use case where BuildStream is used with a native Windows plus WSL. In my opinion, this case will get at least as many support requests upstream than the one on a Windows VM.

So I would consider as well: Windows 10 -> WSL -> BuildStream

This use case will also help us to contrast performance hits and other undetermined issues related with WSL, a technology that is fairly recent, compared to the VM case. I would expect small issues when we push it.

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @tristanvb] on Aug 7, 2018, 11:45

I missed the WSL part of this.

To be frank, I think that if we support the Windows 10 with WSL case, it should not be for remote execution only; we should ideally be able to run builds on the Windows 10 machine using the WSL layer.

For remote execution only, it seems to me that we shouldn't really need WSL, it would be nicer to be able to just run natively on windows.

I can see how running BuildStream with remote execution is the lowest effort possible, but I worry about what the longer term support expectations will be for this. For instance, if we duct tape it all together now and declare this setup "supported"; do we need to keep supporting this indefinitely, even when we have BuildStream running natively on windows without WSL ?

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @juergbi] on Aug 7, 2018, 14:07

For instance, if we duct tape it all together now and declare this setup "supported"; do we need to keep supporting this indefinitely, even when we have BuildStream running natively on windows without WSL ?

I don't expect much, if any, work to be required for only remote execution on WSL - except for general support of installing BuildStream without the local build feature, but we want that anyway for macOS. Due to this I don't expect any duct taping or big support costs. If it's indeed such a low hanging fruit, I think it makes sense to support it.

For local execution I expect the main blockers to be FUSE and bubblewrap. For the latter, see containers/bubblewrap#272

For a native Windows port I expect a lot more effort to be required, even just for remote execution. E.g., BuildStream's use of fork() could be a big issue. Due to this I suspect this is still a long way off and the low effort WSL support could be very useful.

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @LaurenceUrhegyi] on Aug 8, 2018, 09:06

changed the description

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @sstriker] on Aug 8, 2018, 15:10

changed the description

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @knownexus] on Aug 8, 2018, 15:30

[Gitlab user @toscalix] apologies if i did not make myself clear, I do not intend to make it work on MacOs by using a windows VM,

i was going to make it work on MacOS and to save from needing a 3rd computer, set up a VM and make it work on windows through that

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @knownexus] on Aug 8, 2018, 15:33

[Gitlab user @tristanvb] That is my goal, however i am unsure of how compatible it will be, either way i need to find a way past the lack of a fuse layer, but if possible, i intend to make windows WSL run buildstream natively

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @knownexus] on Sep 10, 2018, 13:25

mentioned in merge request !726

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @knownexus] on Sep 17, 2018, 17:14

This can now run on remote execution, further testing and documentation is required

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @knownexus] on Sep 18, 2018, 10:15

assigned to [Gitlab user @knownexus]

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @toscalix] on Sep 18, 2018, 15:17

marked this issue as related to #644

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @juergbi] on Sep 27, 2018, 15:55

mentioned in commit 261f65c

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @juergbi] on Sep 27, 2018, 16:10

Fixed in !726.

@BuildStream-Migration-Bot
Copy link
Author

In GitLab by [Gitlab user @juergbi] on Sep 27, 2018, 16:10

closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant