Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

code-oss: init at 1.89.1 #318673

Closed
wants to merge 8 commits into from
Closed

code-oss: init at 1.89.1 #318673

wants to merge 8 commits into from

Conversation

inclyc
Copy link
Member

@inclyc inclyc commented Jun 10, 2024

Description of changes

This PR adds a package named vscode-oss that build https://github.com/microsoft/vscode from source. (Currently we have vscode + vscodium but they are not built by nix).

Not only does this provide vscode desktop, but also remote-ssh server and web server in separated outputs. That is, users of this package can use vscode-oss.reh for their remote-ssh extension host, instead of downloading a large unknown binary from microsoft.

Electron, nodejs, are reused from nixpkgs.

The package also allows user changing the product.json, to customize their own vscode variant.

I don't know how to use by-name technology as well as reusing vscode directory, sorry ;)

Initially when I wrote these expressions, vscode is on 1.89.1 and I tested it. Automatic version bump, 1.90.0, I think I can file another subsequent PRs.

Fixes: #26698
Closes: #86122

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@inclyc inclyc marked this pull request as ready for review June 10, 2024 02:41
@inclyc inclyc requested a review from malob June 10, 2024 02:41
@inclyc inclyc requested a review from amaxine June 10, 2024 03:50
@amaxine amaxine removed their request for review June 10, 2024 07:10
@SuperSandro2000
Copy link
Member

Did you explore how hard it would be to do this for vscodium?

@inclyc
Copy link
Member Author

inclyc commented Jun 11, 2024

Did you explore how hard it would be to do this for vscodium?

VSCodium seems to be a set of patches (which aims to remove some MS branding stuff) combined with a LOT of scripts.

Patches: https://github.com/VSCodium/vscodium/tree/master/patches

maybe for nixpkgs we can just apply their patches, but not reusing their scripts. Some of them basically look scary:

https://github.com/VSCodium/vscodium/blob/ffd7e631b05feef1b9142adb9e63303549cf4e6c/package_linux_bin.sh#L37

};

meta = {
description = ''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

meta.description should fit on a single line without being to long to fit in on search.nixos.org

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

meta.description should fit on a single line without being to long to fit in on search.nixos.org

These are copied from previous nixpkgs version.

Open source source code editor developed by Microsoft for Windows,

Should I update this then?

Linux and macOS
'';
longDescription = ''
Open source source code editor developed by Microsoft for Windows,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

longDescription should not repeat description, as they are often displayed very close to each other.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, these words have already presented in nixpkgs before:

Open source source code editor developed by Microsoft for Windows,

}:

let
inherit (nodePackages) node-gyp;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Task for another PR: we should package this package outside of nodePackages

pkgs/applications/editors/vscode/vscode-oss.nix Outdated Show resolved Hide resolved
'';

outputHashMode = "recursive";
outputHashAlgo = "sha256";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is redundant with outputHash

"out"

# Remote extension host, for vscode remote development
"reh"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would write those out, then the outputs are longer but it is easier to understand what they are. As a long term vscode user that fiddled a lot with remote extensions in nixos wsl, i have never seen this abbreviation before.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would write those out, then the outputs are longer but it is easier to understand what they are. As a long term vscode user that fiddled a lot with remote extensions in nixos wsl, i have never seen this abbreviation before.

Could you suggest a new name? maybe "remote-extension-host" ?

I don't have strong opinion about this but personally I want to keep the name because there are same abbreviation in VSCodium community, and VSCode gulp task. (Yes, microsoft just named it reh)

xorg.libxkbfile
] ++ lib.optionals stdenv.isDarwin [ darwin.apple_sdk.frameworks.Cocoa ];

patches = map (name: ./vscode-oss/patches/${name}) (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we might need to use filterSource here to not copy the entire tree again. There should be an issue about this.

licenseFileName = "LICENSE.txt";
reportIssueUrl = "https://github.com/microsoft/vscode/issues/new";
nodejsRepository = "https://nodejs.org";
urlProtocol = "code-oss";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this break erb links from eg gitlab?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what's the meaning of this field actually, basically reading some comments from microsoft/vscode-docs#5215. As per my understanding, it seems to be OK with code-oss, right?

Copy link
Contributor

@Pandapip1 Pandapip1 Jun 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can attest that this does break some extensions. Please override it to make it vscode (or at the least, a parameter to override it would be greatly appreciated!)

extensionAllowedBadgeProvidersRegex = [
"^https:\\/\\/github\\.com\\/[^/]+\\/[^/]+\\/(actions\\/)?workflows\\/.*badge\\.svg"
];
extensionEnabledApiProposals = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems rather aggressive to put this into nix code. Can't we read that from the downloaded source file?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we read that from the downloaded source file?

The source file is rather empty, and these items listed mainly comes from VSCodium community.

https://github.com/microsoft/vscode/blob/main/product.json

@inclyc inclyc changed the title vscode-oss: init at 1.89.1 vscode-oss: init at 1.90.0 Jun 11, 2024
This reverts commit c738881.

code-oss> [05:53:03] 'compile-extensions-build' errored after 2.71 s
code-oss> [05:53:03] Error: Command failed: yarn list --prod --json
code-oss>     at genericNodeError (node:internal/errors:984:15)
code-oss>     at wrappedFn (node:internal/errors:538:14)
code-oss>     at ChildProcess.exithandler (node:child_process:422:12)
code-oss>     at ChildProcess.emit (node:events:518:28)
code-oss>     at ChildProcess.emit (node:domain:551:15)
code-oss>     at maybeClose (node:internal/child_process:1105:16)
code-oss>     at Socket.<anonymous> (node:internal/child_process:457:11)
code-oss>     at Socket.emit (node:events:518:28)
code-oss>     at Socket.emit (node:domain:551:15)
code-oss>     at Pipe.<anonymous> (node:net:337:12)
code-oss> error Command failed with exit code 1.
@inclyc
Copy link
Member Author

inclyc commented Jun 12, 2024

1.90.0 cannot be built from linux, revert it for now.

@inclyc inclyc changed the title vscode-oss: init at 1.90.0 vscode-oss: init at 1.89.1 Jun 12, 2024
@linsui
Copy link
Contributor

linsui commented Jun 21, 2024

Does this work with both extensions installed from nix and the market at the same time?

@inclyc
Copy link
Member Author

inclyc commented Jun 21, 2024

Does this work with both extensions installed from nix and the market at the same time?

Yes.

@batonac
Copy link

batonac commented Jul 1, 2024

@inclyc you're a hero, I'm so happy for this!

Now that 1.90.2 is out, could we try the latest version again?

@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 4, 2024
@inclyc
Copy link
Member Author

inclyc commented Jul 5, 2024

Hi @batonac,

I have a personal testing code-oss package in my flake and I recently bumped it to version 1.90.2.

I'd like to test it for days before updating this PR.

@superherointj superherointj marked this pull request as draft July 20, 2024 12:13
@batonac
Copy link

batonac commented Sep 30, 2024

Hey @inclyc, apparently this stalled?

@inclyc
Copy link
Member Author

inclyc commented Oct 1, 2024

Hey @inclyc, apparently this stalled?

Hi, I'm testing this package in my own flake, it is on 1.93.1

@inclyc inclyc changed the title vscode-oss: init at 1.89.1 code-oss: init at 1.89.1 Oct 10, 2024
@inclyc
Copy link
Member Author

inclyc commented Oct 10, 2024

Because microsoft/vscode 1.94.0 has migrated to npm again, currently it is impossible to package vscode from source. Reasons are:

  • It uses npm, and the offline cache produced by npm itself cannot be reproduced.
  • prefetch-npm-deps cannot deal with such complex project, (i.e. vscode), there are many "package-lock.json" needs to be prefetched, but optional dependencies must be ignored, prefetch-npm-deps seems to be confused by that. Also, npm's "git" deps are causing trouble (e.g. https://registry.npmjs.org/@emmetio%2Fstream-reader).

I'd like to find out a workaround and file another PR, since the build system has changed significantly.

@inclyc inclyc closed this Oct 10, 2024
inclyc added a commit to inclyc/flakes that referenced this pull request Oct 10, 2024
The "Code - OSS" repository has altered its dependency manager to "npm"
which caused numerous issues for nixpkgs.
The most critical issue is the offline cache produced by npm is
completely non-reproducible.
As a result, it is currently challenging to update the version of
"code-oss" (build-from-source version).

For now, I am switching from "code-oss" to "vscodium" the binary
releases maintained by the community.


Link: microsoft/vscode#196795
Link: microsoft/vscode#226927
Link: NixOS/nixpkgs#318673
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: merge conflict This PR has merge conflicts with the target branch 6.topic: vscode 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 10.rebuild-linux: 1 11.by: package-maintainer This PR was created by the maintainer of the package it changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build vscodium (non-branded VSCode) from Source
7 participants