Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

aria2: refactor to avoid python2 dependency #101978

Merged
merged 1 commit into from
Oct 29, 2020
Merged

aria2: refactor to avoid python2 dependency #101978

merged 1 commit into from
Oct 29, 2020

Conversation

braunse
Copy link
Contributor

@braunse braunse commented Oct 28, 2020 •
edited
Loading

Use python3 Sphinx package instead of python2 Sphinx package,
to avoid build-time dependency on python 2.

Motivation for this change

To remove the build dependency on python2, as per #101964

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@braunse
aria2: refactor to avoid python2 dependency
aa8525a 
Use python3 Sphinx package instead of python2 Sphinx package,
to avoid build-time dependency on python 2, as per
NixOS#101964
@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Oct 28, 2020 •
edited
Loading

Result of nixpkgs-review pr 101978 run on x86_64-linux 1

6 packages failed to build:
  • aria
  • persepolis
  • python37Packages.aria2p
  • python38Packages.aria2p
  • uget
  • uget-integrator

From aria.log:

Can't exec "/bin/sh": No such file or directory at /nix/store/38vxlxrvg3yji3jms44qn94lxdysbj5j-perl-5.32.0/lib/perl5/5.32.0/x86_64-linux-thread-multi/IO/File.pm line 186.

@braunse
Copy link
Contributor Author

braunse commented Oct 28, 2020

Hmmm.... Here's what happens for me, on Debian if that matters:

❯ nix-shell -p nixpkgs-review --run "nixpkgs-review pr 101978"
$ git -c fetch.prune=false fetch --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/101978/head:refs/nixpkgs-review/1
remote: Enumerating objects: 10, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 10 (delta 5), reused 7 (delta 5), pack-reused 0
Unpacking objects: 100% (10/10), 3.20 KiB | 252.00 KiB/s, done.
From https://github.com/NixOS/nixpkgs
   0dc7ad6f1fc..4a64cdd39ba  master                -> refs/nixpkgs-review/0
 * [new ref]                 refs/pull/101978/head -> refs/nixpkgs-review/1
$ git worktree add /home/seb/.cache/nixpkgs-review/pr-101978/nixpkgs 4a64cdd39ba5efc692205df8f15bc0924c5e0bf1
Preparing worktree (detached HEAD 4a64cdd39ba)
Updating files: 100% (23067/23067), done.
HEAD is now at 4a64cdd39ba Merge pull request #101952 from mweinelt/p10k
$ nix-env -f /home/seb/.cache/nixpkgs-review/pr-101978/nixpkgs -qaP --xml --out-path --show-trace
$ git merge --no-commit aa8525a8ec73d832b0f7b2f1a0f0fe0d670c2f0d
Automatic merge went well; stopped before committing as requested
$ nix-env -f /home/seb/.cache/nixpkgs-review/pr-101978/nixpkgs -qaP --xml --out-path --show-trace --meta
6 packages updated:
aria2 persepolis python3.7-aria2p python3.8-aria2p uget uget-integrator

$ nix build --no-link --keep-going --option build-use-sandbox relaxed -f /home/seb/.cache/nixpkgs-review/pr-101978/build.nix
[6 built, 482 copied (1316.6 MiB), 293.5 MiB DL]
https://github.com/NixOS/nixpkgs/pull/101978
6 packages built:
aria persepolis python37Packages.aria2p python38Packages.aria2p uget uget-integrator

@zopieux
Copy link
Contributor

zopieux commented Oct 28, 2020

Tested working on x86_64 Archlinux starting with an empty /nix directory. Managed to download a magnet torrent with the built aria2c.

Logs 
$ sudo mount -t tmpfs -o size=20G tmpfs /nix && curl -L https://nixos.org/nix/install | sh && source ~/.nix-profile/etc/profile.d/nix.sh
$ nix-shell -p nixpkgs-review
$ nixpkgs-review pr 101978
(snip)
[7 built, 490 copied (1341.5 MiB), 297.0 MiB DL]
https://github.com/NixOS/nixpkgs/pull/101978
1 package marked as broken and skipped:
kiwix

6 packages built:
aria persepolis python37Packages.aria2p python38Packages.aria2p uget uget-integrator

$ aria2c 'magnet:?xt=urn:btih:dd8255ecdc7ca55fb0bbf81323d87062db1f6d1c&dn=Big+Buck+Bunny&tr=udp%3A%2F%2Fexplodie.org%3A6969&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Ftracker.empire-js.us%3A1337&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337&tr=wss%3A%2F%2Ftracker.btorrent.xyz&tr=wss%3A%2F%2Ftracker.fastcast.nz&tr=wss%3A%2F%2Ftracker.openwebtorrent.com&ws=https%3A%2F%2Fwebtorrent.io%2Ftorrents%2F&xs=https%3A%2F%2Fwebtorrent.io%2Ftorrents%2Fbig-buck-bunny.torrent'

10/28 23:54:45 [NOTICE] Downloading 1 item(s)
10/28 23:54:45 [NOTICE] IPv4 DHT: listening on UDP port 6960
10/28 23:54:45 [NOTICE] IPv4 BitTorrent: listening on TCP port 6909
10/28 23:54:45 [NOTICE] IPv6 BitTorrent: listening on TCP port 6909
[#e226df 20KiB/20KiB(100%) CN:20 SD:1]                                                                                                                                                                                                                                                                                      
10/28 23:54:55 [NOTICE] Download complete: [MEMORY][METADATA]Big+Buck+Bunny
[#a6cd3e SEED(0.0) CN:1 SD:0]                                                                                                                                                                                                                                                                                               
10/28 23:55:40 [NOTICE] Shutdown sequence commencing... Press Ctrl-C again for emergency shutdown.
10/28 23:55:40 [NOTICE] Download complete: Big Buck Bunny
10/28 23:55:40 [NOTICE] Your share ratio was 0.0, uploaded/downloaded=0B/263MiB

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
e226df|OK  |       0B/s|[MEMORY][METADATA]Big+Buck+Bunny
a6cd3e|OK  |    11MiB/s|Big Buck Bunny/Big Buck Bunny.en.srt (2more)

Status Legend:
(OK):download completed.

@SuperSandro2000
Copy link
Member

Could it be that my fail is related to an error in the perl package rather than aria2?

@braunse
Copy link
Contributor Author

braunse commented Oct 29, 2020

I tested it again on a freshly installed NixOS 20.09 VM, and could not reproduce @SuperSandro2000's problem.

Logs:

[seb@alfons-nixdev:~/dev/nixpkgs]$ nix-shell -p nixpkgs-review --run "nixpkgs-review pr 101978"
$ git -c fetch.prune=false fetch --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/101978/head:refs/nixpkgs-review/1
$ git worktree add /home/seb/.cache/nixpkgs-review/pr-101978-1/nixpkgs df6068e934ad2ea8db02609c335b23ac96c3d286
Preparing worktree (detached HEAD df6068e934a)
Updating files: 100% (23069/23069), done.
HEAD is now at df6068e934a Merge pull request #101985 from rnhmjoj/monero
$ git merge --no-commit aa8525a8ec73d832b0f7b2f1a0f0fe0d670c2f0d
Auto-merging pkgs/top-level/all-packages.nix
Automatic merge went well; stopped before committing as requested
$ nix build --no-link --keep-going --option build-use-sandbox relaxed -f /home/seb/.cache/nixpkgs-review/pr-101978-1/build.nix
[7 built, 558 copied (1807.6 MiB), 397.7 MiB DL]
https://github.com/NixOS/nixpkgs/pull/101978
1 package marked as broken and skipped:
kiwix

6 packages built:
aria persepolis python37Packages.aria2p python38Packages.aria2p uget uget-integrator

[0.0 MiB DL]
$ nix-shell /home/seb/.cache/nixpkgs-review/pr-101978-1/shell.nix
these paths will be fetched (0.51 MiB download, 2.47 MiB unpacked):
  /nix/store/82bsy6f0bz7avmy8y62j0z2dnxgi8pxw-bash-interactive-4.4-p23-doc
  /nix/store/b6mp5xw5pjr3bsmk3fphkyxw73vc806j-bash-interactive-4.4-p23-info
  /nix/store/hfvjs6abgzadrv9f387v1ip5mr206jh9-bash-interactive-4.4-p23-dev
  /nix/store/xpmrr7kyzlbny0540kzd1vlqsca69y13-bash-interactive-4.4-p23-man
copying path '/nix/store/82bsy6f0bz7avmy8y62j0z2dnxgi8pxw-bash-interactive-4.4-p23-doc' from 'https://cache.nixos.org'...
copying path '/nix/store/xpmrr7kyzlbny0540kzd1vlqsca69y13-bash-interactive-4.4-p23-man' from 'https://cache.nixos.org'...
copying path '/nix/store/hfvjs6abgzadrv9f387v1ip5mr206jh9-bash-interactive-4.4-p23-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/b6mp5xw5pjr3bsmk3fphkyxw73vc806j-bash-interactive-4.4-p23-info' from 'https://cache.nixos.org'...

[nix-shell:~/.cache/nixpkgs-review/pr-101978-1]$ aria2
aria2c  aria2p  

[nix-shell:~/.cache/nixpkgs-review/pr-101978-1]$ aria2c http://google.com

10/29 08:45:40 [NOTICE] Downloading 1 item(s)

10/29 08:45:40 [NOTICE] CUID#7 - Redirecting to http://www.google.com/

10/29 08:45:40 [NOTICE] Download complete: /home/seb/.cache/nixpkgs-review/pr-101978-1/index.html

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
18ab42|OK  |    71KiB/s|/home/seb/.cache/nixpkgs-review/pr-101978-1/index.html

Status Legend:
(OK):download completed.

[nix-shell:~/.cache/nixpkgs-review/pr-101978-1]$ exit
$ git worktree prune

@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Oct 29, 2020 •
edited
Loading

Result of nixpkgs-review pr 101978 run on x86_64-linux 1

1 package marked as broken and skipped:
  • kiwix
2 packages failed to build:
  • uget
  • uget-integrator
4 packages built:
  • aria (aria2)
  • persepolis
  • python37Packages.aria2p
  • python38Packages.aria2p
configuring
configure flags: --disable-dependency-tracking --prefix=/nix/store/pjsfmk5gq3n7vr17znhjrzxmf4d08hra-uget-2.2.3-1
/nix/store/q1zjp9grl4w92qalkdqjs2bj5d0pf8ih-stdenv-linux/setup: ./configure: /bin/sh: bad interpreter: No such file or directory

Uget depends on Aria and would be broken with this PR https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/networking/uget/default.nix#L2. Please fix.

@braunse
Copy link
Contributor Author

braunse commented Oct 29, 2020 •
edited
Loading

AFAICT, there should be nothing in this PR that relates to whether uget's configure script can find /bin/sh in any way, so I'm unsure how to go about fixing this. Any hints would be appreciated.

FWIW, uget builds fine for me with this PR, both on my Debian machine with Nix installed and on a NixOS 20.09 VM. @SuperSandro2000 can you check that you are able to build uget on the master branch without this PR?

Update: I have a fix for this in https://github.com/braunse/nixpkgs/tree/patch-configure-shebang-in-uget.
Since the fix (patchShebangs . in preConfigure) seems unrelated to aria2, I'm unsure whether I should add it to this PR, or open a fresh PR?

braunse added a commit to braunse/nixpkgs that referenced this pull request Oct 29, 2020
@braunse
uget: Patch /bin/sh reference in configure
1a7fcb8 
To fix build failure referenced in
NixOS#101978 (comment)
@SuperSandro2000
Copy link
Member

Result of nixpkgs-review pr 101978 run on x86_64-linux 1

1 package marked as broken and skipped:
  • kiwix
1 package failed to build:
  • persepolis
5 packages built:
  • aria (aria2)
  • python37Packages.aria2p
  • python38Packages.aria2p
  • uget
  • uget-integrator

@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Oct 29, 2020 •
edited
Loading

Uget gets pulled from cache for me so I can't check if it build or not but I suppose it should.

Edit: Now Uget builds after collecting nix store garbage. The persepolis build failure is unrelated to this PR.

error: can't copy 'man/persepolis.1.gz': doesn't exist or not a regular file

But this is fixed on master.

jonringer
jonringer approved these changes Oct 29, 2020
View reviewed changes
Copy link
Contributor

@jonringer jonringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Result of nixpkgs-review pr 101978 1

1 package marked as broken and skipped:
  • kiwix
6 packages built:
  • aria (aria2)
  • persepolis
  • python37Packages.aria2p
  • python38Packages.aria2p
  • uget
  • uget-integrator

@jonringer jonringer merged commit 8bdb520 into NixOS:master Oct 29, 2020
@braunse braunse deleted the update-aria2-python2-dep branch October 30, 2020 08:13
@Janik-Haag Janik-Haag added the 12. first-time contribution This PR is the author's first one; please be gentle! label Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jonringer jonringer jonringer approved these changes

@SuperSandro2000 SuperSandro2000 SuperSandro2000 approved these changes

Assignees
No one assigned
Labels
10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 12. first-time contribution This PR is the author's first one; please be gentle!
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@braunse @SuperSandro2000 @zopieux @jonringer @Janik-Haag