Can't build python37.pytorchWithCuda with nix-build

[GuillaumeDesforges]

Describe the bug
I tried to build pytorch with CUDA on my computer to find a way to patch #69424.

To Reproduce
Steps to reproduce the behavior:

1. git clone https://github.com/NixOS/nixpkgs
2. cd nixpkgs
3. nix-build --verbose -A python37Packages.pytorchWithCuda

Expected behavior
Pytorch should build the same way as if I did nix-env -iA python37Packages.pytorchWithCuda, but instead I have:

...
-- Build files have been written to: /build/source/build
cmake: enabled parallel building
building
Executing setuptoolsBuildPhase
Traceback (most recent call last):
  File "nix_run_setup", line 8, in <module>
    exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/tokenize.py", line 447, in open
    buffer = _builtin_open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'setup.py'
builder for '/nix/store/fqbxsjmp0lh5760h6khabhzydsqi755b-python3.7-pytorch-1.0.0.drv' failed with exit code 1
error: build of '/nix/store/fqbxsjmp0lh5760h6khabhzydsqi755b-python3.7-pytorch-1.0.0.drv' failed

Additional context

I'm a noob here, sorry if this is just basic nixpkgs things that I'm doing wrong.

Metadata

❯ nix run nixpkgs.nix-info -c nix-info -m                                                                         ⏎
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.69, NixOS, 19.03.173426.541d9cce8af (Koi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.2.2`
 - channels(arsleust): `"home-manager-19.03"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[andersk]

git bisect points to f7e28bf “Split buildPythonPackage into setup hooks” (#64997, rebased and merged in #68244) as the first bad commit. @FRidh any thoughts?