Why aren't certificates from security.pki used in the cacert package or some fetchers?

[expipiplus1]

Describe the bug

Some fetchers (at least fetchgit) do not use extra certificates set in the nixos option security.pki.certificates or certificateFiles. This causes builds to fail when used with a TLS intercepting proxy

see also #89526

To Reproduce

Steps to reproduce the behavior:

• be behind a TLS intercepting proxy

• Set the nixos option security.pki.certificates to include the MITM certs and switch

• Observe that a user can use curl and git and successfully navigate through the proxy

• Observe that builtins.fetchurl works

  nix-build $(nix-instantiate --expr 'builtins.fetchurl{url="https://github.com"; sha256="1111111111111111111111111111111111111111111111111111111111111111";}')
  # Fails after fetching the file
  

• Observe that fetchgit fails

   nix-build $(nix-instantiate --expr 'with (import <nixpkgs>{}); fetchgit{url="https://github.com/nixos/nixpkgs";sha256="1111111111111111111111111111111111111111111111111111111111111111";rev="foo";}')
  warning: you did not specify '--add-root'; the result might be removed by the garbage collector
  these derivations will be built:
    /nix/store/iskxcb2sqzfqicgdx7dryrj5kiacm6ma-nixpkgs.drv
  building '/nix/store/iskxcb2sqzfqicgdx7dryrj5kiacm6ma-nixpkgs.drv'...
  exporting https://github.com/nixos/nixpkgs (rev foo) into /nix/store/wqfh9mzx0c2c47dz59yc2wrck49zslrm-nixpkgs
  Initialized empty Git repository in /nix/store/wqfh9mzx0c2c47dz59yc2wrck49zslrm-nixpkgs/.git/
  fatal: unable to access 'https://github.com/nixos/nixpkgs/': SSL certificate problem: unable to get local issuer certificate
  fatal: unable to access 'https://github.com/nixos/nixpkgs/': SSL certificate problem: unable to get local issuer certificate
  fatal: unable to access 'https://github.com/nixos/nixpkgs/': SSL certificate problem: unable to get local issuer certificate
  Unable to checkout refs/tags/foo from https://github.com/nixos/nixpkgs.
  builder for '/nix/store/iskxcb2sqzfqicgdx7dryrj5kiacm6ma-nixpkgs.drv' failed with exit code 1
  

• Modify the cacert package to include the MITM certs in ca-bundle.crt

• Observe that fetchgit works

  nix-build $(nix-instantiate --expr 'with (import <nixpkgs>{}); fetchgit{url="https://github.com/nixos/nixpkgs";sha256="1111111111111111111111111111111111111111111111111111111111111111";rev="foo";}')                         ~
  warning: you did not specify '--add-root'; the result might be removed by the garbage collector
  these derivations will be built:
    /nix/store/bwfscaj59r1vjhrvx6myny762s3szkkc-nixpkgs.drv
  building '/nix/store/bwfscaj59r1vjhrvx6myny762s3szkkc-nixpkgs.drv'...
  exporting https://github.com/nixos/nixpkgs (rev foo) into /nix/store/wqfh9mzx0c2c47dz59yc2wrck49zslrm-nixpkgs
  Initialized empty Git repository in /nix/store/wqfh9mzx0c2c47dz59yc2wrck49zslrm-nixpkgs/.git/
  fatal: couldn't find remote ref refs/tags/foo
  remote: Enumerating objects: 28, done.
  remote: Counting objects: 100% (28/28), done.
  remote: Compressing objects: 100% (28/28), done.
  

Expected behavior

The certs used for fetchers should be consistent. And if the MITM certs aren't used, then documentation should explain what a user must do to use them in fetchers.

Metadata

 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.70, NixOS, 20.03.git.d3784204ba1 (Markhor)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.6`
 - channels(root): `"nixos-19.09.2152.790970f6b1c"`
 - nixpkgs: `/home/j/src/nixpkgs`

[rnhmjoj]

It's a known problem: many libraries and applications use their own set of CAs instead of relying on the system trust store. This is main tracking issue: #8247.

[roberth]

#8247 only discusses applications, whereas this is about a fetcher.

Fetchers aren't builds and cryptographic keys inherently need to be mutable and unreproducible, because you always need the latest ones. Nix needs to acknowledge this and provide fetchers with an up to date cacert bundle, such as (presumably) the system certificate bundle.

Purity is not a concern for fetchers so we can make the system certificates available in the sandbox as long as the derivation is fixed-output.

This should be a Nix sandbox feature. It can't be solved properly in nixpkgs alone.
@expipiplus1 would you mind creating an issue on the nix repo?

[expipiplus1]

@roberth, done. Thanks