Skip to content

Latest commit

 

History

History
228 lines (145 loc) · 12.4 KB

2024.md

File metadata and controls

228 lines (145 loc) · 12.4 KB

Ideas

Welcome to our Google Summer of Code ideas list!

small (90 hours)

Nix cleanup: merge ParsedDerivation into BasicDerivation, improve JSON format

Nix currently special-cases some environment variables in derivations to convey extra meaning ("system features", "prefer local build", etc.). We're stuck with this in the .drv A-Term format and builtins.derivation because of backwards compatibility requirements, but that doesn't mean our internal data structures and (currently experimental, i.e. not yet stabilized and immobile) JSON format cannot be improved.

This project would be about doing just that: Fix the internal data structures, and then use that refactor to make the JSON format better.

As a stretch goal, we can then start proposing/prototyping a worthy successor to builtins.derivation that would better support this stuff. (C.f. the "structured attrs" work in Nixpkgs.)

See NixOS/nix#9846 and NixOS/nix#9866.

Possible mentors:

nixpkgs library networking functions

Many things in NixOS use the Internet Protocol and other network related technologies, thus a lot of modules and service have to deal with them. Currently this is mostly done just passing strings as arguments, but it would be a lot more convenient to have things like lib.types.ipv6, lib.types.ipv4 and other functions to for example convert between cidr notation and the bit representation of netmasks. Some inspiration for functions can be drawn from similar libraries like the python ipaddress lib This would not just be useful for NixOS it self but also for its users since a lot of folks use it to power network appliances and build infrastructure on top of it.

Skills required/preferred: Some prior networking knowledge would be useful but is not required. Prior knowledge of the nix programming language is not required, any new-comer wanting to pick this up should easily be able to complete a nix tutorial like a tour of Nix in one or two afternoons and be good to go.

Possible mentors:

Rating: medium

Prior efforts: there was some prior efforts in nixpkgs#258250 but it is stuck for a few months now and also only adds function for legacy ip, it's also quite bare bones and we might want some more advanced things like a function to check if a ip matches a given subnet.

First class support for Crystal applications

Currently, our support for applications written in crystal consists of 2 main pieces:

  1. crystal2nix for converting lock files created by the shards tool to nix.
  2. the nixpkgs build infrastructure that supports a range of different means to build crystal applications (incl consuming the outputs of item 1).

The existing support is brittle and unpleasant to read and only supports dependencies fetched via git.

This project would be interesting as it touches several different types of projects:

  1. redesigning the build infrastructure to be easier to hack on (nixpkgs)
  2. implementing the new build infrastructure (nixpkgs)
  3. extending the crystal2nix tool to understand other VCSs (crystal2nix)

Skills preferred: a. nix and bash for the build infrastructure b. crystal or ruby for crystal2nix (if you know ruby, you know crystal)

Possible Mentors:

Difficulty: easy

Prior efforts: We have both working crystal build infrastructure as well as a working crystal2nix, so you will not be starting from scratch.

Centralized on-failure reporting for failed systemd units

A systemd service supports triggering another unit if it fails through the OnFailure option. In NixOS we have centralized configuration and yet no way to easily opt in to common/shared OnFailure handlers.

With such infrastructure in place, it becomes trivial to have hosts react properly to failure and outages in addition to whatever other monitoring might be in place.

This project consist of a few different items:

  1. mapping out and understanding the various failure modes that running software is exposed to
  2. implementing both the common NixOS infrastructure but also various types of handlers that will react to failures
  3. test infrastructure through NixOS tests for verification

Skills preferred: a. basic nix for NixOS modules

Possible mentors:

Rating: easy

Prior efforts: We have boot counting that is one part of the puzzle and with what we would want to integrate the standard failure handlers.

medium (175 hours)

Nix Internals: Use std::filesystem::path for Path

Nix currently has a bunch of home-grown file system and file path utilities. However in recent years (C++17) the standard library has gained a new std::fileystem library which has much of the same functionality.

The purpose of this project is to use std::filesystem where possible, with a particular emphasis on using the std::filesystem::path type rather than our own Path (which is just an alias for std::string).

This shrinks Nix (no need to maintain redundant code!), and will also aid in porting Nix to Windows, since an explicit design goal of std::filesystem is to be portable between Unix and Windows.

See NixOS/nix#9205 for details.

Possible mentors:

nixpkgs pnpm tooling

pnpm is a package manager that claims to be up to 2x faster than npm and more efficient than npm when it comes to disk space usage, because of this benefits more and more projects are using pnpm. We have tooling for the npm but it's mostly incompatible, this is leading to hacky workarounds and makes packaging pnpm projects a pain. If you decide to work on this project you will gain a lot of knowledge about the gears that make nixpkgs turn, namely things like setup hooks and fetcher.

Skills required/preferred: Having experience with javascript/frontend tooling like nodejs would most certainly be a plus.

Possible mentors:

Rating: hard

Prior efforts: There is a tool called pnpm2nix but it didn't get updates in a few years because of pnpm#1035, this should be a non problem for us if we use a fod fetcher like the nixpkgs npm tooling. Some packages use a fetcher without strong reproducibility guarantees.

large (350 hours)

outpath cache for faster automation

OfBorg and nixpkgs-update rely on a tool that calculates the output directories for every derivation in the top level of nixpkgs. The tool is run before and after a change to detect how many packages will be affected by the change. It is currently very slow. In fact, it is probably about 80% of the total computation nixpkgs-update does. This project would be to cache information about what kind of edits affect other derivations, or statically determine it in a faster way than calculating the outpath of every derivation. A successful project would speed up OfBorg and/or nixpkgs-update significantly, leading to faster Nixpkgs PR automation and CI velocity.

Skills preferred: Rust (OfBorg) or Haskell (nixpkgs-update)

Possible mentors:

Rating: medium

Prior efforts:

Testing and enhancing Dynamic Derivations

Existing work on an experimental feature known as "dynamic derivations", which refers to derivation outputs that can themselves be derivations, lacks the adequate testing to be upstreamed. Moreover, functionality can be added to this feature in order to enable "lang2nix" tools to act in a more seamless manner than they do now.

The goal of this project is thus to thoroughly test this feature, in addition to adding new functionalities to it. Such functionalities include but are not limited to:

  • Methods for converting foreign packages to Nix, such as with graphs for various package managers, we can also create conversion tools for Ninja and Make graphs. This could greatly benefit large builds like Chromium and LLVM.

  • Additionally, determine a path for upstream tools like CMake, Meson, etc., to integrate with Nix.

  • We can also explore modifying language-specific package managers to utilize Nix for the actual building process.

Possible mentors:

Rating: Hard

References:

Nixpkgs: Build GCC libraries and compiler separately

This project aims to redo the way we build GCC so it is the same as LLVM: build libraries separately from compiler, let Nixpkgs control the bootstrapping. Exherbo Linux builds GCC in this way, so there is precedent.

Doing this will enable major simplifications in bootstrapping, cross compiling, cc-wrapper, etc. These benefits however are fine to leave until after the GSOC project ends.

This is a very advanced project because it requires building a very old convoluted project (GCC) in ways that the upstream project doesn't anticipate. The ability to be self-motivated, researching obscure error messages that few other people have encountered before, and generally trace issues to their root causes is required. Being able to read GCC's source code, and potentially create patches (in C, C++, etc.) and upstream them is also a useful skill.

Possible mentors:

Rating: Hard

devenv language support improvements

There's a lot of room for improving languages support in devenv project.

The main focus areas are:

  • add more languages that are commonly used
  • provide more fine grained support for languages, for example specifying what LSP to use
  • extending testing infrastructure
  • write glue code for mapping nixpkgs packages to language specific libraries

Rating: Medium

Possible mentors

Nixpkgs analytics: nixpkgs-review with a time-budget

Nixpkgs and its infrastructure feature fifteen years of history of the open source software: in the form of build and test logs, dependency graphs, and conversations. Compared to the opportunities offered by this data, we'll attempt a modest task: write a version of nixpkgs-review or nix-fast-build that can follow a fixed time-budget.

The problem decomposes into at least three tasks:

  • Obtaining the data from Hydra and preparing it for redistribution and downstream usage.
  • Working out a statistical model for the build times, including online update rules.
  • Writing the evaluation and build scheduler that can consult such a model.

Rating: Hard

Possible mentors

Skills: designing and implementing algorithms; experience of working with data and the basic numerical sympathy will be helpful.

References: