diff --git a/source/conf.py b/source/conf.py index a59a52a66..0e4e15d81 100644 --- a/source/conf.py +++ b/source/conf.py @@ -169,7 +169,7 @@ # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ['_static'] +html_static_path = ['_static', 'tutorials/module-system/files'] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied diff --git a/source/tutorials/module-system/geocode b/source/tutorials/module-system/files/geocode similarity index 100% rename from source/tutorials/module-system/geocode rename to source/tutorials/module-system/files/geocode diff --git a/source/tutorials/module-system/map b/source/tutorials/module-system/files/map similarity index 100% rename from source/tutorials/module-system/map rename to source/tutorials/module-system/files/map diff --git a/source/tutorials/module-system/icat b/source/tutorials/module-system/icat deleted file mode 100755 index cf7109313..000000000 --- a/source/tutorials/module-system/icat +++ /dev/null @@ -1,3 +0,0 @@ -#!/usr/bin/env bash - -feh - diff --git a/source/tutorials/module-system/module-system.md b/source/tutorials/module-system/module-system.md index 8aa0d4541..ac9beb85c 100644 --- a/source/tutorials/module-system/module-system.md +++ b/source/tutorials/module-system/module-system.md @@ -1,260 +1,370 @@ -# Module system introduction +# Deep dive demo: Wrapping the world in modules -Note: This tutorial was created from https://github.com/tweag/summer-of-nix-modules, as presented by @infinisil at Summer of Nix 2021, presentation can be seen here (might not work in browser, `mpv` should work): https://infinisil.com/modules.mp4 +Much of the power in Nixpkgs and NixOS comes from the module system. +It provides mechanisms for conveniently declaring and automatically merging interdependent attribute sets that follow dynamic type constraints, making it easy to express modular configurations. + +In this tutorial you'll learn +- what a module is +- how to define one +- what options are +- how to declare them +- how to express dependencies between modules +and follow extensive demonstration of how to wrap an existing API with Nix modules. + +Concretely, you'll write modules to interact with the Google Maps API, declaring options which represent map geometry, location pins, and more. + +:::{warning} +To run the examples in this tutorial, you will need a [Google API key](https://developers.google.com/maps/documentation/maps-static/start#before-you-begin) in `$XDG_DATA_HOME/google-api/key`. +::: + +Be prepared to see some Nix errors: during the tutorial, you will first write some *incorrect* configurations, creating opportunities to discuss the resulting error messages and how to resolve them, particularly when discussing type checking. + +:::{note} +This tutorial follows [@infinisil](https://github.com/infinisil)'s [presentation on modules](https://infinisil.com/modules.mp4) [(source)](https://github.com/tweag/summer-of-nix-modules) for 2021 Summer of Nix. +::: + +You will use need two helper scripts for this exercise. +Download {download}`map ` and {download}`geocode ` into your working directory. ## Empty module -The simplest module you can have is just an empty attribute set, which -as you might expect, doesn't do anything! +We have to start somewhere. +The simplest module is just a function that takes any attributes and returns an empty attribute set. -```diff -diff --git a/default.nix b/default.nix -new file mode 100644 -index 0000000..1797133 ---- /dev/null -+++ b/default.nix -@@ -0,0 +1,3 @@ -+{ -+ -+} -``` +Write the following into a file called `default.nix`: -## Module arguments: lib +```nix +# default.nix +{ ... }: +{ -Modules can be just an attribute set. But if you want access to some -arguments you need to change it into a function taking an attribute set -with an ellipsis (that's the "..."). In this case we only match on the -`lib` argument, which gives us access to nixpkgs library functions. +} +``` -Note that the ellipsis is necessary since arbitrary arguments can be -passed to modules. +## Module Arguments + +We will need some helper functions, which will come from the Nixpkgs library. +Start by changing the first line in `default.nix`: ```diff -diff --git a/default.nix b/default.nix -index 1797133..f7569bd 100644 ---- a/default.nix -+++ b/default.nix -@@ -1,3 +1,3 @@ --{ -+{ lib, ... }: { - - } +# default.nix +- { ... }: ++ { lib, ... }: +{ + +} ``` -## Declaring generate.script option +Now the module is a function which takes *at least* one argument, called `lib`, and may accept other arguments (expressed by the ellipsis `...`). + +On NixOS, `lib` argument is passed automatically. +This will make Nixpkgs library functions available within the function body. -In order for modules to be useful, we need to have options, so let's -declare one. Options are declared by defining an attribute with -`lib.mkOption` under the `options` attribute. Here we're defining the -option `generate.script`. +:::{note} +The ellipsis `...` is necessary because arbitrary arguments can be passed to modules. +::: -While there are many attributes to customize options, the most -important one is `type`, which specifies what values are valid for an -option, and how/whether multiple values should be merged together. +## Declaring Options -In the nixpkgs library there are a number of types available under -`lib.types`. Here we're using the `lines` type, which specifies that: -- Only strings are valid values -- Multiple strings are joined with newlines +To set any values, the module system first has to know which ones are allowed. + +This is done by declaring *options* that specify which values can be set and used elsewhere. +Options are declared by adding an attribute under the top-level `options` attribute, using `lib.mkOption`. + +In this section, you will define the `generate.script` option. + +Change `default.nix` to include the following declaration: ```diff -diff --git a/default.nix b/default.nix -index f7569bd..cb423a9 100644 ---- a/default.nix -+++ b/default.nix -@@ -1,3 +1,9 @@ +# default.nix { lib, ... }: { -+ -+ options = { -+ generate.script = lib.mkOption { -+ type = lib.types.lines; -+ }; -+ }; - + ++ options = { ++ generate.script = lib.mkOption { ++ type = lib.types.lines; ++ }; ++ }; + } ``` -Let's try to evaluate this with this file: +While many attributes for customizing options are available, the most important one is `type`, which specifies which values are valid for an option. +There are several types available under [`lib.types`](https://nixos.org/manual/nixos/stable/#sec-option-types-basic) in the Nixpkgs library. + +You have just declared `generate.script` with the `lines` type, which specifies that the only valid values are strings, and that multiple definitions should be joined with newlines. + +Write a new file, `eval.nix`, which you will use to evaluate `default.nix`: ```nix # eval.nix -(import ).evalModules { +let + nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-22.11"; + pkgs = import nixpkgs { config = {}; overlays = []; }; +in +pkgs.lib.evalModules { modules = [ ./default.nix ]; } ``` -Then we run +[`evalModules`] is the function that evaluates modules, applies type checking, and merges values into the final attribute set. +It expects a `modules` attribute that takes a list, where each element can be a path to a module or an expression that follows the [module schema](https://nixos.org/manual/nixos/stable/#sec-writing-modules). + +Now run the following command: + ```bash nix-instantiate --eval eval.nix -A config.generate.script ``` -Trying to evaluate the `generate.script` option however, we get an error -that the option is used but not defined, indicating that we need to -actually give a value to the option. +You will see an error message indicating that the `generate.script` option is used but not defined; you will need to assign a value to the option before using it. + +## Type Checking + +As previously mentioned, the `lines` type only permits string values. +:::{warning} +In this section, you will set an invalid value and encounter a type error. +::: -## Type checking: Assigning integer to generate.script +What happens if you instead try to assign an integer to the option? -We can try to assign an integer to our option of type `lines`, but the -module system correctly throws an error saying that our definition -doesn't match the options type. +Add the following lines to `default.nix`: ```diff -diff --git a/default.nix b/default.nix -index cb423a9..0a9162e 100644 ---- a/default.nix -+++ b/default.nix -@@ -5,5 +5,9 @@ - type = lib.types.lines; - }; - }; -+ -+ config = { -+ generate.script = 42; -+ }; - +# default.nix + { lib, ... }: { + + options = { + generate.script = lib.mkOption { + type = lib.types.lines; + }; + }; + ++ config = { ++ generate.script = 42; ++ }; } ``` -## Successful evaluation: Assigning a string to generate.script +Now try to execute the previous command, and witness your first module error: + +```console +$ nix-instantiate --eval eval.nix -A config.generate.script +error: +... + error: A definition for option `generate.script' is not of type `strings concatenated with "\n"'. Definition values: + - In `/home/nix-user/default.nix': 42 +``` -We can make type checking pass by assigning a string to the option, -giving us our first successful evaluation of the `generate.script` -option. +This assignment of `generate.script = 42;` caused a type error: integers are not strings concatenated with the newline character. -In this case, we assign a script which calls the Google Maps Static API -to generate a world map, then displaying the result using icat -(image-cat), both of which are helper scripts. +## Successful Type-checking + +To make this module pass the type-checker and successfully evaluate the `generate.script` option, you will now assign a string to `generate.script`. + +In this case, you will assign a `map` script which first calls the Google Maps Static API to generate a world map, then displays the result using `icat` (image-cat), both of which are helper scripts. + +Update `default.nix` by changing the value of `generate.script` to the following string: ```diff -diff --git a/default.nix b/default.nix -index 0a9162e..24f9c34 100644 ---- a/default.nix -+++ b/default.nix -@@ -7,7 +7,9 @@ - }; - +# default.nix config = { - generate.script = 42; + generate.script = '' -+ map size=640x640 scale=2 | icat ++ ./map size=640x640 scale=2 | feh - + ''; }; - +``` + +## Interlude: Reproducible scripts + +The simple script will likely not work as intended on your system, as it may lack the required dependencies. +We can solve this by packaging the raw {download}`map ` script with `pkgs.writeShellApplication`. + +First, make available a `pkgs` argument in your module evaluation by adding a module that sets `config._module.args`: + +```diff +# eval.nix + pkgs.lib.evalModules { + modules = [ ++ ({ config, ... }: { config._module.args = { inherit pkgs; }; }) + ./test.nix + ]; } ``` -TODO: Create derivations to get these commands +Then change `default.nix` to have the following contents: + +```nix +# default.nix +{ pkgs, lib, ... }: { + + options = { + generate.script = lib.mkOption { + type = lib.types.package; + }; + }; + + config = { + generate.script = pkgs.writeShellApplication { + name = "map"; + runtimeInputs = with pkgs; [ curl feh ]; + text = '' + ${./map} size=640x640 scale=2 | feh - + ''; + }; + }; +} +``` + +This will access the previously added `pkgs` argument so we can use dependencies, and copy the `map` file in the current directory into the Nix store so it's available to the wrapped script, which will also live in the Nix store. + +Run the script with: + +```console +nix-build eval.nix -A config.generate.script +./result/bin/map +``` + +To iterate more quickly, open a new terminal and set up [`entr`](https://github.com/eradman/entr) to re-run the script whenever any source file in the current directory changes: + +```console +nix-shell -p entr findutils bash --run \ + "ls *.nix | \ + entr -rs ' \ + nix-build eval.nix -A config.generate.script --no-out-link \ + | xargs printf -- \"%s/bin/map\" \ + | xargs bash \ + ' \ + " +``` + +This command does the following: +- List all `.nix` files +- Make `entr` watch them for changes. Terminate the invoked command on each change with `-r`. +- On each change: + - Run the `nix-build` invocation as above, but without adding a `./result` symlink + - Take the resulting store path and append `/bin/map` to it + - Run the executable at the path constructed this way + +## Declaring More Options + +Rather than setting all script parameters directly, we will to do that through the module system. +This will not just add some safety through type checking, but also allows to build abstractions in order to manage growing complexity and changing requirements. + +In this section, you will introduce another option: `generate.requestParams`. -## A new list option: Declaring generate.requestParams +For its type, you should use `listOf `, which is a list type where each element must have the specified type. -Let's introduce another option, generate.requestParams. For its type, -we'll use `listOf `, which is a generic list type where each -element has to match the given nested type. In our case we want `str` to -be the nested type, which is a generic string type. +Instead of `lines`, in this case you will want the type of the list elements to be `str`, a generic string type. -Note that the difference between `str` and `lines` is in their merging -behavior: -- For `lines`, multiple definitions get merged by concatenation with - newlines -- For `str`, multiple definitions are not allowed. Which in this case is - mostly irrelevant however, since we can't really define a list element - multiple times. +The difference between `str` and `lines` is in their merging behavior: +Module option types not only check for valid values, but also specify how multiple definitions of an option are to be combined into one. +- For `lines`, multiple definitions get merged by concatenation with newlines. +- For `str`, multiple definitions are not allowed. This is not a problem here, since one can't define a list element multiple times. + +Make the following additions to your `default.nix` file: ```diff -diff --git a/default.nix b/default.nix -index 24f9c34..fd5027a 100644 ---- a/default.nix -+++ b/default.nix -@@ -4,12 +4,21 @@ +# default.nix generate.script = lib.mkOption { - type = lib.types.lines; + type = lib.types.package; }; + + generate.requestParams = lib.mkOption { + type = lib.types.listOf lib.types.str; + }; }; - - config = { - generate.script = '' - map size=640x640 scale=2 | icat - ''; + + config = { + generate.script = pkgs.writeShellApplication { + name = "map"; + runtimeInputs = with pkgs; [ curl feh ]; + text = '' + ${./map} size=640x640 scale=2 | feh - + ''; + }; + + generate.requestParams = [ + "size=640x640" + "scale=2" + ]; }; - } ``` -## Dependencies between options: Using generate.requestParams +## Dependencies Between Options + +A given module generally declares one option that produces a result to be used elsewhere, which in this case is `generate.script`. + +Options can depend on other options, making it possible to build more useful abstractions. -A collection of modules generally only has a single option that is meant -to be evaluated. That's the option that generates the final result we're -interested in, which in our case is `generate.script`. +Here, we want the `generate.script` option to use the values of `generate.requestParams` as arguments to the `map` command. -In order to build up abstractions on that, we have the ability for -options to depend on other options. In this case we want the -`generate.script` option to use the values of `generate.requestParams`. -We can access the values of options by adding the `config` argument to -the argument list at the top and using e.g. -`config.generate.requestParams` to access that options value. +### Accessing Option Values -We're then using `lib.concatStringsSep " "` to join each list element -of the option together into an argument list. +To make option values available, the argument of the module accessing them must include the `config` attribute. + +Update `default.nix` to add the `config` attribute: ```diff -diff --git a/default.nix b/default.nix -index fd5027a..050a98e 100644 ---- a/default.nix -+++ b/default.nix -@@ -1,4 +1,4 @@ --{ lib, ... }: { -+{ lib, config, ... }: { - - options = { - generate.script = lib.mkOption { -@@ -12,7 +12,9 @@ - +# default.nix +-{ pkgs, lib, ... }: { ++{ pkgs, lib, config, ... }: { +``` + +When a module that sets options is evaluated, the resulting values can be accessed by their corresponding attribute names under `config`. + +:::{note} +Option values can't be accessed directly from the same module. + +The module system evaluates all modules it receives, and any of them can define a particular option's value. +What happens when an option is set by multiple modules is determined by that option's type. + +The `config` argument is *not the same* as the `config` attribute where option values are set: +- The `config` argument holds the module system's evaluation result that takes into account all modules passed to `evalModules` and their `imports`. +- The `config` attribute of a module exposes that particular module's option values to the module system for evaluation. +::: + +Now make the following changes to `default.nix`: + +```diff +# default.nix + config = { - generate.script = '' -- map size=640x640 scale=2 | icat -+ map ${lib.concatStringsSep " " -+ config.generate.requestParams -+ } | icat - ''; - - generate.requestParams = [ + generate.script = pkgs.writeShellApplication { + name = "map"; + runtimeInputs = with pkgs; [ curl feh ]; + text = '' +- ${./map} size=640x640 scale=2 | feh - ++ builtins.map ${lib.concatStringsSep " " ++ config.generate.requestParams ++ } | feh - + ''; ``` -## Conditional definitions: Introducing map.zoom option +Here, the value of the `config.generate.requestParams` attribute is populated by the module system based on the definitions in the same file. + +:::{note} +Lazy evaluation in the Nix language allows the module system to make a value available in the `config` argument passed to the module which defines that value. +::: + +`lib.concatStringsSep " "` is then used to join each list element from the value of `config.generate.requestParams` into a single string, with the list elements of `requestParams` separated by a space character. -We now introduce the new option `map.zoom` in order to control the zoom -level of the map. We'll use a new type for it, `nullOr `, which -accepts the value `null`, but also values of its argument type. We're -using `null` here to mean an inferred zoom level. +The result of this represents the list of command line arguments to pass to the `map` script. -For this option, we'll set a default value which should be used if it's -not defined otherwise, which we can do using `mkOption`'s `default` -argument. +## Conditional Definitions -Now we want to use this option to define another element in -`generate.requestParams`, but we only want to add this element if its -value is non-null. We can do this using the `mkIf -` function, which only adds a definition if the condition -holds. +In this section, you will define a new option, `map.zoom`, to control the zoom level of the map. + +Since the Google Maps API will infer a zoom level if no corresponding argument is passed, we want to represent that at the module level. +To do that, you will use a new type, `nullOr `, which can take either values of its argument type or `null`. + +Add the `map` attribute set with the `zoom` option into the top-level `options` declaration, like so: ```diff -diff --git a/default.nix b/default.nix -index 050a98e..6b6e1e1 100644 ---- a/default.nix -+++ b/default.nix -@@ -8,6 +8,13 @@ +# default.nix generate.requestParams = lib.mkOption { type = lib.types.listOf lib.types.str; }; @@ -262,13 +372,16 @@ index 050a98e..6b6e1e1 100644 + map = { + zoom = lib.mkOption { + type = lib.types.nullOr lib.types.int; -+ default = 2; + }; + }; }; - - config = { -@@ -20,6 +27,8 @@ +``` + +To make use of this, use the `mkIf ` function, which only adds the definition if the condition evaluates to `true`. +Make the following additions to the `generate.requestParams` list in the `config` block: + +```diff +# default.nix generate.requestParams = [ "size=640x640" "scale=2" @@ -276,23 +389,38 @@ index 050a98e..6b6e1e1 100644 + "zoom=${toString config.map.zoom}") ]; }; - ``` -## Declaring map.center option +This will will only add a `zoom` parameter to the script call if the value is non-null. + +## Default values + +Let's say that in our application we want to have a different default behavior that sets the the zoom level to `2`, such that automatic zoom has to be enabled explicitly. + +This can be done with the `default` argument to [`mkOption`](https://github.com/NixOS/nixpkgs/blob/master/lib/options.nix). +Its value will be used if the value of the option declaring it is otherwise specified. + +Add the corresponding line: + +```diff +# default.nix + map = { + zoom = lib.mkOption { + type = lib.types.nullOr lib.types.int; ++ default = 2; + }; + }; + }; +``` + +## Centering the Map -Similarly, let's declare a map.center option, declaring where the map -should be centered. +You have now declared options controlling the map dimensions and zoom level, but have not provided a way to specify where the map should be centered. -We'll be using a small utility for geocoding location names, aka turning -them from names into coordinates +Add the `center` option now, possibly with your own location as default value: ```diff -diff --git a/default.nix b/default.nix -index 6b6e1e1..098d135 100644 ---- a/default.nix -+++ b/default.nix -@@ -14,6 +14,11 @@ +# default.nix type = lib.types.nullOr lib.types.int; default = 2; }; @@ -303,78 +431,107 @@ index 6b6e1e1..098d135 100644 + }; }; }; - -@@ -29,6 +34,10 @@ +``` + +To implement this behavior, you will use the {download}`geocode ` utility, which turns location names into coordinates. +There are multiple ways of making a new package accessible, but as an exercise, you will add it as an option in the module system. + +First, add a new option to accommodate the package: + + +```diff +# default.nix + options = { + generate.script = lib.mkOption { + type = lib.types.package; + }; ++ ++ helpers.geocode = lib.mkOption { ++ type = lib.types.package; ++ }; +``` + +Then define the value for that option where you make the raw script reproducible by wrapping a call to it in `writeShellApplication`: + +```diff +# default.nix + config = { ++ helpers.geocode = pkgs.writeShellApplication { ++ name = "geocode"; ++ runtimeInputs = with pkgs; [ curl jq ]; ++ text = "exec ${./geocode}"; ++ }; ++ + generate.script = pkgs.writeShellApplication { + name = "map"; + runtimeInputs = with pkgs; [ curl feh ]; +``` + +Add another `mkIf` call to the list of `requestParams` now where you access the wrapped package through `config.helpers.geocode`, and run the executable `/bin/geocode` inside: + +```diff +# default.nix "scale=2" (lib.mkIf (config.map.zoom != null) "zoom=${toString config.map.zoom}") + (lib.mkIf (config.map.center != null) -+ "center=\"$(geocode ${ ++ "center=\"$(${config.helpers.geocode}/bin/geocode ${ + lib.escapeShellArg config.map.center + })\"") ]; }; - ``` -## Splitting modules: importing marker.nix +This time, you've used `escapeShellArg` to pass the `config.map.center` value as a command-line argument to `geocode`, string interpolating the result back into the `requestParams` string which sets the `center` value. + +Wrapping shell command execution in Nix modules is a helpful technique for controlling system changes, as it uses the more ergonomic attributes and values interface rather than dealing with the peculiarities of escaping manually. + +## Splitting Modules -The module system allows you to split your config into multiple files -via the `imports` attribute, which can define further modules to import. -This allows us to logically separate options and config for different -parts. +The [module schema](https://nixos.org/manual/nixos/stable/#sec-writing-modules) includes the `imports` attribute, which allows incorporating further modules, for example to split a large configuration into multiple files. -Here we create a new module `marker.nix`, where we will declare options -for defining markers on the map +In particular, this allows you to separate option declarations from where they are used in your configuration. + +Create a new module, `marker.nix`, where you can declare options for defining location pins and other markers on the map: + +```diff +# marker.nix +{ lib, config, ... }: { + +} +``` + +Reference this new file in `default.nix` using the `imports` attribute: ```diff -diff --git a/default.nix b/default.nix -index 098d135..9d9f29d 100644 ---- a/default.nix -+++ b/default.nix -@@ -1,5 +1,9 @@ - { lib, config, ... }: { - +# default.nix + { pkgs, lib, config ... }: { + + imports = [ + ./marker.nix + ]; + - options = { - generate.script = lib.mkOption { - type = lib.types.lines; -diff --git a/marker.nix b/marker.nix -new file mode 100644 -index 0000000..035c28d ---- /dev/null -+++ b/marker.nix -@@ -0,0 +1,3 @@ -+{ lib, config, ... }: { -+ -+} ``` -## Submodule types: Declaring map.markers option +## The `submodule` Type + +We want to set multiple markers on the map. +A marker is a complex type with multiple fields. + +This is wher one of the most useful types included in the module system's type system comes into play: `submodule`. +This type allows you to define nested modules with their own options. -One of the most useful types of the module system is the `submodule` -type. This type allows you to define a nested module system evaluation, -with its own options. Every value of such a type is then interpreted -(by default) as a `config` assignment of the nested module evaluation. +Here, you will define a new `map.markers` option whose type is a list of submodules, each with a nested `location` type, allowing you to define a list of markers on the map. -In this case we're defining a `map.markers` option, whose type is a list -of submodules with a nested `location` type, allowing us to define a -list of markers on the map, where each assignment is type checked -according to the submodule. +Each assignment of markers will be type-checked during evaluation of the top-level `config`. + +Make the following changes to `marker.nix`: ```diff -diff --git a/marker.nix b/marker.nix -index 035c28d..6a6c686 100644 ---- a/marker.nix -+++ b/marker.nix -@@ -1,3 +1,20 @@ --{ lib, config, ... }: { -+{ lib, config, ... }: +# marker.nix +-{ pkgs, lib, config, ... }: { ++{ pkgs, lib, config, ... }: +let -+ + markerType = lib.types.submodule { + options = { + location = lib.mkOption { @@ -390,26 +547,18 @@ index 035c28d..6a6c686 100644 + type = lib.types.listOf markerType; + }; + }; - - } ``` -## Single option namespace: Defining markers in generate.requestParams +## Setting Option Values Within Other Modules + +Because of the way the module system composes option definitions, you can freely assign values to options defined in other modules. + +In this case, you will use the `map.markers` option to produce and add new elements to the `requestParams` list, making your declared markers appear on the returned map – but from the module declared in `marker.nix`. -Since all modules in `imports` are treated the same, we can also freely -assign an option defined in our initial module. In this case we want to -add some request parameters derived from the `map.markers` option, so -that markers actually show up on the map. +To implement this behavior, add the following `config` block to `marker.nix`: ```diff -diff --git a/marker.nix b/marker.nix -index 6a6c686..c2c5da2 100644 ---- a/marker.nix -+++ b/marker.nix -@@ -17,4 +17,26 @@ in { - }; - }; - +# marker.nix + config = { + + map.markers = [ @@ -428,27 +577,26 @@ index 6a6c686..c2c5da2 100644 + in "markers=${ + lib.concatStringsSep "\\|" attributes + }"; -+ in map paramForMarker config.map.markers; -+ -+ }; -+ - } ++ in builtins.map paramForMarker config.map.markers; ``` -## Set map.{center,zoom} for more than {1,2} markers +:::{warning} +To avoid confusion with the `map` option setting and the evaluated `config.map` configuration value, here we use the `map` function explicitly as `builtins.map`. +::: + -Let's let the API infer the center if we have more than one marker, and -let's let it infer the zoom as well if there's more than 2. +Here, you again used `escapeShellArg` and string interpolation to generate a Nix string, this time producing a pipe-separated list of geocoded location attributes. + +The `generate.requestParams` value was also set to the resulting list of strings, which gets appended to the `generate.requestParams` list defined in `default.nix`, thanks to the default merging behavior of the `list` type. + +## Dealing with multiple markers + +When defining multiple markers, determining an appropriate center or zoom level for the map may be challenging; it's easier to let the API do this for you. + +To achieve this, make the following additions to `marker.nix`, above the `generate.requestParams` declaration: ```diff -diff --git a/marker.nix b/marker.nix -index c2c5da2..b80ddd0 100644 ---- a/marker.nix -+++ b/marker.nix -@@ -23,6 +23,14 @@ in { - { location = "new york"; } - ]; - +# marker.nix + map.center = lib.mkIf + (lib.length config.map.markers >= 1) + null; @@ -462,30 +610,24 @@ index c2c5da2..b80ddd0 100644 let ``` -## Nested submodules: Introducing users option +In this case, the default behavior of the Google Maps API when not passed a center or zoom level is to pick the geometric center of all the given markers, and to set a zoom level appropriate for viewing all markers at once. -We will now introduce the option `users`, where we make use of a very -useful type, `lib.types.attrsOf `. This type lets us specify an -attribute set as a value, where the keys can be arbitrary, but each -value has to conform to the given . +## Nested Submodules -In this case, we'll use another submodule as the subtype, one that -allows declaring a departure marker, which notably also makes use of our -`markerType` submodule, giving us a nested structure of submodules. +Next, we want to allow multiple named users to define a list of markers each. -We're now propagating each of the users marker definitions to the -`map.markers` option. +For that you'll add a `users` option with type `lib.types.attrsOf `, which will allow you to define `users` as an attribute set, whose values have type ``. + +Here, that subtype will be another submodule which allows declaring a departure marker, suitable for querying the API for the recommended route for a trip. + +This will again make use of the `markerType` submodule, giving a nested structure of submodules. + +To propagate marker definitions from `users` to the `map.markers` option, make the following changes. + +In the `let` block: ```diff -diff --git a/marker.nix b/marker.nix -index b80ddd0..3c54ad8 100644 ---- a/marker.nix -+++ b/marker.nix -@@ -9,9 +9,24 @@ let - }; - }; - }; -+ +# marker.nix + userType = lib.types.submodule { + options = { + departure = lib.mkOption { @@ -496,20 +638,27 @@ index b80ddd0..3c54ad8 100644 + }; + in { - - options = { -+ +``` + +This defines a submodule type for a user, with a `departure` option of type `markerType`. + +In the `options` block, above `map.markers`: + +```diff +# marker.nix + users = lib.mkOption { + type = lib.types.attrsOf userType; + }; -+ - map.markers = lib.mkOption { - type = lib.types.listOf markerType; - }; -@@ -19,9 +34,11 @@ in { - +``` + +That allows adding a `users` attribute set to `config` in any submodule that imports `marker.nix`, where each attribute will be of type `userType` as declared in the previous step. + +In the `config` block, above `map.center`: + +```diff +# marker.nix config = { - + - map.markers = [ - { location = "new york"; } - ]; @@ -518,24 +667,35 @@ index b80ddd0..3c54ad8 100644 + (lib.concatMap (user: [ + user.departure + ]) (lib.attrValues config.users)); - + map.center = lib.mkIf (lib.length config.map.markers >= 1) ``` -## Introducing style.label and strMatching type +This takes all the `departure` markers from all users in the `config` argument, and adds them to `map.markers` if their `location` attribute is not `null`. + +The `config.users` attribute set is passed to `attrValues`, which returns a list of values of each of the attributes in the set (here, the set of `config.users` you've defined), sorted alphabetically (this how attribute names are stored in the Nix language). + +Back in `default.nix`, the resulting `map.markers` option value is still accessed by `generate.requestParams`, which in turn is used to generate arguments to the script that ultimately calls the Google Maps API. + +Defining the options in this way allows you to set multiple `users..departure.location` values and generate a map with the appropriate zoom and center, with pins corresponding to the set of `departure.location` values for *all* `users`. + +In the 2021 Summer of Nix, this formed the basis of an interactive multi-person map demo. + +## Labeling Markers + +Now that the map can be rendered with multiple markers, it's time to add some style customizations. -Let's add an option to customize markers with a label. We can do so by -just adding another option in our markerType submodule. The API states -that this has to be an uppercase letter or a number, which we can -implement with the `strMatching ""` type. +To tell the markers apart, add another option to the `markerType` submodule, to allow labeling each marker pin. + +The API documentation states that [these labels must be either an uppercase letter or a number](https://developers.google.com/maps/documentation/maps-static/start#MarkerStyles). + +You can implement this with the `strMatching ""` type, where `` is a regular expression that will accept any matching values, in this case an uppercase letter or number. + +In the `let` block: ```diff -diff --git a/marker.nix b/marker.nix -index 3c54ad8..1c9a043 100644 ---- a/marker.nix -+++ b/marker.nix -@@ -7,6 +7,12 @@ let +# marker.nix type = lib.types.nullOr lib.types.str; default = null; }; @@ -547,8 +707,14 @@ index 3c54ad8..1c9a043 100644 + }; }; }; - -@@ -52,7 +58,10 @@ in { +``` + +Again, `types.nullOr` allows for `null` values, and the default has been set to `null`. + +In the `paramForMarker` function: + +```diff +# marker.nix paramForMarker = marker: let attributes = @@ -562,27 +728,18 @@ index 3c54ad8..1c9a043 100644 })" ``` -## Using the attribute name in the submodule to define a default label +Here, the label for each `marker` is only propagated to the CLI parameters if `marker.style.label` is set. -Let's set a default label by deriving it from the username. By -transforming the submodule's argument into a function, we can access -arguments within it. One special argument available to submodules is the -`name` argument, which when used in `attrsOf`, gives you the name of the -attribute the submodule is defined under. +## Defining a Default Label -In this case, we don't easily have access to the name from the marker -submodules label option (where we could set a `default =`). Instead we -will use the `config` section of the user submodule to set a default. We -can do so using the `lib.mkDefault` modifier, which has lower precedence -than if no modifier were used. +Right now, if a label is not explicitly set, none will show up. +But since every `users` attribute has a name, we could use that as an automatic value instead. + +This `firstUpperAlnum` function allows you to retrieve the first character of the username, with the correct type for passing to `departure.style.label`: ```diff -diff --git a/marker.nix b/marker.nix -index 1c9a043..53860f1 100644 ---- a/marker.nix -+++ b/marker.nix -@@ -1,5 +1,11 @@ - { lib, config, ... }: +# marker.nix +{ lib, config, ... }: let + # Returns the uppercased first letter + # or number of a string @@ -590,13 +747,17 @@ index 1c9a043..53860f1 100644 + lib.mapNullable lib.head + (builtins.match "[^A-Z0-9]*([A-Z0-9]).*" + (lib.toUpper str)); - + markerType = lib.types.submodule { options = { -@@ -16,14 +22,19 @@ let - }; - }; - +``` + +By transforming the argument to `lib.types.submodule` into a function, you can access arguments within it. + +One special argument automatically available to submodules is `name`, which when used in `attrsOf`, gives you the name of the attribute the submodule is defined under: + +```diff +# marker.nix - userType = lib.types.submodule { + userType = lib.types.submodule ({ name, ... }: { options = { @@ -606,34 +767,50 @@ index 1c9a043..53860f1 100644 }; }; - }; +``` + +In this case, you don't easily have access to the name from the marker submodules `label` option, where you otherwise could set a `default` value. + +Instead you can use the `config` section of the `user` submodule to set a default, like so: + +```diff +# marker.nix + + config = { + departure.style.label = lib.mkDefault + (firstUpperAlnum name); + }; + }); - + in { - + ``` -## Marker colors +:::{note} +Module options have a *priority*, represented as an integer, which determines the precedence for setting the option to a particular value. +When merging values, the priority with lowest numeric value wins. + +The `lib.mkDefault` modifier sets the priority of its argument value to 1000, the lowest precedence. + +This ensures that other values set for the same option will prevail. +::: + +## Marker Styling: Color + +For better visual contrast, it would be helpful to have a way to change the *color* of a marker. + +Here you will use two new type-functions for this: +- `either `, which takes two types as arguments, and allows either of them +- `enum [ ]`, which takes a list of allowed values, and allows any of them -Let's allow markers to change their color as well. We'll use some new -type functions for this, namely -- `either `: Takes two types as arguments, allows either of - them -- `enum [ ]`: Takes a list of allowed values +In the `let` block, add the following `colorType` option, which can hold strings containing either some given color names or an RGB value add the new compound type: ```diff -diff --git a/marker.nix b/marker.nix -index 53860f1..df0d08b 100644 ---- a/marker.nix -+++ b/marker.nix -@@ -7,6 +7,13 @@ let +# marker.nix + ... (builtins.match "[^A-Z0-9]*([A-Z0-9]).*" (lib.toUpper str)); - + + # Either a color name or `0xRRGGBB` + colorType = lib.types.either + (lib.types.strMatching "0x[0-9A-F]{6}") @@ -644,7 +821,14 @@ index 53860f1..df0d08b 100644 markerType = lib.types.submodule { options = { location = lib.mkOption { -@@ -19,6 +26,11 @@ let +``` + +This allows either strings that matche a 24-bit hexadecimal number or are equal to one of the specified color names. + +At the bottom of the `let` block, add the `style.color` option and specify a default value: + +```diff +# marker.nix (lib.types.strMatching "[A-Z0-9]"); default = null; }; @@ -655,8 +839,12 @@ index 53860f1..df0d08b 100644 + }; }; }; - -@@ -73,6 +85,7 @@ in { +``` + +Now add an entry to the `paramForMarker` list which makes use of the new option: + +```diff +# marker.nix (marker.style.label != null) "label:${marker.style.label}" ++ [ @@ -666,16 +854,14 @@ index 53860f1..df0d08b 100644 })" ``` -## Marker size +## Marker Styling: Size -Let's also allow changing of marker sizes. +In case you set many different markers, it would be helpful to have the ability to change their size individually. + +Add a new `style.size` option to `marker.nix`, allowing you to choose from the set of pre-defined sizes: ```diff -diff --git a/marker.nix b/marker.nix -index df0d08b..2c0c1a8 100644 ---- a/marker.nix -+++ b/marker.nix -@@ -31,6 +31,12 @@ let +# marker.nix type = colorType; default = "red"; }; @@ -687,8 +873,12 @@ index df0d08b..2c0c1a8 100644 + }; }; }; - -@@ -80,10 +86,20 @@ in { +``` + +Now add a mapping for the size parameter in `paramForMarker`, which selects an appropriate string to pass to the API: + +```diff +# marker.nix generate.requestParams = let paramForMarker = marker: let @@ -699,6 +889,12 @@ index df0d08b..2c0c1a8 100644 + large = null; + }.${marker.style.size}; + +``` + +Finally, add another `lib.optional` call to the `attributes` string, making use of the selected size: + +``` +# marker.nix attributes = lib.optional (marker.style.label != null) @@ -711,83 +907,76 @@ index df0d08b..2c0c1a8 100644 "$(geocode ${ ``` -## Initial path module +## The `pathType` Submodule + +So far, you've created an option for declaring a *destination* marker, as well as several options for configuring the marker's visual representation. + +Now we want to compute and display a route from the user's location to some destination. + +The new option defined in the next section will allow you to set an *arrival* marker, which together with a destination allows you to draw *paths* on the map using the new module defined below. + +To start, create a new `path.nix` file with the following contents: + +```diff +# path.nix +{ lib, config, ... }: +let + pathType = lib.types.submodule { + options = { + locations = lib.mkOption { + type = lib.types.listOf lib.types.str; + }; + }; + }; +in { + options = { + map.paths = lib.mkOption { + type = lib.types.listOf pathType; + }; + }; + + config = { + generate.requestParams = let + attrForLocation = loc: + "$(geocode ${lib.escapeShellArg loc})"; + paramForPath = path: + let + attributes = + builtins.map attrForLocation path.locations; + in "path=${ + lib.concatStringsSep "\\|" attributes + }"; + in builtins.map paramForPath config.map.paths; + }; +} +``` + +The `path.nix` module defines an option for declaring a list of paths on our `map`, where each path is a list of strings for geographic locations. -Let's introduce a new module for declaring paths on the map. We'll -import a new `path.nix` module from our `marker.nix` module. -In the path module we'll define an option for declaring paths, and will -use the same `generate.requestParams` to influence the API call to -include our defined paths +In the `config` attribute we augment the API call by setting the `generate.requestParams` option value with the coordinates transformed appropriately, which will be concatenated with request paremeters set elsewhere. + +Now import this new `path.nix` module from your `marker.nix` module: ```diff -diff --git a/marker.nix b/marker.nix -index 2c0c1a8..ffb8185 100644 ---- a/marker.nix -+++ b/marker.nix -@@ -56,6 +56,10 @@ let - +# marker.nix in { - + + imports = [ + ./path.nix + ]; + options = { - + users = lib.mkOption { -diff --git a/path.nix b/path.nix -new file mode 100644 -index 0000000..554a88b ---- /dev/null -+++ b/path.nix -@@ -0,0 +1,32 @@ -+{ lib, config, ... }: -+let -+ pathType = lib.types.submodule { -+ -+ options = { -+ locations = lib.mkOption { -+ type = lib.types.listOf lib.types.str; -+ }; -+ }; -+ -+ }; -+in { -+ options = { -+ map.paths = lib.mkOption { -+ type = lib.types.listOf pathType; -+ }; -+ }; -+ -+ config = { -+ generate.requestParams = let -+ attrForLocation = loc: -+ "$(geocode ${lib.escapeShellArg loc})"; -+ paramForPath = path: -+ let -+ attributes = -+ map attrForLocation path.locations; -+ in "path=${ -+ lib.concatStringsSep "\\|" attributes -+ }"; -+ in map paramForPath config.map.paths; -+ }; -+} ``` -## Arrival marker +## The Arrival Marker -Now in order for users to be able to draw a path in their definitions, -we need to allow them to specify another marker. We will copy the -`departure` option declaration to a new `arrival` option for that. +Copy the `departure` option declaration to a new `arrival` option in `marker.nix`, to complete the initial path implementation: ```diff -diff --git a/marker.nix b/marker.nix -index ffb8185..940b8f8 100644 ---- a/marker.nix -+++ b/marker.nix -@@ -46,11 +46,18 @@ let +# marker.nix type = markerType; default = {}; }; @@ -797,7 +986,12 @@ index ffb8185..940b8f8 100644 + default = {}; + }; }; - +``` + +Next, add an `arrival.style.label` attribute to the `config` block, mirroring the `departure.style.label`: + +```diff +# marker.nix config = { departure.style.label = lib.mkDefault (firstUpperAlnum name); @@ -805,34 +999,33 @@ index ffb8185..940b8f8 100644 + (firstUpperAlnum name); }; }); - -@@ -76,7 +83,7 @@ in { +``` + +Finally, update the return list in the function passed to `concatMap` in `map.markers` to also include the `arrival` marker for each user: + +```diff +# marker.nix map.markers = lib.filter (marker: marker.location != null) (lib.concatMap (user: [ - user.departure + user.departure user.arrival ]) (lib.attrValues config.users)); - + map.center = lib.mkIf ``` -## Connecting user paths +Now you have the basesis to define paths on the map, connecting pairs of departure and arrival points. + +## Connecting Markers by Paths -In our path module, we can now define a path spanning from every users -departure location to their arrival location. +In the path module, define a path connecting every user's departure and arrival locations: ```diff -diff --git a/path.nix b/path.nix -index 554a88b..d4a3a84 100644 ---- a/path.nix -+++ b/path.nix -@@ -17,6 +17,17 @@ in { - }; - +# path.nix config = { + -+ map.paths = map (user: { ++ map.paths = builtins.map (user: { + locations = [ + user.departure.location + user.arrival.location @@ -847,24 +1040,21 @@ index 554a88b..d4a3a84 100644 "$(geocode ${lib.escapeShellArg loc})"; ``` -## Introducing path weight option +The new `map.paths` attribute contains a list of all valid paths defined for all users. + +A path is valid only if the `departure` and `arrival` attributes are set for that user. -Let's also allow some customization of path styles with a `weight` option. -As already done before, we'll declare a submodule for the path style. +## Path Styling: Weight -While we could also directly define the style.weight option in this -case, we will use the submodule in a future change to reuse the path -style definitions. +Your users have spoken, and they demand the ability to customize the styles of their paths with a `weight` option. -Note how we're using a new type for this, `ints.between -`, which allows integers in the given inclusive range. +As before, you'll now declare a new submodule for the path style. +While you could also directly declare the `style.weight` option, in this case you should use the submodule to be able reuse the path style type later. + +Add the `pathStyleType` submodule option to the `let` block in `path.nix`: ```diff -diff --git a/path.nix b/path.nix -index d4a3a84..88766a8 100644 ---- a/path.nix -+++ b/path.nix -@@ -1,11 +1,26 @@ +# path.nix { lib, config, ... }: let + @@ -878,7 +1068,18 @@ index d4a3a84..88766a8 100644 + }; + pathType = lib.types.submodule { - +``` + +:::{note} +The `ints.between ` type allows integers in the given (inclusive) range. +::: + +The path weight will default to 5, but can be set to any integer value in the 1 to 20 range, with higher weights producing thicker paths on the map. + +Now add a `style` option to the `options` set further down the file: + +```diff +# path.nix options = { locations = lib.mkOption { type = lib.types.listOf lib.types.str; @@ -889,39 +1090,38 @@ index d4a3a84..88766a8 100644 + default = {}; + }; }; - + }; -@@ -34,7 +49,10 @@ in { +``` + +Finally, update the `attributes` list in `paramForPath`: + +```diff +# path.nix paramForPath = path: let attributes = -- map attrForLocation path.locations; +- builtins.map attrForLocation path.locations; + [ + "weight:${toString path.style.weight}" + ] -+ ++ map attrForLocation path.locations; ++ ++ builtins.map attrForLocation path.locations; in "path=${ lib.concatStringsSep "\\|" attributes }"; ``` -## User path styles +## The `pathStyle` Submodule + +Users still can't actually customize the path style yet. +Introduce a new `pathStyle` option for each user. -Now users can't actually customize the path style yet, so let's -introduce a new `pathStyle` option for each user. +The module system allows you to declare values for an option multiple times, and if the types permit doing so, takes care of merging each declaration's values together. -But wait! Didn't we already define the `user` option in the `marker.nix` -module? Yes we did, but the module system actually allows us to declare -an option multiple times, and the module system takes care of merging -each declarations types together (if possible). +This makes it possible to have a definition for the `user` option in the `marker.nix` module, as well as a `user` definition in `path.nix`: ```diff -diff --git a/path.nix b/path.nix -index 88766a8..8b56782 100644 ---- a/path.nix -+++ b/path.nix -@@ -26,6 +26,16 @@ let - }; +# path.nix in { options = { + @@ -937,7 +1137,12 @@ index 88766a8..8b56782 100644 map.paths = lib.mkOption { type = lib.types.listOf pathType; }; -@@ -38,6 +48,7 @@ in { +``` + +Then add a line using the `user.pathStyle` option in `map.paths` where each user's paths are processed: + +```diff +# path.nix user.departure.location user.arrival.location ]; @@ -947,20 +1152,19 @@ index 88766a8..8b56782 100644 && user.arrival.location != null ``` -## Introducing path color option +## Path Styling: Color + +As with markers, paths should have customizable colors. + +You can accomplish this using types you've already encountered by now. -Very similar to markers, let's allow customization of the path color, -using types we've seen before already. +Add a new `colorType` block to `path.nix`, specifying the allowed color names and RGB/RGBA hexadecimal values: ```diff -diff --git a/path.nix b/path.nix -index 8b56782..d2073fe 100644 ---- a/path.nix -+++ b/path.nix -@@ -1,12 +1,25 @@ +# path.nix { lib, config, ... }: let - + + # Either a color name, `0xRRGGBB` or `0xRRGGBBAA` + colorType = lib.types.either + (lib.types.strMatching "0x[0-9A-F]{6}[0-9A-F]{2}?") @@ -970,8 +1174,12 @@ index 8b56782..d2073fe 100644 + ]); + pathStyleType = lib.types.submodule { - options = { - weight = lib.mkOption { +``` + +Under the `weight` option, add a new `color` option to use the new `colorType` value: + +```diff +# path.nix type = lib.types.ints.between 1 20; default = 5; }; @@ -982,8 +1190,12 @@ index 8b56782..d2073fe 100644 + }; }; }; - -@@ -62,6 +75,7 @@ in { +``` + +Finally, add a line using the `color` option to the `attributes` list: + +```diff +# path.nix attributes = [ "weight:${toString path.style.weight}" @@ -993,17 +1205,16 @@ index 8b56782..d2073fe 100644 in "path=${ ``` -## Introducing geodesic path option +## Further Styling -Finally, another option for the path style, using a new but very simple -type, `bool`, which just allows `true` and `false`. +Now that you've got this far, to further improve the aesthetics of the rendered map, add another style option allowing paths to be drawn as *geodesics*, the shortest "as the crow flies" distance between two points on Earth. + +Since this feature can be turned on or off, you can do this using the `bool` type, which can be `true` or `false`. + +Make the following changes to `path.nix` now: ```diff -diff --git a/path.nix b/path.nix -index d2073fe..ebd9561 100644 ---- a/path.nix -+++ b/path.nix -@@ -20,6 +20,11 @@ let +# path.nix type = colorType; default = "blue"; }; @@ -1014,8 +1225,12 @@ index d2073fe..ebd9561 100644 + }; }; }; - -@@ -76,6 +81,7 @@ in { +``` + +Make sure to also add a line to use that value in `attributes` list, so the option value is included in the API call: + +```diff +# path.nix [ "weight:${toString path.style.weight}" "color:${path.style.color}" @@ -1025,3 +1240,12 @@ index d2073fe..ebd9561 100644 in "path=${ ``` +## Wrapping Up + +In this tutorial, you've learned how to write custom Nix modules to bring external services under declarative control, with the help of several new utility functions from the Nixpkgs `lib`. + +You defined several modules in multiple files, each with separate submodules making use of the module system's type checking. + +These modules exposed features of the external API in a declarative way. + +You can now conquer the world with Nix.