Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support remote-building to macOS hosts #714

Merged
merged 12 commits into from
Nov 10, 2023
47 changes: 3 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ A fast, friendly, and reliable tool to help you use Nix with Flakes everywhere.
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
```

The `nix-installer` has successfully completed over 500,000 installs in a number of environments, including [Github Actions](#as-a-github-action):
The `nix-installer` has successfully completed over 1,000,000 installs in a number of environments, including [Github Actions](#as-a-github-action):

| Platform | Multi User | `root` only | Maturity |
|------------------------------|:------------------:|:-----------:|:-----------------:|
Expand Down Expand Up @@ -265,49 +265,6 @@ This is especially useful when using the installer in non-interactive scripts.

While `nix-installer` tries to provide a comprehensive and unquirky experience, there are unfortunately some issues which may require manual intervention or operator choices.

### Using MacOS remote SSH builders, Nix binaries are not on `$PATH`

When connecting to a Mac remote SSH builder users may sometimes see this error:

```bash
$ nix store ping --store "ssh://$USER@$HOST"
Store URL: ssh://$USER@$HOST
zsh:1: command not found: nix-store
error: cannot connect to '$USER@$HOST'
```

The way MacOS populates the `PATH` environment differs from other environments. ([Some background](https://gist.github.com/Linerre/f11ad4a6a934dcf01ee8415c9457e7b2))

There are two possible workarounds for this:

* **(Preferred)** Update the remote builder URL to include the `remote-program` parameter pointing to `nix-store`. For example:
```bash
nix store ping --store "ssh://$USER@$HOST?remote-program=/nix/var/nix/profiles/default/bin/nix-store"
```
If you are unsure where the `nix-store` binary is located, run `which nix-store` on the remote.
* Update `/etc/zshenv` on the remote so that `zsh` populates the Nix path for every shell, even those that are neither *interactive* or *login*:
```bash
# Nix
if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
. '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
fi
# End Nix
```
<details>
<summary>This strategy has some behavioral caveats, namely, <code>$PATH</code> may have unexpected contents</summary>

For example, if `$PATH` gets unset then a script invoked, `$PATH` may not be as empty as expected:
```bash
$ cat example.sh
#! /bin/zsh
echo $PATH
$ PATH= ./example.sh
/Users/ephemeraladmin/.nix-profile/bin:/nix/var/nix/profiles/default/bin:
```
This strategy results in Nix's paths being present on `$PATH` twice and may have a minor impact on performance.

</details>

### Using MacOS after removing `nix` while `nix-darwin` was still installed, network requests fail

If `nix` was previously uninstalled without uninstalling `nix-darwin` first, users may experience errors similar to this:
Expand Down Expand Up @@ -469,6 +426,7 @@ Subtle differences in the shell implementations and tool used in the scripts mak

The Determinate Nix installer has numerous advantages:

* survives macOS upgrades
* keeping an installation receipt for easy uninstallation
* offering users a chance to review an accurate, calculated install plan
* having 'planners' which can create appropriate install plans for complicated targets
Expand All @@ -477,6 +435,7 @@ The Determinate Nix installer has numerous advantages:
* supporting a expanded test suite including 'curing' cases
* supporting SELinux and OSTree based distributions without asking users to make compromises
* operating as a single, static binary with external dependencies such as `openssl`, only calling existing system tools (like `useradd`) where necessary
* As a MacOS remote build target, ensures `nix` is not absent from path

It has been wonderful to collaborate with other participants in the Nix Installer Working Group and members of the broader community. The working group maintains a [foundation owned fork of the installer](https://github.com/nixos/experimental-nix-installer/).

Expand Down
96 changes: 96 additions & 0 deletions src/action/macos/configure_remote_building.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
use crate::action::base::{create_or_insert_into_file, CreateOrInsertIntoFile};
use crate::action::{Action, ActionDescription, ActionError, ActionTag, StatefulAction};

use std::path::Path;
use tracing::{span, Instrument, Span};

const PROFILE_NIX_FILE_SHELL: &str = "/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh";

/**
Configure macOS's zshenv to load the Nix environment when ForceCommand is used.
This enables remote building, which requires `ssh host nix` to work.
*/
#[derive(Debug, serde::Deserialize, serde::Serialize, Clone)]
pub struct ConfigureRemoteBuilding {
create_or_insert_into_file: StatefulAction<CreateOrInsertIntoFile>,
}

impl ConfigureRemoteBuilding {
#[tracing::instrument(level = "debug", skip_all)]
pub async fn plan() -> Result<StatefulAction<Self>, ActionError> {
let shell_buf = format!(
r#"
# Set up Nix only on SSH connections
# See: https://github.com/DeterminateSystems/nix-installer/pull/714
if [ -e '{PROFILE_NIX_FILE_SHELL}' ] && [ -n "${{SSH_CONNECTION}}" ] && [ "${{SHLVL}}" -eq 1 ]; then
. '{PROFILE_NIX_FILE_SHELL}'
fi
# End Nix
"#
);

let create_or_insert_into_file = CreateOrInsertIntoFile::plan(
Path::new("/etc/zshenv"),
None,
None,
0o644,
shell_buf.to_string(),
create_or_insert_into_file::Position::Beginning,
)
.await
.map_err(Self::error)?;

Ok(Self {
create_or_insert_into_file,
}
.into())
}
}

#[async_trait::async_trait]
#[typetag::serde(name = "configure_remote_building")]
impl Action for ConfigureRemoteBuilding {
fn action_tag() -> ActionTag {
ActionTag("configure_remote_building")
}
fn tracing_synopsis(&self) -> String {
"Configuring zsh to support using Nix in non-interactive shells".to_string()
}

fn tracing_span(&self) -> Span {
span!(tracing::Level::DEBUG, "configure_remote_building",)
}

fn execute_description(&self) -> Vec<ActionDescription> {
vec![ActionDescription::new(
self.tracing_synopsis(),
vec!["Update `/etc/zshenv` to import Nix".to_string()],
)]
}

#[tracing::instrument(level = "debug", skip_all)]
async fn execute(&mut self) -> Result<(), ActionError> {
let span = tracing::Span::current().clone();
self.create_or_insert_into_file
.try_execute()
.instrument(span)
.await
.map_err(Self::error)?;

Ok(())
}

fn revert_description(&self) -> Vec<ActionDescription> {
vec![ActionDescription::new(
"Remove the Nix configuration from zsh's non-login shells".to_string(),
vec!["Update `/etc/zshenv` to no longer import Nix".to_string()],
)]
}

#[tracing::instrument(level = "debug", skip_all)]
async fn revert(&mut self) -> Result<(), ActionError> {
self.create_or_insert_into_file.try_revert().await?;

Ok(())
}
}
2 changes: 2 additions & 0 deletions src/action/macos/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
*/

pub(crate) mod bootstrap_launchctl_service;
pub(crate) mod configure_remote_building;
pub(crate) mod create_apfs_volume;
pub(crate) mod create_fstab_entry;
pub(crate) mod create_nix_hook_service;
Expand All @@ -16,6 +17,7 @@ pub(crate) mod set_tmutil_exclusions;
pub(crate) mod unmount_apfs_volume;

pub use bootstrap_launchctl_service::BootstrapLaunchctlService;
pub use configure_remote_building::ConfigureRemoteBuilding;
pub use create_apfs_volume::CreateApfsVolume;
pub use create_nix_hook_service::CreateNixHookService;
pub use create_nix_volume::{CreateNixVolume, NIX_VOLUME_MOUNTD_DEST};
Expand Down
20 changes: 17 additions & 3 deletions src/cli/subcommand/repair.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,23 @@ impl CommandExecute for Repair {

if let Err(err) = reconfigure.try_execute().await {
println!("{:#?}", err);
Ok(ExitCode::FAILURE)
} else {
Ok(ExitCode::SUCCESS)
return Ok(ExitCode::FAILURE);
}
// TODO: Using `cfg` based on OS is not a long term solution.
// Make this read the planner from the `/nix/receipt.json` to determine which tasks to run.
grahamc marked this conversation as resolved.
Show resolved Hide resolved
#[cfg(target_os = "macos")]
Hoverbear marked this conversation as resolved.
Show resolved Hide resolved
{
let mut reconfigure = crate::action::macos::ConfigureRemoteBuilding::plan()
.await
.map_err(PlannerError::Action)?
.boxed();

if let Err(err) = reconfigure.try_execute().await {
println!("{:#?}", err);
return Ok(ExitCode::FAILURE);
}
}

Ok(ExitCode::SUCCESS)
}
}
10 changes: 9 additions & 1 deletion src/planner/macos.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ use crate::{
action::{
base::RemoveDirectory,
common::{ConfigureInitService, ConfigureNix, CreateUsersAndGroups, ProvisionNix},
macos::{CreateNixHookService, CreateNixVolume, SetTmutilExclusions},
macos::{
ConfigureRemoteBuilding, CreateNixHookService, CreateNixVolume, SetTmutilExclusions,
},
StatefulAction,
},
execute_command,
Expand Down Expand Up @@ -166,6 +168,12 @@ impl Planner for Macos {
.map_err(PlannerError::Action)?
.boxed(),
);
plan.push(
ConfigureRemoteBuilding::plan()
.await
.map_err(PlannerError::Action)?
.boxed(),
);

if self.settings.modify_profile {
plan.push(
Expand Down
Loading