Skip to content

Commit

Permalink
ssh-ng: also store build logs to make them accessible by nix log
Browse files Browse the repository at this point in the history
Right now when building a derivation remotely via

    $ nix build -j0 -f . hello -L --builders 'ssh://builder'

it's possible later to read through the entire build-log by running
`nix log -f . hello`. This isn't possible however when using `ssh-ng`
rather than `ssh`.

The reason for that is that there are two different ways to transfer
logs in Nix through e.g. an SSH tunnel (that are used by `ssh`/`ssh-ng`
respectively):

* `ssh://` receives its logs from the fd pointing to `builderOut`. This
  is directly passed to the "log-sink" (and to the logger on each `\n`),
  hence `nix log` works here.
* `ssh-ng://` however expects JSON-like messages (i.e. `@nix {log data
  in here}`) and passes it directly to the logger without doing anything
  with the `logSink`. However it's certainly possible to extract
  log-lines from this format as these have their own message-type in the
  JSON payload (i.e. `resBuildLogLine`).

  This is basically what I changed in this patch: if the code-path for
  `builderOut` is not reached and a `logSink` is initialized, the
  message was successfully processed by the JSON logger (i.e. it's in
  the expected format) and the line is of the expected type (i.e.
  `resBuildLogLine`), the line will be written to the log-sink as well.

Closes #5079
  • Loading branch information
Ma27 committed Jan 31, 2022
1 parent 59b6afe commit 06ff9d8
Showing 1 changed file with 10 additions and 3 deletions.
13 changes: 10 additions & 3 deletions src/libstore/build/derivation-goal.cc
Original file line number Diff line number Diff line change
Expand Up @@ -1175,10 +1175,10 @@ bool DerivationGoal::isReadDesc(int fd)
return fd == hook->builderOut.readSide.get();
}


void DerivationGoal::handleChildOutput(int fd, const string & data)
{
if (isReadDesc(fd))
auto isWrittenToLog = isReadDesc(fd);
if (isWrittenToLog)
{
logSize += data.size();
if (settings.maxLogSize && logSize > settings.maxLogSize) {
Expand Down Expand Up @@ -1207,7 +1207,14 @@ void DerivationGoal::handleChildOutput(int fd, const string & data)
if (hook && fd == hook->fromHook.readSide.get()) {
for (auto c : data)
if (c == '\n') {
handleJSONLogMessage(currentHookLine, worker.act, hook->activities, true);
auto s = handleJSONLogMessage(currentHookLine, worker.act, hook->activities, true);
if (s && !isWrittenToLog && logSink) {
auto json = nlohmann::json::parse(std::string(currentHookLine, 5));
if (json["type"] == resBuildLogLine) {
auto f = json["fields"];
(*logSink)((f.size() > 0 ? f.at(0).get<std::string>() : "") + "\n");
}
}
currentHookLine.clear();
} else
currentHookLine += c;
Expand Down

0 comments on commit 06ff9d8

Please sign in to comment.