linux_slave_on_master will not run jobs

[wjwwood]

I am now getting:

containerd: container did not start before the specified timeout

-- http://ci.ros2.org/job/ci_linux/2722/console

It has happened twice in a row, so I'm guessing it borked until we figure out why. I'm just disabling it for now. I'll look into it as soon as I can.

[mikaelarguedas]

that's an issue we faced several times in the past. It's a flaky behavior that almost always requires a reboot of the machine. Maybe restarting the master machine just before the nightly restart would be needed

[wjwwood]

@dirk-thomas restarted the master and it is up and running again, but I think it's probably still a temporary fix.

[clalancette]

If I remember properly, this is something that we believe is fixed in Ubuntu 16.04. Maybe one of the tasks we should consider doing after beta2 is upgrading the Linux build farm machines to 16.04.

[dirk-thomas]

Maybe one of the tasks we should consider doing after beta2 is upgrading the Linux build farm machines to 16.04.

The goal is to use a ROS 2 build farm based on the existing ROS 1 infrastructure. I don't think we should spend any time in between to update the existing custom build infrastructure.

[sloretz]

Saw again on linux_3

[sloretz]

It's been a while without an instance of this, and the machines it was seen on are no longer in use. Is it safe to close?

[wjwwood]

No idea, but you can close it and let it get reopened in the future if you want.

[nuclearsandwich]

I haven't yet done enough profiling of the jenkins manager host to determine whether or not running a separate executor process on it is reasonable. It's easier now to scale linux executors so it's less important that we run an executor on the jenkins host. In theory the chef roles are co-installable but I didn't have time to test this before doing the migration. It would be a good thing to verify but not directly associated with this issue.

I'll close this one out.