LXC Support #1699
LXC Support #1699
Conversation
diptanu
force-pushed
the
f-lxc-driver
branch
2 times, most recently
from
9fb4534 to
be33c11
Compare
| return nil, fmt.Errorf("unable to start container: %v", err) | ||
| } | ||
| // Set the resource limits | ||
| if err := c.SetMemoryLimit(lxc.ByteSize(task.Resources.MemoryMB) * lxc.MB); err != nil { |
There was a problem hiding this comment.
Should this come before c.Start()?
|
Neither of my comments are blockers. LGTM |
diptanu
force-pushed
the
f-lxc-driver
branch
from
2a4c13f to
27b0774
Compare
|
@schmichael Addressed all your comments. |
|
@dadgar I would recommend merging this PR since the basic features have been implemented. We can add support for downloading support of rootfs and MacVLAN in subsequent PRs. Can you please review this? |
diptanu
force-pushed
the
f-lxc-driver
branch
2 times, most recently
from
6b22537 to
e512245
Compare
| if err := c.SetConfigItem("lxc.mount.entry", allocDirMount); err != nil { | ||
| return nil, fmt.Errorf("error setting alloc dir bind mounts configuration: %v", err) | ||
| } | ||
| if err := c.SetConfigItem("lxc.mount.entry", taskDirMount); err != nil { |
There was a problem hiding this comment.
Calling SetConfigItem("lxc.mount.entry",... twice will append the second entry, not overwrite the first?
I couldn't tell from looking at the lxc docs.
There was a problem hiding this comment.
No it won't, if you look at lxc configuration there can be multiple lxc.mount.entry - as many fstab entries you want.
| func (h *lxcDriverHandle) Kill() error { | ||
| h.logger.Printf("[INFO] driver.lxc: shutting down container %q", h.container.Name()) | ||
| if err := h.container.Shutdown(h.killTimeout); err != nil { | ||
| h.logger.Printf("[INFO] driver.lxc: shutting down container %q failed: %v", h.container.Name(), err) |
| ) | ||
|
|
||
| const ( | ||
|
|
| ) | ||
|
|
||
| // Add the lxc driver to the list of builtin drivers | ||
| func init() { |
There was a problem hiding this comment.
Why is this here, do it like the other drivers?
There was a problem hiding this comment.
@dadgar So that we don't have to build the driver on other platforms. This driver needs the lxc-dev package which is unavailable on other platforms.
| if err := mapstructure.WeakDecode(task.Config, &driverConfig); err != nil { | ||
| return nil, err | ||
| } | ||
| lxcPath := lxc.DefaultConfigPath() |
There was a problem hiding this comment.
Can we set the path to be the task_dir?
There was a problem hiding this comment.
I am not very sure if we can have multiple Config Paths on a machine, usually all the containers live in /var/lib/lxc. Having it in the task dir is desirable since then we can make use of disk space accounting.
There was a problem hiding this comment.
LXC allows unprovileged containers, i.e. I can run nomad as nomad user & without any root privileges to spin LXC containers owned by nomad user. In such setup, generally the containers will stay at /home/nomad/.local/share/lxc (or $HOME/.local/share/lxc).. but this is configurable. I am not aware of any case where the nomad client would be running as different user /or same user with different lxc path.
There was a problem hiding this comment.
So the lxc path can be configurable by the operator. I think @dadgar is suggesting using a different path per container.
There was a problem hiding this comment.
Yes I was suggesting specific path per container. I was looking at lxc docs and it seems like you can
|
@diptanu i took this for a spin during the weekend. Its working on my cluster. Could not get to went through all the code. I think you should enable wantdaemonize + close file descriptor by default. |
schmichael
force-pushed
the
f-lxc-driver
branch
from
e512245 to
1566b97
Compare
|
|
||
| const ( | ||
| // The option that enables this driver in the Config.Options map. | ||
| lxcConfigOption = "driver.lxc.enable" |
There was a problem hiding this comment.
Is this documented on the website?
There was a problem hiding this comment.
comments should start with the variable/constant/method/struct name
There was a problem hiding this comment.
Fixed the comment.
@diptanu did you get a chance to write any docs? If not I don't mind doing it.
| return nil, err | ||
| } | ||
| lxcPath := lxc.DefaultConfigPath() | ||
| if path := d.config.Read("lxc.path"); path != "" { |
There was a problem hiding this comment.
shouldn't this be driver.lxc.path and document it
| taskLocalDir, ok := ctx.AllocDir.TaskDirs[task.Name] | ||
| if !ok { | ||
| return nil, fmt.Errorf("failed to find task local directory: %v", task.Name) | ||
| } |
| } | ||
|
|
||
| // Set the resource limits | ||
| if err := c.SetMemoryLimit(lxc.ByteSize(task.Resources.MemoryMB) * lxc.MB); err != nil { |
There was a problem hiding this comment.
Set resources before starting?
|
I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions. |
This PR implements support for the LXC driver. The driver is enabled only in Linux.