Puppet (apply) provisioning plugin #629

igalic · 2019-02-03T12:54:46Z

This patch introduces Puppet (apply) as provisioning method, addressing #625

So far the design requires a (unique) name, a source (the control-repo) and an optional list of packages to be pre-installed.
By default that List of packages is puppet6, and if the source is a git repo, rubygem-r10k.

To begin the provisioning, we

install puppet
optionall install r10k
(optionally) clone the control repo
mount the control-repo
(optionally, if the repo was cloned) run r10k
run puppet

we could also consider to run puppet more than once, to guarantee idempotence.

libioc/Provisioning/puppet.py

igalic

😻

libioc/Provisioning/__init__.py

igalic · 2019-02-03T17:14:42Z

libioc/Provisioning/puppet.py

+        self.url = url
+        self.name = name
+
+        _pkgs = ['puppet6']  # make this a Global Varialbe


we can make the 6 a provisioning.puppet.version=6 default.

libioc/Provisioning/puppet.py

libioc/ZFS.py

libioc/Provisioning/puppet.py

igalic

help with nomenclature please

libioc/Provisioning/puppet.py

igalic

there's a lot of potential here

libioc/Provisioning/puppet.py

igalic

👀

libioc/Provisioning/puppet.py

igalic

provisioning plugins: dedicated dataset for caching / sharing.

libioc/Provisioning/puppet.py

igalic

👀

libioc/Provisioning/puppet.py

igalic

provisioning.name has been re-removed.

libioc/Config/Jail/Defaults.py

libioc/Provisioning/__init__.py

igalic

local control-repo no longer works.

libioc/Provisioning/puppet.py

gronke · 2019-02-07T13:03:24Z

@igalic we still have to test remote repositories. Also it would be nice to provide authentication information for remote sources. Any ideas?

igalic

👀

igalic · 2019-02-07T12:32:47Z

libioc/Config/Jail/File/Fstab.py

-                filepath=path,
-                resource=self.jail
-            )
+            self.jail._require_relative_path(path)


a method so important for security could be made public…

libioc/Config/Jail/File/__init__.py

libioc/Jail.py

libioc/Provisioning/__init__.py

libioc/Provisioning/puppet.py

igalic · 2019-02-07T13:55:32Z

@gronke wrote:

@igalic we still have to test remote repositories. Also it would be nice to provide authentication information for remote sources. Any ideas?

so, right now, the repo i'm using is on gitlab and is only accessible with the correct SSH keys

however, it has no secretes and i could expose it publicly

as for authentication:

the easiest way would be to mount an (root's?) ~/.ssh/?
It would also be nice if this could happen temporarily — i.e.: only during provisioning.

gronke · 2019-02-07T14:12:36Z

the easiest way would be to mount an (root's?) ~/.ssh/?

Nein! We need proper key management to allow authenticated sources. How about an .ssh directory in a jails dataset (next to the config.json file) and the provisioning.key=gronke with .ssh/gronke and .ssh/gronke.pub as the key files.

this is mostly copied from the iX plugin and renames / removes a bunch of things we don't need. There's still a lot of confusion wrt the naming. We also need to generalize the plugin so we'll be able to work with both `puppet apply` and `puppet agent`

@gronke

address review by @gronke

regardless of what type of plugin it was created from

use these for generate_postscript which, so far, is still just a hollow ghost.

let r10k handle the rest! and, to let r10k handle the rest, we need to mount the dataset as rw

This fixes a regression with previous functionality!

…nd Types.Path

…t/bin globally in Jail.env

- do *not* git clone the control repo - instead, let r10k do the work - do do so, install git(-lite) Finally, fix r10k's config location (create dir), name (yml -> yaml), and content (indentation)

gronke reviewed Feb 3, 2019

View reviewed changes

libioc/Provisioning/puppet.py Outdated Show resolved Hide resolved

igalic commented Feb 3, 2019

View reviewed changes

gronke requested changes Feb 3, 2019

View reviewed changes

libioc/Provisioning/puppet.py Outdated Show resolved Hide resolved

igalic commented Feb 3, 2019

View reviewed changes

libioc/Provisioning/puppet.py Outdated Show resolved Hide resolved

igalic force-pushed the feat/provision-puppet branch 4 times, most recently from 1669aa9 to 43fd53a Compare February 4, 2019 22:25

igalic commented Feb 4, 2019

View reviewed changes

libioc/Provisioning/puppet.py Outdated Show resolved Hide resolved

libioc/Provisioning/puppet.py Outdated Show resolved Hide resolved

libioc/Provisioning/puppet.py Outdated Show resolved Hide resolved

gronke force-pushed the feat/provision-puppet branch from 0fa7f04 to ad92f11 Compare February 5, 2019 14:26

igalic commented Feb 5, 2019

View reviewed changes

libioc/Provisioning/puppet.py Outdated Show resolved Hide resolved

igalic commented Feb 5, 2019

View reviewed changes

libioc/Provisioning/puppet.py Show resolved Hide resolved

igalic commented Feb 5, 2019

View reviewed changes

libioc/Provisioning/puppet.py Outdated Show resolved Hide resolved

libioc/Provisioning/puppet.py Outdated Show resolved Hide resolved

gronke force-pushed the feat/provision-puppet branch from bf347cb to f87af03 Compare February 5, 2019 21:52

igalic commented Feb 5, 2019

View reviewed changes

libioc/Config/Jail/Defaults.py Outdated Show resolved Hide resolved

libioc/Provisioning/__init__.py Outdated Show resolved Hide resolved

igalic commented Feb 5, 2019

View reviewed changes

libioc/Provisioning/puppet.py Outdated Show resolved Hide resolved

igalic force-pushed the feat/provision-puppet branch from 1a079e3 to 68857fb Compare February 6, 2019 12:34

gronke force-pushed the feat/provision-puppet branch from 6e18f7f to 67081a4 Compare February 7, 2019 12:46

gronke approved these changes Feb 7, 2019

View reviewed changes

igalic commented Feb 7, 2019

View reviewed changes

gronke force-pushed the feat/provision-puppet branch from f6c00b9 to b911599 Compare February 7, 2019 13:51

gronke force-pushed the feat/provision-puppet branch from b911599 to 91f1bed Compare February 7, 2019 14:40

igalic and others added 5 commits February 8, 2019 13:32

remove iocage (c) from copy/pasted code

e15d43e

add puppet to available provisioning modules

b979db8

break cli example in puppet provisioner in multiple lines

4cb9e05

pkgs is a property and needs to be accessed by .

c5063d6

address review by @gronke

igalic and others added 24 commits February 8, 2019 13:32

be consistent with naming: use pluginDefinition as variable name

312ed37

regardless of what type of plugin it was created from

add local & remote property to our plugin definition

0ea62f4

use these for generate_postscript which, so far, is still just a hollow ghost.

puppet: use persistent dataset

c6a45c5

remove unused copy/pasted function

3e925a3

only clone the repo if it doesnt already exist

8fa883d

let r10k handle the rest! and, to let r10k handle the rest, we need to mount the dataset as rw

only mount plugin dataset as rw when it needs to be rw

ca19958

write postinstall (provisioning) script

5a266d1

fix flake8 warnings

35359c7

fix scope and name in puppet provisioning

e19087f

puppet provisioning uses urllib.parse

f48a1b5

rename url to source and allow it to be AbsolutePath

87e3a43

This fixes a regression with previous functionality!

add --debug to puppet, libioc will filter it out anyway

010047c

correct puppet provisioner url to source property

12f538a

use interpolation for used variables

0b22678

raise correct ValueError instead of TypeError in Types.AbsolutePath a…

8c946e5

…nd Types.Path

move path validation methods from config to Resource

c6ec4e9

ToDo: remove unused jailed argument

b98a7d9

docstring on JailGenerator._ensure_script_dir

c671d88

refactor Provisioning

c14904f

set /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/roo…

5adbb3b

…t/bin globally in Jail.env

fix typo in variable name in Fstab.insert

d56ca12

puppet: fix for remote sources

4d5ae50

- do *not* git clone the control repo - instead, let r10k do the work - do do so, install git(-lite) Finally, fix r10k's config location (create dir), name (yml -> yaml), and content (indentation)

Better documentation! for provisioning.method=puppet

ad97934

fix flake8 complaints

a0f8d2a

igalic force-pushed the feat/provision-puppet branch from bf1f327 to a0f8d2a Compare February 8, 2019 12:34

gronke merged commit a0f8d2a into bsdci:master Feb 8, 2019

igalic deleted the feat/provision-puppet branch February 8, 2019 12:50

igalic mentioned this pull request Feb 8, 2019

(help me) add provisioning method puppet #625

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Puppet (apply) provisioning plugin #629

Puppet (apply) provisioning plugin #629

igalic commented Feb 3, 2019

igalic left a comment

igalic Feb 3, 2019

igalic left a comment

igalic left a comment

igalic left a comment

igalic left a comment

igalic left a comment

igalic left a comment

igalic left a comment

gronke commented Feb 7, 2019

igalic left a comment

igalic Feb 7, 2019

igalic commented Feb 7, 2019

gronke commented Feb 7, 2019

Puppet (apply) provisioning plugin #629

Puppet (apply) provisioning plugin #629

Conversation

igalic commented Feb 3, 2019

igalic left a comment

Choose a reason for hiding this comment

igalic Feb 3, 2019

Choose a reason for hiding this comment

igalic left a comment

Choose a reason for hiding this comment

igalic left a comment

Choose a reason for hiding this comment

igalic left a comment

Choose a reason for hiding this comment

igalic left a comment

Choose a reason for hiding this comment

igalic left a comment

Choose a reason for hiding this comment

igalic left a comment

Choose a reason for hiding this comment

igalic left a comment

Choose a reason for hiding this comment

gronke commented Feb 7, 2019

igalic left a comment

Choose a reason for hiding this comment

igalic Feb 7, 2019

Choose a reason for hiding this comment

igalic commented Feb 7, 2019

gronke commented Feb 7, 2019