✨ [WIP] Introduce bootstrap provider for ignition #3437

dongsupark · 2020-07-31T18:10:29Z

An experimental bootstrap provider for ignition, to support OS like Flatcar Container Linux or Ignition.

As discussed with @vincepri , the code is located under exp, so it can be isolated from other parts.

It originally comes from
https://github.com/fossabot/cluster-api-bootstrap-provider-kubeadm-ignition.

k8s-ci-robot · 2020-07-31T18:10:37Z

Hi @dongsupark. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot · 2020-07-31T18:10:39Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign cecilerobertmichon
You can assign the PR to them by writing /assign @cecilerobertmichon in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ncdc · 2020-07-31T18:12:03Z

Make sure you add an OWNERS or OWNERS_ALIASES file

vincepri · 2020-07-31T18:45:20Z

exp/kubeadm-ignition/api/v1alpha2/conversion.go

@@ -0,0 +1,164 @@
+/*


We can probably remove the v1alpha2 APIs completely

We can probably remove the v1alpha2 APIs completely

Removed v1alpha2 completely.

detiber · 2020-07-31T19:09:36Z

exp/kubeadm-ignition/api/v1alpha3/kubeadmbootstrapconfig_types.go

+// +kubebuilder:subresource:status
+
+// KubeadmConfig is the Schema for the kubeadmconfigs API
+type KubeadmConfig struct {


I suspect this type will need to be renamed to avoid conflicting with the cloud-init based bootstrap provider.

I suspect this type will need to be renamed to avoid conflicting with the cloud-init based bootstrap provider.

Right. I renamed related structures to KubeadmIgnition*.

detiber · 2020-07-31T19:14:59Z

exp/kubeadm-ignition/types/README.md

@@ -0,0 +1,9 @@
+# Kubeadm types


If these are the same types being used by the cloud-init based bootstrap provider, you might be able to just import those rather than keeping a separate copy here as well.

If these are the same types being used by the cloud-init based bootstrap provider, you might be able to just import those rather than keeping a separate copy here as well.

Right.
I removed unnecessary types, and tried to use existing types outside of exp.
Though types related to systemd units should stay here under exp/kubeadm-ignition/types.

detiber · 2020-07-31T19:16:46Z

exp/util/secret/consts.go

@@ -0,0 +1,54 @@
+/*


Similar with these util/secret helpers, if they are the same as the cloud-init based bootstrap provider, might be worth it to just import those instead.

Removed util/secret from exp, and just used one in the upper-level.

fabriziopandini · 2020-08-04T11:46:42Z

@dongsupark let me know when this one is ready for a first pass

dongsupark · 2020-08-10T15:32:17Z

Make sure you add an OWNERS or OWNERS_ALIASES file

Added exp/kubeadm-ignition/OWNERS.

An experimental bootstrap provider for ignition. It originally comes from https://github.com/fossabot/cluster-api-bootstrap-provider-kubeadm-ignition See also #3430

ncdc · 2020-08-25T16:10:48Z

#3430 is currently in the v0.3.9 milestone, due in a few days. Do we anticipate this is ready to merge by then, or should we push this back?

CecileRobertMichon · 2020-08-25T17:47:43Z

exp/kubeadm-ignition/internal/ignition/s3backend_backend.go

+	"bytes"
+	"encoding/json"
+	"errors"
+	"github.com/aws/aws-sdk-go/aws"


https://github.com/kubernetes-sigs/cluster-api-provider-aws why are there AWS specific things in here? Any AWS provider code should go in CAPA https://github.com/kubernetes-sigs/cluster-api-provider-aws

From what I understand, Ignition needs a source from which it pulls its configuration. This implementation is using S3 at the moment. We may want/need to support multiple sources, and S3 could/would be a reasonable choice when running on AWS. I don't know there's any simple way to keep source-specific code out of the bootstrap provider, unless you went with a plugin model, where the plugins lived in separate repos. That, however, would make deployment a lot more complicated.

A bootstrap provider that lives in the Cluster API repo being infrastructure agnostic should be a requirement IMO otherwise we're setting a precedent and breaking one of the core principles of Cluster API. If it's not straightforward it should come with a design proposal.

"I don't know there's any simple way to keep source-specific code out of the bootstrap provider" --> sounds like we need to do a bit more research around this.

https://github.com/flatcar-linux/ignition/blob/master/doc/supported-platforms.md gives a good overview of using ignition with different platforms

+1 making this agnostic, let's discuss and design a solution that is agnostic. We have prior art with object references, or we could make this based on the infrastructure provider used (infra providers would provide the code that runs here)

would this be okay to switch to a generic S3 API? Like minio?

Possibly 🤔 We try to be as agnostic as possible, what that really comes down to is provider interfaces that are satisfied with third party installations.

For example, we could abstract away this interface to be a metadata provider. Then, the AWS provider could contribute the AWS code, GCP, Azure can do the same with time.

What do you all think?

this might should be part of the #3761 design

CecileRobertMichon · 2020-08-25T17:48:56Z

exp/kubeadm-ignition/main.go

+
+	templateBackend, err := ignition.NewS3TemplateBackend(userdataDir, userDataBucket)
+	if err != nil {
+		setupLog.Error(err, "unable to create aws s3 session")


same comment

cc @vincepri

dongsupark · 2020-08-26T13:31:39Z

@ncdc

#3430 is currently in the v0.3.9 milestone, due in a few days. Do we anticipate this is ready to merge by then, or should we push this back?

There are a couple of issues left in individual infrastructure providers that have cloud-init-specific implementations.
Let's push back it to the next milestone.

vincepri · 2020-08-26T20:27:44Z

/milestone Next

k8s-ci-robot · 2020-08-26T20:27:51Z

@dongsupark: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

vbatts · 2020-12-02T18:25:17Z

WIP for rebasing this on v1alpha4 https://github.com/kinvolk/cluster-api/pull/1

Regardless, I'm hoping we can find a short term way forward while we figure out the architecture needed for #3761

bgilbert · 2020-12-02T18:23:38Z

exp/kubeadm-ignition/internal/ignition/s3backend_backend.go

+
+const (
+	KubernetesDefaultVersion = "v1.17.4"
+	IngitionSchemaVersion    = "2.2.0"


Ignition config spec 2.x is unmaintained and unsupported upstream. New code should use 3.x, either with or without compatibility code for 2.x. One approach is to generate 3.x structs and then use ign-converter to downconvert when needed.

just to add here: ign-converter also goes the other way, i.e. we're using it to migrate clusters provisioned with Ign0.x/spec 2.2.0 to Ign2.x/spec 3

mariusgrigoriu · 2020-12-13T18:14:14Z

Are the AWS and ignition version issues the only thing blocking merge?

dongsupark · 2021-02-11T14:55:43Z

Thanks a lot for all comments.
@invidian and I have reworked the whole PR into a new one #4172.
For more contexts, please see the comment.
We will try to address comments in this PR as well.

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jul 31, 2020

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jul 31, 2020

k8s-ci-robot requested review from CecileRobertMichon and JoelSpeed July 31, 2020 18:10

k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jul 31, 2020

vincepri reviewed Jul 31, 2020

View reviewed changes

detiber reviewed Jul 31, 2020

View reviewed changes

k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 4, 2020

dongsupark mentioned this pull request Aug 19, 2020

Support multi part mime for user-data for cluster-api coreos/ignition#1072

Closed

Introduce bootstrap provider for ignition

81108b5

An experimental bootstrap provider for ignition. It originally comes from https://github.com/fossabot/cluster-api-bootstrap-provider-kubeadm-ignition See also #3430

dongsupark mentioned this pull request Aug 20, 2020

Support userdata privacy beyond cloudinit kubernetes-sigs/cluster-api-provider-aws#1875

Closed

CecileRobertMichon reviewed Aug 25, 2020

View reviewed changes

k8s-ci-robot added this to the Next milestone Aug 26, 2020

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 26, 2020

CecileRobertMichon mentioned this pull request Sep 16, 2020

Flatcar support for CAPZ kubernetes-sigs/cluster-api-provider-azure#629

Closed

ncdc mentioned this pull request Sep 29, 2020

Support Flatcar linux for nodes kubernetes-sigs/cluster-api-provider-aws#1979

Closed

bgilbert reviewed Dec 2, 2020

View reviewed changes

This was referenced Feb 10, 2021

Add ignition support in bootstrap provider #3430

Closed

✨ Add support for generating bootstrap data in Ignition format to CABPK #4172

Merged

dongsupark closed this Feb 11, 2021

dongsupark deleted the dongsu/bootstrap-ignition branch February 11, 2021 14:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ [WIP] Introduce bootstrap provider for ignition #3437

✨ [WIP] Introduce bootstrap provider for ignition #3437

dongsupark commented Jul 31, 2020 •

edited by ncdc

Loading

k8s-ci-robot commented Jul 31, 2020

k8s-ci-robot commented Jul 31, 2020

ncdc commented Jul 31, 2020

vincepri Jul 31, 2020

dongsupark Aug 10, 2020

detiber Jul 31, 2020

dongsupark Aug 10, 2020

detiber Jul 31, 2020

dongsupark Aug 10, 2020

detiber Jul 31, 2020

dongsupark Aug 10, 2020

fabriziopandini commented Aug 4, 2020

dongsupark commented Aug 10, 2020

ncdc commented Aug 25, 2020

CecileRobertMichon Aug 25, 2020 •

edited

Loading

ncdc Aug 25, 2020

CecileRobertMichon Aug 25, 2020

CecileRobertMichon Aug 25, 2020

vincepri Aug 25, 2020

vbatts Nov 2, 2020

vincepri Nov 30, 2020

vbatts Dec 2, 2020

CecileRobertMichon Aug 25, 2020

dongsupark commented Aug 26, 2020

vincepri commented Aug 26, 2020

k8s-ci-robot commented Aug 26, 2020

vbatts commented Dec 2, 2020

bgilbert Dec 2, 2020

LorbusChris Dec 2, 2020 •

edited

Loading

mariusgrigoriu commented Dec 13, 2020

dongsupark commented Feb 11, 2021

✨ [WIP] Introduce bootstrap provider for ignition #3437

✨ [WIP] Introduce bootstrap provider for ignition #3437

Conversation

dongsupark commented Jul 31, 2020 • edited by ncdc Loading

k8s-ci-robot commented Jul 31, 2020

k8s-ci-robot commented Jul 31, 2020

ncdc commented Jul 31, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabriziopandini commented Aug 4, 2020

dongsupark commented Aug 10, 2020

ncdc commented Aug 25, 2020

CecileRobertMichon Aug 25, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dongsupark commented Aug 26, 2020

vincepri commented Aug 26, 2020

k8s-ci-robot commented Aug 26, 2020

vbatts commented Dec 2, 2020

Choose a reason for hiding this comment

LorbusChris Dec 2, 2020 • edited Loading

Choose a reason for hiding this comment

mariusgrigoriu commented Dec 13, 2020

dongsupark commented Feb 11, 2021

dongsupark commented Jul 31, 2020 •

edited by ncdc

Loading

CecileRobertMichon Aug 25, 2020 •

edited

Loading

LorbusChris Dec 2, 2020 •

edited

Loading