Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CAPI Book versioning #6017

Closed
sbueringer opened this issue Jan 28, 2022 · 36 comments
Closed

CAPI Book versioning #6017

sbueringer opened this issue Jan 28, 2022 · 36 comments
Labels
kind/documentation Categorizes issue or PR as related to documentation.
Milestone

Comments

@sbueringer
Copy link
Member

sbueringer commented Jan 28, 2022

Hey folks,
let's discuss how we want to version the CAPI book.

Currently we have the following book versions:

  • cluster-api.sigs.k8s.io => main
  • main.cluster-api.sigs.k8s.io => main (redirects to cluster-api.sigs.k8s.io)
  • release-0-4.cluster-api.sigs.k8s.io => release-0-4
  • release-0-3.cluster-api.sigs.k8s.io => release-0.3

I don't think this is ideal:

  • cluster-api.sigs.k8s.io is based on main. Which means that users will be confused when there are features documented there which don't exist in our latest stable release. In my opinion we should use the latest release branch for that URL (i.e. release-1.1 now).
  • There is no book for release-1-0.

I think we have (at least) the following options.

Option 1: One book per version/release

  • cluster-api.sigs.k8s.io => release-1-1
  • main.cluster-api.sigs.k8s.io => main
  • release-1-1.cluster-api.sigs.k8s.io => release-1-1
  • release-1-0.cluster-api.sigs.k8s.io => release-1-0
  • release-0-4.cluster-api.sigs.k8s.io => release-0-4
  • release-0-3.cluster-api.sigs.k8s.io => release-0.3

Option 2: One book per contract version (~ current state)

  • cluster-api.sigs.k8s.io => release-1-1
  • main.cluster-api.sigs.k8s.io => main
  • release-0-4.cluster-api.sigs.k8s.io => release-0-4
  • release-0-3.cluster-api.sigs.k8s.io => release-0.3

In my opinion we should implement Option 1. With Option 2 the books are mostly correct, but because we are always writing documentation for the current state, it's unrealistic that one version of the book can apply to all versions which use the same contract. For example the current ClusterClass documentation reflects the state of the feature now, with
1.0 a lot of what we documented doesn't work/apply.

When we last talked about it in Slack, there was a concern that when we have a book for every release we have to get a new certificate with every release. Can we maybe mitigate this by either:

  • using a wildcard certificate
  • get a new certificate which already includes URLs for release-1-0 until release-1-x (x big enough that we don't have to do it again for a few years)

xref: previous Slack discussion: https://kubernetes.slack.com/archives/C8TSNPY4T/p1639069957464900?thread_ts=1639068761.463700&cid=C8TSNPY4T

/kind documentation

@k8s-ci-robot k8s-ci-robot added the kind/documentation Categorizes issue or PR as related to documentation. label Jan 28, 2022
@sbueringer
Copy link
Member Author

@alexander-demicev
Copy link
Contributor

+1 for the first option, there will be feature differences between minor releases and it would be nice to have stable documentation for every release, I think it provides a good user experience.

@CecileRobertMichon
Copy link
Contributor

I agree option 1 is better if we can have it for free but unfortunately, it might add overhead to the release process.

The main issue is that we have no control over the certificates for each release branch because this is something we have to email Netlify support for (there is no option to do it ourselves in the UI). It would be good to email them to ask if the two options you suggested are possible so we know how much cost we're talking about. Ideally, we want to make the release process as easy and automated as possible so we can release early and release often. Having a human manually email Netlify for every single minor release will definitely go against that goal.

If we figure out how to get this working, it would be great to have a page in the book with links to documentation for other releases, or even better a dropdown, kind of how k8s does it (https://v1-21.docs.kubernetes.io/ - versions dropdown on the top right).

@fabriziopandini
Copy link
Member

+1 to ask email Netify the best curse of actions.
Given that we know in advance what the next release branch will look like, we can eventuallly ask for release-1.2, and release-1.3 in the same email, so we will be ok for ~one year

@sbueringer
Copy link
Member Author

sbueringer commented Jan 31, 2022

Sounds great! I don't really know anything about our Netlify integration / account (?) / ... (apart from a few netlify.toml's). But let me know if I can help.

@CecileRobertMichon
Copy link
Contributor

@sbueringer I just sent a support request to Netlify and added you in the email thread.

@sbueringer
Copy link
Member Author

@sbueringer I just sent a support request to Netlify and added you in the email thread.

Perfect, thank you!

@CecileRobertMichon
Copy link
Contributor

Sorry for the delay, I missed the reply because it got filtered out of my Inbox. Here's the response I got from Netlify:


Thanks for reaching out! We cannot provision a wildcard certificate for the site because we do not host your DNS. You will need to bring your own wildcard custom SSL cert.

The second option you've provided would work as well, but the above would be our recommendation.

  1. Add a couple of releases that don't exist yet to the cert (eg. release-1-0 until release-1-5) so we don't have to do this again for a while

I've also noticed that you have some misconfigured DNS records for domain k8s.io that you may want to remove here: https://app.netlify.com/teams/kubernetes-docs/dns/k8s.io#delete-dns-zone

I hope this helps. Please let us know if you have any followup questions.

@sbueringer
Copy link
Member Author

sbueringer commented Feb 15, 2022

I assume we don't have a good way to provide "our own" custom SSL cert.

So I think either Netlify has some kind of Let's encrypt support or our only implementation option is 2, if we want to make Option 1 from above happen.

@schrej
Copy link
Member

schrej commented Feb 15, 2022

They do automatically create LE certificates for custom domains: https://docs.netlify.com/domains-https/https-ssl/#netlify-managed-certificates

https://cluster-api.sigs.k8s.io is also using LE certificates right now, so I'm not sure why support tickets are needed to add further domains to the certificate.

@sbueringer
Copy link
Member Author

sbueringer commented Feb 15, 2022

Wow, thx for the info.

@CecileRobertMichon Do you see a reason why that documentation might not apply to us?

There's the following section

If your domain uses Netlify DNS, we’ll automatically provision a wildcard certificate, which ensures instant HTTPS for all of the Netlify sites using subdomains of that domain.

But I think that doesn't apply to us as we don't have and need a wild card certificate if they provision certificates automatically.

@CecileRobertMichon
Copy link
Contributor

As far as I can tell, we don't have branch subdomains setup:

Screen Shot 2022-02-15 at 11 18 25 AM

If I click on "Setup Netlify DNS", it says k8s.io is already registered:

Screen Shot 2022-02-15 at 11 20 15 AM

I'm guessing this has to do with the fact that we're using the k8s.io domain but we share it with the rest of Kubernetes so we can't register it to this site.

Regarding the cert, I do see that it's using Let's Encrypt and has all our release branches as domains but it doesn't let me edit the cert since they manage it for us:

Screen Shot 2022-02-15 at 11 24 10 AM

@vincepri @detiber might be able to provide more history / context here.

@fabriziopandini
Copy link
Member

Might be I'm missing something, but can we follow their suggestion and add SANS for a couple of releases that don't exist yet to the cert (eg. from release-1-2 up release-1-5); If I'm not wrong, this will give 404 until the branch exists, and start responding whenever the branch get created.
Assuming we release every 4 months, the next request to netlify team will be in 1 year from now..

@CecileRobertMichon
Copy link
Contributor

CecileRobertMichon commented Feb 15, 2022

@fabriziopandini they said that would work but they recommend using a custom SSL cert instead: "The second option you've provided would work as well, but the above would be our recommendation."

+1 on going that route for now unless we are missing something and there is a better way to have them provision wildcard certificates (doesn't look like there is though).

@sbueringer
Copy link
Member Author

sbueringer commented Feb 15, 2022

+1 let's try that.

I think even in the worst case that we have to write a mail on every release that is preferable to Option 2 above.
We would add it to our house keeping tasks for releases and I would volunteer to sent that mail once I have access.

@CecileRobertMichon
Copy link
Contributor

I just emailed netlify asking them to add release-1-0.cluster-api.sigs.k8s.io, release-1-1.cluster-api.sigs.k8s.io, release-1-2.cluster-api.sigs.k8s.io, release-1-3.cluster-api.sigs.k8s.io, release-1-4.cluster-api.sigs.k8s.io, and release-1-5.cluster-api.sigs.k8s.io to the cert

@CecileRobertMichon
Copy link
Contributor

Cecile,

We have added those 6 subdomains to the SSL certificate on the kubernetes-sigs-cluster-api site.

Hope that helps! Let us know if anything else is needed at this time.

We should be good to go

@sbueringer
Copy link
Member Author

Thank you, /hold cancel'd kubernetes/k8s.io#3404 (waiting for approval from dims)

@sbueringer
Copy link
Member Author

sbueringer commented Feb 16, 2022

@CecileRobertMichon
New DNS names seem to be available.

We're getting cert issues on:

Not sure why https://cluster-api.sigs.k8s.io/ is still pointing to main (https://cluster-api.sigs.k8s.io/developer/providers/v1.1-to-v1.2.html does not exist on 1.1). I think I changed the DNS name accordingly in kubernetes/k8s.io#3404, but I wonder if there is anything in the Netlify UI which can explain this?
DNS would be correct:

 nslookup cluster-api.sigs.k8s.io
Server:         10.93.9.100
Address:        10.93.9.100#53

Non-authoritative answer:
cluster-api.sigs.k8s.io canonical name = release-1-1--kubernetes-sigs-cluster-api.netlify.app.

Maybe some redirect based on Host header?

@CecileRobertMichon
Copy link
Contributor

@sbueringer I don't see the updated cert in the UI... maybe it takes some time? If I still don't see the new branches tomorrow I'll email support again.

@vincepri
Copy link
Member

Same here, I still see the old certificate

@CecileRobertMichon
Copy link
Contributor

emailed support

@CecileRobertMichon
Copy link
Contributor

Answer:
Laura did add all those names in our database.  It seems your DNS was not answering well for some of them, though, since we did not get a certificate for all of them after adding them.  This is the error I saw when I tried renewing the first time and is presumably also what blocked Laura's renewal from being a complete success:Unable to verify challenge for release-1-0.cluster-api.sigs.k8s.io: DNS problem: SERVFAIL looking up CAA for sigs.k8s.io - the domain's nameservers may be  malfunctioning To fix this, I have just performed the same operation you could - renewing the certificate (from this button in our UI) - took me two tries but did work the second time: https://app.netlify.com/sites/kubernetes-sigs-cluster-api/settings/domain#ssl-tls-certificateMy favorite 3rd-party DNS error checking tool (I use this anytime I see a SERVFAIL response) is https://dnsviz.net/d/sigs.k8s.io/dnssec/ and in this case does show a DNS problem (though it doesn't seem like a fatal error to me).  Might be related, and thus might be worth reviewing with your DNS admins.

@CecileRobertMichon
Copy link
Contributor

Looks like release-1.0 and release-1.1 are good now. We might have to click the "renew certificate" button every time we release now. Better than emailing support though.

@sbueringer
Copy link
Member Author

sbueringer commented Feb 17, 2022

Thx! Certificates look good. I think one last problem.
For some reason https://cluster-api.sigs.k8s.io/ is main instead of release-1.1

DNS looks okay:

 nslookup cluster-api.sigs.k8s.io
Server:         10.16.142.68
Address:        10.16.142.68#53

Non-authoritative answer:
cluster-api.sigs.k8s.io canonical name = release-1-1--kubernetes-sigs-cluster-api.netlify.app.

I have no idea how to figure that out / where that could be configured. I suspect there might be something in the Netlify UI?

@CecileRobertMichon
Copy link
Contributor

@sbueringer can you check if it looks good now? There is a "production branch" setting in the UI, I just changed that from main to release-1.1. Hoping that did the trick.

@sbueringer
Copy link
Member Author

@sbueringer can you check if it looks good now? There is a "production branch" setting in the UI, I just changed that from main to release-1.1. Hoping that did the trick.

Interesting. Now the release-1-1.cluster-api.sigs.k8s.io page redirects to cluster-api.sigs.k8s.io. This would be fine in general, but unfortunately both are the book from main (the v1.1-v1.2 page doesn't exist on 1.1)

@sbueringer
Copy link
Member Author

@CecileRobertMichon Okay not sure if you had to do something. But I think it works now (https://cluster-api.sigs.k8s.io is based on release-1.1)

I'll open a few PRs to update the cross-references to other version of the book tomorrow and then we should be finally done here :)

(+ I'll update the housekeeping issue so that next time we ~ now what we have to do then)

@CecileRobertMichon
Copy link
Contributor

Interesting I didn't do anything since then... Glad it works now

@vincepri how do we get @sbueringer added to Netlify so he can access this stuff too?

@sbueringer
Copy link
Member Author

Interesting I didn't do anything since then... Glad it works now

Maybe it was triggered by a merge on release-1.1 after your change. Absolutely no idea how it works, but I assume they are running the book build after every merge (as specified in the netlify.toml), so maybe this triggered a rollout of the config.

@fabriziopandini
Copy link
Member

I can help with Netlify too, even if it seems there is a bit of magic in it

@sbueringer
Copy link
Member Author

sbueringer commented Feb 23, 2022

I can help with Netlify too, even if it seems there is a bit of magic in it

Thx that's very good to know. Just to summarize, now Netfliy behaves as expected. It's just about future changes/issues.

@sbueringer
Copy link
Member Author

Short update, last PR for this issue: #6194 :)

@sbueringer
Copy link
Member Author

sbueringer commented Feb 24, 2022

/close

Thx to everyone who contributed, especially @CecileRobertMichon !

@k8s-ci-robot
Copy link
Contributor

@sbueringer: Closing this issue.

In response to this:

/close

Thx for everyone who contributed, especially @CecileRobertMichon !

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@CecileRobertMichon
Copy link
Contributor

CecileRobertMichon commented Nov 22, 2023

For the record, here is the email I just sent to Netlify via their support request UI:

Subject:
Add subdomains to the cert

Site:
https://cluster-api.sigs.k8s.io

Message:
Hello,

Could you please help us add the following subdomains to our site certificate?

release-1-6.cluster-api.sigs.k8s.io
release-1-7.cluster-api.sigs.k8s.io
release-1-8.cluster-api.sigs.k8s.io
release-1-9.cluster-api.sigs.k8s.io
release-1-10.cluster-api.sigs.k8s.io

In the past we were able to have the Netlify support engineers add the subdomains to the DB even before the DNS records existed. When we then proceeded to create the release branch (and thus the DNS record), all we had to do was click "renew certificate" in the Netlify UI to add the new domains to the cert. This avoids us having to email Netlify support for each release. Is that something you'd be able to do?

Previous support case # is 81938 for reference.

Thank you,
Cecile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/documentation Categorizes issue or PR as related to documentation.
Projects
None yet
Development

No branches or pull requests

7 participants