From 8505b278435a9ec6e8c4322393c21f82a0891284 Mon Sep 17 00:00:00 2001 From: liz Date: Tue, 29 Oct 2019 17:09:00 -0400 Subject: [PATCH] v1alpha3-compliant cluster API provideri implementor's guide --- docs/book/SUMMARY.md | 30 -- docs/book/src/SUMMARY.md | 7 +- .../building_running_and_testing.md | 132 +++++++++ .../implementers-guide/cluster-email.png | Bin 0 -> 28253 bytes .../providers/implementers-guide/configure.md | 159 +++++++++++ .../controllers_and_reconciliation.md | 268 ++++++++++++++++++ .../implementers-guide/create_api.md | 65 +++++ .../implementers-guide/generate_crds.md | 87 ++++++ .../providers/implementers-guide/naming.md | 54 ++++ .../providers/implementers-guide/overview.md | 70 +++++ 10 files changed, 841 insertions(+), 31 deletions(-) delete mode 100644 docs/book/SUMMARY.md create mode 100644 docs/book/src/providers/implementers-guide/building_running_and_testing.md create mode 100644 docs/book/src/providers/implementers-guide/cluster-email.png create mode 100644 docs/book/src/providers/implementers-guide/configure.md create mode 100644 docs/book/src/providers/implementers-guide/controllers_and_reconciliation.md create mode 100644 docs/book/src/providers/implementers-guide/create_api.md create mode 100644 docs/book/src/providers/implementers-guide/generate_crds.md create mode 100644 docs/book/src/providers/implementers-guide/naming.md create mode 100644 docs/book/src/providers/implementers-guide/overview.md diff --git a/docs/book/SUMMARY.md b/docs/book/SUMMARY.md deleted file mode 100644 index 4271c113e75c..000000000000 --- a/docs/book/SUMMARY.md +++ /dev/null @@ -1,30 +0,0 @@ -# Managing Kubernetes Infrastructure with the Cluster API - -* [Introduction](README.md) - -## Getting Started - -* [Abbreviations](ABBREVIATIONS.md) -* [Glossary](GLOSSARY.md) -* [Interoperability](getting_started/interoperability.md) -* [Existing Providers](getting_started/existing_providers.md) - -## Common Code - -* [Architecture](common_code/architecture.md) -* [Repository Layout](common_code/repository_layout.md) -* [Cluster Controller](common_code/cluster_controller.md) -* [Machine Controller](common_code/machine_controller.md) -* [MachineSet Controller](common_code/machineset_controller.md) -* [MachineDeployment Controller](common_code/machinedeployment_controller.md) - -## Creating a New Provider - -* [Overview](provider_implementations/overview.md) -* [Naming](provider_implementations/naming.md) -* [Generate CRDs](provider_implementations/generate_crds.md) -* [Register Schemes](provider_implementations/register_schemes.md) -* [Create Actuators](provider_implementations/create_actuators.md) -* [Register Controllers](provider_implementations/register_controllers.md) -* [Building, Running, and Testing](provider_implementations/building_running_and_testing.md) - diff --git a/docs/book/src/SUMMARY.md b/docs/book/src/SUMMARY.md index 3fb709448ec6..74f20a8e54af 100644 --- a/docs/book/src/SUMMARY.md +++ b/docs/book/src/SUMMARY.md @@ -19,7 +19,12 @@ - [Provider Implementers](./providers/implementers.md) - [v1alpha1 to v1alpha2](./providers/v1alpha1-to-v1alpha2.md) - [Cluster Infrastructure](./providers/cluster-infrastructure.md) - + - [Implementer's Guide](./providers/implementers-guide/overview.md) + - [Naming](./providers/implementers-guide/naming.md) + - [Create Repo and Generate CRDs](./providers/implementers-guide/generate_crds.md) + - [Create API](./providers/implementers-guide/create_api.md) + - [Controllers and Reconciliation](./providers/implementers-guide/controllers_and_reconciliation.md) + - [Building, Running, Testing](./providers/implementers-guide/building_running_and_testing.md) - [Reference](./reference/reference.md) - [Glossary](./reference/glossary.md) - [Provider List](./reference/providers.md) diff --git a/docs/book/src/providers/implementers-guide/building_running_and_testing.md b/docs/book/src/providers/implementers-guide/building_running_and_testing.md new file mode 100644 index 000000000000..039a629d16b1 --- /dev/null +++ b/docs/book/src/providers/implementers-guide/building_running_and_testing.md @@ -0,0 +1,132 @@ +# Building, Running, Testing + +## Docker Image Name + +The patch in `config/default/manager_image_patch.yaml` will be applied to the manager pod. +Right now there is a placeholder `IMAGE_URL`, which you will need to change to your actual image. + +### Development Images +It's likely that you will want one location and tag for release development, and another during development. + +The approach most Cluster API projects is using [a `Makefile` that uses `sed` to replace the image URL][sed] on demand during development. + +[sed]: https://github.com/kubernetes-sigs/cluster-api/blob/e0fb83a839b2755b14fbefbe6f93db9a58c76952/Makefile#L201-L204 + +## Deployment + +### cert-manager + +Cluster API uses [cert-manager] to manage the certificates it needs for its webhooks. +Before you apply Cluster API's yaml, you should [install `cert-manager`][cm-install] + +[cert-manager]: https://github.com/jetstack/cert-manager +[cm-install]: https://docs.cert-manager.io/en/latest/getting-started/install/kubernetes.html + +``` +kubectl apply -f https://github.com/jetstack/cert-manager/releases/download//cert-manager.yaml +``` + +### Cluster API + +Before you can deploy the infrastructure controller, you'll need to deploy Cluster API itself. + +You can [use a precompiled manifest][install], or clone [`cluster-api`][capi] and apply its manifests using `kustomize`. + +``` shell +cd cluster-api +kustomize build config/default | kubectl apply -f- +``` + +Check the status of the manager to make sure it's running properly + +```shell +$ kubectl describe -n capi-system pod | grep -A 5 Conditions +Conditions: + Type Status + Initialized True + Ready True + ContainersReady True + PodScheduled True +``` + + +[install]: https://cluster-api.sigs.k8s.io/tasks/installation.html#install-cluster-api + +### Your provider + +Now you can apply your provider as well: + +``` +$ cd cluster-api-provider-mailgun +$ kustomize build config/default | envsubst | kubectl apply -f- +$ kubectl describe -n cluster-api-provider-mailgun-system pod | grep -A 5 Conditions +Conditions: + Type Status + Initialized True + Ready True + ContainersReady True + PodScheduled True +``` + +### Tiltfile +Cluster API development requires a lot of iteration, and the "build, tag, push, update deployment" workflow can be very tedious. +[Tilt](https://tilt.dev) makes this process much simpler by watching for updates, then automatically building and deploying them. + +You can visit [some example repositories][capidev], but you can get started by writing out a yaml manifest and using the following [`Tiltfile`][tiltfile] +`kustomize build config/default | envsubst > capm.yaml` + +[capidev]: https://github.com/chuckha/capi-dev +[tiltfile]: https://docs.tilt.dev/tiltfile_concepts.html + +```starlark +allow_k8s_contexts('kubernetes-admin@kind) + +k8s_yaml('capm.yaml') + +docker_build('/cluster-api-controller-mailgun-amd64', '.') +``` + +You can then use Tilt to watch the logs coming off your container + + +## Your first Cluster + +Let's try our cluster out. We'll make some simple YAML: + +```yaml +apiVersion: cluster.x-k8s.io/v1alpha2 +kind: Cluster +metadata: + name: hello-mailgun +spec: + clusterNetwork: + pods: + cidrBlocks: ["192.168.0.0/16"] + infrastructureRef: + apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3 + kind: MailgunCluster + name: hello-mailgun +--- +apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3 +kind: MailgunCluster +metadata: + name: hello-mailgun +spec: + priority: "ExtremelyUrgent" + request: "Please make me a cluster, with sugar on top?" + requester: "cluster-admin@example.com" +``` + +We apply it as normal with `kubectl apply -f .yaml`. + +If all goes well, you should be getting an email to the address you configured when you set up your management cluster: + +![An email from mailgun urgently requesting a cluster](cluster-email.png) + +## Conclusion + +Obviously, this is only the first step. +We need to implement our Machine object too, and log events, handle updates, and many more things. + +Hopefully you feel empowered to go out and create your own provider now. +The world is your Kubernetes-based oyster! diff --git a/docs/book/src/providers/implementers-guide/cluster-email.png b/docs/book/src/providers/implementers-guide/cluster-email.png new file mode 100644 index 0000000000000000000000000000000000000000..896d3b5afb93d74eed8bfd1b429d839b997cb803 GIT binary patch literal 28253 zcmd43^+VNL_bt2;1q%yBKokoRQ94A@Mx+Fk5C!S(mNKvbm2Ol*q(nLd6#?lKq!9@b z0i~OFEYH2~`xm_5d!HYka}I3w{_M5pnsdxC##|2-<Co-muZ97+P4G>mRe#wb9qN zur;!@8>23gpiuTwE}S{3;uQY7(^2#C)XImyJFQx;Qe9Fk4xpc@+P>4LLnBn@h{?0u z>i2Z#ulU&LeQ##r80cDm@4(r!%9l=dJlb$}OY6)Gt)#r>z?$#9*wmAW<5(x<_3z*0FHnyB_x>`g6#u=!3+4Zl zZ)I;b`{(;zFR%I{kCjjIkLuD^rc8^hf1LaD>CNrXl%&?M`S9?ta<;{_wroo+#c~gJmDh^5-oE8=`2FqK&|>lZ&GZZm_x$|) ze3avMD=RB&WSX4q%(D+LPT2eluj0C15Iiz37bRiH7Qc}#i3PNfk zLA9ud2X$vKeXLyeP&CdD)+f3j3+fUS!@TYZZyWpeJ(4b=EzJhd9;JGZ~LR$He+4y zKgr+Ta%xe*g?+)3dDkx6#aV0Y{p_FO%>hHN&Ux=Ydh`k2jmKlz6}w1f)KwsI#)pR-*IwEEk7mP@PfiLbBkIf{Rq!0(}<^1JKk zkKdd=8pL%?rzTv4k%_53`Kqwb9`Oonz_$!zzlPYom*v|PDnEU?&8eDVH8CLfdt^j# zU}?JD5*vJ}&~;8Ts;Ft%?qcB4?>~Gc5ApCk)vpYs>iPNeOK-{JT&KVLcXD6v3$rZz z{+PRPFCKc|>3c?IWyQsJPT${TJ<=w)xVXqSX?oFIB2^_}TbPh}@2{_kEFv~%UWF|7 zmHR!}Ez01D@2Qu^YCbnNx0QuO)_0G%QC;*&pGrf=V`Ads>o#oAo%~r}G2UMhZ~*Tw zxw5DnVxXh*$$OXZES`w3^L|K3h!QowNz>uxlxvBrqbRkJ;+5|Lvt>j0NQ~-NGI$Rq z%q^zZN+`z(ntgjZHQZv0m0Dh2u*A~x^758vvBadzEQ}S=v#?aPrW=Nc*dD_FDi(iN z$XtAI28N0}{R;o@?UqF)-Q7_= zJ=#9IMehX#1(i-0fBJOXaa1$Qn1_!~=hNMF;Sz44_$N2V<)X}V>bn1wBp(hA?d7sp>JjqXl2 ztgW#c`yv;4FMNzwy?$Yt#lj5u0a$Kf!TDqcs zhvYte{CMltdEe*3!4LfW=nD%AGfcid>dbfiF+6+`zgEk4ICdn!K>uw}&`x=I`Ptdo zExUGIL;$Fz8_?nJ3~+M7#hjnxY0{8i^dmaVSofYd5xTT6^C~|6eBza4|Dm&w8DcFu z^A?w8x|xN)cDRhap9vN+XCADL8XTw&L)v_MqnHNUQ-<$;+-34GlGu%&54V__cT2j9 zVI>dl+egL9%8Kj|!l%EU)nhTO)TXqw^r-r~y9hPE9h_Hse*doe)zP0%S`5{wU+`RKW3mo0T;}Yj<;}78Wq4UGlIxy5d#XYOIg2G4f&zX!-MpzBE3;#9X(=r&?FJ5x-Egy9 zNL1L8K|GemtRv^yX352;8+LM8&HlN;vU_(376-Yt>T6cv%JR+8j@;rr`{B2lnfl{> zkpi89IdK*aXP?WNz2ZMMdTAco);1J`&*jbPy?Il!ZMNslYr5d zd-v|KoUmKV$;mlT6A_kZ9uuUTTPZ)ka9bu;r#e({BO{L|a?4wrewjx*xoeOD{dbz_ z>gu{Lj0m5w`x)IKiYTq}q@o^o-6+zj`>t}PB~{B1VT2`p_Vnq8H@E z7%t*No06{v;T^XyGJg44|7t@){*x`etS3xw*UHS_I)L2xWA^V4@QcD)TvdJiNV@>zC(;x$(63A|iGp z2a&X;VJZ0)x!`9*?1s_NQS$7R42w=?q>wE0_US8wS*ReBW8GTovgRj;?mT|H;nuBN zPEJl+SXq4^K3qo%7@nlUg^P=8BeTE*l%o3Bi#vOJd)4ynIH;(oHts&*-|ew-O}+Wc zmj}j;aU7ZVnwy*P20f#rHHa_u45O_$vrP%g_Ya)+`GagUFy}NmIf;ejl7^PP;8H z&WW+A%HtJJUcA^>8F;MHYWdv}S{D}=*(izKnHq8^Z~=v1O`DTmpaLINd+WI{Gsf9| zLF>vp`aOGUu-S@_)7|@6P-nhGQB$hBYh{ zhtZDm0_O>78JSfdkYT&s7rgPSw{8m#$^Z*Iaj zcp4o1rQDC*+RpBCef_iHmQ-rRWDP-Y`=!N2|L)Z5@7I55j8`COf^y-)1>|%pj_S7D zPWLxO%kz Y^cl-Z*{q*fA=j_N-%n|NbSrI;>kQi(@eU^p=0y2~Q+8j_S`yCQQ>m z1_tg$M(#Q9yQieNnc@8T^GQibYd38wZEa;TH8s_5y?b{xFyD*C2$vL`?By%QwKccb zY{+q25V}J{SMe@aEzI=qV3dcBp5EQWL{1tSnx;fmU;E+at-E$5u8suaJ9OlTOz!AV ztdD#1A*;$npgL+E?GF@}$)C4!bDe#LRg%=Uva_?NTz_AYsG90;YlXZsIW?7_6#J-j ze5g{!p*dB{S0kBb@7}!>siDEp_9cU$Wu#lHAEk6`v#WFLhZ)ZMuv|*n8RK8n5POkR za?_?wOj$u$V(|_S(f{l{bcjaE*jTz@gV|3%s*A9EZj>REG)EDw+iE-o4v-@+Uc4D#;o%B|t;xcj*#k+@5anJQ50Y_Te6-*I@PnZSQ3kwUcTD1yb ziR%8Q-MxStA0$7@J==el@0z6G-Y1VAHwL@Uj9&9DQeFs&OTFc?G@n`8ogl~z$XuOK zbnS0%$-^g4Rx9l}RbF0RQSj}X8f7CLUHXZRY=)gX%MgF9Z*Ncoz7k+p7o0khdE9;R zxVXz?G^*`Ec6ME3<8quEYez@bRdr|=tb0GNi)A?ao?Ri5!Vy(*Z9gS|g~2#zNMJfNa}`tZS%cJrRY znaT6>ZAIDffw}lEBUuHl0h?j-ifi)=r+(#L&oH`lPhUCzrJlVw3cl^LUr!dQM)&vy zxu(js3JVB)uBdnv9LyLgelw0&tI%NY-O=qXl+XwO-5 zffT8utINy7qy4AnqYRz=n@Qo99WG%bmYsP4_}a+9J|Q9M_XcK~>=7?%+1Z0bjqw!) zK0J3`waU) zC#n3}aC+WSIy59iU(Er5YS#MB3viRqWzxu?CcGS(!I(c|!{f zgK;#%OLMMaO9L4ZRjZntG%A3ps5z=zg@lAE7PF64J{>>BYf!~l6D8@3EkHL_2DFFV zGiV~3!2L`lK%}l}a{_mETwI*<1^29m?wv}tQ{f`EInEPtr3FpPAGU7YDpeZPGg&>> zHehT%#+zF`)8@Z~obvMJOCDk2vbs9O17q!NZ3cPQ_MKV28BbjYY_5Cb#;y2xb}f%3 zLG-7^EiJ0Kb1}^_v2L?^>^wY*L#qA`AD-vt&RuPhE~uU^E-qfTc5O5&quSN06Te&e zy$=bB*jW7(({)OX0a-&5l6a&02GybZCmzUznQq;&qXelu z;c_AafTWB^9x5O&mKx2>t)vByQ!>Yv5Mf5$h2p?4dO%am)nP!t$B!S^YrDJBS5E)j z&ku>^V^fn#g0X^mmYlw+Y31Xc+)fh%bYa4lanjHBKVjm(XY%!p^{=J`*~*hvRy>z3 zUGhJ2`4OstX@NjrKeGvE(e%y!iXD-X9`dm=0amuQ$`z-PHc20e_pWf;o2cDq?$4>Z zNnBhUffKtrFF|fO_gR}jSAsG-5X(7~#`*dAkmh-D*V!$|!Edf4SA_5zdU}KwZfL+p z7`J6`;HjD2j!jKX1#s`weD4x~zw7<}{UcB=7LCMWj-ydnJSt+J|JXh@DIs3ok4Q7? z+))j&#H1JqnYSJ8Tr1;$c;B6;ck5o9+AL*Q6P~a&{Jrbka~z?%_ctAZ;3{rW)2`jP zQ4eT!c77h^GkNk^!Oa=2b?eqOC8_&iu?J_P{2913rBQcghvK6M(9gE)j#Eq15io6j z0%T|Kn(0e&cIz@8(WoJ29e|wynsKJv<<1iWX(nHfXk=T+X>0FA15=FZW@~GU{E*J_ z#IG5kcL*pQXT)%@CPK5wJz}%OtS63wX_G*Nq(`J~nNNL^I{zkCi6>D}tR|Vvs;a7F zLBAF@$c%8dBWJO&upGDUJrlU{l|vYz9el#xByOefQoE@&dVcI;tm`Ok531AW&z}ht zvOwSULn`Qk-=Xh)eP?i5)d7A1Cl82;g}*%exEGx_nteVxDf(^O2qXbhR$5kepPF`4 z&%i+73*Yb2C-1zzmM;V($;ilf_>Py7=?_3N1dsumNrI5@(k)C(7trs#efu^V)!q5T zpvXgC-`b$1xJ^gZ-)d78Esk$M{w{D|Vn&gXm6Z(^G!5Lw$Z!_{<`*7b1^iqEOqQ0F z6$glu+f@TVLLdjo6+Iw7(tx6%nhHe4x8*x>qvH4Hj}__y!TEDt(_mleG%3DEH#yoF zRx7da1P9UP`=>QWj~+E@dc76A)pwxxB=z)r%S#K5Xnw#@os^bdOSem?4`Ii=a!amV zEV)|zP`vAAj+sFG^*e_V4bVnaX!J6UEL;rYioO2cg}}M-{z@=AXYoJ}A3Qi*w>*66 zfyegwy)ozp8)Gie_s3s)b?#exJHPX|-Zpyr5Mj$bPx;i0SC*G@a&iv8YD6g}Eg>pX zhFL3LakfCCsK>Gxny+wCM<3EuDM&8)uitUx4^Z@A<=*FruJ6V71KM6xohynuZhae1 z@qKJe0WAd>6ShN#?g32H;dCX33LV(L|0jY9q=G-URwQcT1m1&FPTP3l>83qm!N_hq zPM<#QGX48ue!eIkzsp?+g+=6#!$a8t zS__@Iw(O+$TAZ5@Flk!r=;%oJtY3+#ID}`ZVkJs&gvcM`&=OogV7mL_teANv@HAox z4dH2Y*CHUR(5cf1gcS>$efyAFwEPI{1WIoyiVO13K;>UOc0Ru6fa-TaBq#%cqc#!@ zg#Ju9&%eOw?`;xQ=*SdboLYm^d*9brAx?|B?vTRinwjA6aq$mmhjJZ8n7<~f#@qB) zcsl5>AmYG81;#5xjn556HD;Qs4pfD_+UUCN)OIoPojZ35#ysJ&x3(^y+3ou0BQ0r@ z5jy&Fyq33=_??#K?ITtc#?ZJ&rd-SC)A{@$dyjL}RS5|R(x>7HhV6<@ZI1)m(8O=&+NGGZe3uFiBxq}75eb%uWK7to>NiNXlrYKK%{OkV?_lm znCZI7+^Bi&nlZ`-R&Y&25EmynDsY<&KaHICb*%|Fa=BqPpLB~JDitXnY2X&ahLSeF z&dwI@>gqxSD-F>fRtkMu;m>gz)C;;3o#E!>HCfz1@yN3qVeSV1D zyMAiT2=Jexgv+rgiLEOJI9+T)LV+k;+Ba^TM7DkV?p^QXP$P5q@g1igtUO<}S&Cb$ zFzK(Hg!zhXxrY~EKX9ob_6B=?pz-6nxqd7=l6V8!5R<0YCviGJAnG9>Xsj3pE%pu% zCoVp&xxM%xe~Xtn9+1QtmJA(X4iY{XADz~B`fD`0ytedSyVB$Kzu0~0izoE{XJvud zit0~<^abP0&K`eT#`mAfFdI~PnQ{U^B2GR+L(zEaf0_kVuF377P=$Z0!{4l!&Hppn z#N}`6>dv9Y4S{<=aD4jqE%@{2GpA0S!ue-M{o?-Jw~jp015KxRYDG6UH=0c>(#aZ` z-vIl&-DY+_eE1OcL-Ex)Izmk#4}lk9myjrY>&N*|NXhskFZ|JCd2aAqYb&qbmn{Sr zA&pW2pZ0d-+JYi{{@}q{5ap_0IR2Am+FT5&>t?z=qR=kz;wZ5FSEySiE_86ZG$PUR2Gu11kmyr-|4qIm%prxT1AIa_>Fg=0d7?Ik(ApF?JX8`Bm%hMIU zAO96#wQP&syM!%bph7SOrX9N$+4lavL2#E%ukYOR>(_D?T6p)YjyOV;guAmlo$$p` z5N)xN|Fi`E8yYBSXhwey4Grx#W-cfwm<5>~BIe8oJ|g~Fz5{4I`dz!iMqGS+d`Qkw zUyeZE4|F{tn2ah=*)p=!F7EV4DzTrGE|e*^`NG0k($ZgeI-8mV8v9Y+P`?LoJl8}Bc%`TFU(2&26#oDYXvO5yr(60R z(ea~i^Y!+oGsz5Pb)UV30E$zH^0{+|8b?zXhjt^&amBftpr(j*d>Sfbll8n8!s$Nu{l-u6ACW<)xrI>jS~VYRZ;y^9E{04;@n2VLLkx$%AutwOO43`e7(GeBO_)zkYN%&Ysr z+dCHht?T?yJeC$M?gm2L0l);7H5Yk!n3aO=c9cFMZC}Ow+o`PRoX9LG7{t_c~k2*mg5xMS8qTCkUA3o=;4QQYF zRQGwKTfjb#wLZ99R^S4Y}QhoT=5E?38z>~&xN~dEN*O*#m zSWqp8M%;e=u&nd7A3TM*Bdv|w*Ha#;CSOdLua;8q{T5W4oAdSle59_E&C-5{{zRL> z$xBHx{)Z&ycR1FvP=AP1QSY+s+^uVSaX)L<_g|6T2iXrrr}Fi=%~+_31fHptDtVv2 zEVlMU$gu7HyIXp@ha>2u{GLDO;o;fK!Eqt6KRG#B@V3;o*UIsmc5-QcaVSE8A$SW| z0sUqP0Kib!dr`!y;m^7ku768*QaXw}w0?};quXU-7LVrBb*?|?^;Z)D*ngCIi-1=k zG8u9e4mfFz!Rkcqn_d00F@7`XgcyY=2?~nRgH5|n{3+R~wS}JkG>!nlYd`?;s6y&B z7v<%fzomG2tpP^Y1qzw`-Rfsqv=j)40;pS#{>-@k<(iw*zrQypUv-^sH6(OBdfv7y z^GcwpcVbyMb7`R9-DkhkqIVu?%fwMwuA5(6MG&IS=e5W{`12cR&t)UUm<$aK1COe| zTpdlq7`dtp?8C9ESv|wUmEigK-$%Z8nd*hUW%lhY6_Aa0P|Wm5JK^b37((^kzxcd( z$WgDb$7-gE7dsQDlpnd+KXEK}Sop_Wbz+Bz+jhi6xrYC`e47A~e20AxEp-m7XLIam zpYHio8tZRaACj0a{O4D__{NRixBN$`Vs~Zrb=-@Re(>;|{&?-h$coQT(h~=%wJ*QS zD3wTh6tv%})7O)^B_-1T&0y(>-~GewlLuaY8lx|8q@O=>nt2m*?ZjJc9j*6Y8l6$I z{*F!5&_wUsw-355pt%=IkiCsfFVX^2Q?7s5|Mmjhes$Zx$msRfy^bi+*n}HqW<(|- z01l#w9>|>vJ|L9WD!Gy)*u^u+SE#K3mBJ$;I5aX26B2Zw4aIgolT}i$HTB?mhCkv~(K_%SI`1(NG3}{g?dU&qohJ@untbg=Eav3q8c#m+@~80UPeAVw*&F+C5m-zn$On|PuBBya z68-Moa@~b)9O~~51_lOVzYUo@BZ2X66yMvrdv`UOPk%8(z|$xRx1U&rW^OqVyR8>w z9*8#Hl;U7`EvuG@LT{M=@C?t-?fs$ll%Bb2 z?JmbLd9@Ezk(~vP_VlRbGC0=GKifhdVN=*3kS_WDuVdhXlhyZ_=Yg~*;}5G3o~E!I zxx&sX#1PNCva&5U)i)tI;-A@J`G>yRVLMn#)~PQrd<#AI*Wm&p2&t=hx|t8#nq# zb0;^24kiu6201+S(MWEd6%`R-3Xt&%31I>c3#LO55N_MntuLO{RQb?p1KvT{TMsIM zFe`X}ewNb?gTihae7h7?GEFX{X4If(MOSX*(3jE>1rx$WuZrS%J$q zdZ%`_A*;6XQ_=aq6lO;ZM3&qVC7HaBtn=&N`^!ZTx3$bxn%8=LwMu zA)BL9Eu+&$Vm^D+Bt+h#y0Eem(ed<^=-VTgLL2Bb6!Y9|YgV!t-A*59QjZbkkl?Dc zedhnCHrsLP@)3Pj7nk|P_ij6Qw<^5TxV(1XXO9k1(X~TBFK5o2;W4V)>pcVlzHT-j z49<4$>*1&@LwwE@%|chAjA9wQA|fLAIy*X8Jj;Ap(Y@>fw4jSwEsI#B*~`VX>EgwU z`Vo^Jra#aaLVLdiHDPb8f(a5-x~2qvzhT1$3MjpwNSDArQZkOW!6_o9gGUMqtnBO# z3JEcwCZhbox$x%7@Z7=zo@BSg{LfdiVcqGXMQv{&s)I54{w+-xkBpATi^Z1>L;>6Y z;3TOl0*`I@09Kt}PWMgl1NNUoyA)r#-0@%MGVee4_L#zHuEF4Q^uzl(t#!sA$tG`qz?e(AHCT`g+yMj?-43QuiM)$lc3DPqN|3tXJ;SBf@kk4)}C@`&`h z{z7kL^ChM*3dgN9q2Cp8;iiHdZqmxWY`xQTGG8s@es{O!KDp9PkHpgbx2D;1)gnv| zPgo`QSXx?MN!B2JVSsZ@O${LgK(+F_&D%u?7sZG2+*n0Tvr!vmfsvURD5n^L0fccD zU=gCDx-UARaXya#E?S!21>TYk?tm{}q=_VerU&VZ*~9dJh)4*S6r#Q(tHVKo>>^ZY zn`S?J1)!xE{Z(t4E-iShBp66?a*kgwaC+fdw}RzVuc>@|HQRz%7|^PIMo%h0Gor`7 zZ{IBJSfAKu4+ZpimK+)gvm=#601%-tT z(Yx4*tM==vgO7LjO5D>=*8T=1*ZYMa5)d-J#~Q((*F< zC%?%*r>A#vof3-^I-Ah4V)}gBHm9kdTpw==4+L`^*ueS5Vf68f1d-G|o&ELw=|7(- zRca)dD=>SAc*Wdxy52#@Htw$G^WT=e80_tRdC!0E$C47Zb}#gZM7^_n{x$fr=zcbRz+odzQc$PQmGe%YvE%Q-->tom9q46C z@wy9XK`vP>v#+XCK|Ji_);nmffdc%wy*-RHnBGyDOMmcZ%g0*uU+*0Vo%Y)n*D!2 zczVd`f5M%nlCrY1Acc`Vq<4!dOS{v}quY{s6yALntKa6?^5*bMy6b;6c9d**6)z^_ zE>&<+b6>pLzx6Tt5wTuM?s?VWo!7j!S<2jqP&-)JZ?^4`j-R56Zrg@p!@FS}2by}1 zq|xxMiBS3XMV_4>J3}#fu2;H{s>!iSc^$QV*K@rv?fd4pckg(aXx%6lJRBHwaLd0A z(TR4439F*tLH;*IMpF5I_NX{rJ68MAP=m%~2VPsxRY3byOX$P#@`J2OZz2e6gEkZ~ zlDgLocoL{gL}3`9>^UKmOijaT6I`-Ijvt3fK}`dh}@lyXf}VYpCE;zChMVIllOvfrV`!bKpa z(7uRUpdIIU|?VF>jj5@$2^dQ5QItu4xF1&)Zr)&&?=?dq0p@19JE1mTH?p9 zuwEqgB@glQpbSBNzy9q)eFFaG5_Dr^K%~&osGV6^Sx@$eheA{Oi_cpnePbF8YZ;#P zXLnIi%}j(=mPMzGrsim=Ey>e|cal=j$f3c=-lBYXy)trW_rrw;DXs18w!_WGV1|JBUo|xkOX8XH z=O@rpiY2dPB_$`PL62|D8>^TqbYIFlw%j3EZFNk~M(4!@+x-c2|H4R|np`Cjc6Q>>{yu zg3A3fHPs)a<$h@(KR=(}sz>Vg@83CKHaD|Ms0_r8>Vm`+uR~_uEuBPMW=EnQ3rK2lB zas_54^r*|ss2qG+Sos*BWEAsAVwQ-L4LRzYZ{D6oJTagOFXh_kL)-ZXaf1r2AT>&b zX1;?rXk0kLq){CKwfH@jrgp-@`mwDo6!ruXRS-$|VpO%XBrAsluY+SGUXb@QT?gRQ z0%Ragr?+plak>am$?!aNcjf;#^Kmi4b)xZEU26c?#CeDxB@Q#>V3IcF{^3I;8}wTIBAAq=Nm)8=d-#^ZiTgjggEFOpxF$GQKTPU;4^q9|4vNkf-4H?x}Sbz z(0Fck_AUIFP#xCBXc9-v#P|EG;ypJ6o36n1V?1msD0HSfw0ZHq1zEh8{|=c?bK6V>xsP-$p$KDN{q-? znEO8B5F}hlW`;{}&*nX1+Fb?Cgp4?T;sgLpiDS2088oH<0deQ?yISsZUf6H4P>{G4 z&`lEcAMxMfWZ^dTRaG|PL;%EUWr@uPWW8De>9cu4g1R612TPI?F*4&bOhC_a(E)`AY&i90o>Ql(k^^&!d$k2=W;Y zyK@kJi02KQZS9m$p54$XL?UT3(7F-cPD-YzAUb+4{CbbW!gfK=adL4P$6oXf4PAhl z1n$HB#MHuuZ`UCnlamW8q;1?nI$|H`7Y?>iBF@3(eB?^fS}BkvObGIB*9kfzlY*N= zH+ZDc{0SrPZ3(yeI_OQHJt~N$38GiFMW+r7nCg2h;mZpF$S0!Fe}u78NK2gDE(A}& zkZ+;83_~(13qsofWF0xKIABCUA+~ns;@1!1S~p&Qw!%GA-7$EAmnX(>Aj2nr{ZEJF()2qc`#=48&n0$`l^Qw(P^NK;T&?-aA5H%tJ&TfcGpKz4q$Wt9yR@D8|DgS?Qql z+PFuI5%lNY*wUU_$(4h=yhOzyWIEI|qPD^NyKn#gRh}r@I2UjVin%Ysx7t<*#1Y#a z4kiowtKF}rhlDpuQsZTND>t>gFvbrq8;AmZ=WawZZ?s$FFkE5?RW>m9z384uZ}nQ` z4a>g_^S-0+779EH-j+l1;SaFW5T%%hYTjsGRaYnY03FL|;Fmws)9`_afcfAYHc&lB zmH4 zhWPk3vdqfid4ij5BT!MJ$w~;>d$^EaLU1QJ1`gK8I9rBcT^S(ZE~04|!snq4bcM6d z)4_SWHXiQi^#VqiIFXqaw+{ftY=}A+CKG>)+jt)DnbbW0GM|!Rs!vJjr zO|V}{=&`co zNGPAZY_rmC{4%D1s1(uygB^kI|ACJWO%JG7XxtK&y3+00Wwo`wG6BoEn~+80k+MUU zp=NUqYJ&!<$nDCpK7efVQm_0u1?op5=2U1l?}08Ez3rgBV+RGkT@{}N2*PlBQNW7q zjpaP4sQ9$h8Zw9{pwg=7D~TA2aElY}Y!Vo>>!e65{N@K}eNQoOH zMsnD)ZQCbMmsDt}i-;u8>iXx_JBus&pG!)-kh8|&frn^x(ah{9_OD`6O=&j(rnR*1}MF|O1Dqk!#sw^TRslwR;rUoTo6j0-Ee+|w}42E#U%SDRX4s1k{ugJP$ zipqcls|?o^(q~+F5y*Ff*_UU3Y=+4RI#8NYWpFbRwzRzbA&>*WQnuiTQ?!awes{x% zL+@8byP-SR_PjF_5Ks>c9Wp#2)7P8x>`k$Qubzmm-Mm?KR39V@p!a#TJily-7% zi0llM;OQ`T808g=^x>g4Kw5zDf|cb3rka!TwS_9?Z5hbIw*kfjK)j(130UwIau&|6 z{j?aJe#pu1pt8xC0R_6g_SaimJ@3akjxRopgA(rxS&)N?GZmHUEc}*gg{~qv>qkLr zH|}i$8@OxNF2J#Py(?fupLlx<&X2@&{l0eX8U{<^_}s84L;y#ngX;7NUpW`bHgSr1 z-%U@qy(bX{1wF>-Qnh7roVkI#`G7Rc>0w7L_b0Sp+(A!&2bRKSP)FdbV4zY+X!Z1u zF0HBY0rIeh+Y^4=ljs4BI`cB$$vTwL8BN|&N&JdXHDXaUGfTp0fjo8KT77*zMkBP% zl3L4-#}5adpi>%GF%aZplkx8Xx5i{Lg78h{ofjG5MQnitPv(8FeCyRF#^h5oJHtPkBg2rP*%$ve@24pB=)#Ldt5;hQI zPK}5}bS>V81~V!R4lOOhtqG868^F!4YhZBC+gtA5C_lXU@!%CjMkP6isMCOgbvu8sJD>St#M+QO(I)aLFsR|@2U}69?qhl^4Ld4JD`)IC*sw_;3{+tN zXS~XmlCrW1xV^Q35P>WYtERAkTqH{>Cfv}{V-U#QC?QPToMOUzW2d>&n;%ZL--Oxr zxi6Yrfl+;9&f5)OlQ;5Ds}s=aF3#CvVvCoL zui{<#DkY1YAxT4Bef?OyyuLm?!mgtJ|NZkv8=wq>ZiY_uZcFomNJ>u5&KzEc`4fZ( z(Wlh|p?Nz)tbv`u8`cx6sb81(_K$FW#U=*08z$fb0@})MH)AeEHY|>cIOlC__{fL@ znEyqf$)1z&j+NHTmHNJuwY1PIw-s3AxY;B z?(byI7rh{COFZzj28 z08cd0jU_iO2ssNRFCHI77BpCpxha9PFF0`j-aS4*Et$ZhUQ8s8@~%59A++W~y} zt7iWEp+sZ7kf_j3k#@q<-!){}X*`sBq8Deb;{9(NbK)2xwx+}@$FS(uR#qQiNrr;M zS^Y~Sf;61apuQEliNdGA1n&hq%5(SlB@Mt)8BcR#;|pmzC49+SYY{;Xx%cS!uZ-_d z38~(E!uYnZLPm_N>_=axmMbAGJ9sZtT%*%`gI*Y8pQSn0KX~~0Ph<81&5WMfC{`7U zI;UGEv^v3137kN5!UvF^L2}>F(~BvnFG4*4czdOk07#<^?1sZH>%3M(M8tThnU2Ft zCRT|fGYc^4a+lq{*xV!}r;2t}tlT@ZD_m>a1U`;+=I;bpi)}4#XiyqTjZ00f#nJln z_wNn#Z^ZKh@@dei!}$_)bP3qwVaz3f1vSWkbdy|Ro%V^ExqMLLXq7^*-<XJx|E954WfV z_kb)D{Pg+rT_g$kxZ;|oF_y_W*j!pl1L&LF_`*DLxG~;$JRkOBg85L05mia!tBoQT zI*0@S$@e8@4y9zGG|Vfe5TQoj-o9u^GBYxg#(+%beA`(Vb#e46JkOynkTHWa*>rO>@*(%s|rn8AMFPz&QY%8>)?Qqs-V;mItSLj&NQ9v2c%hWwG~Txk zi=sT=Y5wi)EzBp^XPIAxI#UMSnlxo#NlxJupt}DBk82V!0MJMb0Ym^oC!K5kg9e5N z#euPJX?Nh_$6;HqKiLCl^UjsB;~oJ>FfWj?I-~>Vxp6l5gH|h@v!Sk_eHw&;P((P< zkF&}mcEa=a8IJ`Fj9HjUv^D80&%!(HO*``h2L=W-3!I(-gBC7~iW2_~)*q07NE?ur z_|flTw)n*)_j*j?z&jZnTwuR17c(~~O0cxNbzgEpB**efrhmx7Dv>@5*oKf~C~^Tm zc7Y^DnI>LZG_7!)`!ktOXgWC^M;$0>Xdny*hu}`+m3G@7LqqHIBH0@d-)+b_uq$CI zay4XNAnKdbu70l5?N2M;dwy#C_Ax%p=%#NLy7XFujT<45aA`@jRjF(gY^xD(Fx3` z$fzhM?9G^KXGOAy9|+C-+wl(`VGG^uef5)F(>dv!gT~2PMUgP-q#HLL+P{DQao5?m zrHd$TJtHG%0xgd_kH<9Bo&eyLl$3Q~A4H(r6tEh}3fvhNf zWdHo*rqF5_h&UZ9O89C*>MNu|(&T~wPcB`oqA4*@-DC=1BY3iaTh9qCj%M*7=99E0 zF1^8*dk5taK@K;2)w?S~`G#631hc)5xrvh-g|h@3N~oZ(s#;s2-J^KtqD{Z$-?|w6 zpA1*k*W3Gi-aV_$u@_enjh6YL~ee!L_zrLWUZIBhwSg3rvu(el`03nP7_@D+~VCslbN|*(i zfazwJ!R382^R?TTgLXL4z1F{7ExOZnl^R>CPKfqt{~Oa0b6&IAfZ#Vkdk|{K`bX&v zW8;U=OYp-A;ZL^N=ksG+xmTWrXV$mg&+9>Zuh$y$>(tX zV7TI-<|xTe_&}`~FC06eFB;1n5Iz0j&GE1HpPNd9b6z_|Uia3XdHd>&xNF2<#x*~R z$|5ZrJ9|3pS1YX^Ga71GVxopX20zHfC2t&8W2}<5bgbgu-UPvo^s%=<#e_Ex18!L zMKc^Tx3sdu39^>|RCag9rF2Vu=8ef8A76boJ1zPleZ_RF=9OvQ%!;q1pWm7N^cU6! z>>n`Ya<|ZIu(U`xck!ztzhKOzM@ku5q1W$j61dmgbcN#*@0u5{nHyAe%HxM7t*_LS_Z00ETqqfEOyVaWKCy%V_v^v)mYq2 zqJ016(Py-ZgBx2GclCcNZNA6wDoneI=iRT&v0R(au6))88izuRbImj<>!Zo5n`gds zQ_u74tIdzx$@V^6y2Fw8{Or;7@*y>I!mxL2eYSt<3$;<4-1j26vJ%l@{lCIB5*;nq zAE6oh^5e%d+B(rJubWb9_l`I;wj}^%4f^39y8xN>27IFfvkYcSCZ6j zMMPXK98tkr_C5UjrLo?AfzvA9L)(Ab2!&tbQ2x>&iCiCD0Aer(M@D6w+1 ztih+-=6iOfS8DQ>{Ra+QP*7m!=Qnj9#yDOL&`ot)OiT=q>#S8nW8<-Df-$b&91$c_ zh8VDrtyE1?J85F_X2=#dKC~J_ayTR#tUd2D;ITBUwULSME~Jj_uUE{#)f)g`pnd;a zy33MqJ|DFSX%#p2#eE>eknw~aaXFD$2oB{qt3mlIMvhii(WvPH#tnBt3Q@E%*8}R9 znAag*kx}h0(v@0{CdLb9m*1$|0Pl*H@&PHEWm#z3Vo=0jRJFzoBpe_IfKf`76~s$O z#s&7hwc@UthJ6vcPS)K{Cc4Wz&Z}#=D_J1WPVM}UW&4EAH0A&eF|Ycy*<76iwO`9} z+y@3uf4!oHyn0APq_y`1+SVuM4j^--=PY|H4|)*)9ST*(EZq^fOUvzx;7w&ea%8Q+Dak6Qbyh+7VK-$jN+xdqb>9;orRY$Pz=s2~|~>KB9;j7ic) zqZ;Fq2u~~n1rz-j(auna#{sUQYHL?WV7zWYvjG3#k7BXaBGXar(-gA%2(+55qqu_P zE()(_!gaJS$f(5_LZ)D_3xH-9gReTy#~#)zA<<6t99y&f8=Ajez9nAull=quxAHGv zC_V5q(?lc9PL8w(VRRBhNj9C+so6~{72sLGRKtw<1$9w9*Cw_#d1)~6rVo4+%F@+^ zyRX+R?=!U%!p$F$`uf#cFv3DW5e3^{iRMgh7`MYbzyzRFO5=JmNdQ8bSj-X9(1g@k z*3_6U6!;iJi;U|T0!NN+!wL3Wd=Q)D(j+~FFp3jZ{Nxe|^cTFi@Wcml;&36!Ax>&+ zo_p-)?i|TpzV*@QZRzNEzzx5TyxEd_-uS*aeLvK@;WDHu3_m{g_gDETffiaG;Y)=Z zbhZ(|jO%Io(R@iYkIbOMl2qvgK>h)&-%&8vov?xC&Ml1ZKbs18`c%r*RgCm8 z|K@7h*h1E%eIm>|1S26g2W#tpx2`}Fdq~e7dQKHM@JV)dcEUCEQZ^Bdp~u9qM)om@Mf9um zEcjtxAg1GJ!*P*9)T`>H-ySRX}}?sw82DR0Ks zkxcLQd)pI7CbITJK>1dF#-^be6A2|jdUgugub??YMo3{M?rLZ9_ z+)g6gLl+ObH7!(ULv*Wr4ojcSL)#|l1clhEUwQvnaVORu^jkgnip62hJ8Zh;@oO3K zVvZvvd!(o=+O!O{hSZkrmO?ayclF%NZ>IJzRmziN#zR1r|88*+tVL6(+<@1nM4mCJ>jq1D$$Te}WaPjdzB=6d?m z?s?y~pvSo`|330T*%8m>uRHEP*0grYNsZW!A3iHnNE2JbO}^%eBZnu$R6w>u!7=l$I_k){k`0;0C#tZZSx=U1dqi0Rcm|}v`Wo&&70Rvq* zw8S3Ow6vroR@wyGz}E6gT~XS}3x1K2HT{*Fr}s9YpMa`j`>eShe~BCnjp3?~J@M4S z&j8!SLoh@kfkkrzVZ^TEzkS$EGyoB{*d0sJID?s$b5NBT1x>c28^yFW712S6jto6> zLsKofSRn3?xa@-1KY?}l&>i_rB@mV?K8aY}tBiiowEC zpwY>?ktA+eICl>-sXCUHwHORD#x)c8?8)I4Uwka^mNl&8AhbY6{s6g$3>%{B0fkT( z4urTZzj}K=LC7HEyGX(P_(C0dv5v2SH^01!6^ZthhIIxfRRA>)T1ANeR$8f__mRe z5i;0^g+g8F{q;)**O@>a|Dm-qzmt-Xkbpci&^D-b+6WZY_J5D}OMo`xDYm`5s%o`1 zJVm%>WR)W9HpDCh4n)>gk~lSyCD&(0VJ9a>^T!%cB>4FGTcNYzgV0#>;VLvfK0YiH zaSBmjvnLjWcUIp|%}Q0gGHfFZm5IuFP3%=*gGmX?;aRuGgR!cUkL7*sPi z7W0yWwBPH+)j#B)I+y?JE&gx#XUSoxJLqpITefVGQc+RiRh|5qo0EeJR)`SzIN$%& z!mvF(5&~^JMPn@{oc2Kvs7>pDLlB$^uB%H@Dnm=~6Nx&~90k&OI63>lqenWJP$H(j z|7BuMY??g`H=1Pq$ty@%D@WGUv8$asM}yI>+!RoUgSd956v`*zn{Z+Akr_sZkeBt5 z1u$0VgDZ>v0C7OLV^t0VF|A2WjwNpY(FGCC(84O_gh{)N7;e3x(F}_Wp7%WHQ#E)C z&>Y3R?ZZqsh+tf35`#Bof!QAyEGlc~b;G$xuKa;XJ;S^`5b0`AL?bK=(nPk^ zLX0#pu{7XkWWpEF*NkB~z2DIrtV=Z@(@=MjBq%n_V*0CIcV0sblZ^;Kv^T{-SyiM# zUfP5v7$1Q9&(7d06JUm*yxB9szJLGtaV5&J7;xk17*9!;1o8&_%#fU>cB-whQmRKhPCwxuKESX4fr-#`$9{maGpqOi9wRk zKyx(9SUsw@7})}Qd=|z53dUXraLL~R7%JelJ(g zg-l#$hrC}M2n(|hQc8$K%j-*bfVsmgHdCW|*r2BtTgXaFdx3QkImhjRTh=-{JD~yY zg*;PBaRHeL;9-Tlgg+8#Zf|Y9^hSjv-5$g5LQCO5Drg%HU&|8!cY(W&0uWE3)w}HN z>|${F8ZK`9oHT~5$CH8tP>3v3--1;!B-4~GF7Ncpx49!NYPMsxf*x7{+DC~RI7%7^ zQC$b}Yw=RFt8|TnSY3Rz1jQGtU@S{lZ(mGFgQJOzzJff&rBBcB>>)J>l&iSkjh5(s zZ~&XNXL+OQV&vua`}gl#xychA7Zv@{+j~kzMy9ZAFNr>7L?3APzO~P1?w`E#@dGRa zxaCOTYGnJDt3tpz8);7aTy4gUk(9W8s5mv5sd%*KAtB}bwPLyqU*3I9P^Lvk0K{*k zVPNcmk06o>h#Iu5ghEyi16b4O@HlCiZD?RHkdwL8QVZDe3c*MwJFvg+;C&<3lgDmM z8)>i{j_A(~v&4=atDc-#nz*q+BOk7z65EJw!ANi%7$C*;#I`+qlpr^5&bWq}17j~?mx#9} zP#9et>}8?nj!9x*NOK_vLs!uKNec%OikJl&Phdr0LO2C?NJ{q4u~c>dgR-AG7KlwR zT-XS1nlG{YIleJ`fyVP@JdCoN&*@*pIR(vyo5TFLix!`Qs<6h@Rd_CSH92YN9#Ge8 zpp$+(vHC-^#rKQ5REJ7Oymh3pe`S8X1ry`{Pj%=1m-F4n@v>r;X(go$bJ!T#9HUek z$|1>VDybYgYBX1rD_P3scwZ4|>Z;gHVIq~pI?_70E8#+#DGWKrghs0S`L^9Z?EVk- z!yX=UsIKq#^L@Wx@7MeFdVS-iuD~Q{27Lvjt^8$wYN{bdO;KRN5&07?vr)fKXFfaIWi-|OM5uUndUqC>34XIHpmQPCv79aK=?Fj27fh=2Z(G zP@23H;IOo8OrzbR5@#6Otp^l=LvI)fj|X*$zAMghR{UE@3+52V`H@ovr3O zMSN|EIPvuKl&l^RB-Q0RVrnE8)4KJ!2DDsuOVg+nwsO>{ST>+@&1ZXdX|#my%LG!Xjxv2zbEAb|4G7 z#OUnbo0}V`a&KNgOgC;3zkha(XaBKdcZb#~JF(RzjQ0$>F& z{<77(e?3);`mls*m*$!g1)X}#uevuJ$hmP(@`~vZCvku>yD`UQ6qR*$YcRL6N2U~tDwhq!uWLocqX1n6Su7}2W* zwP!g$T?O31mJ9y3afMeVT6SAUh|eq@-~U}vReE~5!40JS$b9bBuI9KBfmwtc-Do3Q zJ$}aJVPBGsu*t~gdXx!+fx+R~THXKl>&1yv`b8LY>o#Yvixb&3Q*H+62mwARZJPE! z?N=s8?mNUyiAQOFsnyV3pi%S2*=d9S1m{MfruL-_t(+2ciHc=Sq2qiF43yS`))}9X zXh;45H&X02mFh9uXj&$rZqIc1txp>b+2Re5C7Z*(3&+zCe3!Uh*m<_DuAn@^bYzb% zB1wY3%Mzf<(K&FyFQ#$gr2S(V`7h?wf8z*@SAuL2$9W@vMW$GAvSPLQ+Zm9X;WWV&gzH!r(6`4aC zJ0#D=t=s@q*COV18={$~XY@Wl#w>Zue0uLS_-&MRFV7LKpsGFSGm8RmU+1?%Ik%yF z$K(}t!Y{r5{^+%y(0HQ3d)CTvF;T5EjzLg$$Ew4FJ9q3+EfVD=VKiWz_$!t$lFa*} z-b4Th>`P(JOnrfRo^gFQE8nM3z%t@$v3M7GEdGln+z6lsy9@6X?X|%cd#Gf_LtO<^ zif{|#M0;y}`^SR^Z{adZ>`;fA7cRte;BAdYB^kc0`5W{zlkDv;NNO0;REhXg2Ng)R z)E|$ss$E&5@-6nUDERL^Fc+o3AxtgkO&Tks)gS1yRS z3|q$STXf=4+o{61qaI*Ho{6G`!x-a&%sN*+4D#~ z8r5?d!@*Dy_M$CK4b~^}xD*?gB$#oEWRN+0Gf&&8ZxctbU& zZXxQXM|JsXv7NnXs#ag|+aoLt)3W>vv1S9g7e~;2=}TffW~Uv17C?)ZfhIE+#19VZ zTJXm$JxYK4Z&%3|<%kZTrr|g1!J=$y3q{!v5BRC%P=lC(st*1JhhV{FeRFa-Em0%| zn2f|W0t|PcT4@nXy`QoxK`nfo9UwnRIsF=SEPwyojT@UTI(|N!+W5A2ZSOnrAc1!1 zH)MeTsv5~|#5xd#+W-pxAe*3KUN}%8?+BpsOVEMXWuQ-#cXqk#=83^9`vbO;S65!R zINQ81V#pC;0vg2dZ8e6h*w6zwabrc@&93Pt{g)xqQU)OsNJuZHLH+E z<%mki1+Q2&HvN6KQ}oJyM(by2_R0zjo2UDbysGQ6<8Z5)(}#Wjlo z1HJ`!X9U&TVs=63kPzU(6omKQ1Oi%45FdLE;>fL|CAdXG9Oc4uQ_m&axbJQ9f%B%- zi&#dcYrZe4h}uV1mydd=p^-F&5fM@A*URBd&wKWtcarvzsDA#(FNerTEO8a+t~yg1 z)Wsf1Z_P&e+i?IzjRT3ckB8Q{g?LbN3IpnhJ+vF#_}W-uJ0e}v(bfH(#hZd@hv|sO zm#nwgLBjDN(JGM)Y!YZ1v4p!U;EDm$zi=5UvEprw-B%j-ju*K*D0c{%c%_saU1EtMSVp{!V^3YFvlGfJBa4q(^iQ@B4e2E z(%0JhL)>n8`T6E2*8Zj%Lys2Wr1<@rQ+CQOuDWUI)a5mwf#!|cDAHuzhD4?}fbyhQ zfObwbhSF71wU}I^ng7^IrZCdj&C|LL_KJSYW=7P$rlzKBpuZLf{uAdLhH4T{N8}-; zn3`B<%>T7;+ctgiweuxEvxUjrCTor}U{u*3;L8Q%H zdre!qbZO+d6xbEJU^~4-EaO%_sq4i|2DKqX+R2soXGe$N2ZskL8)fybTaG(BGT3JMNDc+2q^uhs}p?DB&jIV?}+Yzqjfm4#b~x;Fgkv>ZEaguV2B0 zSvnkG`LQ84cTK@69|_@Q{PxY6r>DqkW`^mrDD#ROYsiF2xi9P2ZXjHm9VHCvmXj&8xdEpkV(1qe08aLVU(;DFO62Jw~M# z$-evmET5*b4E<|b zw#AI$A_jt6L_>j&Hp&C?&!zyEsl2jSu2iVt&Bo?giGaTE=#IKDWj zQym>ER<-$Yh`2DHp7(cAC=}7nBj$&F@SrWQlVx$>EU)TE*Pe=@h|7{Vc0~uKDX1`^ zf{v~=Q#oW6xww*3c@rUqALp5^kA8W3coGvIoKd`o8W6nu+)h@>;uAy%>|aRv5a>-rc~AD1xso7Vxto5GGprd_M3Z8uPcg(cVw_vDm$D(`fsT}{R! zBGee22hLDPO1@*j)nk1nGZ8Ht7sbTwy7KZa^N}NWGGyLhK?TT|?!2gBRi6DKdZIK{ zAe|VpMH-7ucIXL|z=L@*{|&l=>iGSxI_Pd~Gco}o2$c6zz8?e}r68Q>F$MFp3kxsb zgr10R{jPkL6WGaZpUAn%AC98@jDPwj#I8 z8zOvjF#}OnQV^wrLO-c)idFMojydqguo@Hjj80{V^{*(=J+Y1CwKV3++kdDuf=cGN z&Tw+N1JRy3@GKfZNAP-|gD0qki>Re|f|JwbvUajM%hIZN%_K6;BQ3L{xwK;U4i%?`K*n!ku^K@^yYmcgvyasygr zI_Dd|!d>VFO_XG{YB5_y`+1?Bt6DGf2+7b>ZQay`y;f!O>a7!qIA-b={7VzUqIYmX z(2#U;V6eg+TYlY&R)ec|q}pY5AvbFTFZTdKY-am|A<#F+DNBwu=zBLRX&JgOs?|>U z(zB)!=0fh7fWK5_n!dv3;iW^x^ks$32IebM2$C!d!3B(~%8uu0{~|9KQy(t;dSYO1 z9-$LzJTAQ5DVn;+!P0&<-iwB}m{xR&WHbtmtVU>N!_8FSo*gkUr+Lj!j}hMQBIrNB z`A_r6eHrhR|mCi5F{RB!PTQ!lqfH)r2SGRg_QZf+YdWPLkS>Uze<$g0r0nQ>X**J94k=>}VnRa(&l%P)D z$@1oL@j&nVi$g3eUj>=Sx>I}Z?adMrOhl#YuRcbW=#@yIn8b62icH=~gc#^q)tGvt z&;Ur_ckhj!jwAvQgYIN>{?hxmzbvQf$Ry^zkG zU8(2!kD^%~M0W7UMMd+7n41k+-BjMLZNZIjScgcJELFAN67Vb Controllers are the core of Kubernetes, and of any operator. +> +> It’s a controller’s job to ensure that, for any given object, the actual state of the world (both the cluster state, and potentially external state like running containers for Kubelet or loadbalancers for a cloud provider) matches the desired state in the object. +> Each controller focuses on one root Kind, but may interact with other Kinds. +> +> We call this process reconciling. + +[controller]: https://book.kubebuilder.io/cronjob-tutorial/controller-overview.html#whats-in-a-controller + +Right now, we can create objects in our API but we won't do anything about it. Let's fix that. + +# Let's see the Code + +Kubebuilder has created our first controller in `controllers/mailguncluster_controller.go`. Let's take a look at what got generated: + +```go +// MailgunClusterReconciler reconciles a MailgunCluster object +type MailgunClusterReconciler struct { + client.Client + Log logr.Logger +} + +// +kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=mailgunclusters,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=mailgunclusters/status,verbs=get;update;patch + +func (r *MailgunClusterReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) { + _ = context.Background() + _ = r.Log.WithValues("mailguncluster", req.NamespacedName) + + // your logic here + + return ctrl.Result{}, nil +} +``` + +## RBAC Roles + +Those `// +kubebuilder...` lines tell kubebuilder to generate [RBAC] roles so the manager we're writing can access its own managed resources. +We also need to add roles that will let it retrieve (but not modify) Cluster API objects. +So we'll add another annotation for that: + +``` +// +kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=mailgunclusters,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=mailgunclusters/status,verbs=get;update;patch +// +kubebuilder:rbac:groups=cluster.x-k8s.io,resources=clusters;clusters/status,verbs=get;list;watch +``` + +Make sure to add the annotation to both `MailgunClusterReconciler` and `MailgunMachineReconciler`. + +Regenerate the RBAC roles after you are done: + +``` +make manifests +``` + + +[RBAC]: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole + +## State + +Let's focus on that `struct` first. +First, a word of warning: no guarantees are made about parallel access, both on one machine or multiple machines. +That means you should not store any important state in memory: if you need it, write it into a Kubernetes object and store it. + +We're going to be sending mail, so let's add a few extra fields: + +```go +// MailgunClusterReconciler reconciles a MailgunCluster object +type MailgunClusterReconciler struct { + client.Client + Log logr.Logger + Mailgun mailgun.Mailgun + Recipient string +} +``` + +## Reconciliation + +Now it's time for our Reconcile function. +Reconcile is only passed a name, not an object, so let's retrieve ours. + +Here's a naive example: + +``` +func (r *MailgunClusterReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) { + ctx := context.Background() + _ = r.Log.WithValues("mailguncluster", req.NamespacedName) + + var cluster infrastructurev1alpha3.MailgunCluster + if err := r.Get(ctx, req.NamespacedName, &cluster); err != nil { + return ctrl.Result{}, err + } + + return ctrl.Result{}, nil +} +``` + +By returning an error, we request that our controller will get `Reconcile()` called again. +That may not always be what we want - what if the object's been deleted? So let's check that: + +``` +var cluster infrastructurev1alpha3.MailgunCluster +if err := r.Get(ctx, req.NamespacedName, &cluster); err != nil { + // import apierrors "k8s.io/apimachinery/pkg/api/errors" + if apierrors.IsNotFound(err) { + return ctrl.Result{}, nil + } + return ctrl.Result{}, err +} +``` + +Now, if this were any old `kubebuilder` project we'd be done, but in our case we have one more object to retrieve. +Cluster API splits a cluster into two objects: the [`Cluster` defined by Cluster API itself][cluster]. +We'll want to retrieve that as well. +Luckily, cluster API [provides a helper for us][getowner]. + +```go +cluster, err := util.GetOwnerCluster(ctx, r.Client, &mg) +if err != nil { + return ctrl.Result{}, err + +} +``` + +### client-go versions +At the time this document was written, `kubebuilder` pulls `client-go` version `1.14.1` into `go.mod` (it looks like `k8s.io/client-go v11.0.1-0.20190409021438-1a26190bd76a+incompatible`). + +If you encounter an error when compiling like: + +``` +../pkg/mod/k8s.io/client-go@v11.0.1-0.20190409021438-1a26190bd76a+incompatible/rest/request.go:598:31: not enough arguments in call to watch.NewStreamWatcher + have (*versioned.Decoder) + want (watch.Decoder, watch.Reporter)` +``` + +You may need to bump `client-go`. At time of writing, that means `1.15`, which looks like: `k8s.io/client-go v11.0.1-0.20190409021438-1a26190bd76a+incompatible`. +## The fun part + +_More Documentation: [The Kubebuilder Book][book] has some excellent documentation on many things, including [how to write good controllers!][implement]_ + +[book]: https://book.kubebuilder.io/ +[implement]: https://book.kubebuilder.io/cronjob-tutorial/controller-implementation.html + +Now that we have our objects, it's time to do something with them! +This is where your provider really comes into it's own. +In our case, let's try sending some mail: + +```go +subject := fmt.Sprintf("[%s] New Cluster %s requested", mgCluster.Spec.Priority, cluster.Name) +body := fmt.Sprint("Hello! One cluster please.\n\n%s\n", mgCluster.Spec.Request) + +msg := mailgun.NewMessage(mgCluster.Spec.Requester, subject, body, r.Recipient) +_, _, err = r.Mailgun.Send(msg) +if err != nil { + return ctrl.Result{}, err +} +``` + +## Idempotency + +But wait, this isn't quite right. +`Reconcile()` gets called periodically for updates, and any time any updates are made. +That would mean we're potentially sending an email every few minutes! +This is an important thing about controllers: they need to be [*idempotent*][idempotent]. + +So in our case, we'll store the result of sending a message, and then check to see if we've sent one before. + +``` +if mgCluster.Status.MessageID != nil { + // We already sent a message, so skip reconcilation + return ctrl.Result{}, nil +} + +subject := fmt.Sprintf("[%s] New Cluster %s requested", mgCluster.Spec.Priority, cluster.Name) +body := fmt.Sprintf("Hello! One cluster please.\n\n%s\n", mgCluster.Spec.Request) + +msg := mailgun.NewMessage(mgCluster.Spec.Requester, subject, body, r.Recipient) +_, msgID, err := r.Mailgun.Send(msg) +if err != nil { + return ctrl.Result{}, err +} + +// patch from sigs.k8s.io/cluster-api/util/patch +helper, err := patch.NewHelper(&mgCluster, r.Client) +if err != nil { + return ctrl.Result{}, err +} +mgCluster.Status.MessageID = &msgID +if err := helper.Patch(ctx, &mgCluster); err != nil { + return ctrl.Result{}, errors.Wrapf(err, "couldn't patch cluster %q", mgCluster.Name) +} + +return ctrl.Result{}, nil +``` + +[cluster]: https://godoc.org/sigs.k8s.io/cluster-api/api/v1alpha3#Cluster +[getowner]: https://godoc.org/sigs.k8s.io/cluster-api/util#GetOwnerMachine +[idempotent]: https://stackoverflow.com/questions/1077412/what-is-an-idempotent-operation + +#### A note about the status + +Usually, the `Status` field should only be fields that can be _computed from existing state_. +Things like whether a machine is running can be retrieved from an API, and cluster status can be queried by a healthcheck. +The message ID is ephemeral, so it should properly go in the `Spec` part of the object. +Anything that can't be recreated, either with some sort of deterministic generation method or by querying/observing actual state, needs to be in Spec. +This is to support proper disaster recovery of resources. +If you have a backup of your cluster and you want to restore it, Kubernetes doesn't let you restore both spec & status together. + +We use the MessageID as a `Status` here to illustrate how one might issue status updates in a real application. + + +## Update `main.go` with your new fields + +If you added fields to your reconciler, you'll need to update `main.go`. + +Right now, it probably looks like this: + +```go +if err = (&controllers.MailgunClusterReconciler{ + Client: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("MailgunCluster"), +}).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "MailgunCluster") + os.Exit(1) +} +``` + +Let's add our configuration. +We're going to use environment variables for this: + +```go +domain := os.Getenv("MAILGUN_DOMAIN") +if domain == "" { + setupLog.Info("missing required env MAILGUN_DOMAIN") + os.Exit(1) +} + +apiKey := os.Getenv("MAILGUN_API_KEY") +if apiKey == "" { + setupLog.Info("missing required env MAILGUN_API_KEY") + os.Exit(1) +} + +recipient := os.Getenv("MAIL_RECIPIENT") +if recipient == "" { + setupLog.Info("missing required env MAIL_RECIPIENT") + os.Exit(1) +} + +mg := mailgun.NewMailgun(domain, apiKey) + +if err = (&controllers.MailgunClusterReconciler{ + Client: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("MailgunCluster"), + Mailgun: mg, + Recipient: recipient, +}).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "MailgunCluster") + os.Exit(1) +} +``` + +If you have some other state, you'll want to initialize it here! + diff --git a/docs/book/src/providers/implementers-guide/create_api.md b/docs/book/src/providers/implementers-guide/create_api.md new file mode 100644 index 000000000000..37b9cc4276ea --- /dev/null +++ b/docs/book/src/providers/implementers-guide/create_api.md @@ -0,0 +1,65 @@ +# Defining your API + +The API generated by Kubebuilder is just a shell, your actual API will likely have more fields defined on it. + +Kubernetes has a lot of conventions and requirements around API design. +The [Kubebuilder docs][apidesign] have some helpful hints on how to design your types. + +[apidesign]: https://book.kubebuilder.io/cronjob-tutorial/api-design.html#designing-an-api + + +Let's take a look at what was generated for us: + +```go +// MailgunClusterSpec defines the desired state of MailgunCluster +type MailgunClusterSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file +} + +// MailgunClusterStatus defines the observed state of MailgunCluster +type MailgunClusterStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file +} +``` + +Our API is based on Mailgun, so we're going to have some email based fields: + +```go +type Priority string + +const ( + // PriorityUrgent means do this right away + PriorityUrgent = Priority("Urgent") + + // PriorityUrgent means do this immediately + PriorityExtremelyUrgent = Priority("ExtremelyUrgent") + + // PriorityBusinessCritical means you absolutely need to do this now + PriorityBusinessCritical = Priority("BusinessCritical") +) + +// MailgunClusterSpec defines the desired state of MailgunCluster +type MailgunClusterSpec struct { + // Priority is how quickly you need this cluster + Priority Priority `json:"priority"` + // Request is where you ask extra nicely + Request string `json:"request"` + // Requester is the email of the person sending the request + Requester string +} + +// MailgunClusterStatus defines the observed state of MailgunCluster +type MailgunClusterStatus struct { + // MessageID is set to the message ID from Mailgun when our message has been sent + MessageID *string `json:"response"` +} +``` + +As the deleted comments request, run `make manager manifests` to regenerate some of the generated data files afterwards. + +```bash +git add . +git commit -m "Added cluster types" +``` diff --git a/docs/book/src/providers/implementers-guide/generate_crds.md b/docs/book/src/providers/implementers-guide/generate_crds.md new file mode 100644 index 000000000000..85a22e65b050 --- /dev/null +++ b/docs/book/src/providers/implementers-guide/generate_crds.md @@ -0,0 +1,87 @@ +### Create a repository + +```bash +mkdir cluster-api-provider-mailgun +cd src/sigs.k8s.io/cluster-api-provider-mailgun +git init +``` + +You'll then need to set up [go modules][gomod] + +```bash +$ go mod init github.com/liztio/cluster-api-provider-mailgun +go: creating new go.mod: module github.com/liztio/cluster-api-provider-mailgun +``` + +### Generate scaffolding + +```bash +kubebuilder init --domain cluster.x-k8s.io +``` + +`kubebuilder init` will create the basic repository layout, including a simple containerized manager. +It will also initialize the external go libraries that will be required to build your project. + +Commit your changes so far: + +```bash +git commit -m "Generate scaffolding." +``` +### Generate provider resources for Clusters and Machines + +Here you will be asked if you want to generate resources and controllers. +You'll want both of them: + +```bash +kubebuilder create api --group infrastructure --version v1alpha3 --kind MailgunCluster +kubebuilder create api --group infrastructure --version v1alpha3 --kind MailgunMachine +``` + +``` +Create Resource under pkg/apis [y/n]? +y +Create Controller under pkg/controller [y/n]? +y +``` + + +### Add Status subresource + +The [status subresource][status] lets Spec and Status requests for custom resources be addressed seperately so requests don't conflict with each other. +It also lets you split RBAC rules between Spec and Status. +It's stable in Kubernetes as of [v1.16][rbac], but you will have to [manually enable it in Kubebuilder][kbstatus]. + +Add the `subresource:status` annotation to your `cluster_types.go` `machine_types.go` + +```go +// +kubebuilder:subresource:status +// +kubebuilder:object:root=true + +// MailgunCluster is the Schema for the mailgunclusters API +type MailgunCluster struct { +``` + +```go +// +kubebuilder:subresource:status +// +kubebuilder:object:root=true + +// MailgunMachine is the Schema for the mailgunmachines API +type MailgunMachine struct { +``` + +And regenerate the CRDs: +```shell +make manifests +``` + +[status]: https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#status-subresource +[rbac]: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/#customresourcesubresources-v1beta1-apiextensions-k8s-io +[kbstatus]: https://book.kubebuilder.io/reference/generating-crd.html?highlight=status#status + + +### Commit your changes + +```bash +git add . +git commit -m "Generate Cluster and Machine resources." +``` diff --git a/docs/book/src/providers/implementers-guide/naming.md b/docs/book/src/providers/implementers-guide/naming.md new file mode 100644 index 000000000000..dbf513e76b06 --- /dev/null +++ b/docs/book/src/providers/implementers-guide/naming.md @@ -0,0 +1,54 @@ +# Repository Naming + +The naming convention for new Cluster API [_provider_ repositories][repo-naming] +is generally of the form `cluster-api-provider-${env}`, where `${env}` is a, +possibly short, name for the _environment_ in question. For example +`cluster-api-provider-gcp` is an implementation for the Google Cloud Platform, +and `cluster-api-provider-aws` is one for Amazon Web Services. Note that an +environment may refer to a cloud, bare metal, virtual machines, or any other +infrastructure hosting Kubernetes. Finally, a single environment may include +more than one [_variant_][variant-naming]. So for example, +`cluster-api-provider-aws` may include both an implementation based on EC2 as +well as one based on their hosted EKS solution. + + +## A note on Acronyms + +Because these names end up being so long, developers of Cluster API frequently refer to providers by acronyms. +Cluster API itself becomes [CAPI], pronounced "Cappy." +cluster-api-provider-aws is [CAPA], pronounced "KappA." +cluster-api-provider-gcp is [CAPG], pronounced "Cap Gee," [and so on][letterc]. + +[CAPI]: https://cluster-api.sigs.k8s.io/reference/glossary.html#capi +[CAPA]: https://cluster-api.sigs.k8s.io/reference/glossary.html#capa +[CAPG]: https://cluster-api.sigs.k8s.io/reference/glossary.html#capg +[letterc]: https://cluster-api.sigs.k8s.io/reference/glossary.html#c + +# Resource Naming + +For the purposes of this guide we will create a provider for a +service named **mailgun**. Therefore the name of the repository will be +`cluster-api-provider-mailgun`. + +Every Kubernetes resource has a *Group*, *Version* and *Kind* that uniquely +identifies it. + +* The resource *Group* is similar to package in a language. + It disambiguates different APIs that may happen to have identically named *Kind*s. + Groups often contain a domain name, such as k8s.io. + The domain for Cluster API resources is `cluster.x-k8s.io`, and infrastructure providers generally use `infrastructure.cluster.x-k8s.io`. +* The resource *Version* defines the stability of the API and its backward compatibility guarantees. + Examples include v1alpha1, v1beta1, v1, etc. and are governed by the Kubernetes API Deprecation Policy [^1]. + Your provider should expect to abide by the same policies. +* The resource *Kind* is the name of the objects we'll be creating and modifying. + In this case it's `MailgunMachine` and `MailgunCluster`. + +For example, our cluster object will be: +```yaml +apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3 +kind: MailgunCluster +``` +[repo-naming]: https://github.com/kubernetes-sigs/cluster-api/issues/383 +[variant-naming]: https://github.com/kubernetes-sigs/cluster-api/issues/480 + +[^1]: https://kubernetes.io/docs/reference/using-api/deprecation-policy/ diff --git a/docs/book/src/providers/implementers-guide/overview.md b/docs/book/src/providers/implementers-guide/overview.md new file mode 100644 index 000000000000..8d489ed7f660 --- /dev/null +++ b/docs/book/src/providers/implementers-guide/overview.md @@ -0,0 +1,70 @@ +# Overview + +In order to demonstrate how to develop a new Cluster API provider we will use +`kubebuilder` to create an example provider. For more information on `kubebuilder` +and CRDs in general we highly recommend reading the [Kubebuilder Book][kubebuilder-book]. +Much of the information here was adapted directly from it. + +This is an _infrastructure_ provider - tasked with managing provider-specific resources for clusters and machines. +There are also [bootstrap providers][bootstrap], which turn machines into Kubernetes nodes. + +[bootstrap]: https://cluster-api.sigs.k8s.io/reference/providers.html?highlight=bootstrap#bootstrap + +## Prerequisites + +- Install [`kubectl`][kubectl-install] +- Install [`kustomize`][install-kustomize] +- Install [`kubebuilder`][install-kubebuilder] + +### tl;dr + +{{#tabs name:"kubectl and kustomize" tabs:"MacOS,Linux"}} +{{#tab MacOS}} + +```bash +# Install kubectl +brew install kubernetes-cli + +# Install kustomize +brew install kustomize +``` +{{#/tab }} +{{#tab Linux}} + +```bash +# Install kubectl +KUBECTL_VERSION=$(curl -sf https://storage.googleapis.com/kubernetes-release/release/stable.txt) +curl -fLO https://storage.googleapis.com/kubernetes-release/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl + +# Install kustomize +OS_TYPE=linux +curl -sf https://api.github.com/repos/kubernetes-sigs/kustomize/releases/latest |\ + grep browser_download |\ + grep ${OS_TYPE} |\ + cut -d '"' -f 4 |\ + xargs curl -f -O -L +mv kustomize_*_${OS_TYPE}_amd64 /usr/local/bin/kustomize +chmod u+x /usr/local/bin/kustomize +``` + +{{#/tab }} +{{#/tabs }} + +``` +# Install Kubebuilder +os=$(go env GOOS) +arch=$(go env GOARCH) + +# download kubebuilder and extract it to tmp +curl -sL https://go.kubebuilder.io/dl/2.1.0/${os}/${arch} | tar -xz -C /tmp/ + +# move to a long-term location and put it on your path +# (you'll need to set the KUBEBUILDER_ASSETS env var if you put it somewhere else) +sudo mv /tmp/kubebuilder_2.1.0_${os}_${arch} /usr/local/kubebuilder +export PATH=$PATH:/usr/local/kubebuilder/bin +``` + +[kubebuilder-book]: https://book.kubebuilder.io/ +[kubectl-install]: http://kubernetes.io/docs/user-guide/prereqs/ +[install-kustomize]: https://github.com/kubernetes-sigs/kustomize/blob/master/docs/INSTALL.md +[install-kubebuilder]: https://book.kubebuilder.io/quick-start.html#installation