Any luck with manually creating EC2 Spot Fleet ?

[zytek]

I'd like to ask if anyone had luck with overcoming #1784 and manually creating Spot Fleet using userdata from regular kops-created AutoScaling groups. From a quick glance I don't see why this would not work and I am willing to try to do this later this week. It seems that there is nothing instance-specific in userdata apart from NodeLabels. Correct me if I'm wrong. ;-)

Until #1784 is implemented I could live with a script/terraform config that would create Spot Fleets from regular ASGs to ease capacity planning.

[zytek]

I will answer myself: I have successfully created Spot Fleet by manually supplying userdata and tags configuration. This method is klunky and less flexible, but with some terraforming should do the job.

The main problem I wish to fix by using this is assuring capacity of our small clusters consisting of only spot instances.

[rverma-nikiai]

@zytek Would you mind sharing your steps, possibly in a gist to help the community. If there are any guide you followed please share.
We are also running a complete spot cluster and sometimes the required instance types are are not there in instance pools. With spot fleet (or ec2 fleet) we want to address this.

Thanks

[zytek]

@rverma-nikiai here it is - terraform manifest that creates spot fleet: https://gist.github.com/zytek/d1e49e97c0ff1dcb965ab712351b035c

FYI this could be cleaner with terraform 0.12 when they fix tags with dots in names, so copying tags from an instance to launch configuration would be easier without so much copy-pasting :)

[rverma-nikiai]

@zytek Thanks for this, wondering if you got this working with cluster-autoscaler. Linking kubernetes/autoscaler#838 may be relevant

[zytek]

@rverma-nikiai no - for cluster-autoscaler I have provisioned separate ASG through kops and only those are autoscaled - and I use spot fleets for "static" portion of the cluster. I plan to experiment with EC2 Fleets and ASG (as they are supported now) in the near future - it will probably end up in similar hack - creating Fleet ASG via terraform based on "normal" ASG from kops, at least until kops gains support for launch templates and fleets.

[rverma-nikiai]

@zytek Thanks for insight.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[wanghanlin]

I think this should be resolved now by #6277
/close

[k8s-ci-robot]

@wanghanlin: Closing this issue.

In response to this:

I think this should be resolved now by #6277
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.