Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fallback to fetch limits from EC2 API #782

Merged
merged 1 commit into from
Dec 31, 2019

Conversation

mogren
Copy link
Contributor

@mogren mogren commented Dec 31, 2019

Description of changes:

  • For unknown instance types, try calling the EC2 API
  • Note: This requires the nodes to allow calls to DescribeInstanceTypes or the fallback won't work. We need to update the managed CNI policy.

I deleted m5.xlarge from the generated limits file in order to test this. The following error will be seen:

2019-12-31T02:50:53.787Z [ERROR] 	%!(EXTRA *awserr.requestError=UnauthorizedOperation: You are not authorized to perform this operation.
	status code: 403, request id: 2e2986ac-985d-4de3-8e83-4b8c25ddd63c)
2019-12-31T02:50:53.787Z [ERROR] 	Failed to get ENI limit
2019-12-31T02:50:53.787Z [ERROR] 	Initialization failure: Failed calling DescribeInstanceTypes for `m5.xlarge`: UnauthorizedOperation: You are not authorized to perform this operation.
	status code: 403, request id: 2e2986ac-985d-4de3-8e83-4b8c25ddd63c

After adding "ec2:DescribeInstanceTypes" permission it starts up correctly, even with missing instance data:

2019-12-31T02:56:56.098Z [INFO] 	Starting L-IPAMD v1.6.0-fallback-dirty  ...
2019-12-31T02:56:56.122Z [INFO] 	Testing communication with server
2019-12-31T02:56:56.123Z [INFO] 	Running with Kubernetes cluster version: v1.14+. git version: v1.14.9-eks-c0eccc. git tree state: clean. commit: c0eccca51d7500bb03b2f163dd8d534ffeb2f7a2. platform: linux/amd64
2019-12-31T02:56:56.123Z [INFO] 	Communication with server successful
2019-12-31T02:56:56.123Z [INFO] 	Starting Pod controller
2019-12-31T02:56:56.123Z [INFO] 	Waiting for controller cache sync
2019-12-31T02:56:56.124Z [DEBUG] 	Discovered region: us-west-2
2019-12-31T02:56:56.125Z [DEBUG] 	Found availability zone: us-west-2d
2019-12-31T02:56:56.125Z [DEBUG] 	Discovered the instance primary ip address: 10.10.20.28
2019-12-31T02:56:56.126Z [DEBUG] 	Found instance-id: i-0d51a749cc2eeefb0
2019-12-31T02:56:56.126Z [DEBUG] 	Found instance-type: m5.xlarge
2019-12-31T02:56:56.127Z [DEBUG] 	Found primary interface's MAC address: 0e:c8:41:0f:8d:64
2019-12-31T02:56:56.127Z [DEBUG] 	Discovered 2 interfaces.
2019-12-31T02:56:56.128Z [DEBUG] 	Found device-number: 0
2019-12-31T02:56:56.128Z [DEBUG] 	Found account ID: 973117571331
2019-12-31T02:56:56.129Z [DEBUG] 	Found eni: eni-0ebb53b6e6774b4a7
2019-12-31T02:56:56.129Z [DEBUG] 	Found ENI eni-0ebb53b6e6774b4a7 is a primary ENI
2019-12-31T02:56:56.129Z [DEBUG] 	Found security-group id: sg-08f1036b507b30a9f
2019-12-31T02:56:56.129Z [DEBUG] 	Found security-group id: sg-0aa43d9d01804ccbb
2019-12-31T02:56:56.129Z [DEBUG] 	Found subnet-id: subnet-02d0c4ea9bd25d02f
2019-12-31T02:56:56.130Z [DEBUG] 	Found vpc-ipv4-cidr-block: 10.10.0.0/16
2019-12-31T02:56:56.130Z [DEBUG] 	Found VPC CIDR: 10.10.0.0/16
2019-12-31T02:56:56.130Z [DEBUG] 	Found VPC CIDR: 100.66.0.0/16
2019-12-31T02:56:56.130Z [DEBUG] 	Start node init

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@mogren mogren force-pushed the add-instance-type-limit-fallback branch from abfe898 to 4f73d60 Compare December 31, 2019 04:06
Copy link
Contributor

@jaypipes jaypipes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice little patch, Claes, thanks! :)

@jaypipes jaypipes merged commit 3b0f876 into aws:master Dec 31, 2019
mogren pushed a commit to mogren/amazon-vpc-cni-k8s that referenced this pull request Jan 31, 2020
jaypipes pushed a commit that referenced this pull request Jan 31, 2020
@mogren mogren deleted the add-instance-type-limit-fallback branch June 16, 2020 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants