Support DISABLE_NETWORK_RESOURCE_PROVISIONING

[jayanthvn]

What type of PR is this?
feature

Which issue does this PR fix:
Support DISABLE_NETWORK_RESOURCE_PROVISIONING

What does this PR do / Why do we need it:
Certain users doesn't want to attach AmazonEKS_CNI_Policy and make IPAMD behave like in a readonly mode. This toggle was added but never supported. This PR will avoid IPAMD to attach ENIs, IPs/prefixes, tag ENIs and refresh SGs if DISABLE_NETWORK_RESOURCE_PROVISIONING set to true.

If an issue # is not available please add repro steps and logs from IPAMD/CNI showing the issue:
n/a

Testing done on this change:

Yes

Startup ->

kgpsys
NAME                       READY   STATUS              RESTARTS   AGE   IP               NODE                                           NOMINATED NODE   READINESS GATES
aws-node-hcdfq             1/1     Running             0          17m   192.168.66.135   ip-192-168-66-135.us-west-2.compute.internal   <none>           <none>
coredns-86d9946576-4kntd   0/1     ContainerCreating   0          20m   <none>           ip-192-168-66-135.us-west-2.compute.internal   <none>           <none>
coredns-86d9946576-pft88   0/1     ContainerCreating   0          20m   <none>           ip-192-168-66-135.us-west-2.compute.internal   <none>           <none>
kube-proxy-x4xfk           1/1     Running             0          17m   192.168.66.135   ip-192-168-66-135.us-west-2.compute.internal   <none>           <none>

Manually Added an ENI ->

{"level":"debug","ts":"2021-08-26T20:33:27.018Z","caller":"ipamd/ipamd.go:561","msg":"Reconcile and add a new ENI {eni-0481c298cbb5bfc6d 0e:2e:f4:9d:91:61 %!s(int=1) 192.168.64.0/19 [{\n  Primary: true,\n  PrivateIpAddress: \"192.168.80.254\"\n}] []}"}
{"level":"debug","ts":"2021-08-26T20:33:27.018Z","caller":"ipamd/ipamd.go:885","msg":"DataStore Add an ENI eni-0481c298cbb5bfc6d"}

dev-dsk-varavaj-2b-fdf1da64 % kgpsys
NAME                       READY   STATUS              RESTARTS   AGE    IP              NODE                                          NOMINATED NODE   READINESS GATES
aws-node-xr7bh             1/1     Running             0          10m    192.168.90.70   ip-192-168-90-70.us-west-2.compute.internal   <none>           <none>
coredns-86d9946576-kzvkp   0/1     ContainerCreating   0          2d1h   <none>          ip-192-168-90-70.us-west-2.compute.internal   <none>           <none>
coredns-86d9946576-mbwzc   0/1     ContainerCreating   0          2d1h   <none>          ip-192-168-90-70.us-west-2.compute.internal   <none>           <none>
kube-proxy-h4brm           1/1     Running             0          10m    192.168.90.70   ip-192-168-90-70.us-west-2.compute.internal   <none>           <none>

Added 2 IPs to the ENIs and coreDNS pods came up

kgpsys
NAME                       READY   STATUS    RESTARTS   AGE    IP               NODE                                          NOMINATED NODE   READINESS GATES
aws-node-mt6bx             1/1     Running   0          24s    192.168.90.70    ip-192-168-90-70.us-west-2.compute.internal   <none>           <none>
coredns-86d9946576-fqw8c   1/1     Running   0          27m    192.168.81.10    ip-192-168-90-70.us-west-2.compute.internal   <none>           <none>
coredns-86d9946576-mbwzc   1/1     Running   0          2d1h   192.168.80.255   ip-192-168-90-70.us-west-2.compute.internal   <none>           <none>
kube-proxy-h4brm           1/1     Running   0          42m    192.168.90.70    ip-192-168-90-70.us-west-2.compute.internal   <none>           <none>

Automation added to e2e:

No

Will this break upgrades or downgrades. Has updating a running cluster been tested?:
No

Does this change require updates to the CNI daemonset config files to work?:

No

Does this PR introduce any user-facing change?:

No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[M00nF1sh]

/lgtm