You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Attempt to provision node using CAPA control plane
Node becomes stuck in Provisioning state, cloud-init init phase fails with the following error while executing user-data:
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/cloudinit/cmd/main.py", line 653, in status_wrapper
ret = functor(name, args)
File "/usr/lib/python3.9/site-packages/cloudinit/cmd/main.py", line 377, in main_init
init.update()
File "/usr/lib/python3.9/site-packages/cloudinit/stages.py", line 363, in update
self._store_userdata()
File "/usr/lib/python3.9/site-packages/cloudinit/stages.py", line 390, in _store_userdata
processed_ud = self.datasource.get_userdata()
File "/usr/lib/python3.9/site-packages/cloudinit/sources/__init__.py", line 385, in get_userdata
self.userdata = self.ud_proc.process(self.get_userdata_raw())
File "/usr/lib/python3.9/site-packages/cloudinit/user_data.py", line 90, in process
self._process_msg(convert_string(blob), accumulating_msg)
File "/usr/lib/python3.9/site-packages/cloudinit/user_data.py", line 160, in _process_msg
self._do_include(payload, append_msg)
File "/usr/lib/python3.9/site-packages/cloudinit/user_data.py", line 258, in _do_include
_handle_error(message, urle)
File "/usr/lib/python3.9/site-packages/cloudinit/user_data.py", line 74, in _handle_error
raise Exception(error_message) from source_exception
Exception: [Errno 2] No such file or directory: '/etc/secret-userdata.txt' for url: file:///etc/secret-userdata.txt
cloudinit.features.ERROR_ON_USER_DATA_FAILURE = True
If there is a failure in obtaining user data (i.e., #include or decompress fails) and ERROR_ON_USER_DATA_FAILURE is False, cloud-init will log a warning and proceed. If it is True, cloud-init will instead raise an exception.
As of 20.3, ERROR_ON_USER_DATA_FAILURE is True.
(This flag can be removed after Focal is no longer supported.)
Previously this would be just a warning as is indicated by CAPA documentation.
I am not entirely sure why CAPA user-data script is shipping /etc/secret-userdata.txt as x-include-url in addition to extracting it from SecretManager but one workaround that comes to mind is to ship images with an empty /etc/secret-userdata.txt and test for empty file in addition to missing file in
Another option would be to tweak cloud-init in the node image to disable hard fail but that appears more difficult as it can't be done at runtime based on my reading of cloud-init docs.
Happy to send patch if you agree on the approach.
Environment:
Any image with cloud-init version 20.3+ on latest CAPA control plane
/kind bug
What steps did you take and what happened:
Provisioning
state, cloud-init init phase fails with the following error while executing user-data:What did you expect to happen:
User-data execution stage should not fail.
Anything else you would like to add:
So this appears to happen because cloud-init made failures during user-data execution hard failures (https://cloudinit.readthedocs.io/en/latest/topics/hacking.html):
Previously this would be just a warning as is indicated by CAPA documentation.
I am not entirely sure why CAPA user-data script is shipping /etc/secret-userdata.txt as
x-include-url
in addition to extracting it from SecretManager but one workaround that comes to mind is to ship images with an empty /etc/secret-userdata.txt and test for empty file in addition to missing file incluster-api-provider-aws/pkg/cloud/services/secretsmanager/secret_fetch_script.go
Line 178 in 159c13f
Another option would be to tweak cloud-init in the node image to disable hard fail but that appears more difficult as it can't be done at runtime based on my reading of cloud-init docs.
Happy to send patch if you agree on the approach.
Environment:
Any image with cloud-init version 20.3+ on latest CAPA control plane
kubectl version
): n/a/etc/os-release
):Bonus - cloud-init version:
The text was updated successfully, but these errors were encountered: