pulumi_kubernetes.yaml.ConfigFile with CRD raises TypeError #800

adir-intsights · 2019-09-12T09:41:14Z

I've tried to apply the Elastic Cloud on Kubernetes (ECK) YAML using ConfigFile and got TypeError: unexpected keyword argument 'status'.

pulumi_kubernetes.yaml.ConfigFile(
    name='eck',
    file_id='https://download.elastic.co/downloads/eck/0.9.0/all-in-one.yaml',
    opts=pulumi.ResourceOptions(
        provider=k8s_provider,
    ),
)

Trackback:

Diagnostics:
  pulumi:pulumi:Stack (tip-logserver-dev):
    error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/home/adir/.pulumi/bin/pulumi-language-python-exec", line 85, in <module>
        loop.run_until_complete(coro)
      File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
        return future.result()
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi/runtime/stack.py", line 72, in run_in_stack
        raise RPC_MANAGER.unhandled_exception.with_traceback(RPC_MANAGER.exception_traceback)
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi/runtime/rpc_manager.py", line 67, in rpc_wrapper
        result = await rpc_function(*args, **kwargs)
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi/runtime/resource.py", line 414, in do_register_resource_outputs
        serialized_props = await rpc.serialize_properties(outputs, {})
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi/runtime/rpc.py", line 67, in serialize_properties
        result = await serialize_property(v, deps, input_transformer)
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi/runtime/rpc.py", line 159, in serialize_property
        value = await serialize_property(value.future(), deps, input_transformer)
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi/runtime/rpc.py", line 147, in serialize_property
        future_return = await asyncio.ensure_future(value)
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi/output.py", line 149, in run
        value = await self._future
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi/output.py", line 311, in gather_futures
        return await asyncio.gather(*value_futures)
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi/output.py", line 149, in run
        value = await self._future
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi/output.py", line 161, in run
        transformed: Input[U] = func(value)
      File "/home/adir/.virtualenvs/pulumi/lib/python3.7/site-packages/pulumi_kubernetes/yaml.py", line 218, in <lambda>
        CustomResourceDefinition(f"{x}", opts, **obj)))]
    TypeError: __init__() got an unexpected keyword argument 'status'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

I'm using Pulumi 1.1.0 and pulumi-kubernetes 1.0.1.

The text was updated successfully, but these errors were encountered:

brews · 2020-09-23T19:39:40Z

I'm getting comparable TypeError for status on pulumi-kubernetes v2.3.1 and v2.6.1 trying to deploy cert-manager v0.15, v0.16, and v1.0.3. This is with pulumi CLI v2.10.2.

lblackstone · 2020-09-23T20:06:49Z

This happens if the CRD YAML is setting the .status field, which isn't technically a valid input. We could probably add a workaround to ignore status if it's set in the code generators, but in the meantime, you can fix the error with a transformation to remove the erroneous status field.

Something along these lines should do the trick:

# Remove the .status field from CRDs
def remove_status(obj, opts):
    if obj["kind"] == "CustomResourceDefinition":
        del obj["status"]

example = ConfigFile(
    "example",
    files=["cert-manager.yaml"],
    transformations=[remove_status],
)

brews · 2020-09-23T20:44:54Z

Ah, that's a good workaround. I was able to deploy with the above transformation. Appreciate the suggestion!

kmr0877 · 2020-10-06T22:33:42Z

Similar issue #1226

ppawiggers · 2021-01-22T08:51:24Z

For a Helm chart, you can use the same transformation function as shown above, and connect it in the ChartOps:

Chart("myChart",
  config=ChartOpts(
    transformations=[remove_status],
  ),
)

However, I applied it to the kube-prometheus-stack Helm chart and it makes a pulumi up very slow (Python CPU usage too 100%).

Edit: It seems to be related to (my?) MacOS, when I run it in Docker (for Mac) it is fast.

repudi8or · 2021-06-02T02:34:17Z

so above it was stated:

This happens if the CRD YAML is setting the .status field, which isn't technically a valid input

however:
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#customresourcedefinitionstatus-v1beta1-apiextensions-k8s-io

it seems like status field is valid. Or is the suggestion that it is not valid as an input and only as output?

lblackstone · 2021-06-02T03:21:31Z

Or is the suggestion that it is not valid as an input and only as output?

Correct. It's an output-only field.

dz-pyps · 2021-10-29T12:57:00Z

Any updates? It has been a while

pasmon · 2022-03-02T08:13:02Z

I'm not sure if I have the same issue or not, but the workaround doesn't seem to work in my ConfigFile.

Here's the traceback:

    error: Traceback (most recent call last):
      File "/home/pasmon/.pulumi/bin/pulumi-language-python-exec", line 107, in <module>
        loop.run_until_complete(coro)
      File "/usr/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
        return future.result()
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 126, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 51, in run_pulumi_func
        await wait_for_rpcs()
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 73, in wait_for_rpcs
        await RPC_MANAGER.rpcs.pop()
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/rpc_manager.py", line 68, in rpc_wrapper
        result = await rpc
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/resource.py", line 685, in do_register_resource_outputs
        serialized_props = await rpc.serialize_properties(outputs, {})
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/rpc.py", line 172, in serialize_properties
        result = await serialize_property(
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/rpc.py", line 343, in serialize_property
        value = await serialize_property(
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/rpc.py", line 326, in serialize_property
        future_return = await asyncio.ensure_future(awaitable)
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/output.py", line 123, in get_value
        val = await self._future
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/output.py", line 169, in run
        value = await self._future
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/output.py", line 447, in gather_futures
        return await asyncio.gather(*value_futures_list)
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/output.py", line 123, in get_value
        val = await self._future
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/output.py", line 169, in run
        value = await self._future
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/output.py", line 194, in run
        transformed: Input[U] = func(value)
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi_kubernetes/yaml/yaml.py", line 543, in <lambda>
        CustomResourceDefinition(f"{x}", opts, **obj)))]
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi_kubernetes/apiextensions/v1/CustomResourceDefinition.py", line 126, in __init__
        __self__._internal_init(resource_name, *args, **kwargs)
    TypeError: CustomResourceDefinition._internal_init() got an unexpected keyword argument 'status'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

The workaround in my case:

# Remove the .status field from CRDs
def remove_status(obj, opts):
    if obj["kind"] == "CustomResourceDefinition":
        del obj["status"]

argocd = ConfigFile(
    "argocd",
    file="external/install.yaml",
    transformations=[define_namespace, remove_status],
    opts=pulumi.ResourceOptions(depends_on=[traefik_release,argocd_namespace])
    )

This is the outcome with the workaround:

    error: Traceback (most recent call last):
      File "/home/pasmon/.pulumi/bin/pulumi-language-python-exec", line 107, in <module>
        loop.run_until_complete(coro)
      File "/usr/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
        return future.result()
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 126, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 49, in run_pulumi_func
        func()
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 126, in <lambda>
        await run_pulumi_func(lambda: Stack(func))
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 149, in __init__
        func()
      File "/home/pasmon/.pulumi/bin/pulumi-language-python-exec", line 106, in <lambda>
        coro = pulumi.runtime.run_in_stack(lambda: runpy.run_path(args.PROGRAM, run_name='__main__'))
      File "/usr/lib/python3.10/runpy.py", line 286, in run_path
        return _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/home/pasmon/git/homelab-python/./__main__.py", line 369, in <module>
        argocd = ConfigFile(
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi_kubernetes/yaml/yaml.py", line 351, in __init__
        self.resources = _parse_yaml_document(__ret__, opts, transformations, resource_prefix)
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi_kubernetes/yaml/yaml.py", line 419, in _parse_yaml_document
        file_objects = _parse_yaml_object(obj, opts, transformations, resource_prefix)
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi_kubernetes/yaml/yaml.py", line 448, in _parse_yaml_object
        t(obj, opts)
      File "/home/pasmon/git/homelab-python/./__main__.py", line 367, in remove_status
        del obj["status"]
    KeyError: 'status'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

wallflower762 · 2022-06-02T06:37:12Z

This issue has been open for a few years now, is there any plan to fix it?

While the workaround worked fine for me, it would be nice if it worked as expected out of the box.

pbarker · 2022-07-08T18:28:27Z

It would be nice if this were handled internally, I've hit a lot of OSS packages that fail on this, which is pretty bad ux, seems like a nominal thing to ignore

nicholasgibson2 · 2022-09-20T21:47:38Z

+1 for a real fix to this issue

This is the outcome with the workaround:

    error: Traceback (most recent call last):
      File "/home/pasmon/.pulumi/bin/pulumi-language-python-exec", line 107, in <module>
        loop.run_until_complete(coro)
      File "/usr/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
        return future.result()
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 126, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 49, in run_pulumi_func
        func()
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 126, in <lambda>
        await run_pulumi_func(lambda: Stack(func))
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 149, in __init__
        func()
      File "/home/pasmon/.pulumi/bin/pulumi-language-python-exec", line 106, in <lambda>
        coro = pulumi.runtime.run_in_stack(lambda: runpy.run_path(args.PROGRAM, run_name='__main__'))
      File "/usr/lib/python3.10/runpy.py", line 286, in run_path
        return _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/home/pasmon/git/homelab-python/./__main__.py", line 369, in <module>
        argocd = ConfigFile(
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi_kubernetes/yaml/yaml.py", line 351, in __init__
        self.resources = _parse_yaml_document(__ret__, opts, transformations, resource_prefix)
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi_kubernetes/yaml/yaml.py", line 419, in _parse_yaml_document
        file_objects = _parse_yaml_object(obj, opts, transformations, resource_prefix)
      File "/home/pasmon/.local/share/virtualenvs/homelab/lib/python3.10/site-packages/pulumi_kubernetes/yaml/yaml.py", line 448, in _parse_yaml_object
        t(obj, opts)
      File "/home/pasmon/git/homelab-python/./__main__.py", line 367, in remove_status
        del obj["status"]
    KeyError: 'status'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

@pasmon I was getting the same issue, this happens when not all of the CRDs contain status. Adding "status" in objcheck to the previous example works for me:

# Remove the .status field from CRDs
def remove_status(obj, opts):
    if obj["kind"] == "CustomResourceDefinition" and "status" in obj:
        del obj["status"]

example = ConfigFile(
    "example",
    files=["cert-manager.yaml"],
    transformations=[remove_status],
)

lblackstone · 2022-09-20T22:36:54Z

As far as I can tell, this issue only affects the Python SDK. I fixed the Python issue in #2183 and this change should be included in the next provider release.

adir-intsights changed the title ~~pulumi_kubernetes.yaml.ConfigFile raises TypeError with CRD~~ pulumi_kubernetes.yaml.ConfigFile with CRD raises TypeError Sep 12, 2019

lblackstone mentioned this issue Sep 12, 2019

We should not populate .status in the input types #444

Closed

lblackstone assigned hausdorff Sep 12, 2019

lukehoban assigned lblackstone and unassigned hausdorff Nov 27, 2019

lblackstone mentioned this issue Oct 5, 2020

[kedacore/keda] Install fails on helm3 and Kubernetes v1.18.8 #1339

Closed

lblackstone mentioned this issue Dec 9, 2020

Error templating helm chart with python SDL #1400

Closed

misterkramer mentioned this issue Dec 28, 2020

Generated CRDs have status fields in their output set kubernetes-sigs/controller-tools#456

Closed

leezen mentioned this issue Jan 21, 2021

Deploy cert-manager with Helm: unexpected keyword argument 'status' #1440

Closed

lukehoban mentioned this issue Jan 25, 2021

pulumi up fails with TypeError: __init__() got an unexpected keyword argument 'status' pulumi/pulumi#6188

Closed

georgyo mentioned this issue Apr 23, 2021

cert-manager helm chart returns type error when passing values while using python #1541

Closed

lblackstone mentioned this issue Aug 16, 2022

Spark Operator Helm Deployment Error #2142

Closed

lblackstone added this to the 0.78 milestone Sep 20, 2022

lblackstone mentioned this issue Sep 20, 2022

[sdk/python] Handle CRDs with status field input #2183

Merged

lblackstone added kind/bug Some behavior is incorrect or out of spec language/python labels Sep 20, 2022

lblackstone closed this as completed in #2183 Sep 21, 2022

pulumi-bot added the resolution/fixed This issue was fixed label Sep 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pulumi_kubernetes.yaml.ConfigFile with CRD raises TypeError #800

pulumi_kubernetes.yaml.ConfigFile with CRD raises TypeError #800

adir-intsights commented Sep 12, 2019

brews commented Sep 23, 2020 •

edited

Loading

lblackstone commented Sep 23, 2020

brews commented Sep 23, 2020

kmr0877 commented Oct 6, 2020

ppawiggers commented Jan 22, 2021 •

edited

Loading

repudi8or commented Jun 2, 2021

lblackstone commented Jun 2, 2021

dz-pyps commented Oct 29, 2021

pasmon commented Mar 2, 2022

wallflower762 commented Jun 2, 2022

pbarker commented Jul 8, 2022

nicholasgibson2 commented Sep 20, 2022

lblackstone commented Sep 20, 2022

pulumi_kubernetes.yaml.ConfigFile with CRD raises TypeError #800

pulumi_kubernetes.yaml.ConfigFile with CRD raises TypeError #800

Comments

adir-intsights commented Sep 12, 2019

brews commented Sep 23, 2020 • edited Loading

lblackstone commented Sep 23, 2020

brews commented Sep 23, 2020

kmr0877 commented Oct 6, 2020

ppawiggers commented Jan 22, 2021 • edited Loading

repudi8or commented Jun 2, 2021

lblackstone commented Jun 2, 2021

dz-pyps commented Oct 29, 2021

pasmon commented Mar 2, 2022

wallflower762 commented Jun 2, 2022

pbarker commented Jul 8, 2022

nicholasgibson2 commented Sep 20, 2022

lblackstone commented Sep 20, 2022

brews commented Sep 23, 2020 •

edited

Loading

ppawiggers commented Jan 22, 2021 •

edited

Loading