Spark Operator Helm Deployment Error

[David-N-Perkins]

What happened?

I'm trying to install the Spark Operator Helm chart, https://github.com/GoogleCloudPlatform/spark-on-k8s-operator, with Pulumi + Python, but getting an error.

Steps to reproduce

Here's the resource declaration:

import pulumi_kubernetes as kubernetes

aks_spark_operator = kubernetes.helm.v3.Chart('spark-operator',
    kubernetes.helm.v3.ChartOpts(
        chart='spark-operator',
        version='1.1.25',
        namespace='spark-test',
        fetch_opts=kubernetes.helm.v3.FetchOpts(
            repo='https://googlecloudplatform.github.io/spark-on-k8s-operator',
            version='spart-operator-chart-1.1.25',
        ),
        values={
            'sparkJobNamespace': 'spark-test',
            'metrics.enable': False, 
            'serviceAccounts.spark.name': 'spark-test',
            'serviceAccounts.sparkoperator.name': 'spark-operator-test',
            'uiService.enable': False,
        },
    ),
    pulumi.ResourceOptions(provider=k8s_provider),
)

Expected Behavior

The Spark Operator Helm chart to get installed.

Actual Behavior

I got an error and stack trace:

Diagnostics:
  pulumi:pulumi:Stack (myproject-dev):
    error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/opt/homebrew/bin/pulumi-language-python-exec", line 107, in <module>
        loop.run_until_complete(coro)
      File "/opt/homebrew/Cellar/[email protected]/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
        return future.result()
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 126, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 51, in run_pulumi_func
        await wait_for_rpcs()
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 73, in wait_for_rpcs
        await RPC_MANAGER.rpcs.pop()
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/runtime/rpc_manager.py", line 68, in rpc_wrapper
        result = await rpc
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/runtime/resource.py", line 685, in do_register_resource_outputs
        serialized_props = await rpc.serialize_properties(outputs, {})
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/runtime/rpc.py", line 207, in serialize_properties
        result = await serialize_property(
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/runtime/rpc.py", line 378, in serialize_property
        value = await serialize_property(
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/runtime/rpc.py", line 361, in serialize_property
        future_return = await asyncio.ensure_future(awaitable)
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 169, in run
        value = await self._future
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 123, in get_value
        val = await self._future
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 206, in run
        return await transformed.future(with_unknowns=True)
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 123, in get_value
        val = await self._future
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 206, in run
        return await transformed.future(with_unknowns=True)
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 123, in get_value
        val = await self._future
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 169, in run
        value = await self._future
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 447, in gather_futures
        return await asyncio.gather(*value_futures_list)
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 123, in get_value
        val = await self._future
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 169, in run
        value = await self._future
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi/output.py", line 194, in run
        transformed: Input[U] = func(value)
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi_kubernetes/yaml/yaml.py", line 545, in <lambda>
        CustomResourceDefinition(f"{x}", opts, **obj)))]
      File "/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/lib/python3.10/site-packages/pulumi_kubernetes/apiextensions/v1/CustomResourceDefinition.py", line 127, in __init__
        __self__._internal_init(resource_name, *args, **kwargs)
    TypeError: CustomResourceDefinition._internal_init() got an unexpected keyword argument 'status'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

Versions used

Current pulumi project: processing
CLI
Version      3.37.2
Go Version   go1.18.4
Go Compiler  gc

Plugins
NAME          VERSION
azure         5.14.0
azure-native  1.67.0
kubernetes    3.20.3
python        unknown

Host
OS       darwin
Version  12.5
Arch     arm64

This project is written in python: executable='/Users/dperkins/Library/Caches/pypoetry/virtualenvs/pulumi-yuX4ywwK-py3.10/bin/python3' version='3.10.5'

Current Stack: dev

Found no resources associated with dev

Found no pending operations associated with dev

Backend
Name           MBP
URL            azblob://processing
User           dperkins
Organizations

Dependencies:
NAME              VERSION
myproj-pulumi  0.1.0
pip               22.0.3
wheel             0.37.1

Pulumi locates its logs in /var/folders/jr/_lr1djrn3_39vfty473tsn2w0000gp/T/ by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[dixler]

Hi, thanks for reporting this. It looks like a pulumi-kubernetes issue so I'll transfer you there. I'm working on improving error messages(and tracebacks) so I really appreciate you posting this.

[lblackstone]

This appears to be another instance of #800, so I'll close this as a duplicate. See #800 (comment) for a workaround.

[David-N-Perkins]

Didn't realize there was separate project for the Kubernetes package. Thanks for the reference.