Unable to create tag template with dictionary

[mik-laj]

Hi.
I'm working on updating to the latest version of the libraries. Previously I was using 0.8.0 and 1.0.0 where I had a similar problem, but found a simple workaround - some values as an object, but these workarounds no longer work in the newer version of the library.

In datacatalog 0.8.0, it was possible to create an tag template using a dictionary, but template fields must be passed as a TagTemplateField object.

        tag_template={
            "display_name": "Awesome Tag Template",
            "fields": {
                FIELD_NAME_1: TagTemplateField(
                    display_name="first-field", type_=dict(primitive_type="STRING")
                )
            },
        },

In a newer version, such a trick can no longer be used. If we want to create an object, we must always pass the fully typed object.

    request_good = datacatalog.CreateTagTemplateRequest(
        parent=LOCATION_PATH,
        tag_template_id=TEMPLATE_ID,
        tag_template=datacatalog.TagTemplate(
            display_name="Awesome Tag template",
            fields={
                FIELD_NAME_1: datacatalog.TagTemplateField(
                    display_name="first-field", type_=dict(primitive_type="STRING")
                )
            },
        ),
    )

However, I would expect that it would be possible to pass the object as a dictionary.

    request = dict(
        parent=LOCATION_PATH,
        tag_template_id=TEMPLATE_ID,
        tag_template=dict(
            display_name="Awesome Tag template",
            fields={FIELD_NAME_1: dict(display_name="first-field", type_=dict(primitive_type="STRING"))},
        ),
    )

As a workaround, I can convert this object on my side for now, but it would be great if the library handles it correctly.

    request = datacatalog.CreateTagTemplateRequest(
        parent=request.get("parent"),
        tag_template_id=request.get("tag_template_id"),
        tag_template=(
            datacatalog.TagTemplate(
                display_name=request.get("tag_template", {}).get("display_name"),
                fields={
                    k: datacatalog.TagTemplateField(**v)
                    for k, v in request.get("tag_template", {}).get("fields", {}).items()
                },
            )
            if request.get("tag_template")
            else None
        ),
    )

Environment details

• OS type and version: Mac OS 11.01
• Python version: Python 3.6.8
• pip version: pip 20.2.4 from /Users/XXXXXXXXX/.virtualenvs/airflow/lib/python3.6/site-packages/pip (python 3.6)
• google-cloud-datacatalog version: 3.0.0

Steps to reproduce

1. Run code from "code example" section or full example

Code example

    request = dict(
        parent=LOCATION_PATH,
        tag_template_id=TEMPLATE_ID,
        tag_template=dict(
            display_name="Awesome Tag template",
            fields={FIELD_NAME_1: dict(display_name="first-field", type_=dict(primitive_type="STRING"))},
        ),
    )
    log.info("create_tag_template(request=%s)", request)
    tag_template = client.create_tag_template(request=request)

Full example: https://gist.github.com/mik-laj/4d2e4701b0f827dff5a712300646f024

Stack trace

Traceback (most recent call last):
  File "files/datacatalog_lookup_entry.py", line 105, in <module>
    tag_template = client.create_tag_template(request=request)
  File "/Users/X/.virtualenvs/airflow/lib/python3.6/site-packages/google/cloud/datacatalog_v1beta1/services/data_catalog/client.py", line 1582, in create_tag_template
    request = datacatalog.CreateTagTemplateRequest(request)
  File "/Users/X/.virtualenvs/airflow/lib/python3.6/site-packages/proto/message.py", line 461, in __init__
    pb_value = marshal.to_proto(pb_type, value)
  File "/Users/X/.virtualenvs/airflow/lib/python3.6/site-packages/proto/marshal/marshal.py", line 205, in to_proto
    pb_value = rule.to_proto(value)
  File "/Users/X/.virtualenvs/airflow/lib/python3.6/site-packages/proto/marshal/rules/message.py", line 32, in to_proto
    return self._descriptor(**value)
TypeError: Parameter to MergeFrom() must be instance of same class: expected google.cloud.datacatalog.v1beta1.TagTemplateField got dict.

CC: @busunkim96

[meredithslota]

@busunkim96 I'm not totally sure if this would be a bug, but is this an expected change?

[mik-laj]

@meredithslota While working on migrating to this library, I noticed that the problem also occurs with the Tag type.
Reference:
https://github.com/apache/airflow/blob/9235af594c54d4a2991c494929deef37eee33ac6/airflow/providers/google/cloud/hooks/datacatalog.py#L232-L245
https://github.com/apache/airflow/blob/9235af594c54d4a2991c494929deef37eee33ac6/airflow/providers/google/cloud/hooks/datacatalog.py#L295-L306

Both objects contain proto.MapField in their definition.

python-datacatalog/google/cloud/datacatalog_v1/types/tags.py

Line 82 in 6ca679e

fields = proto.MapField(proto.STRING, proto.MESSAGE, number=3, message="TagField",)

python-datacatalog/google/cloud/datacatalog_v1/types/tags.py

Lines 185 to 187 in 6ca679e

	fields = proto.MapField(
	proto.STRING, proto.MESSAGE, number=3, message="TagTemplateField",
	)

[busunkim96]

I believe this is googleapis/proto-plus-python#161

[tswast]

Closing as a duplicate of googleapis/proto-plus-python#161, as I don't believe there will be any changes required to this library once the issue is resolved in proto-plus.