Expose get_entities_v2 endpoint in python client

[noggi]

Checklist

• The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues (if applicable)
• Tests for the changes have been added/updated (if applicable)
• Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
• For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@jjoyce0510

[jjoyce0510]

LGTM. Let's get @hsheth2 signoff !

[hsheth2]

does this always work? I thought there's some differences between the openapi format and the restli format that we currently use?

note that from_obj ignores unknown fields, so it's possible this is silently dropping data - needs to be tested more completely

[noggi]

What is the alternative though? We can return raw dict if that's preferred, but if the field is unknown to the model, what else can we do?

[noggi]

I guess the alternative might be to raise if the dict has unknown fields? That means that this API becomes unusable for everyone every time a new field is introduced but python model is not up to date, whereas in the current implementation it's unusable only for those who depend on specific new field(s) that have to be added to the model anyway. In such case I'd vote for the current behavior.

[noggi]

Also as a compromise, we can return both the raw dict and the native objects.

[hsheth2]

My point was simply that from_obj is going to crash for a bunch of object types, because the formatting openapi uses is slightly different from the format used by restli. The python classes only support restli format and avro.

As a simple example, try this on dev01

from pprint import pp

import datahub.metadata.schema_classes as models
from datahub.ingestion.graph.client import get_default_graph

graph = get_default_graph()

urn = "urn:li:assertion:81c2e13a-1f41-43d5-8637-1171977badee"
info = graph.get_entities_v2(
    entity_name="assertion",
    urns=[urn],
    aspects=[models.AssertionInfoClass],
)
assert info is not None
pp(info[urn])

[noggi]

For posterity: we discussed offline and agreed that at this point this method should return raw dict with aspects and it's up to the user to deserialize it as appropriate. This approach seems the most reliable and is not affected by silent data loss issue. As soon as data model consistency situation improves, we can implement proper deserialization support, perhaps as a wrapper method.

[hsheth2]

Suggested change

	aspects: List[str],
	aspects: Optional[List[str]],