Allow to construct AnyVector and AnyDictionary with values from Python #1574
Conversation
…n. This is mainly meant to facilitate developing OTIO itself. Signed-off-by: Jean-Christophe Morin <[email protected]>
Signed-off-by: Jean-Christophe Morin <[email protected]>
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #1574 +/- ##
==========================================
+ Coverage 80.04% 80.06% +0.01%
==========================================
Files 197 197
Lines 21720 21757 +37
Branches 4309 4316 +7
==========================================
+ Hits 17386 17419 +33
+ Misses 2182 2180 -2
- Partials 2152 2158 +6
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 2 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
| .def(py::init([](const py::iterable &it) { | ||
| any a; | ||
| py_to_any(it, &a); | ||
| AnyVector v = safely_cast_any_vector_any(a); | ||
| auto proxy = new AnyVectorProxy; | ||
| proxy->fetch_any_vector().swap(*v.get_or_create_mutation_stamp()->any_vector); | ||
| return proxy; | ||
| })) |
There was a problem hiding this comment.
is this copying the data?
There was a problem hiding this comment.
If by data you mean everything (like a deep copy in Python), no. If you mean the top level list/vector, yes. See https://github.com/AcademySoftwareFoundation/OpenTimelineIO/pull/1574/files#r1165693980.
What should be the behavior?
| 'list': [1, 2.5, 'asd'], | ||
| 'dict': {'map1': [345]}, | ||
| 'AnyVector': v, | ||
| 'AnyDictionary': d1, |
There was a problem hiding this comment.
related to the question about copies, can you edit d1 in place and have d2["AnyDictionary"] change?
There was a problem hiding this comment.
Items removed or added in d1 in place won't be added/removed in d2, but values are shared. As far as I see, this is an existing behavior. For example, this passes:
def test_construct_with_values(self):
# Contruct with values
d1 = opentimelineio.core._core_utils.AnyDictionary(
{'key1': 1234, 'key_2': {'asdasdasd': 5.6}}
)
# Construct without values and only add a value after instantiation.
d3 = opentimelineio.core._core_utils.AnyDictionary()
d3['key4'] = 'value4'
# Create an SO and add it to d1 and d3.
so = opentimelineio._otio.SerializableObjectWithMetadata('initial name')
d1['so'] = so
d3['so'] = so
d2 = opentimelineio.core._core_utils.AnyDictionary(
{
'd1': d1,
'd3': d3,
'so': so
}
)
self.assertEqual(d2['d1'], d1)
self.assertEqual(d2['d3'], d3)
# Edit d1 in place
d1['key3'] = 'value3'
d1['so'].name = 'new name set on d1["so"]'
# Confirm that d2['d1'] is not updated
self.assertEqual(d2['d1'], {'key1': 1234, 'key_2': {'asdasdasd': 5.6}, 'so': so})
self.assertEqual(d1, {'key1': 1234, 'key_2': {'asdasdasd': 5.6}, 'key3': 'value3', 'so': so})
# But values that were already there are shared.
# We previously updated the name of so by doing d1['so'].name = '...'
# and d2['so'] got updated.
self.assertEqual(d2['so'].name, 'new name set on d1["so"]')
# Same behavior here.
d3['key5'] = 'value5'
d3['so'].name = 'new name set on d3["so"]'
self.assertEqual(d2['d3'], {'key4': 'value4', 'so': so})
self.assertEqual(d3, {'key4': 'value4', 'key5': 'value5', 'so': so})
self.assertEqual(d2['so'].name, 'new name set on d3["so"]')There was a problem hiding this comment.
But just to be sure, I'll do some A/B testing with the main branch.
There was a problem hiding this comment.
Same result on the main branch (with slight modification in the code example to not construct with values):
def test_construct_with_values(self):
d1 = opentimelineio.core._core_utils.AnyDictionary()
d1['key1'] = 1234
d1['key_2'] = {'asdasdasd': 5.6}
# Construct without values and only add a value after instantiation.
d3 = opentimelineio.core._core_utils.AnyDictionary()
d3['key4'] = 'value4'
# Create an SO and add it to d1 and d3.
so = opentimelineio._otio.SerializableObjectWithMetadata('initial name')
d1['so'] = so
d3['so'] = so
d2 = opentimelineio.core._core_utils.AnyDictionary()
d2['d1'] = d1
d2['d3'] = d3
d2['so'] = so
self.assertEqual(d2['d1'], d1)
self.assertEqual(d2['d3'], d3)
# Edit d1 in place
d1['key3'] = 'value3'
d1['so'].name = 'new name set on d1["so"]'
# Confirm that d2['d1'] is not updated
self.assertEqual(d2['d1'], {'key1': 1234, 'key_2': {'asdasdasd': 5.6}, 'so': so})
self.assertEqual(d1, {'key1': 1234, 'key_2': {'asdasdasd': 5.6}, 'key3': 'value3', 'so': so})
# But values that were already there are shared.
# We previously updated the name of so by doing d1['so'].name = '...'
# and d2['so'] got updated.
self.assertEqual(d2['so'].name, 'new name set on d1["so"]')
# Same behavior here.
d3['key5'] = 'value5'
d3['so'].name = 'new name set on d3["so"]'
self.assertEqual(d2['d3'], {'key4': 'value4', 'so': so})
self.assertEqual(d3, {'key4': 'value4', 'key5': 'value5', 'so': so})
self.assertEqual(d2['so'].name, 'new name set on d3["so"]')
|
C++ folks, @darbyjohnston, @davidbaraff, or @meshula -- can anyone else scan this? Thanks @JeanChristopheMorinPerso! |
|
My question is whether or not this opens a path to creating malformed constructions? (By opens a path, I mean, can you do something now to build something malformed, that could not be built before?) |
|
An important invariant is not being able to build something in Python that causes C++ to crash. :) I’m fine with programming in C++ and doing illegal things in OTIO that cause a core dump. That’s the nature of C++.But the goal is that if you type something in in Python that can cause a core dump, it’s a bug on the implementors of the bridge (i.e., primary me) that should be fixed. The corollary is never to open up any new capabilities that permit this to happen. The bridging regarding dictionaries/vectors is extremely complicated because of things like modifying these structures during traversal, which could invalidate a C++ iterator. A lot of care was taken to get to where we are now (I don’t know that it’s perfect right now of course but I don’t know of any existing mechanisms that can cause a crash).So just saying that any new capabilities have to keep this front of mind. This stuff is NOT easy.Sent from my iPadOn Apr 13, 2023, at 9:37 AM, Nick Porcino ***@***.***> wrote:
My question is whether or not this opens a path to creating malformed constructions? (By opens a path, I mean, can you do something now to build something malformed, that could not be built before?)
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
@meshula By that you mean will accept values/types that couldn't used before? Basically, when we do Also, I'm not aware of anyone actually using the Also, thanks @davidbaraff! There is one known crash with AnyDictionary, see #1474. But that's existing behavior. And there is also a known exception (at least to me): This particular wasn't reported by anyone, except me. I tend to poke around quite a bit with the OTIO internals and I sometimes find these gems. But that's all existing behaviour. |
Allow to construct
AnyVectorandAnyDictionarywith values from Python.Note that it only useful to OTIO developers who have to play with the Python bindings internals. For example I often have to create these objects when I work on the bindings, and it's quite annoying to have to do
while I could simply do
d = otio._otio.AnyDictionary({'a': 'a'}).This was extracted from #1490.