extract_paths, use get_object_state

[albertz]

This fixes the same problems as #207 but now for extract_paths, by using the same shared code (get_object_state).

[albertz]

@NeoLegends @michelwi @curufinwe what is the status here?

[michelwi]

This change leads to a RecursionError in one of our gmm training setups
cf. https://bitbucket.org/omnifluent/apptek_asr/pull-requests/1268

(I did not start any debugging besides triggering the pipeline, sorry)

[albertz]

Can you tell me for what object this recursion error happens? I checked the AppTek PR pipeline error, but there is no information on the variables, so I cannot tell from that. Maybe you can also enable better_exchook (maybe better_exchook.setup_all()) for that, so then I can see it?

[michelwi]

if obj is of type np.float then it is an instance of float so get_object_state simply returns it.

But sis_hash_helper(obj) checks for type(obj) in (int, float, bool, str, complex):, which is False and therefore we end up in the else case byte_list.append(sis_hash_helper(get_object_state(obj))) which leads to an infinite recursion.

[albertz]

I pushed some change for that which makes this check more consistent. Can you check again?

[albertz]

Note, np.float is a bad example: This will actually break depending on your Numpy version. In (very old) Numpy versions, yes, np.float is not float but derived from float. However, in newer Numpy versions, np.float is just an alias to float.

DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

So it means, if you really used np.float in your setup, the hash will break when you update Numpy...

A better example is a namedtuple. And here I accidentally also broke some hash now, but this is fixed now. Specifically, due to the type(obj) in (tuple, list) check, which was False, it also falls back to the get_object_state logic for namedtuples, so it's important that get_object_state behaves the same as before for namedtuples. This is what I do now.

[curufinwe]

AppTek pipeline no longer crashes.

[albertz]

So it means ok to merge?

[curufinwe]

LGTM