monitor containers for modified entities #51
Conversation
|
Why not use walk? It basically does the same thing as what you've written in _traverse. I think there is missing functionality like include_types for synapseutils.walk |
|
re: Why not use synapseutils.walk? The synapseutils.walk returns a dictionary in which one of the values is a nested list of descendant entities that requires parsing to extract just the descendant entity Synapse IDs as a simple list. The extra parsing step in addition to the lack of the include_types functionality made implementing the _traverse function here explicitly more straight-forward and flexible. |
synapsemonitor/monitor.py
Outdated
| synid_children = syn.getChildren(parent=synid_root, includeTypes=include_types_mod) | ||
| for synid_child in synid_children: | ||
| synid_desc.extend( | ||
| _traverse( |
There was a problem hiding this comment.
I just remembered, in Python there is a max recursion limit - limit of calling a function recursively ~1000 times. So you may get into trouble here because if you do a getChildren on a folder with 1000 files, you're actually calling _traverse 1000 times. We may want to do something similar as synapseutils.walk where we only call _traverse when it is a container.
Then again - for those cases, we should encourage the use of file views...
There was a problem hiding this comment.
Added a check for the entity type of the children to limit recursive calls. Now, if a child entity is neither a folder nor a project, the child entity type is added directly to the descendant list after checking that it is a requested entity type.
synapsemonitor/monitor.py
Outdated
|
|
||
| synid_children = syn.getChildren(parent=synid_root, includeTypes=include_types_mod) | ||
| for synid_child in synid_children: | ||
| entity = syn.get(synid_child["id"], downloadFile=False) |
There was a problem hiding this comment.
I'm not sure this syn.get isn't necessary. Print out synid_child. I think concreteType is part of it?
There was a problem hiding this comment.
Almost. Child entity reference 'concreteType' as 'type'. But I've removed the redundant syn.get
Fixes #23
_traverse: traversal function that returns any descendants of a given entity type (retained to enhance later functionality)_find_modified_entities_container: checks container and descendants for modification