Search: Returning CachedIrodsPath for data objects

[chStaiger]

To give some performance boost when searching for data objects the search function will now return a CachedIrodsPath and set the the checksum and the size to the value the search returns.

That gives a splendid improvement.

import time

search_start = time.time()
data = search_data(session, path_pattern="%.txt")
search_end = time.time()
print(f"Search time {search_end-search_start}")
for d in data:
    d.size
size_end = time.time()
print(f"Size time {size_end-search_end}")

• Returning and IrodsPath for data objects and fetching the size

  Search time 2.733164072036743
  Size time 67.01198482513428
  

• Returning a CachedIrodsPath with values from search

  Search time 2.3439738750457764
  Size time 0.0004801750183105469
  

[chStaiger]

@jjkoehorst Does this solve your performance issue?

[qubixes]

Given that there seems almost no performance impact generally for returning an extra column, this looks like a good idea. Just one small comment, otherwise seems like a sensible implementation!

[qubixes]

What about simplifying it to simply looping over item.keys()? Or am I missing something?

[chStaiger]

Also possible. I can then transform the key object to a string in the loop.

[qubixes]

It just saves one line of code, right?

[chStaiger]

Yes indeed.