Chown gateway #195

will-moore · 2020-03-06T13:31:10Z

Started to add Chown functionality to Blitz Gateway to allow for support in webclient.

joshmoore · 2020-03-06T13:36:45Z

@will-moore : please see #153 for a possible leak with this usage.

src/omero/gateway/__init__.py

will-moore · 2020-03-17T12:32:40Z

@josh Is there a workaround for the leak?
We do the same in conn.deleteObjects() so I guess we need to apply the same fix there too.

joshmoore · 2020-03-17T13:01:04Z

@will-moore : the block from ome/openmicroscopy#6082 (comment)

@@ -339,6 +339,7 @@ def forgotten_password(request, **kwargs):
                     try:
                         conn._waitOnCmd(handle)
                     finally:
+                        # This likely leaks a callback servant
                         handle.close()

should be:

                     try:
                         cb = conn._waitOnCmd(handle)
                     finally:
                         cb.close(True)

i.e. if you are using a callback, close via the callback and not via the handle with the issue being that we don't generally consider _waitOnCmd to be using a callback, but it is internally.

will-moore · 2020-03-17T17:09:52Z

Travis failing with tests

INTERNALERROR>     self.writer = self._tw
INTERNALERROR> AttributeError: can't set attribute

Full error

py36 run-test: commands[2] | pytest -n4 -m 'not broken' --reruns 5 -rf test
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 187, in wrap_session
INTERNALERROR>     config._do_configure()
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/_pytest/config/__init__.py", line 820, in _do_configure
INTERNALERROR>     self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/hooks.py", line 308, in call_historic
INTERNALERROR>     res = self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pytest_sugar.py", line 176, in pytest_configure
INTERNALERROR>     sugar_reporter = SugarTerminalReporter(standard_reporter)
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pytest_sugar.py", line 214, in __init__
INTERNALERROR>     self.writer = self._tw
INTERNALERROR> AttributeError: can't set attribute
ERROR: InvocationError for command /home/travis/build/ome/omero-py/.tox/py36/bin/pytest -n4 -m 'not broken' --reruns 5 -rf test (exited with code 3)

manics · 2020-03-17T17:22:17Z

Probably needs the same fix as omero-web ome/omero-web#152

will-moore · 2020-03-18T09:13:53Z

Thanks @manics - that helped get a bit further.
Now tests are failing for python 3.6 only with:

INTERNALERROR> AssertionError: ('test/unit/tablestest/test_servants.py::TestTables::testTablePreInitialized', <WorkerController gw0>)
INTERNALERROR> assert not 'test/unit/tablestest/test_servants.py::TestTables::testTablePreInitialized'

Full error

........................................................................ [ 92%]
........ss.sss.s.ss.ss.sssssssssssss.sssssss............................ [ 97%]
...........x....R.INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 191, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 247, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/xdist/dsession.py", line 112, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/xdist/dsession.py", line 135, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "/home/travis/build/ome/omero-py/.tox/py36/lib/python3.6/site-packages/xdist/dsession.py", line 177, in worker_workerfinished
INTERNALERROR>     assert not crashitem, (crashitem, node)
INTERNALERROR> AssertionError: ('test/unit/tablestest/test_servants.py::TestTables::testTablePreInitialized', <WorkerController gw0>)
INTERNALERROR> assert not 'test/unit/tablestest/test_servants.py::TestTables::testTablePreInitialized'

joshmoore · 2020-03-18T09:14:51Z

Let's concentrate on #198 to try to fix all open PRs.

joshmoore · 2020-04-03T12:59:59Z

Re-opening with #198 in.

joshmoore · 2020-04-03T13:07:28Z

Green!

sbesson · 2020-06-09T09:55:11Z

As I am looking into cutting a patch release of omero-py with all included fixes (see https://github.com/ome/omero-py/milestone/4?closed=1), this PR contains the only change that worries me as it adds a new API to the omero.gateway and the discussion suggests this is still in-progress work.

@will-moore how confident do you feel about making this a public gateway API and supporting it? Additionally, is anything in our own stack consuming it yet?
One potential solution would be to mark this API as internal using the leading underscore convention (https://www.python.org/dev/peps/pep-0008/#public-and-internal-interfaces) i.e. def _chownObjects, release omero-py 5.6.3 and start consuming this API until we are happy to declare it as public API?

will-moore · 2020-06-09T09:59:39Z

I'm pretty happy with releasing this.
It's a thin wrapper around the underlying API and we have tests.
The usage of it in our own stack is in progress at ome/omero-web#149
but needs this to be in place first, as well as #196 from omero-py.

sbesson · 2020-06-09T10:41:50Z

Thanks. I think options are:

wait until api experimenters groups #196 is integrated and release omero-py 5.7.0
release the current set of PRs immediately as omero-py 5.7.0 and start working towards omero-py 5.8.0 with api experimenters groups #196
open a PR to make this wrapper interface internal, release omero-py 5.6.3 and make the API public together with the one introduced api experimenters groups #196 in omero-py 5.7.0

The first option is my least favorite one as we effectively block a component release for internal reasons while there are fixes that many users could consume immediately. I am equally fine with options 2 and 3.

will-moore · 2020-06-10T10:15:25Z

I don't have a strong preference between those options.
Don't know how long #196 will take. No fixes requested yet...
Happy to open a PR to add a 5.7.0 changelog or make chown "internal" (by just prefixing with _?)...

will-moore added 2 commits March 5, 2020 12:16

Add conn.chownObjects() function

c048117

Remove unused skiphead option from conn.chownObjects()

7b08901

will-moore mentioned this pull request Mar 6, 2020

Chown tests ome/openmicroscopy#6223

Merged

will-moore mentioned this pull request Mar 12, 2020

Chown webclient ome/omero-web#149

Merged

mtbc reviewed Mar 16, 2020

View reviewed changes

src/omero/gateway/__init__.py Show resolved Hide resolved

Improve conn.chownObjects() docstring

91cb5dd

close callback

a5dbda1

manics mentioned this pull request Mar 17, 2020

Remove pytest-sugar (incompatible pytest change) #198

Merged

Remove pytest-sugar from tox.ini

9f9130b

joshmoore closed this Apr 3, 2020

joshmoore reopened this Apr 3, 2020

joshmoore merged commit cc791c2 into ome:master Jun 3, 2020

sbesson added this to the 5.6.3 milestone Jun 9, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Chown gateway #195

Chown gateway #195

will-moore commented Mar 6, 2020 •

edited

Loading

joshmoore commented Mar 6, 2020

will-moore commented Mar 17, 2020

joshmoore commented Mar 17, 2020

will-moore commented Mar 17, 2020

manics commented Mar 17, 2020

will-moore commented Mar 18, 2020

joshmoore commented Mar 18, 2020

joshmoore commented Apr 3, 2020

joshmoore commented Apr 3, 2020

sbesson commented Jun 9, 2020

will-moore commented Jun 9, 2020

sbesson commented Jun 9, 2020

will-moore commented Jun 10, 2020

Chown gateway #195

Chown gateway #195

Conversation

will-moore commented Mar 6, 2020 • edited Loading

joshmoore commented Mar 6, 2020

will-moore commented Mar 17, 2020

joshmoore commented Mar 17, 2020

will-moore commented Mar 17, 2020

manics commented Mar 17, 2020

will-moore commented Mar 18, 2020

joshmoore commented Mar 18, 2020

joshmoore commented Apr 3, 2020

joshmoore commented Apr 3, 2020

sbesson commented Jun 9, 2020

will-moore commented Jun 9, 2020

sbesson commented Jun 9, 2020

will-moore commented Jun 10, 2020

will-moore commented Mar 6, 2020 •

edited

Loading