Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 0.0.1 #13

Merged
merged 87 commits into from
Apr 7, 2022
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
50d4674
adding memory
Mar 11, 2022
b5fe070
added missing errors
Mar 11, 2022
2b102ea
adding utils
Mar 11, 2022
b75bdeb
added basic Makefile
Mar 11, 2022
99854f8
doc work with py 3.6
Mar 11, 2022
b4d84f6
trying to unlock CI
Mar 11, 2022
f896453
reverting
Mar 11, 2022
485d6ba
added in memory transport
Mar 11, 2022
4d4f7fa
renamed
Mar 14, 2022
fab0aba
renaming
Mar 14, 2022
cc4b422
add in_memory and base transport
Mar 15, 2022
2cf73bf
fixed transport tests
Mar 15, 2022
36d31fb
format
Mar 15, 2022
d2afab0
added models
Mar 15, 2022
88f05f2
memory no longer used
Mar 15, 2022
bbbd1d6
reformat
Mar 15, 2022
ab0972e
added tests for commandreply
Mar 15, 2022
fdc0e9b
added zmq base transport
Mar 15, 2022
bed0724
first base working version
Mar 15, 2022
a78d557
updated dependencies
Mar 15, 2022
c07e06e
codestyle
Mar 15, 2022
ff2e3d2
refactor models
Mar 16, 2022
ddcdc4b
more models extension
Mar 16, 2022
0333bdf
adds parameter validation to manifest
Mar 16, 2022
2b03027
models extention
Mar 16, 2022
6adc09c
base working example
Mar 16, 2022
7e8fea2
working version
Mar 16, 2022
e7db3e1
removed unused errors
Mar 16, 2022
712e85d
refator renaming
Mar 16, 2022
b12239c
renamed methods
Mar 16, 2022
04a7dd3
add simple example
Mar 16, 2022
1602e54
end of line
Mar 16, 2022
677f866
updated codestyle
Mar 16, 2022
4663129
pre-commit checks enabled
Mar 16, 2022
3d4760a
fixed issues with mypy
Mar 16, 2022
9d89009
trying to fix mypy errors
Mar 16, 2022
4afb8cd
fix mypy locally
Mar 16, 2022
e49dda6
making docs build on py.36 in CI
Mar 16, 2022
db24b5d
fix CI for noxfile
Mar 16, 2022
b7752b8
this should fix typing issues with noxfile
Mar 16, 2022
7b24392
fixed test_transport
Mar 16, 2022
2081bfe
adding base test_core
Mar 16, 2022
97338b9
fixed coverage
Mar 16, 2022
fd16826
fixed mypy
Mar 16, 2022
faac9f5
raised timeout for macos in CI
Mar 16, 2022
3d70110
updated target code coverage
Mar 16, 2022
abc0577
added some examples
Mar 16, 2022
411b0e3
added missing example files
Mar 16, 2022
206a673
bumped version
Mar 17, 2022
29ac0f8
refactor docs
Mar 17, 2022
e1ace81
fix typo
Mar 17, 2022
e5e2825
fix readme
Mar 31, 2022
d21022c
added context manager support
Mar 31, 2022
7d0d7aa
refacto rename
Mar 31, 2022
3151a18
more refactoring
Mar 31, 2022
e5d01e8
refactor and optimized
Mar 31, 2022
868f572
typo
Mar 31, 2022
5e5adea
enhanced request uniquness by adding a session id
Apr 1, 2022
1940698
added validation
Apr 6, 2022
2b6cbd3
improve parsing of commands
Apr 6, 2022
e30613d
improvde comments
Apr 6, 2022
5b2bab0
renamed ControlInterface to PairedTransmitter
Apr 6, 2022
cf911c7
renamed exposed_interface to exposed_commands
Apr 6, 2022
adf3fd5
making tests more readbale
Apr 6, 2022
c01d89d
timeout has to be explicit when calling fucntiuon
Apr 6, 2022
586020a
refactored the simple example
Apr 6, 2022
1af657c
refactored the simple example
Apr 6, 2022
4c07c16
updated examples
Apr 6, 2022
c147041
added missing examples
Apr 6, 2022
2453207
add examples tester
Apr 6, 2022
77bbe64
add examples tester
Apr 6, 2022
62a26c6
add comment to test
Apr 6, 2022
c824be8
give more time to deliver messages
Apr 6, 2022
3e5d80e
added extra test
Apr 6, 2022
1735dd6
replaced sphinx with docsify
Apr 7, 2022
fe0c6b1
added empty file
Apr 7, 2022
3d51dae
Merge branch 'master' into initial-version
GitHK Apr 7, 2022
7ac3fa3
removed file
Apr 7, 2022
f0349d5
updated docspage link
Apr 7, 2022
dad8dd2
removed unused
Apr 7, 2022
4b5a480
updated docs links and purged unused badge
Apr 7, 2022
49c6943
added docs links on readme
Apr 7, 2022
d9da532
added some more general usage fixtures
Apr 7, 2022
9fbadc3
refacored test structure
Apr 7, 2022
08c698b
new test structure
Apr 7, 2022
ff83b6b
updated coverage badge
Apr 7, 2022
35a5dad
add delay when booting services
Apr 7, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[flake8]
select = B,B9,C,D,DAR,E,F,N,RST,S,W
select = B,B9,C,DAR,E,F,N,RST,S,W
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did you choose flake? How does it compare to what we already know/have ?
My understanding is that this is covered by black and pylint

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it already came in bundled with flake8, it does the same job.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these entries are amazingly clear 🤣

ignore = E203,E501,RST201,RST203,RST301,W503
max-line-length = 80
max-complexity = 10
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
- { python: "3.10", os: "macos-latest", session: "tests" }
- { python: "3.10", os: "ubuntu-latest", session: "typeguard" }
- { python: "3.10", os: "ubuntu-latest", session: "xdoctest" }
- { python: "3.10", os: "ubuntu-latest", session: "docs-build" }
- { python: "3.6", os: "ubuntu-latest", session: "docs-build" }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand the need to use nox together with github actions. in my opinion this makes the whole thing more complicated. I don't see what this brings.

for instance such a matrix would do exactly the same

matrix:
        python: [3.6, 3.7, 3.8, 3.9, 3.10]
        os: [ubuntu-latest, macos-latest, windows-latest]

and you would create 3 jobs which would be more explicit.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree.

Moreover, i thought this library was mainly going to be used in osparc services? why ensuring support in macos and windows?


env:
NOXSESSION: ${{ matrix.session }}
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@
/docs/_build/
/src/*.egg-info/
__pycache__/
*ignore*
42 changes: 42 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Heps setup basic env development
GitHK marked this conversation as resolved.
Show resolved Hide resolved

# poetry is required on your system
# suggested installation method
# or refer to official docs
# https://python-poetry.org/docs/
.PHONY: install-poetry
install-poetry:
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -

# install deve dependecis as sugested by coockicutter
GitHK marked this conversation as resolved.
Show resolved Hide resolved
# https://cookiecutter-hypermodern-python.readthedocs.io/en/2021.11.26/quickstart.html
.PHONY: install-dev
install-dev:
pip install nox nox-poetry

.PHONY: tests
tests: # run tests on lowest python interpreter
nox -r -s tests -p 3.6

.PHONY: nox-36
nox-36: # runs nox with python 3.6
nox -p 3.6

.PHONY: tests-dev
tests-dev:
pytest -vv -s --exitfirst --failed-first --pdb tests/

.PHONY: docs
docs: # runs and displays docs
#runs with py3.6 change the noxfile.py to use different interpreter version
nox -r -s docs


.PHONY: codestyle
codestyle: # runs codestyle enforcement
isort .
black .

.PHONY: mypy
mypy: # runs mypy
nox -p 3.6 -r -s mypy
8 changes: 1 addition & 7 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,6 @@ Features
* TODO


Requirements
------------

* TODO


Installation
------------

Expand All @@ -61,7 +55,7 @@ You can install *Osparc Control* via pip_ from PyPI_:
Usage
-----

Please see the `Command-line Reference <Usage_>`_ for details.
Please have a look at the examples folder.


Contributing
Expand Down
4 changes: 2 additions & 2 deletions codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ coverage:
status:
project:
default:
target: "100"
target: "99"
patch:
default:
target: "100"
target: "99"
1 change: 0 additions & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
"""Sphinx configuration."""
from datetime import datetime


project = "Osparc Control"
author = "Andrei Neagu"
copyright = f"{datetime.now().year}, {author}"
Expand Down
15 changes: 14 additions & 1 deletion docs/reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,18 @@ Reference
osparc_control
--------------

.. automodule:: osparc_control
.. autoclass:: osparc_control.ControlInterface
:members:
:undoc-members:

.. autoclass:: osparc_control.CommandManifest
:members:
:undoc-members:

.. autoclass:: osparc_control.CommandParameter
:members:
:undoc-members:

.. autoclass:: osparc_control.CommnadType
:members:
:undoc-members:
20 changes: 17 additions & 3 deletions docs/usage.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,20 @@
Usage
=====

.. click:: osparc_control.__main__:main
:prog: osparc-control
:nested: full
Please have a look at the examples folder.


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restructured text is definitively a very rich format for doc that we have used it extensively in s4l manuals. There it was necessary because of the complexity of the doc (>200+ pages !) and due to the lack of maturity of other solutions at the time .

Nonetheless, I am not sure it is really justified here. So far all python repos have been using two engines (docsify and mkdocs) both based in markdown doc (which is different from restructured).

Adding restructured to the equation requires to learn yet another tool and language ...and I am not convinced it is really justified here. Can you perhaps point out what is the advantage wrt markdown here? I guess also @KZzizzle should have a say here as well provided most of the doc has been created by her.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would imagine that the user documentation will pull from but won't necessarily be the same as this. It's up to you guys what type of documentation is most idea here.

Minimual examples
=================

In one terminal run ``python examples/basic/requester.py``:

.. literalinclude:: ../examples/basic/requester.py
:language: python



In a second terminal run ``python examples/basic/replyer.py``:

.. literalinclude:: ../examples/basic/replyer.py
:language: python
8 changes: 8 additions & 0 deletions examples/base_time_add/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# About

This example consists of a `time_solver`. Which can add, the current time by a provided value.

# Usage

In one terminal run `sidecar_controller.py`.
In a second terminal run `time_solver.py`. It will load data from the `sidecar_solver.py` to use when communicating with the `sidecar_controller.py`
37 changes: 37 additions & 0 deletions examples/base_time_add/sidecar_controller.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
from osparc_control import ControlInterface


control_interface = ControlInterface(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we not call this module a sidecar? this is not following the sidecar pattern in any way is it?
Also, I am not sure I understand which one is the Controller which one is the Controlled? An interface is an abstraction and here you create a concrete implementation right?
Maybe something like RemoteControl as an analogy to the TV being remote controlled? maybe @KZzizzle @elisabettai have some suggestions there?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for me, the anming inside the examples is not super important - I understand enough about what is happening but also have no problem with the rename.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe instead of control_interface you could call it controlled_solver ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've renamed this to PairedTransmitter, which is the best thing that came to mind. I'm still open for discussion and possible name changes.

remote_host="localhost", exposed_interface=[], remote_port=1234, listen_port=1235
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we have some sensible defaults (for instance for the ports or so?) do we really need to set them each time?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually I see the ports are inversed. so what about having a RemoteControl or Controller implementation and a Controlled that already set up the ports automatically?

Also the remote_host is most of the time localhost, set it as default?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As said below ControlInterface was renamed in PairedTransmitter which makes more sense, but other names are welcomed.

In theory this PairedTransmitter has nothing to do with control. The name should be agnostic from the control terminology

)
control_interface.start_background_sync()
GitHK marked this conversation as resolved.
Show resolved Hide resolved

# add_internal_time

add_params = {"a": 10}
print("Will add ", add_params)
request_id = control_interface.request_with_delayed_reply(
"add_internal_time", params=add_params
)

has_result = False
result = None
while not has_result:
has_result, result = control_interface.check_for_reply(request_id=request_id)

print("result of addition", result)

# get_time

print("getting solver time")
solver_time = control_interface.request_with_immediate_reply("get_time", timeout=1.0)
random_int = control_interface.request_with_immediate_reply(
"random_in_range", timeout=1.0, params={"a": 1, "b": 3}
GitHK marked this conversation as resolved.
Show resolved Hide resolved
)
print("solver time", solver_time)

print("sending command to print internal status")
control_interface.request_without_reply("print_status")


control_interface.stop_background_sync()
36 changes: 36 additions & 0 deletions examples/base_time_add/sidecar_solver.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
from osparc_control import CommandManifest
from osparc_control import CommandParameter
from osparc_control import CommnadType
from osparc_control import ControlInterface


GitHK marked this conversation as resolved.
Show resolved Hide resolved
command_add = CommandManifest(
action="add_internal_time",
description="adds internal time to a provided paramter",
params=[
CommandParameter(name="a", description="param to add to internal time"),
],
command_type=CommnadType.WITH_DELAYED_REPLY,
)

command_get_time = CommandManifest(
action="get_time",
description="gets the time",
params=[],
command_type=CommnadType.WITH_IMMEDIATE_REPLY,
)

command_print_solver_status = CommandManifest(
action="print_status",
description="prints the status of the solver",
params=[],
command_type=CommnadType.WITHOUT_REPLY,
GitHK marked this conversation as resolved.
Show resolved Hide resolved
)


control_interface = ControlInterface(
GitHK marked this conversation as resolved.
Show resolved Hide resolved
remote_host="localhost",
exposed_interface=[command_add, command_get_time, command_print_solver_status],
remote_port=1235,
listen_port=1234,
)
75 changes: 75 additions & 0 deletions examples/base_time_add/time_solver.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
from time import sleep

from sidecar_solver import command_add
from sidecar_solver import command_get_time
from sidecar_solver import command_print_solver_status
from sidecar_solver import control_interface

from osparc_control.core import ControlInterface
from osparc_control.models import CommandRequest


def handle_inputs(time_solver: "TimeSolver", request: CommandRequest) -> None:
if request.action == command_add.action:
sum_result = time_solver._add(**request.params)
time_solver.control_interface.reply_to_command(
request_id=request.request_id, payload=sum_result
)
return

if request.action == command_get_time.action:
time_solver.control_interface.reply_to_command(
request_id=request.request_id, payload=time_solver.time
)
return

if request.action == command_print_solver_status.action:
print("Solver internal status", time_solver)
# finally exit
time_solver.can_continue = False
GitHK marked this conversation as resolved.
Show resolved Hide resolved
return


class TimeSolver:
def __init__(
self, initial_time: float, control_interface: ControlInterface
) -> None:
self.time = initial_time
self.control_interface = control_interface

# internal time tick
self.sleep_internal: float = 0.1
self.can_continue: bool = True

def __repr__(self) -> str:
return (
f"<{self.__class__.__name__} time={self.time}, "
f"sleep_interval={self.sleep_internal}>"
)

def _add(self, a: float) -> float:
return self.time + a

def run(self) -> None:
"""main loop of the solver"""
while self.can_continue:
# process incoming requests from remote
for request in self.control_interface.get_incoming_requests():
handle_inputs(time_solver=self, request=request)

# process internal stuff
self.time += 1
sleep(self.sleep_internal)


def main() -> None:
control_interface.start_background_sync()
GitHK marked this conversation as resolved.
Show resolved Hide resolved

solver = TimeSolver(initial_time=0, control_interface=control_interface)
solver.run()

control_interface.stop_background_sync()


if __name__ == "__main__":
main()
42 changes: 42 additions & 0 deletions examples/basic/replyer.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import random
import time

from osparc_control import CommandManifest
from osparc_control import CommandParameter
from osparc_control import CommnadType
from osparc_control import ControlInterface

random_in_range_manifest = CommandManifest(
action="random_in_range",
description="gets the time",
params=[
CommandParameter(name="a", description="lower bound for random numbers"),
CommandParameter(name="b", description="upper bound for random numbers"),
],
command_type=CommnadType.WITH_IMMEDIATE_REPLY,
)

control_interface = ControlInterface(
remote_host="localhost",
exposed_interface=[random_in_range_manifest],
remote_port=2346,
listen_port=2345,
)
control_interface.start_background_sync()

wait_for_requests = True
while wait_for_requests:
for command in control_interface.get_incoming_requests():
if command.action == random_in_range_manifest.action:
random_int = random.randint( # noqa: S311
command.params["a"], command.params["b"]
)
control_interface.reply_to_command(
request_id=command.request_id, payload=random_int
)
wait_for_requests = False

# allow for message to be delivered
time.sleep(0.01)

control_interface.stop_background_sync()
15 changes: 15 additions & 0 deletions examples/basic/requester.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
from osparc_control import ControlInterface

control_interface = ControlInterface(
remote_host="localhost", exposed_interface=[], remote_port=2345, listen_port=2346
)

control_interface.start_background_sync()

random_int = control_interface.request_with_immediate_reply(
"random_in_range", timeout=10.0, params={"a": 1, "b": 10}
)
print(random_int)
assert 1 <= random_int <= 10 # noqa: S101

control_interface.stop_background_sync()
Loading