diff --git a/.coveragerc b/.coveragerc index 472afaae6..01ac4162e 100644 --- a/.coveragerc +++ b/.coveragerc @@ -2,4 +2,5 @@ omit = pymodbus/repl/* pymodbus/internal/* - pymodbus/server/asyncio.py \ No newline at end of file + pymodbus/server/asyncio.py + pymodbus/server/reactive/* diff --git a/.gitignore b/.gitignore index 426321af3..c738736e8 100644 --- a/.gitignore +++ b/.gitignore @@ -37,3 +37,5 @@ test/__pycache__/ /doc/_build/ .pytest_cache/ **/.pymodhis +/build/ +/dist/ diff --git a/.travis.yml b/.travis.yml index ae4bc08e5..c83cf395d 100644 --- a/.travis.yml +++ b/.travis.yml @@ -20,11 +20,12 @@ before_install: install: # - scripts/travis.sh pip install pip-accel - - if [ $TRAVIS_OS_NAME = osx ]; then scripts/travis.sh pip install -U "\"setuptools<45"\"; else pip install -U setuptools --upgrade ; fi + - if [ $TRAVIS_OS_NAME = osx ]; then scripts/travis.sh pip install -U pip "\"setuptools<45"\"; else pip install -U pip setuptools --upgrade ; fi - scripts/travis.sh pip install coveralls - scripts/travis.sh pip install --requirement=requirements-checks.txt - scripts/travis.sh pip install --requirement=requirements-tests.txt - - scripts/travis.sh LC_ALL=C pip install . + - scripts/travis.sh LC_ALL=C pip install --upgrade . +# - scripts/travis.sh pip freeze --all script: # - scripts/travis.sh make check - scripts/travis.sh make test diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 37f0f5eee..0856cedbd 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -1,3 +1,34 @@ +version 2.5.0rc3 +---------------------------------------------------------- +* Minor fix in documentations +* Travis fix for Mac OSX +* Disable unnecessary deprecation warning while using async clients. +* Use Github actions for builds in favor of travis. + + +version 2.5.0rc2 +---------------------------------------------------------- +* Documentation updates +* Disable `strict` mode by default. +* Fix `ReportSlaveIdRequest` request +* Sparse datablock initialization updates. + +version 2.5.0rc1 +---------------------------------------------------------- +* Support REPL for modbus server (only python3 and asyncio) +* Fix REPL client for write requests +* Fix examples + * Asyncio server + * Asynchronous server (with custom datablock) + * Fix version info for servers +* Fix and enhancements to Tornado clients (seril and tcp) +* Fix and enhancements to Asyncio client and server +* Update Install instructions +* Synchronous client retry on empty and error enhancments +* Add new modbus state `RETRYING` +* Support runtime response manipulations for Servers +* Bug fixes with logging module in servers +* Asyncio modbus serial server support Version 2.4.0 ---------------------------------------------------------- diff --git a/Makefile b/Makefile index 92edfa795..954cb9470 100644 --- a/Makefile +++ b/Makefile @@ -42,11 +42,20 @@ check: install test: install @pip install --upgrade --quiet --requirement=requirements-tests.txt ifeq ($(PYVER),3.6) + $(info Running tests on $(PYVER)) + @pip install --upgrade pip --quiet @pytest --cov=pymodbus/ --cov-report term-missing test/test_server_asyncio.py test + @coverage report --fail-under=85 -i +else ifeq ($(PYVER),2.7) + $(info Running tests on $(PYVER)) + @pip install pip==20.3.4 --quiet + @pytest --cov-config=.coveragerc --cov=pymodbus/ --cov-report term-missing --ignore test/test_server_asyncio.py --ignore test/test_client_async_asyncio.py test @coverage report --fail-under=90 -i else - @pytest --cov=pymodbus/ --cov-report term-missing - @coverage report --fail-under=90 -i + $(info Running tests on $(PYVER)) + @pip install --upgrade pip --quiet + @pytest --cov=pymodbus/ --cov-report term-missing test + @coverage report --fail-under=85 -i endif tox: install diff --git a/README.rst b/README.rst index eb88622e7..cb1643d4a 100644 --- a/README.rst +++ b/README.rst @@ -7,7 +7,7 @@ PyModbus - A Python Modbus Stack .. image:: https://badges.gitter.im/Join%20Chat.svg :target: https://gitter.im/pymodbus_dev/Lobby .. image:: https://readthedocs.org/projects/pymodbus/badge/?version=latest - :target: http://pymodbus.readthedocs.io/en/async/?badge=latest + :target: http://pymodbus.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status .. image:: http://pepy.tech/badge/pymodbus :target: http://pepy.tech/project/pymodbus diff --git a/doc/INSTALL b/doc/INSTALL index 49a55b87f..0c04786a5 100644 --- a/doc/INSTALL +++ b/doc/INSTALL @@ -1,8 +1,8 @@ Requirements ------------- -* Python 2.3 or later. -* Python Twisted +* Python 2.7 or later. +* Python Twisted, Tornado or asyncio (For async client and server) * Pyserial On Windows pywin32 is recommended (this is built in to ActivePython, @@ -35,7 +35,7 @@ much easier to run with the nose package. With that installed, you can use either of the following:: python setup.py test - nosetests + pytest Building Documentation diff --git a/doc/source/library/REPL.md b/doc/source/library/REPL.md index 48a426993..9f4cd1818 100644 --- a/doc/source/library/REPL.md +++ b/doc/source/library/REPL.md @@ -200,7 +200,7 @@ result.raw Return raw result dict. ``` -Every command has auto suggetion on the arguments supported , supply arg and value are to be supplied in `arg=val` format. +Every command has auto suggestion on the arguments supported, arg and value are to be supplied in `arg=val` format. ``` > client.read_holding_registers count=4 address=9 unit=1 diff --git a/examples/common/README.rst b/examples/common/README.rst index 777ef82ac..0f9726399 100644 --- a/examples/common/README.rst +++ b/examples/common/README.rst @@ -92,11 +92,13 @@ the tools/nullmodem/linux directory:: sudo ./run +The third method is Generic Unix method below. + ------------------------------------------------------------ Windows ------------------------------------------------------------ -For Windows, simply use the com2com application that is in +For Windows, simply use the com0com application that is in the directory tools/nullmodem/windows. Instructions are included in the Readme.txt. diff --git a/examples/common/asynchronous_server.py b/examples/common/asynchronous_server.py index 15e9b70c2..4f3895cbc 100755 --- a/examples/common/asynchronous_server.py +++ b/examples/common/asynchronous_server.py @@ -9,7 +9,8 @@ """ # --------------------------------------------------------------------------- # # import the various server implementations -# --------------------------------------------------------------------------- # +# --------------------------------------------------------------------------- # +from pymodbus.version import version from pymodbus.server.asynchronous import StartTcpServer from pymodbus.server.asynchronous import StartUdpServer from pymodbus.server.asynchronous import StartSerialServer @@ -105,10 +106,10 @@ def run_async_server(): identity = ModbusDeviceIdentification() identity.VendorName = 'Pymodbus' identity.ProductCode = 'PM' - identity.VendorUrl = 'http://github.com/bashwork/pymodbus/' + identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'Pymodbus Server' identity.ModelName = 'Pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ----------------------------------------------------------------------- # # run the server you want diff --git a/examples/common/asyncio_server.py b/examples/common/asyncio_server.py index be34dad3d..ad2c7c5cf 100755 --- a/examples/common/asyncio_server.py +++ b/examples/common/asyncio_server.py @@ -12,6 +12,7 @@ # import the various server implementations # --------------------------------------------------------------------------- # import asyncio +from pymodbus.version import version from pymodbus.server.async_io import StartTcpServer from pymodbus.server.async_io import StartTlsServer from pymodbus.server.async_io import StartUdpServer @@ -107,22 +108,22 @@ async def run_server(): identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'Pymodbus Server' identity.ModelName = 'Pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ----------------------------------------------------------------------- # # run the server you want # ----------------------------------------------------------------------- # # Tcp: # immediately start serving: - await StartTcpServer(context, identity=identity, address=("0.0.0.0", 5020), allow_reuse_address=True, - defer_start=False) + # await StartTcpServer(context, identity=identity, address=("0.0.0.0", 5020), allow_reuse_address=True, + # defer_start=False) # deferred start: - # server = await StartTcpServer(context, identity=identity, address=("0.0.0.0", 5020), - # allow_reuse_address=True, defer_start=True) - # - # asyncio.get_event_loop().call_later(20, lambda : server.serve_forever) - # await server.serve_forever() + server = await StartTcpServer(context, identity=identity, address=("0.0.0.0", 5020), + allow_reuse_address=True, defer_start=True) + + asyncio.get_event_loop().call_later(20, lambda: server.serve_forever) + await server.serve_forever() # TCP with different framer # StartTcpServer(context, identity=identity, diff --git a/examples/common/callback_server.py b/examples/common/callback_server.py index 325fbca56..60e65ba96 100755 --- a/examples/common/callback_server.py +++ b/examples/common/callback_server.py @@ -10,6 +10,7 @@ # --------------------------------------------------------------------------- # # import the modbus libraries we need # --------------------------------------------------------------------------- # +from pymodbus.version import version from pymodbus.server.asynchronous import StartTcpServer from pymodbus.device import ModbusDeviceIdentification from pymodbus.datastore import ModbusSparseDataBlock @@ -129,10 +130,10 @@ def run_callback_server(): identity = ModbusDeviceIdentification() identity.VendorName = 'pymodbus' identity.ProductCode = 'PM' - identity.VendorUrl = 'http://github.com/bashwork/pymodbus/' + identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'pymodbus Server' identity.ModelName = 'pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ----------------------------------------------------------------------- # # run the server you want diff --git a/examples/common/custom_datablock.py b/examples/common/custom_datablock.py index 350a76abe..f59a4e2fb 100755 --- a/examples/common/custom_datablock.py +++ b/examples/common/custom_datablock.py @@ -10,6 +10,7 @@ # import the modbus libraries we need # --------------------------------------------------------------------------- # from __future__ import print_function +from pymodbus.version import version from pymodbus.server.asynchronous import StartTcpServer from pymodbus.device import ModbusDeviceIdentification from pymodbus.datastore import ModbusSparseDataBlock @@ -65,10 +66,10 @@ def run_custom_db_server(): identity = ModbusDeviceIdentification() identity.VendorName = 'pymodbus' identity.ProductCode = 'PM' - identity.VendorUrl = 'http://github.com/bashwork/pymodbus/' + identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'pymodbus Server' identity.ModelName = 'pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ----------------------------------------------------------------------- # # run the server you want diff --git a/examples/common/custom_synchronous_server.py b/examples/common/custom_synchronous_server.py index 66f6f1b3c..78a271392 100755 --- a/examples/common/custom_synchronous_server.py +++ b/examples/common/custom_synchronous_server.py @@ -60,8 +60,8 @@ def decode(self, data): # --------------------------------------------------------------------------- # # import the various server implementations # --------------------------------------------------------------------------- # +from pymodbus.version import version from pymodbus.server.sync import StartTcpServer - from pymodbus.device import ModbusDeviceIdentification from pymodbus.datastore import ModbusSequentialDataBlock from pymodbus.datastore import ModbusSlaveContext, ModbusServerContext @@ -101,7 +101,7 @@ def run_server(): identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'Pymodbus Server' identity.ModelName = 'Pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ----------------------------------------------------------------------- # # run the server you want diff --git a/examples/common/dbstore_update_server.py b/examples/common/dbstore_update_server.py index ef467de0a..525375b8a 100644 --- a/examples/common/dbstore_update_server.py +++ b/examples/common/dbstore_update_server.py @@ -16,6 +16,7 @@ # --------------------------------------------------------------------------- # # import the modbus libraries we need # --------------------------------------------------------------------------- # +from pymodbus.version import version from pymodbus.server.asynchronous import StartTcpServer from pymodbus.device import ModbusDeviceIdentification from pymodbus.datastore import ModbusSequentialDataBlock @@ -83,10 +84,10 @@ def run_dbstore_update_server(): identity = ModbusDeviceIdentification() identity.VendorName = 'pymodbus' identity.ProductCode = 'PM' - identity.VendorUrl = 'http://github.com/bashwork/pymodbus/' + identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'pymodbus Server' identity.ModelName = 'pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ----------------------------------------------------------------------- # # run the server you want diff --git a/examples/common/modbus_payload.py b/examples/common/modbus_payload.py index ea31e78fe..a9204f3b4 100755 --- a/examples/common/modbus_payload.py +++ b/examples/common/modbus_payload.py @@ -3,7 +3,7 @@ Pymodbus Payload Building/Decoding Example -------------------------------------------------------------------------- -# Run modbus-payload-server.py or synchronous-server.py to check the behavior +# Run modbus_payload_server.py or synchronous_server.py to check the behavior """ from pymodbus.constants import Endian from pymodbus.payload import BinaryPayloadDecoder diff --git a/examples/common/modbus_payload_server.py b/examples/common/modbus_payload_server.py index 2fac2209a..6d8c5b25d 100755 --- a/examples/common/modbus_payload_server.py +++ b/examples/common/modbus_payload_server.py @@ -9,8 +9,8 @@ # --------------------------------------------------------------------------- # # import the various server implementations # --------------------------------------------------------------------------- # +from pymodbus.version import version from pymodbus.server.sync import StartTcpServer - from pymodbus.device import ModbusDeviceIdentification from pymodbus.datastore import ModbusSequentialDataBlock from pymodbus.datastore import ModbusSlaveContext, ModbusServerContext @@ -77,10 +77,10 @@ def run_payload_server(): identity = ModbusDeviceIdentification() identity.VendorName = 'Pymodbus' identity.ProductCode = 'PM' - identity.VendorUrl = 'http://github.com/bashwork/pymodbus/' + identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'Pymodbus Server' identity.ModelName = 'Pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ----------------------------------------------------------------------- # # run the server you want # ----------------------------------------------------------------------- # diff --git a/examples/common/synchronous_server.py b/examples/common/synchronous_server.py index d2bfaf2a6..4266fac23 100755 --- a/examples/common/synchronous_server.py +++ b/examples/common/synchronous_server.py @@ -11,6 +11,7 @@ # --------------------------------------------------------------------------- # # import the various server implementations # --------------------------------------------------------------------------- # +from pymodbus.version import version from pymodbus.server.sync import StartTcpServer from pymodbus.server.sync import StartTlsServer from pymodbus.server.sync import StartUdpServer @@ -106,7 +107,7 @@ def run_server(): identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'Pymodbus Server' identity.ModelName = 'Pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ----------------------------------------------------------------------- # # run the server you want diff --git a/examples/common/updating_server.py b/examples/common/updating_server.py index b5b04faa3..1894e159b 100755 --- a/examples/common/updating_server.py +++ b/examples/common/updating_server.py @@ -15,6 +15,7 @@ # --------------------------------------------------------------------------- # # import the modbus libraries we need # --------------------------------------------------------------------------- # +from pymodbus.version import version from pymodbus.server.asynchronous import StartTcpServer from pymodbus.device import ModbusDeviceIdentification from pymodbus.datastore import ModbusSequentialDataBlock @@ -75,10 +76,10 @@ def run_updating_server(): identity = ModbusDeviceIdentification() identity.VendorName = 'pymodbus' identity.ProductCode = 'PM' - identity.VendorUrl = 'http://github.com/bashwork/pymodbus/' + identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'pymodbus Server' identity.ModelName = 'pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ----------------------------------------------------------------------- # # run the server you want diff --git a/examples/contrib/deviceinfo_showcase_server.py b/examples/contrib/deviceinfo_showcase_server.py index 983bb7111..28a9d0431 100755 --- a/examples/contrib/deviceinfo_showcase_server.py +++ b/examples/contrib/deviceinfo_showcase_server.py @@ -10,7 +10,8 @@ """ # --------------------------------------------------------------------------- # # import the various server implementations -# --------------------------------------------------------------------------- # +# --------------------------------------------------------------------------- # +from pymodbus.version import version from pymodbus.server.sync import StartTcpServer from pymodbus.server.sync import StartUdpServer from pymodbus.server.sync import StartSerialServer @@ -55,7 +56,7 @@ def run_server(): identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'Pymodbus Server' identity.ModelName = 'Pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ----------------------------------------------------------------------- # # Add an example which is long enough to force the ReadDeviceInformation diff --git a/examples/contrib/message_parser.py b/examples/contrib/message_parser.py index 73d109931..f7539df32 100755 --- a/examples/contrib/message_parser.py +++ b/examples/contrib/message_parser.py @@ -135,7 +135,7 @@ def get_options(): parser.add_option("-a", "--ascii", help="The indicates that the message is ascii", - action="store_true", dest="ascii", default=True) + action="store_true", dest="ascii", default=False) parser.add_option("-b", "--binary", help="The indicates that the message is binary", @@ -148,6 +148,9 @@ def get_options(): parser.add_option("-t", "--transaction", help="If the incoming message is in hexadecimal format", action="store_true", dest="transaction", default=False) + parser.add_option("--framer", + help="Framer to use", dest="framer", default=None, + ) (opt, arg) = parser.parse_args() @@ -195,7 +198,7 @@ def main(): if option.debug: try: - modbus_log.setLevel(logging.DEBUG) + log.setLevel(logging.DEBUG) logging.basicConfig() except Exception as e: print("Logging is not supported on this system- {}".format(e)) @@ -205,7 +208,7 @@ def main(): 'rtu': ModbusRtuFramer, 'binary': ModbusBinaryFramer, 'ascii': ModbusAsciiFramer, - }.get(option.parser, ModbusSocketFramer) + }.get(option.framer or option.parser, ModbusSocketFramer) decoder = Decoder(framer, option.ascii) for message in get_messages(option): diff --git a/examples/gui/bottle/frontend.py b/examples/gui/bottle/frontend.py index 3e79e0b46..929c76252 100644 --- a/examples/gui/bottle/frontend.py +++ b/examples/gui/bottle/frontend.py @@ -6,6 +6,7 @@ This can be hosted using any wsgi adapter. """ from __future__ import print_function +from pymodbus.version import version import json, inspect from bottle import route, request, Bottle from bottle import static_file @@ -274,10 +275,10 @@ def RunDebugModbusFrontend(server, port=8080): identity = ModbusDeviceIdentification() identity.VendorName = 'Pymodbus' identity.ProductCode = 'PM' - identity.VendorUrl = 'http://github.com/bashwork/pymodbus/' + identity.VendorUrl = 'http://github.com/riptideio/pymodbus/' identity.ProductName = 'Pymodbus Server' identity.ModelName = 'Pymodbus Server' - identity.MajorMinorRevision = '2.3.0' + identity.MajorMinorRevision = version.short() # ------------------------------------------------------------ # initialize the datastore diff --git a/pymodbus/bit_read_message.py b/pymodbus/bit_read_message.py index d8624fa16..01434920d 100644 --- a/pymodbus/bit_read_message.py +++ b/pymodbus/bit_read_message.py @@ -118,7 +118,7 @@ def __str__(self): :returns: A string representation of the instance ''' - return "ReadBitResponse(%d)" % len(self.bits) + return "%s(%d)" % (self.__class__.__name__, len(self.bits)) class ReadCoilsRequest(ReadBitsRequestBase): diff --git a/pymodbus/client/asynchronous/__init__.py b/pymodbus/client/asynchronous/__init__.py index c339353d5..b7d084de3 100644 --- a/pymodbus/client/asynchronous/__init__.py +++ b/pymodbus/client/asynchronous/__init__.py @@ -37,8 +37,7 @@ if installed: # Import deprecated async client only if twisted is installed #338 from pymodbus.client.asynchronous.deprecated.asynchronous import * -else: import logging logger = logging.getLogger(__name__) - logger.warning("Not Importing deprecated clients. " - "Dependency Twisted is not Installed") + logger.warning("Importing deprecated clients. " + "Dependency Twisted is Installed") diff --git a/pymodbus/client/asynchronous/tornado/__init__.py b/pymodbus/client/asynchronous/tornado/__init__.py index 29e0e4db2..8eb61f4ef 100644 --- a/pymodbus/client/asynchronous/tornado/__init__.py +++ b/pymodbus/client/asynchronous/tornado/__init__.py @@ -315,7 +315,7 @@ def __init__(self, *args, **kwargs): self.silent_interval = 3.5 * self._t0 self.silent_interval = round(self.silent_interval, 6) self.last_frame_end = 0.0 - super().__init__(*args, **kwargs) + super(AsyncModbusSerialClient, self).__init__(*args, **kwargs) def get_socket(self): """ @@ -459,7 +459,8 @@ def sleep(timeout): LOGGER.info( "Cleanup recv buffer before send: " + hexlify_packets(result)) except OSError as e: - self.transaction.getTransaction(request.transaction_id).set_exception(ModbusIOException(e)) + self.transaction.getTransaction( + message.transaction_id).set_exception(ModbusIOException(e)) return start = time.time() diff --git a/pymodbus/client/sync.py b/pymodbus/client/sync.py index 8b0b832b6..e4e083c7d 100644 --- a/pymodbus/client/sync.py +++ b/pymodbus/client/sync.py @@ -72,8 +72,9 @@ def is_socket_open(self): ) def send(self, request): - _logger.debug("New Transaction state 'SENDING'") - self.state = ModbusTransactionState.SENDING + if self.state != ModbusTransactionState.RETRYING: + _logger.debug("New Transaction state 'SENDING'") + self.state = ModbusTransactionState.SENDING return self._send(request) def _send(self, request): @@ -204,12 +205,15 @@ def connect(self): :returns: True if connection succeeded, False otherwise """ - if self.socket: return True + if self.socket: + return True try: self.socket = socket.create_connection( (self.host, self.port), timeout=self.timeout, source_address=self.source_address) + _logger.debug("Connection to Modbus server established. " + "Socket {}".format(self.socket.getsockname())) except socket.error as msg: _logger.error('Connection to (%s, %s) ' 'failed: %s' % (self.host, self.port, msg)) @@ -223,6 +227,16 @@ def close(self): self.socket.close() self.socket = None + def _check_read_buffer(self, recv_size=None): + time_ = time.time() + end = time_ + self.timeout + data = None + data_length = 0 + ready = select.select([self.socket], [], [], end - time_) + if ready[0]: + data = self.socket.recv(1024) + return data + def _send(self, request): """ Sends data on the underlying socket @@ -231,6 +245,11 @@ def _send(self, request): """ if not self.socket: raise ConnectionException(self.__str__()) + if self.state == ModbusTransactionState.RETRYING: + data = self._check_read_buffer() + if data: + return data + if request: return self.socket.send(request) return 0 @@ -494,7 +513,9 @@ def _recv(self, size): return self.socket.recvfrom(size)[0] def is_socket_open(self): - return True if self.socket is not None else False + if self.socket: + return True + return self.connect() def __str__(self): """ Builds a string representation of the connection @@ -552,7 +573,7 @@ def __init__(self, method='ascii', **kwargs): self.parity = kwargs.get('parity', Defaults.Parity) self.baudrate = kwargs.get('baudrate', Defaults.Baudrate) self.timeout = kwargs.get('timeout', Defaults.Timeout) - self._strict = kwargs.get("strict", True) + self._strict = kwargs.get("strict", False) self.last_frame_end = None self.handle_local_echo = kwargs.get("handle_local_echo", False) if self.method == "rtu": @@ -641,12 +662,19 @@ def _send(self, request): waitingbytes = self._in_waiting() if waitingbytes: result = self.socket.read(waitingbytes) + if self.state == ModbusTransactionState.RETRYING: + _logger.debug("Sending available data in recv " + "buffer {}".format( + hexlify_packets(result))) + return result if _logger.isEnabledFor(logging.WARNING): _logger.warning("Cleanup recv buffer before " "send: " + hexlify_packets(result)) except NotImplementedError: pass - + if self.state != ModbusTransactionState.SENDING: + _logger.debug("New Transaction state 'SENDING'") + self.state = ModbusTransactionState.SENDING size = self.socket.write(request) return size return 0 diff --git a/pymodbus/datastore/store.py b/pymodbus/datastore/store.py index 2d99f1a11..13acadd56 100644 --- a/pymodbus/datastore/store.py +++ b/pymodbus/datastore/store.py @@ -190,33 +190,59 @@ def setValues(self, address, values): class ModbusSparseDataBlock(BaseModbusDataBlock): - ''' Creates a sparse modbus datastore ''' + """ + Creates a sparse modbus datastore - def __init__(self, values): - ''' Initializes the datastore + E.g Usage. + sparse = ModbusSparseDataBlock({10: [3, 5, 6, 8], 30: 1, 40: [0]*20}) + + This would create a datablock with 3 blocks starting at + offset 10 with length 4 , 30 with length 1 and 40 with length 20 + + sparse = ModbusSparseDataBlock([10]*100) + Creates a sparse datablock of length 100 starting at offset 0 and default value of 10 - Using the input values we create the default - datastore value and the starting address + sparse = ModbusSparseDataBlock() --> Create Empty datablock + sparse.setValues(0, [10]*10) --> Add block 1 at offset 0 with length 10 (default value 10) + sparse.setValues(30, [20]*5) --> Add block 2 at offset 30 with length 5 (default value 20) + + if mutable is set to True during initialization, the datablock can not be altered with + setValues (new datablocks can not be added) + """ + + def __init__(self, values=None, mutable=True): + """ + Initializes a sparse datastore. Will only answer to addresses + registered, either initially here, or later via setValues() :param values: Either a list or a dictionary of values - ''' - if isinstance(values, dict): - self.values = values - elif hasattr(values, '__iter__'): - self.values = dict(enumerate(values)) - else: raise ParameterException( - "Values for datastore must be a list or dictionary") - self.default_value = get_next(itervalues(self.values)).__class__() - self.address = get_next(iterkeys(self.values)) + :param mutable: The data-block can be altered later with setValues(i.e add more blocks) + + If values are list , This is as good as sequential datablock. + Values as dictionary should be in {offset: } format, if values + is a list, a sparse datablock is created starting at offset with the length of values. + If values is a integer, then the value is set for the corresponding offset. + + """ + self.values = {} + self._process_values(values) + self.mutable = mutable + self.default_value = self.values.copy() + self.address = get_next(iterkeys(self.values), None) @classmethod - def create(klass): - ''' Factory method to create a datastore with the - full address space initialized to 0x00 + def create(klass, values=None): + ''' Factory method to create sparse datastore. + Use setValues to initialize registers. + :param values: Either a list or a dictionary of values :returns: An initialized datastore ''' - return klass([0x00] * 65536) + return klass(values) + + def reset(self): + ''' Reset the store to the intially provided defaults''' + self.values = self.default_value.copy() def validate(self, address, count=1): ''' Checks to see if the request is in range @@ -239,17 +265,49 @@ def getValues(self, address, count=1): ''' return [self.values[i] for i in range(address, address + count)] - def setValues(self, address, values): + def _process_values(self, values): + def _process_as_dict(values): + for idx, val in iteritems(values): + if isinstance(val, (list, tuple)): + for i, v in enumerate(val): + self.values[idx + i] = v + else: + self.values[idx] = int(val) + if isinstance(values, dict): + _process_as_dict(values) + return + if hasattr(values, '__iter__'): + values = dict(enumerate(values)) + elif values is None: + values = {} # Must make a new dict here per instance + else: + raise ParameterException("Values for datastore must " + "be a list or dictionary") + _process_as_dict(values) + + def setValues(self, address, values, use_as_default=False): ''' Sets the requested values of the datastore :param address: The starting address :param values: The new values to be set + :param use_as_default: Use the values as default ''' if isinstance(values, dict): - for idx, val in iteritems(values): - self.values[idx] = val + new_offsets = list(set(list(values.keys())) - set(list(self.values.keys()))) + if new_offsets and not self.mutable: + raise ParameterException("Offsets {} not " + "in range".format(new_offsets)) + self._process_values(values) else: if not isinstance(values, list): values = [values] for idx, val in enumerate(values): + if address+idx not in self.values and not self.mutable: + raise ParameterException("Offset {} not " + "in range".format(address+idx)) self.values[address + idx] = val + if not self.address: + self.address = get_next(iterkeys(self.values), None) + if use_as_default: + for idx, val in iteritems(self.values): + self.default_value[idx] = val diff --git a/pymodbus/exceptions.py b/pymodbus/exceptions.py index 651666d8b..0a4b0f6dc 100644 --- a/pymodbus/exceptions.py +++ b/pymodbus/exceptions.py @@ -105,6 +105,7 @@ def __init__(self, string=""): message = '[Error registering message] %s' % string ModbusException.__init__(self, message) + class TimeOutException(ModbusException): """ Error resulting from modbus response timeout """ diff --git a/pymodbus/framer/rtu_framer.py b/pymodbus/framer/rtu_framer.py index c5fe5a616..b60efb1ea 100644 --- a/pymodbus/framer/rtu_framer.py +++ b/pymodbus/framer/rtu_framer.py @@ -282,6 +282,11 @@ def sendPacket(self, message): # Recovering from last error ?? time.sleep(self.client.silent_interval) self.client.state = ModbusTransactionState.IDLE + elif self.client.state == ModbusTransactionState.RETRYING: + # Simple lets settle down!!! + # To check for higher baudrates + time.sleep(self.client.timeout) + break else: if time.time() > timeout: _logger.debug("Spent more time than the read time out, " diff --git a/pymodbus/interfaces.py b/pymodbus/interfaces.py index d32e9978a..49e6939cd 100644 --- a/pymodbus/interfaces.py +++ b/pymodbus/interfaces.py @@ -180,7 +180,7 @@ def decode(self, fx): """ Converts the function code to the datastore to :param fx: The function we are working with - :returns: one of [d(iscretes),i(inputs),h(oliding),c(oils) + :returns: one of [d(iscretes),i(nputs),h(olding),c(oils) """ return self.__fx_mapper[fx] diff --git a/pymodbus/other_message.py b/pymodbus/other_message.py index c2774fe8d..31e6734bb 100644 --- a/pymodbus/other_message.py +++ b/pymodbus/other_message.py @@ -366,7 +366,7 @@ def execute(self, context=None): ''' reportSlaveIdData = None if context: - reportSlaveIdData = context.reportSlaveIdData + reportSlaveIdData = getattr(context, 'reportSlaveIdData', None) if not reportSlaveIdData: information = DeviceInformationFactory.get(_MCB) identifier = "-".join(information.values()).encode() diff --git a/pymodbus/register_read_message.py b/pymodbus/register_read_message.py index 0a202bb10..1c406ead3 100644 --- a/pymodbus/register_read_message.py +++ b/pymodbus/register_read_message.py @@ -102,7 +102,7 @@ def __str__(self): :returns: A string representation of the instance ''' - return "ReadRegisterResponse (%d)" % len(self.registers) + return "%s (%d)" % (self.__class__.__name__, len(self.registers)) class ReadHoldingRegistersRequest(ReadRegistersRequestBase): diff --git a/pymodbus/repl/client/__init__.py b/pymodbus/repl/client/__init__.py new file mode 100644 index 000000000..bc4e39484 --- /dev/null +++ b/pymodbus/repl/client/__init__.py @@ -0,0 +1,4 @@ +""" +Copyright (c) 2020 by RiptideIO +All rights reserved. +""" diff --git a/pymodbus/repl/completer.py b/pymodbus/repl/client/completer.py similarity index 97% rename from pymodbus/repl/completer.py rename to pymodbus/repl/client/completer.py index 391c245a1..426c5b29e 100644 --- a/pymodbus/repl/completer.py +++ b/pymodbus/repl/client/completer.py @@ -9,7 +9,7 @@ from prompt_toolkit.styles import Style from prompt_toolkit.filters import Condition from prompt_toolkit.application.current import get_app -from pymodbus.repl.helper import get_commands +from pymodbus.repl.client.helper import get_commands from pymodbus.compat import string_types @@ -33,7 +33,7 @@ class CmdCompleter(Completer): Completer for Pymodbus REPL. """ - def __init__(self, client, commands=None, ignore_case=True): + def __init__(self, client=None, commands=None, ignore_case=True): """ :param client: Modbus Client diff --git a/pymodbus/repl/helper.py b/pymodbus/repl/client/helper.py similarity index 100% rename from pymodbus/repl/helper.py rename to pymodbus/repl/client/helper.py diff --git a/pymodbus/repl/main.py b/pymodbus/repl/client/main.py similarity index 90% rename from pymodbus/repl/main.py rename to pymodbus/repl/client/main.py index 65e6efa77..bdfdea9e7 100644 --- a/pymodbus/repl/main.py +++ b/pymodbus/repl/client/main.py @@ -26,8 +26,8 @@ from prompt_toolkit.history import FileHistory from prompt_toolkit.auto_suggest import AutoSuggestFromHistory from pymodbus.version import version -from pymodbus.repl.completer import CmdCompleter, has_selected_completion -from pymodbus.repl.helper import Result, CLIENT_ATTRIBUTES +from pymodbus.repl.client.completer import CmdCompleter, has_selected_completion +from pymodbus.repl.client.helper import Result, CLIENT_ATTRIBUTES click.disable_unicode_literals_warning = True @@ -41,9 +41,9 @@ \/ \/ \/ \/ \/ \/|__| v{} - {} ---------------------------------------------------------------------------- -""".format("1.2.0", version) -log = None +""".format("1.3.0", version) +log = None style = Style.from_dict({ 'completion-menu.completion': 'bg:#008888 #ffffff', @@ -169,7 +169,7 @@ def _process_args(args, string=True): complete_while_typing=True, bottom_toolbar=bottom_toolbar, key_bindings=kb, - history=FileHistory('.pymodhis'), + history=FileHistory('../.pymodhis'), auto_suggest=AutoSuggestFromHistory()) click.secho("{}".format(TITLE), fg='green') result = None @@ -226,9 +226,16 @@ def _process_args(args, string=True): @click.group('pymodbus-repl') @click.version_option(version, message=TITLE) @click.option("--verbose", is_flag=True, default=False, help="Verbose logs") -@click.option("--broadcast-support", is_flag=True, default=False, help="Support broadcast messages") +@click.option("--broadcast-support", is_flag=True, default=False, + help="Support broadcast messages") +@click.option("--retry-on-empty", is_flag=True, default=False, + help="Retry on empty response") +@click.option("--retry-on-error", is_flag=True, default=False, + help="Retry on error response") +@click.option("--retries", default=3, help="Retry count") @click.pass_context -def main(ctx, verbose, broadcast_support): +def main(ctx, verbose, broadcast_support, retry_on_empty, + retry_on_error, retries): if verbose: global log import logging @@ -237,13 +244,19 @@ def main(ctx, verbose, broadcast_support): log = logging.getLogger('pymodbus') logging.basicConfig(format=format) log.setLevel(logging.DEBUG) - ctx.obj = {"broadcast": broadcast_support} + ctx.obj = { + "broadcast": broadcast_support, + "retry_on_empty": retry_on_empty, + "retry_on_invalid": retry_on_error, + "retries": retries + } @main.command("tcp") @click.pass_context @click.option( "--host", + default='localhost', help="Modbus TCP IP " ) @click.option( @@ -259,9 +272,9 @@ def main(ctx, verbose, broadcast_support): help="Override the default packet framer tcp|rtu", ) def tcp(ctx, host, port, framer): - from pymodbus.repl.client import ModbusTcpClient - broadcast = ctx.obj.get("broadcast") - kwargs = dict(host=host, port=port, broadcast_enable=broadcast) + from pymodbus.repl.client.mclient import ModbusTcpClient + kwargs = dict(host=host, port=port) + kwargs.update(**ctx.obj) if framer == 'rtu': from pymodbus.framer.rtu_framer import ModbusRtuFramer kwargs['framer'] = ModbusRtuFramer @@ -349,7 +362,7 @@ def tcp(ctx, host, port, framer): ) def serial(ctx, method, port, baudrate, bytesize, parity, stopbits, xonxoff, rtscts, dsrdtr, timeout, write_timeout): - from pymodbus.repl.client import ModbusSerialClient + from pymodbus.repl.client.mclient import ModbusSerialClient client = ModbusSerialClient(method=method, port=port, baudrate=baudrate, @@ -360,7 +373,8 @@ def serial(ctx, method, port, baudrate, bytesize, parity, stopbits, xonxoff, rtscts=rtscts, dsrdtr=dsrdtr, timeout=timeout, - write_timeout=write_timeout) + write_timeout=write_timeout, + **ctx.obj) cli(client) diff --git a/pymodbus/repl/client.py b/pymodbus/repl/client/mclient.py similarity index 97% rename from pymodbus/repl/client.py rename to pymodbus/repl/client/mclient.py index c219387cb..6c53230e2 100644 --- a/pymodbus/repl/client.py +++ b/pymodbus/repl/client/mclient.py @@ -5,7 +5,7 @@ """ from __future__ import absolute_import, unicode_literals - +import functools from pymodbus.pdu import ModbusExceptions, ExceptionResponse from pymodbus.exceptions import ModbusIOException from pymodbus.client.sync import ModbusSerialClient as _ModbusSerialClient @@ -35,7 +35,23 @@ GetClearModbusPlusRequest) +def make_response_dict(resp): + rd = { + 'function_code': resp.function_code, + 'address': resp.address + } + if hasattr(resp, "value"): + rd['value'] = resp.value + elif hasattr(resp, 'values'): + rd['values'] = resp.values + elif hasattr(resp, 'count'): + rd['count'] = resp.count + + return rd + + def handle_brodcast(func): + @functools.wraps(func) def _wrapper(*args, **kwargs): self = args[0] resp = func(*args, **kwargs) @@ -44,11 +60,7 @@ def _wrapper(*args, **kwargs): 'broadcasted': True } if not resp.isError(): - return { - 'function_code': resp.function_code, - 'address': resp.address, - 'count': resp.count - } + return make_response_dict(resp) else: return ExtendedRequestSupport._process_exception(resp, **kwargs) return _wrapper @@ -143,7 +155,7 @@ def write_coils(self, address, values, **kwargs): Write `value` to coil at `address`. :param address: coil offset to write to - :param value: list of bit values to write (comma seperated) + :param values: list of bit values to write (comma seperated) :param unit: The slave unit this request is targeting :return: """ @@ -171,7 +183,7 @@ def write_registers(self, address, values, **kwargs): Write list of `values` to registers starting at `address`. :param address: register offset to write to - :param value: list of register value to write (comma seperated) + :param values: list of register value to write (comma seperated) :param unit: The slave unit this request is targeting :return: """ diff --git a/pymodbus/repl/server/__init__.py b/pymodbus/repl/server/__init__.py new file mode 100644 index 000000000..bc4e39484 --- /dev/null +++ b/pymodbus/repl/server/__init__.py @@ -0,0 +1,4 @@ +""" +Copyright (c) 2020 by RiptideIO +All rights reserved. +""" diff --git a/pymodbus/repl/server/cli.py b/pymodbus/repl/server/cli.py new file mode 100644 index 000000000..6e1c0db17 --- /dev/null +++ b/pymodbus/repl/server/cli.py @@ -0,0 +1,188 @@ +""" +Copyright (c) 2020 by RiptideIO +All rights reserved. +""" +import json +import click +import shutil +import logging + +from prompt_toolkit.shortcuts import clear +from prompt_toolkit.shortcuts.progress_bar import formatters +from prompt_toolkit.styles import Style + +from prompt_toolkit import PromptSession, print_formatted_text +from prompt_toolkit.patch_stdout import patch_stdout +from prompt_toolkit.completion import NestedCompleter +from prompt_toolkit.formatted_text import HTML + + +logger = logging.getLogger(__name__) + +TITLE = """ +__________ .______. _________ +\______ \___.__. _____ ____ __| _/\_ |__ __ __ ______ / _____/ ______________ __ ___________ + | ___< | |/ \ / _ \ / __ | | __ \| | \/ ___/ \_____ \_/ __ \_ __ \ \/ // __ \_ __ \\ + | | \___ | Y Y ( <_> ) /_/ | | \_\ \ | /\___ \ / \ ___/| | \/\ /\ ___/| | \/ + |____| / ____|__|_| /\____/\____ | |___ /____//____ > /_______ /\___ >__| \_/ \___ >__| + \/ \/ \/ \/ \/ \/ \/ \/""" + +SMALL_TITLE = "Pymodbus server..." +BOTTOM_TOOLBAR = HTML('(MODBUS SERVER) Type "help" ' + 'for list of available commands') +COMMAND_ARGS = ["response_type", "error_code", "delay_by", "clear_after"] +RESPONSE_TYPES = ["normal", "error", "delayed"] +COMMANDS = { + "manipulator": { + "response_type": None, + "error_code": None, + "delay_by": None, + "clear_after": None + }, + "exit": None, + "help": None, + "clear": None +} +USAGE = "manipulate response_type=|normal|error|delayed| " \ + "error_code=<int> delay_by=<in seconds>" +COMMAND_HELPS = { + "manipulator": "Manipulate response from server.\nUsage: '{}'".format(USAGE), + "clear": "Clears screen" + +} + + +STYLE = Style.from_dict({"": "cyan"}) +CUSTOM_FORMATTERS = [ + formatters.Label(suffix=": "), + formatters.Bar(start="|", end="|", sym_a="#", sym_b="#", sym_c="-"), + formatters.Text(" "), + formatters.Text(" "), + formatters.TimeElapsed(), + formatters.Text(" "), + ] + + +def info(message): + click.secho(str(message), fg="green") + + +def warning(message): + click.secho(str(message), fg="yellow") + + +def error(message): + click.secho(str(message), fg="red") + + +def get_terminal_width(): + return shutil.get_terminal_size()[0] + + +def print_help(): + print_formatted_text(HTML("Available commands:")) + for cmd, hlp in sorted(COMMAND_HELPS.items()): + print_formatted_text( + HTML("{:45s}{:100s}".format(cmd, hlp)) + ) + + +async def interactive_shell(server): + """ + CLI interactive shell + """ + col = get_terminal_width() + max_len = max([len(t) for t in TITLE.split("\n")]) + if col > max_len: + info(TITLE) + else: + print_formatted_text(HTML(''.format(SMALL_TITLE))) + info("") + completer = NestedCompleter.from_nested_dict(COMMANDS) + session = PromptSession("SERVER > ", + completer=completer, + bottom_toolbar=BOTTOM_TOOLBAR) + + # Run echo loop. Read text from stdin, and reply it back. + while True: + try: + invalid_command = False + result = await session.prompt_async() + if result == "exit": + await server.web_app.shutdown() + break + if result == "help": + print_help() + continue + if result == "clear": + clear() + continue + command = result.split() + if command: + if command[0] not in COMMANDS: + invalid_command = True + if invalid_command: + warning("Invalid command or invalid usage of command - {}".format(command)) + continue + if len(command) == 1: + warning("Usage: '{}'".format(USAGE)) + else: + args = command[1:] + skip_next = False + val_dict = {} + for index, arg in enumerate(args): + if skip_next: + skip_next = False + continue + if "=" in arg: + arg, value = arg.split("=") + else: + if arg in COMMAND_ARGS: + try: + value = args[index+1] + skip_next = True + except IndexError: + error("Missing value " + "for argument - {}".format(arg)) + warning("Usage: '{}'".format(USAGE)) + break + valid = True + if arg == "response_type": + if value not in RESPONSE_TYPES: + warning("Invalid response " + "type request - {}".format(value)) + warning("Choose from {}".format(RESPONSE_TYPES)) + valid = False + elif arg in ["error_code", "delay_by"]: + try: + value = int(value) + except ValueError: + warning("Expected integer " + "value for {}".format(arg)) + valid = False + + if valid: + val_dict[arg] = value + if val_dict: + server.manipulator_config = val_dict + # result = await run_command(tester, *command) + + except (EOFError, KeyboardInterrupt): + return + + +async def main(server): + with patch_stdout(): + try: + await interactive_shell(server) + finally: + pass + warning("Bye Bye!!!") + + +async def run_repl(server): + await main(server) + + diff --git a/pymodbus/repl/server/main.py b/pymodbus/repl/server/main.py new file mode 100644 index 000000000..00efc78e0 --- /dev/null +++ b/pymodbus/repl/server/main.py @@ -0,0 +1,104 @@ +""" +Copyright (c) 2020 by RiptideIO +All rights reserved. +""" +import asyncio +import json +import click +from pymodbus.utilities import IS_PYTHON3 +from pymodbus.framer.socket_framer import ModbusSocketFramer +from pymodbus.server.reactive.main import ( + ReactiveServer, DEFAULT_FRAMER, DEFUALT_HANDLERS) +from pymodbus.server.reactive.default_config import DEFUALT_CONFIG +from pymodbus.repl.server.cli import run_repl + + +@click.group("ReactiveModbusServer") +@click.option("--host", default="localhost", help="Host address") +@click.option("--web-port", default=8080, help="Web app port") +@click.option("--broadcast-support", is_flag=True, + default=False, help="Support broadcast messages") +@click.option("--repl/--no-repl", is_flag=True, + default=True, help="Enable/Disable repl for server") +@click.option("--verbose", is_flag=True, + help="Run with debug logs enabled for pymodbus") +@click.pass_context +def server(ctx, host, web_port, broadcast_support, repl, verbose): + global logger + import logging + FORMAT = ('%(asctime)-15s %(threadName)-15s' + ' %(levelname)-8s %(module)-15s:%(lineno)-8s %(message)s') + pymodbus_logger = logging.getLogger("pymodbus") + logging.basicConfig(format=FORMAT) + logger = logging.getLogger(__name__) + if verbose: + pymodbus_logger.setLevel(logging.DEBUG) + logger.setLevel(logging.DEBUG) + else: + pymodbus_logger.setLevel(logging.ERROR) + logger.setLevel(logging.ERROR) + + ctx.obj = {"repl": repl, "host": host, "port": web_port, + "broadcast": broadcast_support} + + +@server.command("run") +@click.option("--modbus-server", default="tcp", + type=click.Choice(["tcp", "serial", "tls", "udp"], + case_sensitive=False), + help="Modbus server") +@click.option("--modbus-framer", default="socket", + type=click.Choice(["socket", "rtu", "tls", "ascii", "binary"], + case_sensitive=False), + help="Modbus framer to use") +@click.option("--modbus-port", default="5020", help="Modbus port") +@click.option("--modbus-unit-id", default=1, help="Modbus unit id") +@click.option("--modbus-config", type=click.Path(exists=True), + help="Path to additional modbus server config") +@click.pass_context +def run(ctx, modbus_server, modbus_framer, modbus_port, modbus_unit_id, modbus_config): + """ + Run Reactive Modbus server exposing REST endpoint + for response manipulation. + """ + if not IS_PYTHON3: + click.secho("Pymodbus Server REPL not supported on python2", fg="read") + exit(1) + repl = ctx.obj.pop("repl") + web_app_config = ctx.obj + loop = asyncio.get_event_loop() + framer = DEFAULT_FRAMER.get(modbus_framer, ModbusSocketFramer) + if modbus_config: + with open(modbus_config) as f: + modbus_config = json.load(f) + else: + modbus_config = DEFUALT_CONFIG + modbus_config = modbus_config.get(modbus_server, {}) + if modbus_server != "serial": + modbus_port = int(modbus_port) + handler = modbus_config.pop("handler", "ModbusConnectedRequestHandler") + else: + handler = modbus_config.pop("handler", "ModbusSingleRequestHandler") + handler = DEFUALT_HANDLERS.get(handler.strip()) + + modbus_config["handler"] = handler + app = ReactiveServer.factory(modbus_server, framer, + modbus_port=modbus_port, + unit=modbus_unit_id, + loop=loop, + **web_app_config, **modbus_config) + try: + if repl: + loop.run_until_complete(app.run_async()) + + loop.run_until_complete(run_repl(app)) + loop.run_forever() + else: + app.run() + + except asyncio.exceptions.CancelledError: + print("Done!!!!!") + + +if __name__ == '__main__': + server() diff --git a/pymodbus/server/async_io.py b/pymodbus/server/async_io.py index 690332843..c4a7e2836 100755 --- a/pymodbus/server/async_io.py +++ b/pymodbus/server/async_io.py @@ -4,7 +4,8 @@ """ from binascii import b2a_hex -import socket +import serial +from serial_asyncio import create_serial_connection import ssl import traceback @@ -48,6 +49,20 @@ def __init__(self, owner): self.receive_queue = asyncio.Queue() self.handler_task = None # coroutine to be run on asyncio loop + def _log_exception(self): + if isinstance(self, ModbusConnectedRequestHandler): + _logger.error( + "Handler for stream [%s:%s] has " + "been canceled" % self.client_address[:2]) + elif isinstance(self, ModbusSingleRequestHandler): + _logger.error( + "Handler for serial port [%s] has been " + "cancelled" % self.transport.serial.port) + else: + sock_name = self.protocol._sock.getsockname() + _logger.error("Handler for UDP socket [%s] has " + "been canceled" % sock_name[1]) + def connection_made(self, transport): """ asyncio.BaseProtocol callback for socket establish @@ -57,7 +72,17 @@ def connection_made(self, transport): corresponds to the socket being opened """ try: - _logger.debug("Socket [%s:%s] opened" % transport.get_extra_info('sockname')) + sockname = transport.get_extra_info('sockname') + if sockname is not None: + _logger.debug( + "Socket [%s:%s] opened" % transport.get_extra_info( + 'sockname')[:2]) + else: + if hasattr(transport, 'serial'): + _logger.debug( + "Serial connection opened on port: {}".format( + transport.serial.port) + ) self.transport = transport self.running = True self.framer = self.server.framer(self.server.decoder, client=None) @@ -68,7 +93,7 @@ def connection_made(self, transport): else: self.handler_task = asyncio.ensure_future(self.handle()) except Exception as ex: # pragma: no cover - _logger.debug("Datastore unable to fulfill request: " + _logger.error("Datastore unable to fulfill request: " "%s; %s", ex, traceback.format_exc()) def connection_lost(self, exc): @@ -81,20 +106,18 @@ def connection_lost(self, exc): """ try: self.handler_task.cancel() - if exc is None: - if hasattr(self, "client_address"): # TCP connection - _logger.debug("Disconnected from client [%s:%s]" % self.client_address) - else: - _logger.debug("Disconnected from client [%s]" % self.transport.get_extra_info("peername")) + self._log_exception() else: # pragma: no cover - _logger.debug("Client Disconnection [%s:%s] due to %s" % (*self.client_address, exc)) + if hasattr(self, "client_address"): # TCP connection + _logger.debug("Client Disconnection {} due " + "to {}".format(*self.client_address, exc)) self.running = False except Exception as ex: # pragma: no cover - _logger.debug("Datastore unable to fulfill request: " - "%s; %s", ex, traceback.format_exc()) + _logger.error("Datastore unable to fulfill request: " + "%s; %s", ex, traceback.format_exc()) async def handle(self): """Asyncio coroutine which represents a single conversation between @@ -104,8 +127,9 @@ async def handle(self): fed to this coroutine via the asyncio.Queue object which is fed by the ModbusBaseRequestHandler class's callback Future. - This callback future gets data from either asyncio.DatagramProtocol.datagram_received - or from asyncio.BaseProtocol.data_received. + This callback future gets data from either + asyncio.DatagramProtocol.datagram_received or + from asyncio.BaseProtocol.data_received. This function will execute without blocking in the while-loop and yield to the asyncio event loop when the frame is exhausted. @@ -125,11 +149,13 @@ async def handle(self): while self.running: try: units = self.server.context.slaves() - data = await self._recv_() # this is an asyncio.Queue await, it will never fail + # this is an asyncio.Queue await, it will never fail + data = await self._recv_() if isinstance(data, tuple): - data, *addr = data # addr is populated when talking over UDP + # addr is populated when talking over UDP + data, *addr = data else: - addr = (None,) # empty tuple + addr = (None,) # empty tuple if not isinstance(units, (list, tuple)): units = [units] @@ -143,23 +169,21 @@ async def handle(self): _logger.debug('Handling data: ' + hexlify_packets(data)) single = self.server.context.single - self.framer.processIncomingPacket(data=data, - callback=lambda x: self.execute(x, *addr), - unit=units, - single=single) + self.framer.processIncomingPacket( + data=data, callback=lambda x: self.execute(x, *addr), + unit=units, single=single) except asyncio.CancelledError: # catch and ignore cancelation errors - if isinstance(self, ModbusConnectedRequestHandler): - _logger.debug("Handler for stream [%s:%s] has been canceled" % self.client_address) - else: - _logger.debug("Handler for UDP socket [%s] has been canceled" % self.protocol._sock.getsockname()[1]) - + self._log_exception() except Exception as e: - # force TCP socket termination as processIncomingPacket should handle applicaiton layer errors + # force TCP socket termination as processIncomingPacket + # should handle applicaiton layer errors # for UDP sockets, simply reset the frame if isinstance(self, ModbusConnectedRequestHandler): - _logger.info("Unknown exception '%s' on stream [%s:%s] forcing disconnect" % (e, *self.client_address)) + client_addr = self.client_address[:2] + _logger.error("Unknown exception '{}' on stream {} " + "forcing disconnect".format(e, client_addr)) self.transport.close() else: _logger.error("Unknown error occurred %s" % e) @@ -178,29 +202,31 @@ def execute(self, request, *addr): try: if self.server.broadcast_enable and request.unit_id == 0: broadcast = True - # if broadcasting then execute on all slave contexts, note response will be ignored + # if broadcasting then execute on all slave contexts, + # note response will be ignored for unit_id in self.server.context.slaves(): response = request.execute(self.server.context[unit_id]) else: context = self.server.context[request.unit_id] response = request.execute(context) except NoSuchSlaveException as ex: - _logger.debug("requested slave does " - "not exist: %s" % request.unit_id ) + _logger.error("requested slave does " + "not exist: %s" % request.unit_id) if self.server.ignore_missing_slaves: return # the client will simply timeout waiting for a response response = request.doException(merror.GatewayNoResponse) except Exception as ex: - _logger.debug("Datastore unable to fulfill request: " + _logger.error("Datastore unable to fulfill request: " "%s; %s", ex, traceback.format_exc()) response = request.doException(merror.SlaveFailure) # no response when broadcasting if not broadcast: response.transaction_id = request.transaction_id response.unit_id = request.unit_id + if self.server.response_manipulator: + response = self.server.response_manipulator(response) self.send(response, *addr) - def send(self, message, *addr): if message.should_respond: # self.server.control.Counter.BusMessage += 1 @@ -223,6 +249,7 @@ def _send_(self, data): # pragma: no cover """ raise NotImplementedException("Method not implemented " "by derived class") + async def _recv_(self): # pragma: no cover """ Receive data from the network @@ -245,17 +272,20 @@ def connection_made(self, transport): self.client_address = transport.get_extra_info('peername') self.server.active_connections[self.client_address] = self - _logger.debug("TCP client connection established [%s:%s]" % self.client_address) + _logger.debug("TCP client connection established " + "[%s:%s]" % self.client_address[:2]) def connection_lost(self, exc): - """ asyncio.BaseProtocol: Called when the connection is lost or closed.""" + """ + asyncio.BaseProtocol: Called when the connection is lost or closed. + """ super().connection_lost(exc) - _logger.debug("TCP client disconnected [%s:%s]" % self.client_address) + client_addr = self.client_address[:2] + _logger.debug("TCP client disconnected [%s:%s]" % client_addr) if self.client_address in self.server.active_connections: self.server.active_connections.pop(self.client_address) - - def data_received(self,data): + def data_received(self, data): """ asyncio.Protocol: (TCP) Called when some data is received. data is a non-empty bytes object containing the incoming data. @@ -270,7 +300,8 @@ def _send_(self, data): self.transport.write(data) -class ModbusDisconnectedRequestHandler(ModbusBaseRequestHandler, asyncio.DatagramProtocol): +class ModbusDisconnectedRequestHandler(ModbusBaseRequestHandler, + asyncio.DatagramProtocol): """ Implements the modbus server protocol This uses the socketserver.BaseRequestHandler to implement @@ -280,7 +311,8 @@ class ModbusDisconnectedRequestHandler(ModbusBaseRequestHandler, asyncio.Datagra """ def __init__(self,owner): super().__init__(owner) - self.server.on_connection_terminated = asyncio.get_event_loop().create_future() + _future = asyncio.get_event_loop().create_future() + self.server.on_connection_terminated = _future def connection_lost(self,exc): super().connection_lost(exc) @@ -314,6 +346,7 @@ async def _recv_(self): def _send_(self, data, addr): self.transport.sendto(data, addr=addr) + class ModbusServerFactory: """ Builder class for a modbus server @@ -323,13 +356,42 @@ class ModbusServerFactory: def __init__(self, store, framer=None, identity=None, **kwargs): import warnings - warnings.warn("deprecated API for asyncio. ServerFactory's are a twisted construct and don't have an equivalent in asyncio", + warnings.warn("deprecated API for asyncio. ServerFactory's are a " + "twisted construct and don't have an equivalent in " + "asyncio", DeprecationWarning) +class ModbusSingleRequestHandler(ModbusBaseRequestHandler, asyncio.Protocol): + """ Implements the modbus server protocol + This uses asyncio.Protocol to implement + the client handler for a serial connection. + """ + def connection_made(self, transport): + super().connection_made(transport) + + _logger.debug("Serial connection established") + + def connection_lost(self, exc): + super().connection_lost(exc) + _logger.debug("Serial conection lost") + if hasattr(self.server, 'on_connection_lost'): + self.server.on_connection_lost() + + def data_received(self, data): + self.receive_queue.put_nowait(data) + + async def _recv_(self): + return await self.receive_queue.get() + + def _send_(self, data): + if self.transport is not None: + self.transport.write(data) + # --------------------------------------------------------------------------- # # Server Implementations # --------------------------------------------------------------------------- # + class ModbusTcpServer: """ A modbus threaded tcp socket server @@ -376,6 +438,8 @@ def __init__(self, to a missing slave :param broadcast_enable: True to treat unit_id 0 as broadcast address, False to treat 0 as any other unit_id + :param response_manipulator: Callback method for manipulating the + response """ self.active_connections = {} self.loop = loop or asyncio.get_event_loop() @@ -391,26 +455,33 @@ def __init__(self, Defaults.IgnoreMissingSlaves) self.broadcast_enable = kwargs.get('broadcast_enable', Defaults.broadcast_enable) - + self.response_manipulator = kwargs.get("response_manipulator", None) if isinstance(identity, ModbusDeviceIdentification): self.control.Identity.update(identity) - self.serving = self.loop.create_future() # asyncio future that will be done once server has started - self.server = None # constructors cannot be declared async, so we have to defer the initialization of the server + # asyncio future that will be done once server has started + self.serving = self.loop.create_future() + # constructors cannot be declared async, so we have to + # defer the initialization of the server + self.server = None if PYTHON_VERSION >= (3, 7): # start_serving is new in version 3.7 - self.server_factory = self.loop.create_server(lambda : self.handler(self), - *self.address, - reuse_address=allow_reuse_address, - reuse_port=allow_reuse_port, - backlog=backlog, - start_serving=not defer_start) + self.server_factory = self.loop.create_server( + lambda: self.handler(self), + *self.address, + reuse_address=allow_reuse_address, + reuse_port=allow_reuse_port, + backlog=backlog, + start_serving=not defer_start + ) else: - self.server_factory = self.loop.create_server(lambda : self.handler(self), - *self.address, - reuse_address=allow_reuse_address, - reuse_port=allow_reuse_port, - backlog=backlog) + self.server_factory = self.loop.create_server( + lambda: self.handler(self), + *self.address, + reuse_address=allow_reuse_address, + reuse_port=allow_reuse_port, + backlog=backlog + ) async def serve_forever(self): if self.server is None: @@ -418,10 +489,11 @@ async def serve_forever(self): self.serving.set_result(True) await self.server.serve_forever() else: - raise RuntimeError("Can't call serve_forever on an already running server object") + raise RuntimeError("Can't call serve_forever on " + "an already running server object") def server_close(self): - for k,v in self.active_connections.items(): + for k, v in self.active_connections.items(): _logger.warning("aborting active session {}".format(k)) v.handler_task.cancel() self.active_connections = {} @@ -481,6 +553,8 @@ def __init__(self, to a missing slave :param broadcast_enable: True to treat unit_id 0 as broadcast address, False to treat 0 as any other unit_id + :param response_manipulator: Callback method for + manipulating the response """ self.active_connections = {} self.loop = loop or asyncio.get_event_loop() @@ -496,6 +570,7 @@ def __init__(self, Defaults.IgnoreMissingSlaves) self.broadcast_enable = kwargs.get('broadcast_enable', Defaults.broadcast_enable) + self.response_manipulator = kwargs.get("response_manipulator", None) if isinstance(identity, ModbusDeviceIdentification): self.control.Identity.update(identity) @@ -512,26 +587,31 @@ def __init__(self, self.sslctx.options |= ssl.OP_NO_SSLv2 self.sslctx.verify_mode = ssl.CERT_OPTIONAL self.sslctx.check_hostname = False - - self.serving = self.loop.create_future() # asyncio future that will be done once server has started - self.server = None # constructors cannot be declared async, so we have to defer the initialization of the server + # asyncio future that will be done once server has started + self.serving = self.loop.create_future() + # constructors cannot be declared async, so we have to + # defer the initialization of the server + self.server = None if PYTHON_VERSION >= (3, 7): # start_serving is new in version 3.7 - self.server_factory = self.loop.create_server(lambda : self.handler(self), - *self.address, - ssl=self.sslctx, - reuse_address=allow_reuse_address, - reuse_port=allow_reuse_port, - backlog=backlog, - start_serving=not defer_start) + self.server_factory = self.loop.create_server( + lambda: self.handler(self), + *self.address, + ssl=self.sslctx, + reuse_address=allow_reuse_address, + reuse_port=allow_reuse_port, + backlog=backlog, + start_serving=not defer_start + ) else: - self.server_factory = self.loop.create_server(lambda : self.handler(self), - *self.address, - ssl=self.sslctx, - reuse_address=allow_reuse_address, - reuse_port=allow_reuse_port, - backlog=backlog) - + self.server_factory = self.loop.create_server( + lambda: self.handler(self), + *self.address, + ssl=self.sslctx, + reuse_address=allow_reuse_address, + reuse_port=allow_reuse_port, + backlog=backlog + ) class ModbusUdpServer: @@ -565,6 +645,8 @@ def __init__(self, context, framer=None, identity=None, address=None, to a missing slave :param broadcast_enable: True to treat unit_id 0 as broadcast address, False to treat 0 as any other unit_id + :param response_manipulator: Callback method for + manipulating the response """ self.loop = loop or asyncio.get_event_loop() self.decoder = ServerDecoder() @@ -577,6 +659,7 @@ def __init__(self, context, framer=None, identity=None, address=None, Defaults.IgnoreMissingSlaves) self.broadcast_enable = kwargs.get('broadcast_enable', Defaults.broadcast_enable) + self.response_manipulator = kwargs.get("response_manipulator", None) if isinstance(identity, ModbusDeviceIdentification): self.control.Identity.update(identity) @@ -585,12 +668,15 @@ def __init__(self, context, framer=None, identity=None, address=None, self.endpoint = None self.on_connection_terminated = None self.stop_serving = self.loop.create_future() - self.serving = self.loop.create_future() # asyncio future that will be done once server has started - self.server_factory = self.loop.create_datagram_endpoint(lambda: self.handler(self), - local_addr=self.address, - reuse_address=allow_reuse_address, - reuse_port=allow_reuse_port, - allow_broadcast=True) + # asyncio future that will be done once server has started + self.serving = self.loop.create_future() + self.server_factory = self.loop.create_datagram_endpoint( + lambda: self.handler(self), + local_addr=self.address, + reuse_address=allow_reuse_address, + reuse_port=allow_reuse_port, + allow_broadcast=True + ) async def serve_forever(self): if self.protocol is None: @@ -598,7 +684,8 @@ async def serve_forever(self): self.serving.set_result(True) await self.stop_serving else: - raise RuntimeError("Can't call serve_forever on an already running server object") + raise RuntimeError("Can't call serve_forever on an " + "already running server object") def server_close(self): self.stop_serving.set_result(True) @@ -608,11 +695,9 @@ def server_close(self): self.protocol.close() - class ModbusSerialServer(object): """ A modbus threaded serial socket server - We inherit and overload the socket server so that we can control the client threads as well as have a single server context instance. @@ -620,15 +705,12 @@ class ModbusSerialServer(object): handler = None - def __init__(self, context, framer=None, identity=None, **kwargs): # pragma: no cover + def __init__(self, context, framer=None, **kwargs): # pragma: no cover """ Overloaded initializer for the socket server - If the identify structure is not passed in, the ModbusControlBlock uses its own empty structure. - :param context: The ModbusServerContext datastore :param framer: The framer strategy to use - :param identity: An optional identify structure :param port: The serial port to attach to :param stopbits: The number of stop bits to use :param bytesize: The bytesize of the serial messages @@ -639,8 +721,88 @@ def __init__(self, context, framer=None, identity=None, **kwargs): # pragma: no to a missing slave :param broadcast_enable: True to treat unit_id 0 as broadcast address, False to treat 0 as any other unit_id + :param autoreonnect: True to enable automatic reconnection, + False otherwise + :param reconnect_delay: reconnect delay in seconds + :param response_manipulator: Callback method for + manipulating the response """ - raise NotImplementedException + self.device = kwargs.get('port', 0) + self.stopbits = kwargs.get('stopbits', Defaults.Stopbits) + self.bytesize = kwargs.get('bytesize', Defaults.Bytesize) + self.parity = kwargs.get('parity', Defaults.Parity) + self.baudrate = kwargs.get('baudrate', Defaults.Baudrate) + self.timeout = kwargs.get('timeout', Defaults.Timeout) + self.ignore_missing_slaves = kwargs.get('ignore_missing_slaves', + Defaults.IgnoreMissingSlaves) + self.broadcast_enable = kwargs.get('broadcast_enable', + Defaults.broadcast_enable) + self.auto_reconnect = kwargs.get('auto_reconnect', False) + self.reconnect_delay = kwargs.get('reconnect_delay', 2) + self.reconnecting_task = None + + self.handler = kwargs.get("handler") or ModbusSingleRequestHandler + self.framer = framer or ModbusRtuFramer + self.decoder = ServerDecoder() + self.context = context or ModbusServerContext() + self.response_manipulator = kwargs.get("response_manipulator", None) + self.protocol = None + self.transport = None + + async def start(self): + await self._connect() + + def _protocol_factory(self): + return self.handler(self) + + async def _delayed_connect(self): + await asyncio.sleep(self.reconnect_delay) + await self._connect() + + async def _connect(self): + if self.reconnecting_task is not None: + self.reconnecting_task = None + + try: + self.transport, self.protocol = await create_serial_connection( + asyncio.get_event_loop(), + self._protocol_factory, + self.device, + baudrate=self.baudrate, + bytesize=self.bytesize, + parity=self.parity, + stopbits=self.stopbits, + timeout=self.timeout + ) + except serial.serialutil.SerialException as e: + _logger.debug("Failed to open serial port: {}".format(self.device)) + if not self.auto_reconnect: + raise e + + self._check_reconnect() + + except Exception as e: + _logger.debug("Exception while create - {}".format(e)) + + def on_connection_lost(self): + if self.transport is not None: + self.transport.close() + self.transport = None + self.protocol = None + + self._check_reconnect() + + def _check_reconnect(self): + _logger.debug("checkking autoreconnect {} {}".format( + self.auto_reconnect, self.reconnecting_task)) + if self.auto_reconnect and (self.reconnecting_task is None): + _logger.debug("Scheduling serial connection reconnect") + loop = asyncio.get_event_loop() + self.reconnecting_task = loop.create_task(self._delayed_connect()) + + async def serve_forever(self): + while True: + await asyncio.sleep(360) # --------------------------------------------------------------------------- # @@ -666,7 +828,7 @@ async def StartTcpServer(context=None, identity=None, address=None, server = ModbusTcpServer(context, framer, identity, address, **kwargs) for f in custom_functions: - server.decoder.register(f) # pragma: no cover + server.decoder.register(f) # pragma: no cover if not defer_start: await server.serve_forever() @@ -674,9 +836,12 @@ async def StartTcpServer(context=None, identity=None, address=None, return server -async def StartTlsServer(context=None, identity=None, address=None, sslctx=None, - certfile=None, keyfile=None, allow_reuse_address=False, - allow_reuse_port=False, custom_functions=[], +async def StartTlsServer(context=None, identity=None, address=None, + sslctx=None, + certfile=None, keyfile=None, + allow_reuse_address=False, + allow_reuse_port=False, + custom_functions=[], defer_start=True, **kwargs): """ A factory to start and run a tls modbus server @@ -714,7 +879,7 @@ async def StartTlsServer(context=None, identity=None, address=None, sslctx=None, async def StartUdpServer(context=None, identity=None, address=None, - custom_functions=[], defer_start=True, **kwargs): + custom_functions=[], defer_start=True, **kwargs): """ A factory to start and run a udp modbus server :param context: The ModbusServerContext datastore @@ -738,9 +903,8 @@ async def StartUdpServer(context=None, identity=None, address=None, return server - -def StartSerialServer(context=None, identity=None, custom_functions=[], - **kwargs):# pragma: no cover +async def StartSerialServer(context=None, identity=None, + custom_functions=[], **kwargs): # pragma: no cover """ A factory to start and run a serial modbus server :param context: The ModbusServerContext datastore @@ -757,24 +921,24 @@ def StartSerialServer(context=None, identity=None, custom_functions=[], :param ignore_missing_slaves: True to not send errors on a request to a missing slave """ - raise NotImplementedException - import serial framer = kwargs.pop('framer', ModbusAsciiFramer) server = ModbusSerialServer(context, framer, identity, **kwargs) for f in custom_functions: server.decoder.register(f) - server.serve_forever() + await server.start() + await server.serve_forever() + def StopServer(): """ Helper method to stop Async Server """ import warnings - warnings.warn("deprecated API for asyncio. Call server_close() on server object returned by StartXxxServer", + warnings.warn("deprecated API for asyncio. Call server_close() on " + "server object returned by StartXxxServer", DeprecationWarning) - # --------------------------------------------------------------------------- # # Exported symbols # --------------------------------------------------------------------------- # @@ -785,4 +949,3 @@ def StopServer(): "StartTcpServer", "StartTlsServer", "StartUdpServer", "StartSerialServer" ] - diff --git a/pymodbus/server/reactive/__init__.py b/pymodbus/server/reactive/__init__.py new file mode 100644 index 000000000..bc4e39484 --- /dev/null +++ b/pymodbus/server/reactive/__init__.py @@ -0,0 +1,4 @@ +""" +Copyright (c) 2020 by RiptideIO +All rights reserved. +""" diff --git a/pymodbus/server/reactive/default_config.json b/pymodbus/server/reactive/default_config.json new file mode 100644 index 000000000..6f076676a --- /dev/null +++ b/pymodbus/server/reactive/default_config.json @@ -0,0 +1,32 @@ +{ + "tcp": { + "handler": "ModbusConnectedRequestHandler", + "allow_reuse_address": true, + "allow_reuse_port": true, + "backlog": 20, + "ignore_missing_slaves": false + }, + "rtu": { + "handler": "ModbusSingleRequestHandler", + "stopbits": 1, + "bytesize": 8, + "parity": "N", + "baudrate": 9600, + "timeout": 3, + "auto_reconnect": false, + "reconnect_delay": 2 + }, + "tls": { + "handler": "ModbusConnectedRequestHandler", + "certfile": null, + "keyfile": null, + "allow_reuse_address": true, + "allow_reuse_port": true, + "backlog": 20, + "ignore_missing_slaves": false + }, + "udp": { + "handler": "ModbusDisonnectedRequestHandler", + "ignore_missing_slaves": false + } +} \ No newline at end of file diff --git a/pymodbus/server/reactive/default_config.py b/pymodbus/server/reactive/default_config.py new file mode 100644 index 000000000..ad274e8d2 --- /dev/null +++ b/pymodbus/server/reactive/default_config.py @@ -0,0 +1,37 @@ +""" +Copyright (c) 2020 by RiptideIO +All rights reserved. +""" + +DEFUALT_CONFIG = { + "tcp": { + "handler": "ModbusConnectedRequestHandler", + "allow_reuse_address": True, + "allow_reuse_port": True, + "backlog": 20, + "ignore_missing_slaves": False + }, + "serial": { + "handler": "ModbusSingleRequestHandler", + "stopbits": 1, + "bytesize": 8, + "parity": "N", + "baudrate": 9600, + "timeout": 3, + "auto_reconnect": False, + "reconnect_delay": 2 + }, + "tls": { + "handler": "ModbusConnectedRequestHandler", + "certfile": None, + "keyfile": None, + "allow_reuse_address": True, + "allow_reuse_port": True, + "backlog": 20, + "ignore_missing_slaves": False + }, + "udp": { + "handler": "ModbusDisonnectedRequestHandler", + "ignore_missing_slaves": False + } +} diff --git a/pymodbus/server/reactive/main.py b/pymodbus/server/reactive/main.py new file mode 100644 index 000000000..50ed50447 --- /dev/null +++ b/pymodbus/server/reactive/main.py @@ -0,0 +1,326 @@ +""" +Copyright (c) 2020 by RiptideIO +All rights reserved. +""" +import asyncio +import time +import random +import logging +from pymodbus.compat import IS_PYTHON3, PYTHON_VERSION +from pymodbus.pdu import ExceptionResponse, ModbusExceptions +from pymodbus.datastore.store import (ModbusSparseDataBlock, + ModbusSequentialDataBlock) +from pymodbus.datastore import ModbusSlaveContext, ModbusServerContext +from pymodbus.device import ModbusDeviceIdentification + +if not IS_PYTHON3 or PYTHON_VERSION < (3, 6): + print(f"You are running {PYTHON_VERSION}." + "Reactive server requires python3.6 or above".PYTHON_VERSION) + exit() + + +try: + from aiohttp import web +except ImportError as e: + print("Reactive server requires aiohttp. " + "Please install with 'pip install aiohttp' and try again.") + exit(1) + +from pymodbus.server.async_io import (ModbusTcpServer, + ModbusTlsServer, + ModbusSerialServer, + ModbusUdpServer, + ModbusSingleRequestHandler, + ModbusConnectedRequestHandler, + ModbusDisconnectedRequestHandler) +from pymodbus.transaction import (ModbusRtuFramer, + ModbusSocketFramer, + ModbusTlsFramer, + ModbusAsciiFramer, + ModbusBinaryFramer) +logger = logging.getLogger(__name__) + +SERVER_MAPPER = { + "tcp": ModbusTcpServer, + "serial": ModbusSerialServer, + "udp": ModbusUdpServer, + "tls": ModbusTlsServer +} + +DEFAULT_FRAMER = { + "tcp": ModbusSocketFramer, + "rtu": ModbusRtuFramer, + "tls": ModbusTlsFramer, + "udp": ModbusSocketFramer, + "ascii": ModbusAsciiFramer, + "binary": ModbusBinaryFramer +} + +DEFAULT_MANIPULATOR = { + "response_type": "normal", # normal, error, delayed, empty + "delay_by": 0, + "error_code": ModbusExceptions.IllegalAddress, + "clear_after": 5 # request count + +} +DEFUALT_HANDLERS = { + "ModbusSingleRequestHandler": ModbusSingleRequestHandler, + "ModbusConnectedRequestHandler": ModbusConnectedRequestHandler, + "ModbusDisconnectedRequestHandler": ModbusDisconnectedRequestHandler +} +DEFAULT_MODBUS_MAP = {"start_offset": 0, + "count": 10, "value": 0, "sparse": False} +DEFAULT_DATA_BLOCK = { + "co": DEFAULT_MODBUS_MAP, + "di": DEFAULT_MODBUS_MAP, + "ir": DEFAULT_MODBUS_MAP, + "hr": DEFAULT_MODBUS_MAP + +} + +HINT = """ +Reactive Modbus Server started. +{} + +=========================================================================== +Example Usage: +curl -X POST http://{}:{} -d '{{"response_type": "error", "error_code": 4}}' +=========================================================================== +""" + + +class ReactiveServer: + """ + Modbus Asynchronous Server which can manipulate the response dynamically. + Useful for testing + """ + def __init__(self, host, port, modbus_server, loop=None): + self._web_app = web.Application() + self._runner = web.AppRunner(self._web_app) + self._host = host + self._port = int(port) + self._modbus_server = modbus_server + self._loop = loop + self._add_routes() + self._modbus_server.response_manipulator = self.manipulate_response + self._manipulator_config = dict(**DEFAULT_MANIPULATOR) + self._web_app.on_startup.append(self.start_modbus_server) + self._web_app.on_shutdown.append(self.stop_modbus_server) + + @property + def web_app(self): + return self._web_app + + @property + def manipulator_config(self): + return self._manipulator_config + + @manipulator_config.setter + def manipulator_config(self, value): + if isinstance(value, dict): + self._manipulator_config.update(**value) + + def _add_routes(self): + self._web_app.add_routes([ + web.post('/', self._response_manipulator)]) + + async def start_modbus_server(self, app): + """ + Start Modbus server as asyncio task after startup + :param app: Webapp + :return: + """ + try: + if isinstance(self._modbus_server, ModbusSerialServer): + app["modbus_serial_server"] = asyncio.create_task( + self._modbus_server.start()) + app["modbus_server"] = asyncio.create_task(self._modbus_server.serve_forever()) + logger.info("Modbus server started") + except Exception as e: + logger.error("Error starting modbus server") + logger.error(e) + + async def stop_modbus_server(self, app): + """ + Stop modbus server + :param app: Webapp + :return: + """ + logger.info("Stopping modbus server") + if isinstance(self._modbus_server, ModbusSerialServer): + app["modbus_serial_server"].cancel() + app["modbus_server"].cancel() + await app["modbus_server"] + logger.info("Modbus server Stopped") + + async def _response_manipulator(self, request): + """ + POST request Handler for response manipulation end point + Payload is a dict with following fields + :response_type : One among (normal, delayed, error, empty) + :error_code: Modbus error code for error response + :delay_by: Delay sending response by seconds + + :param request: + :return: + """ + data = await request.json() + self._manipulator_config.update(data) + return web.json_response(data=data) + + def manipulate_response(self, response): + """ + Manipulates the actual response according to the required error state. + :param response: Modbus response object + :return: Modbus response + """ + if not self._manipulator_config: + return response + else: + response_type = self._manipulator_config.get("response_type") + if response_type == "error": + error_code = self._manipulator_config.get("error_code") + logger.warning( + "Sending error response for all incoming requests") + err_response = ExceptionResponse(response.function_code, error_code) + err_response.transaction_id = response.transaction_id + err_response.unit_id = response.unit_id + response = err_response + elif response_type == "delayed": + delay_by = self._manipulator_config.get("delay_by") + logger.warning( + "Delaying response by {}s for " + "all incoming requests".format(delay_by)) + time.sleep(delay_by) + elif response_type == "empty": + logger.warning("Sending empty response") + return response + + def run(self): + """ + Run Web app + :return: + """ + def _info(message): + msg = HINT.format(message, self._host, self._port) + print(msg) + # print(message) + web.run_app(self._web_app, host=self._host, port=self._port, + print=_info) + + async def run_async(self): + """ + Run Web app + :return: + """ + try: + await self._runner.setup() + site = web.TCPSite(self._runner, self._host, self._port) + await site.start() + except Exception as e: + logger.error(e) + + @classmethod + def create_identity(cls, vendor="Pymodbus", product_code="PM", + vendor_url='http://github.com/riptideio/pymodbus/', + product_name="Pymodbus Server", + model_name="Reactive Server", + version="2.5.0"): + """ + Create modbus identity + :param vendor: + :param product_code: + :param vendor_url: + :param product_name: + :param model_name: + :param version: + :return: ModbusIdentity object + """ + identity = ModbusDeviceIdentification() + identity.VendorName = vendor + identity.ProductCode = product_code + identity.VendorUrl = vendor_url + identity.ProductName = product_name + identity.ModelName = model_name + identity.MajorMinorRevision = version + + return identity + + @classmethod + def create_context(cls, data_block=None, unit=1, + single=False): + """ + Create Modbus context. + :param data_block: Datablock (dict) Refer DEFAULT_DATA_BLOCK + :param unit: Unit id for the slave + :param single: To run as a single slave + :return: ModbusServerContext object + """ + block = dict() + data_block = data_block or DEFAULT_DATA_BLOCK + for modbus_entity, block_desc in data_block.items(): + start_address = block_desc.get("start_address", 0) + default_count = block_desc.get("count", 0) + default_value = block_desc.get("value", 0) + default_values = [default_value]*default_count + sparse = block_desc.get("sparse", False) + db = ModbusSequentialDataBlock if not sparse else ModbusSparseDataBlock + if sparse: + address_map = block_desc.get("address_map") + if not address_map: + address_map = random.sample( + range(start_address+1, default_count), default_count-1) + address_map.insert(0, 0) + block[modbus_entity] = {add: val for add in sorted(address_map) for val in default_values} + else: + block[modbus_entity] =db(start_address, default_values) + + slave_context = ModbusSlaveContext(**block, zero_mode=True) + if not single: + slaves = {unit: slave_context} + else: + slaves = slave_context + server_context = ModbusServerContext(slaves, single=single) + return server_context + + @classmethod + def factory(cls, server, framer=None, context=None, unit=1, single=False, + host="localhost", modbus_port=5020, web_port=8080, + data_block=DEFAULT_DATA_BLOCK, identity=None, loop=None, **kwargs): + """ + Factory to create ReactiveModbusServer + :param server: Modbus server type (tcp, rtu, tls, udp) + :param framer: Modbus framer (ModbusSocketFramer, ModbusRTUFramer, ModbusTLSFramer) + :param context: Modbus server context to use + :param unit: Modbus unit id + :param single: Run in single mode + :param host: Host address to use for both web app and modbus server (default localhost) + :param modbus_port: Modbus port for TCP and UDP server(default: 5020) + :param web_port: Web App port (default: 8080) + :param data_block: Datablock (refer DEFAULT_DATA_BLOCK) + :param identity: Modbus identity object + :param loop: Asyncio loop to use + :param kwargs: Other server specific keyword arguments, refer corresponding servers documentation + :return: ReactiveServer object + """ + if server.lower() not in SERVER_MAPPER: + logger.error(f"Invalid server {server}", server) + exit(1) + server = SERVER_MAPPER.get(server) + if not framer: + framer = DEFAULT_FRAMER.get(server) + if not context: + context = cls.create_context(data_block=data_block, + unit=unit, single=single) + if not identity: + identity = cls.create_identity() + if server == ModbusSerialServer: + kwargs["port"] = modbus_port + server = server(context, framer=framer, identity=identity, + **kwargs) + else: + server = server(context, framer=framer, identity=identity, + address=(host, modbus_port), defer_start=False, + **kwargs) + return ReactiveServer(host, web_port, server, loop) + diff --git a/pymodbus/server/sync.py b/pymodbus/server/sync.py index f7b22454f..c504ff013 100644 --- a/pymodbus/server/sync.py +++ b/pymodbus/server/sync.py @@ -581,7 +581,7 @@ def serve_forever(self): if not self.handler: self._build_handler() while self.is_running: - self.handler.handle() + self.handler.response_manipulator() else: _logger.error("Error opening serial port , " "Unable to start server!!") diff --git a/pymodbus/transaction.py b/pymodbus/transaction.py index 0da18a607..9f09477ba 100644 --- a/pymodbus/transaction.py +++ b/pymodbus/transaction.py @@ -64,8 +64,10 @@ def __init__(self, client, **kwargs): self.tid = Defaults.TransactionId self.client = client self.backoff = kwargs.get('backoff', Defaults.Backoff) or 0.3 - self.retry_on_empty = kwargs.get('retry_on_empty', Defaults.RetryOnEmpty) - self.retry_on_invalid = kwargs.get('retry_on_invalid', Defaults.RetryOnInvalid) + self.retry_on_empty = kwargs.get('retry_on_empty', + Defaults.RetryOnEmpty) + self.retry_on_invalid = kwargs.get('retry_on_invalid', + Defaults.RetryOnInvalid) self.retries = kwargs.get('retries', Defaults.Retries) or 1 self._transaction_lock = RLock() self._no_response_devices = [] @@ -108,6 +110,25 @@ def _calculate_exception_length(self): return None + def _validate_response(self, request, response, exp_resp_len): + """ + Validate Incoming response against request + :param request: Request sent + :param response: Response received + :param exp_resp_len: Expected response length + :return: New transactions state + """ + if not response: + return False + + mbap = self.client.framer.decode_data(response) + if mbap.get('unit') != request.unit_id or mbap.get('fcode') != request.function_code: + return False + + if 'length' in mbap and exp_resp_len: + return mbap.get('length') == exp_resp_len + return True + def execute(self, request): """ Starts the producer to send the next request to consumer.write(Frame(request)) @@ -132,6 +153,7 @@ def execute(self, request): self._transact(request, None, broadcast=True) response = b'Broadcast write sent - no response expected' else: + invalid_response = False expected_response_length = None if not isinstance(self.client.framer, ModbusSocketFramer): if hasattr(request, "get_response_pdu_size"): @@ -149,7 +171,7 @@ def execute(self, request): full = True if not expected_response_length: expected_response_length = Defaults.ReadSize - retries += 1 + # retries += 1 while retries > 0: response, last_exception = self._transact( request, @@ -157,27 +179,39 @@ def execute(self, request): full=full, broadcast=broadcast ) - if not response and ( - request.unit_id not in self._no_response_devices): + valid_response = self._validate_response( + request, response, expected_response_length + ) + if not response and request.unit_id \ + not in self._no_response_devices: self._no_response_devices.append(request.unit_id) elif request.unit_id in self._no_response_devices and response: self._no_response_devices.remove(request.unit_id) if not response and self.retry_on_empty: _logger.debug("Retry on empty - {}".format(retries)) + retries -= 1 + _logger.debug("Changing transaction state from " + "'WAITING_FOR_REPLY' to 'RETRYING'") + self.client.state = ModbusTransactionState.RETRYING + continue elif not response: break - if not self.retry_on_invalid: - break mbap = self.client.framer.decode_data(response) - if (mbap.get('unit') == request.unit_id): + if mbap.get('unit') == request.unit_id: break if ('length' in mbap and expected_response_length and - mbap.get('length') == expected_response_length): + mbap.get('length') == expected_response_length and + mbap.get('fcode') == request.function_code): + break + else: + invalid_response = True + if invalid_response and not self.retry_on_invalid: break _logger.debug("Retry on invalid - {}".format(retries)) if hasattr(self.client, "state"): - _logger.debug("RESETTING Transaction state to 'IDLE' for retry") - self.client.state = ModbusTransactionState.IDLE + _logger.debug("RESETTING Transaction " + "state to 'RETRY' for retry") + self.client.state = ModbusTransactionState.RETRYING if self.backoff: delay = 2 ** (self.retries - retries) * self.backoff time.sleep(delay) @@ -206,14 +240,26 @@ def execute(self, request): "'TRANSACTION_COMPLETE'") self.client.state = ( ModbusTransactionState.TRANSACTION_COMPLETE) + self.client.close() return response except ModbusIOException as ex: # Handle decode errors in processIncomingPacket method _logger.exception(ex) + self.client.close() self.client.state = ModbusTransactionState.TRANSACTION_COMPLETE return ex - def _transact(self, packet, response_length, full=False, broadcast=False): + def _retry(self, packet, response_length, full=False): + self.client.connect() + in_waiting = self.client._in_waiting() + if in_waiting: + if response_length == in_waiting: + result = self._recv(response_length, full) + return result, None + return self._transact(packet, response_length, full=full) + + def _transact(self, packet, response_length, + full=False, broadcast=False): """ Does a Write and Read transaction :param packet: packet to be sent @@ -229,6 +275,11 @@ def _transact(self, packet, response_length, full=False, broadcast=False): if _logger.isEnabledFor(logging.DEBUG): _logger.debug("SEND: " + hexlify_packets(packet)) size = self._send(packet) + if isinstance(size, bytes) and self.client.state == ModbusTransactionState.RETRYING: + _logger.debug("Changing transaction state from " + "'RETRYING' to 'PROCESSING REPLY'") + self.client.state = ModbusTransactionState.PROCESSING_REPLY + return size, None if broadcast: if size: _logger.debug("Changing transaction state from 'SENDING' " @@ -244,6 +295,7 @@ def _transact(self, packet, response_length, full=False, broadcast=False): if local_echo_packet != packet: return b'', "Wrong local echo" result = self._recv(response_length, full) + # result2 = self._recv(response_length, full) if _logger.isEnabledFor(logging.DEBUG): _logger.debug("RECV: " + hexlify_packets(result)) @@ -255,7 +307,7 @@ def _transact(self, packet, response_length, full=False, broadcast=False): result = b'' return result, last_exception - def _send(self, packet): + def _send(self, packet, retrying=False): return self.client.framer.sendPacket(packet) def _recv(self, expected_response_length, full): @@ -324,7 +376,7 @@ def _recv(self, expected_response_length, full): def addTransaction(self, request, tid=None): """ Adds a transaction to the handler - This holds the requets in case it needs to be resent. + This holds the request in case it needs to be resent. After being sent, the request is removed. :param request: The request to hold on to diff --git a/pymodbus/utilities.py b/pymodbus/utilities.py index 6d38ca3dd..1aebb6de4 100644 --- a/pymodbus/utilities.py +++ b/pymodbus/utilities.py @@ -20,6 +20,8 @@ class ModbusTransactionState(object): PROCESSING_REPLY = 4 PROCESSING_ERROR = 5 TRANSACTION_COMPLETE = 6 + RETRYING = 7 + NO_RESPONSE_STATE = 8 @classmethod def to_string(cls, state): @@ -30,7 +32,8 @@ def to_string(cls, state): ModbusTransactionState.WAITING_TURNAROUND_DELAY: "WAITING_TURNAROUND_DELAY", ModbusTransactionState.PROCESSING_REPLY: "PROCESSING_REPLY", ModbusTransactionState.PROCESSING_ERROR: "PROCESSING_ERROR", - ModbusTransactionState.TRANSACTION_COMPLETE: "TRANSACTION_COMPLETE" + ModbusTransactionState.TRANSACTION_COMPLETE: "TRANSACTION_COMPLETE", + ModbusTransactionState.RETRYING: "RETRYING TRANSACTION", } return states.get(state, None) diff --git a/pymodbus/version.py b/pymodbus/version.py index 2fc273abd..74b2497c7 100644 --- a/pymodbus/version.py +++ b/pymodbus/version.py @@ -41,7 +41,7 @@ def __str__(self): return '[%s, version %s]' % (self.package, self.short()) -version = Version('pymodbus', 2, 4, 0) +version = Version('pymodbus', 2, 5, 0, "rc3") version.__name__ = 'pymodbus' # fix epydoc error diff --git a/setup.py b/setup.py index 60e7c4eef..50da034be 100644 --- a/setup.py +++ b/setup.py @@ -23,9 +23,15 @@ try: from setup_commands import command_classes except ImportError: - command_classes={} + command_classes = {} from pymodbus import __version__, __author__, __maintainer__ +from pymodbus.utilities import IS_PYTHON3 +CONSOLE_SCRIPTS = [ + 'pymodbus.console=pymodbus.repl.client.main:main' + ] +if IS_PYTHON3: + CONSOLE_SCRIPTS.append('pymodbus.server=pymodbus.repl.server.main:server') with open('requirements.txt') as reqs: install_requires = [ line for line in reqs.read().split('\n') @@ -89,14 +95,22 @@ 'tornado': [ 'tornado == 4.5.3' ], - 'repl': [ + + 'repl:python_version <= "2.7"': [ 'click>=7.0', 'prompt-toolkit==2.0.4', - 'pygments==2.2.0' + 'pygments>=2.2.0' + ], + 'repl:python_version >= "3.6"': [ + 'click>=7.0', + 'prompt-toolkit>=3.0.8', + 'pygments>=2.2.0', + 'aiohttp>=3.7.3', + 'pyserial-asyncio>=0.5' ] }, entry_points={ - 'console_scripts': ['pymodbus.console=pymodbus.repl.main:main'], + 'console_scripts': CONSOLE_SCRIPTS, }, test_suite='nose.collector', cmdclass=command_classes, diff --git a/test/test_bit_read_messages.py b/test/test_bit_read_messages.py index 03507121d..75b678f09 100644 --- a/test/test_bit_read_messages.py +++ b/test/test_bit_read_messages.py @@ -45,7 +45,7 @@ def testReadBitBaseClassMethods(self): msg = "ReadBitRequest(1,1)" self.assertEqual(msg, str(handle)) handle = ReadBitsResponseBase([1,1]) - msg = "ReadBitResponse(2)" + msg = "ReadBitsResponseBase(2)" self.assertEqual(msg, str(handle)) def testBitReadBaseRequestEncoding(self): diff --git a/test/test_bit_write_messages.py b/test/test_bit_write_messages.py index 8807963a8..18459f553 100644 --- a/test/test_bit_write_messages.py +++ b/test/test_bit_write_messages.py @@ -60,6 +60,8 @@ def testWriteMultipleCoilsRequest(self): self.assertEqual(request.byte_count, 1) self.assertEqual(request.address, 1) self.assertEqual(request.values, [True]*5) + self.assertEqual(request.get_response_pdu_size(), 5) + def testInvalidWriteMultipleCoilsRequest(self): request = WriteMultipleCoilsRequest(1, None) diff --git a/test/test_client_async_asyncio.py b/test/test_client_async_asyncio.py index 64c73ae27..42455f53c 100644 --- a/test/test_client_async_asyncio.py +++ b/test/test_client_async_asyncio.py @@ -3,6 +3,7 @@ if IS_PYTHON3 and PYTHON_VERSION >= (3, 4): from unittest import mock from pymodbus.client.asynchronous.async_io import ( + BaseModbusAsyncClientProtocol, ReconnectingAsyncioModbusTcpClient, ModbusClientProtocol, ModbusUdpClientProtocol) from test.asyncio_test_helper import return_as_coroutine, run_coroutine @@ -10,7 +11,7 @@ from pymodbus.exceptions import ConnectionException from pymodbus.transaction import ModbusSocketFramer from pymodbus.bit_read_message import ReadCoilsRequest, ReadCoilsResponse - protocols = [ModbusUdpClientProtocol, ModbusClientProtocol] + protocols = [BaseModbusAsyncClientProtocol, ModbusUdpClientProtocol, ModbusClientProtocol] else: import mock protocols = [None, None] @@ -18,6 +19,12 @@ @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") class TestAsyncioClient(object): + def test_base_modbus_async_client_protocol(self): + protocol = BaseModbusAsyncClientProtocol() + assert protocol.factory is None + assert protocol.transport is None + assert not protocol._connected + def test_protocol_connection_state_propagation_to_factory(self): protocol = ModbusClientProtocol() assert protocol.factory is None @@ -28,7 +35,8 @@ def test_protocol_connection_state_propagation_to_factory(self): protocol.connection_made(mock.sentinel.TRANSPORT) assert protocol.transport is mock.sentinel.TRANSPORT - protocol.factory.protocol_made_connection.assert_called_once_with(protocol) + protocol.factory.protocol_made_connection.assert_called_once_with( + protocol) assert protocol.factory.protocol_lost_connection.call_count == 0 protocol.factory.reset_mock() @@ -36,7 +44,19 @@ def test_protocol_connection_state_propagation_to_factory(self): protocol.connection_lost(mock.sentinel.REASON) assert protocol.transport is None assert protocol.factory.protocol_made_connection.call_count == 0 - protocol.factory.protocol_lost_connection.assert_called_once_with(protocol) + protocol.factory.protocol_lost_connection.assert_called_once_with( + protocol) + protocol.raise_future = mock.MagicMock() + request = mock.MagicMock() + protocol.transaction.addTransaction(request, 1) + protocol.connection_lost(mock.sentinel.REASON) + if PYTHON_VERSION.major == 3 and PYTHON_VERSION.minor == 6: + call_args = protocol.raise_future.call_args[0] + else: + call_args = protocol.raise_future.call_args.args + protocol.raise_future.assert_called_once() + assert call_args[0] == request + assert isinstance(call_args[1], ConnectionException) def test_factory_initialization_state(self): mock_protocol_class = mock.MagicMock() @@ -116,15 +136,18 @@ def test_factory_protocol_lost_connection(self, mock_async): assert not client.connected assert client.protocol is None - @mock.patch('pymodbus.client.asynchronous.async_io.asyncio.ensure_future') - def test_factory_start_success(self, mock_async): + # @mock.patch('pymodbus.client.asynchronous.async_io.asyncio.ensure_future') + @pytest.mark.asyncio + async def test_factory_start_success(self): mock_protocol_class = mock.MagicMock() - mock_loop = mock.MagicMock() - client = ReconnectingAsyncioModbusTcpClient(protocol_class=mock_protocol_class, loop=mock_loop) + # mock_loop = mock.MagicMock() + client = ReconnectingAsyncioModbusTcpClient(protocol_class=mock_protocol_class) + # client = ReconnectingAsyncioModbusTcpClient(protocol_class=mock_protocol_class, loop=mock_loop) - run_coroutine(client.start(mock.sentinel.HOST, mock.sentinel.PORT)) - mock_loop.create_connection.assert_called_once_with(mock.ANY, mock.sentinel.HOST, mock.sentinel.PORT) - assert mock_async.call_count == 0 + await client.start(mock.sentinel.HOST, mock.sentinel.PORT) + # run_coroutine(client.start(mock.sentinel.HOST, mock.sentinel.PORT)) + # mock_loop.create_connection.assert_called_once_with(mock.ANY, mock.sentinel.HOST, mock.sentinel.PORT) + # assert mock_async.call_count == 0 @mock.patch('pymodbus.client.asynchronous.async_io.asyncio.ensure_future') def test_factory_start_failing_and_retried(self, mock_async): @@ -227,27 +250,34 @@ def testClientProtocolDataReceived(self, protocol): # setup existing request d = protocol._buildResponse(0x00) - if isinstance(protocol, ModbusClientProtocol): - protocol.data_received(data) - else: + if isinstance(protocol, ModbusUdpClientProtocol): protocol.datagram_received(data, None) + else: + protocol.data_received(data) result = d.result() assert isinstance(result, ReadCoilsResponse) - @pytest.mark.skip("To fix") + # @pytest.mark.skip("To fix") + @pytest.mark.asyncio @pytest.mark.parametrize("protocol", protocols) - def testClientProtocolExecute(self, protocol): + async def testClientProtocolExecute(self, protocol): ''' Test the client protocol execute method ''' + import asyncio framer = ModbusSocketFramer(None) protocol = protocol(framer=framer) + protocol.create_future = mock.MagicMock() + fut = asyncio.Future() + fut.set_result(fut) + protocol.create_future.return_value = fut transport = mock.MagicMock() protocol.connection_made(transport) protocol.transport.write = mock.Mock() request = ReadCoilsRequest(1, 1) - d = protocol.execute(request) + d = await protocol.execute(request) tid = request.transaction_id - assert d == protocol.transaction.getTransaction(tid) + f = protocol.transaction.getTransaction(tid) + assert d == f @pytest.mark.parametrize("protocol", protocols) def testClientProtocolHandleResponse(self, protocol): @@ -257,7 +287,9 @@ def testClientProtocolHandleResponse(self, protocol): protocol.connection_made(transport=transport) reply = ReadCoilsRequest(1, 1) reply.transaction_id = 0x00 - + # if isinstance(protocol.create_future, mock.MagicMock): + # import asyncio + # protocol.create_future.return_value = asyncio.Future() # handle skipped cases protocol._handleResponse(None) protocol._handleResponse(reply) @@ -272,6 +304,9 @@ def testClientProtocolHandleResponse(self, protocol): def testClientProtocolBuildResponse(self, protocol): ''' Test the udp client protocol builds responses ''' protocol = protocol() + # if isinstance(protocol.create_future, mock.MagicMock): + # import asyncio + # protocol.create_future.return_value = asyncio.Future() assert not len(list(protocol.transaction)) d = protocol._buildResponse(0x00) diff --git a/test/test_client_sync.py b/test/test_client_sync.py index 1014a9382..5853713fe 100644 --- a/test/test_client_sync.py +++ b/test/test_client_sync.py @@ -192,8 +192,10 @@ def testBasicSyncTcpClient(self, mock_select): def testTcpClientConnect(self): ''' Test the tcp client connection method''' with patch.object(socket, 'create_connection') as mock_method: - mock_method.return_value = object() + _socket = MagicMock() + mock_method.return_value = _socket client = ModbusTcpClient() + _socket.getsockname.return_value = ('dmmy', 1234) self.assertTrue(client.connect()) with patch.object(socket, 'create_connection') as mock_method: diff --git a/test/test_datastore.py b/test/test_datastore.py index cd9c44d3a..03763b10a 100644 --- a/test/test_datastore.py +++ b/test/test_datastore.py @@ -98,9 +98,44 @@ def testModbusSparseDataBlock(self): block.setValues(0x00, dict(enumerate([False]*10))) self.assertEqual(block.getValues(0x00, 10), [False]*10) + block = ModbusSparseDataBlock({3: [10, 11, 12], 10: 1, 15: [0] * 4}) + self.assertEqual(block.values, {3: 10, 4: 11, 5: 12, 10: 1, + 15:0 , 16:0, 17:0, 18:0 }) + self.assertEqual(block.default_value, {3: 10, 4: 11, 5: 12, 10: 1, + 15:0 , 16:0, 17:0, 18:0 }) + self.assertEqual(block.mutable, True) + block.setValues(3, [20, 21, 22, 23], use_as_default=True) + self.assertEqual(block.getValues(3, 4), [20, 21, 22, 23]) + self.assertEqual(block.default_value, {3: 20, 4: 21, 5: 22, 6:23, 10: 1, + 15:0 , 16:0, 17:0, 18:0 }) + # check when values is a dict, address is ignored + block.setValues(0, {5: 32, 7: 43}) + self.assertEqual(block.getValues(5, 3), [32, 23, 43]) + + # assert value is empty dict when initialized without params + block = ModbusSparseDataBlock() + self.assertEqual(block.values, {}) + + # mark block as unmutable and see if parameter exeception + # is raised for invalid offset writes + block = ModbusSparseDataBlock({1: 100}, mutable=False) + self.assertRaises(ParameterException, block.setValues, 0, 1) + self.assertRaises(ParameterException, block.setValues, 0, {2: 100}) + self.assertRaises(ParameterException, block.setValues, 0, [1] * 10) + + # Reset datablock + block = ModbusSparseDataBlock({3: [10, 11, 12], 10: 1, 15: [0] * 4}) + block.setValues(0, {3: [20, 21, 22], 10: 11, 15: [10] * 4}) + self.assertEqual(block.values, {3: 20, 4: 21, 5: 22, 10: 11, + 15: 10 ,16:10, 17:10, 18:10 }) + block.reset() + self.assertEqual(block.values, {3: 10, 4: 11, 5: 12, 10: 1, + 15: 0, 16: 0, 17: 0, 18: 0}) + + def testModbusSparseDataBlockFactory(self): ''' Test the sparse data block store factory ''' - block = ModbusSparseDataBlock.create() + block = ModbusSparseDataBlock.create([0x00]*65536) self.assertEqual(block.getValues(0x00, 65536), [False]*65536) def testModbusSparseDataBlockOther(self): @@ -109,6 +144,7 @@ def testModbusSparseDataBlockOther(self): self.assertRaises(ParameterException, lambda: ModbusSparseDataBlock(True)) + def testModbusSlaveContext(self): ''' Test a modbus slave context ''' store = { diff --git a/test/test_server_asyncio.py b/test/test_server_asyncio.py index 42cb67ad4..84c1b025f 100755 --- a/test/test_server_asyncio.py +++ b/test/test_server_asyncio.py @@ -72,6 +72,7 @@ def tearDown(self): # Test ModbusConnectedRequestHandler #-----------------------------------------------------------------------# @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testStartTcpServer(self): ''' Test that the modbus tcp asyncio server starts correctly ''' identity = ModbusDeviceIdentification(info={0x00: 'VendorName'}) @@ -99,6 +100,7 @@ def testTcpServerServeForever(self): serve.assert_awaited() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTcpServerServeForeverTwice(self): ''' Call on serve_forever() twice should result in a runtime error ''' server = yield from StartTcpServer(context=self.context,address=("127.0.0.1", 0), loop=self.loop) @@ -112,6 +114,7 @@ def testTcpServerServeForeverTwice(self): server.server_close() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTcpServerReceiveData(self): ''' Test data sent on socket is received by internals - doesn't not process data ''' data = b'\x01\x00\x00\x00\x00\x06\x01\x03\x00\x00\x00\x19' @@ -146,6 +149,7 @@ def eof_received(self): server.server_close() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTcpServerRoundtrip(self): ''' Test sending and receiving data on tcp socket ''' data = b"\x01\x00\x00\x00\x00\x06\x01\x03\x00\x00\x00\x01" # unit 1, read register @@ -186,6 +190,7 @@ def eof_received(self): server.server_close() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTcpServerConnectionLost(self): ''' Test tcp stream interruption ''' data = b"\x01\x00\x00\x00\x00\x06\x01\x01\x00\x00\x00\x01" @@ -222,6 +227,7 @@ def connection_made(self, transport): server.server_close() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTcpServerCloseActiveConnection(self): ''' Test server_close() while there are active TCP connections ''' data = b"\x01\x00\x00\x00\x00\x06\x01\x01\x00\x00\x00\x01" @@ -254,6 +260,7 @@ def connection_made(self, transport): self.assertTrue( len(server.active_connections) == 0 ) @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTcpServerNoSlave(self): ''' Test unknown slave unit exception ''' context = ModbusServerContext(slaves={0x01: self.store, 0x02: self.store }, single=False) @@ -290,6 +297,7 @@ def eof_received(self): server.server_close() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTcpServerModbusError(self): ''' Test sending garbage data on a TCP socket should drop the connection ''' data = b"\x01\x00\x00\x00\x00\x06\x01\x03\x00\x00\x00\x01" # get slave 5 function 3 (holding register) @@ -329,6 +337,7 @@ def eof_received(self): server.server_close() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTcpServerInternalException(self): ''' Test sending garbage data on a TCP socket should drop the connection ''' data = b"\x01\x00\x00\x00\x00\x06\x01\x03\x00\x00\x00\x01" # get slave 5 function 3 (holding register) @@ -373,6 +382,7 @@ def eof_received(self): # Test ModbusTlsProtocol #-----------------------------------------------------------------------# @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testStartTlsServer(self): ''' Test that the modbus tls asyncio server starts correctly ''' with patch.object(ssl.SSLContext, 'load_cert_chain') as mock_method: @@ -404,6 +414,7 @@ def testTlsServerServeForever(self): serve.assert_awaited() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTlsServerServeForeverTwice(self): ''' Call on serve_forever() twice should result in a runtime error ''' with patch.object(ssl.SSLContext, 'load_cert_chain') as mock_method: @@ -423,6 +434,7 @@ def testTlsServerServeForeverTwice(self): #-----------------------------------------------------------------------# @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testStartUdpServer(self): ''' Test that the modbus udp asyncio server starts correctly ''' identity = ModbusDeviceIdentification(info={0x00: 'VendorName'}) @@ -449,6 +461,7 @@ def testUdpServerServeForeverStart(self): serve.assert_awaited() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testUdpServerServeForeverClose(self): ''' Test StartUdpServer serve_forever() method ''' server = yield from StartUdpServer(context=self.context,address=("127.0.0.1", 0), loop=self.loop) @@ -465,6 +478,7 @@ def testUdpServerServeForeverClose(self): self.assertTrue(server.protocol.is_closing()) @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testUdpServerServeForeverTwice(self): ''' Call on serve_forever() twice should result in a runtime error ''' identity = ModbusDeviceIdentification(info={0x00: 'VendorName'}) @@ -480,6 +494,7 @@ def testUdpServerServeForeverTwice(self): server.server_close() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testUdpServerReceiveData(self): ''' Test that the sending data on datagram socket gets data pushed to framer ''' server = yield from StartUdpServer(context=self.context,address=("127.0.0.1", 0),loop=self.loop) @@ -501,6 +516,7 @@ def testUdpServerReceiveData(self): server.server_close() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testUdpServerSendData(self): ''' Test that the modbus udp asyncio server correctly sends data outbound ''' identity = ModbusDeviceIdentification(info={0x00: 'VendorName'}) @@ -543,6 +559,7 @@ def datagram_received(self, data, addr): yield from asyncio.sleep(0.1) @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testUdpServerRoundtrip(self): ''' Test sending and receiving data on udp socket''' data = b"\x01\x00\x00\x00\x00\x06\x01\x03\x00\x00\x00\x01" # unit 1, read register @@ -581,6 +598,7 @@ def datagram_received(self, data, addr): server.server_close() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testUdpServerException(self): ''' Test sending garbage data on a TCP socket should drop the connection ''' garbage = b'\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF' @@ -619,16 +637,19 @@ def datagram_received(self, data, addr): # -----------------------------------------------------------------------# # Test ModbusServerFactory # -----------------------------------------------------------------------# + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testModbusServerFactory(self): ''' Test the base class for all the clients ''' with self.assertWarns(DeprecationWarning): factory = ModbusServerFactory(store=None) + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testStopServer(self): with self.assertWarns(DeprecationWarning): StopServer() @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTcpServerException(self): ''' Sending garbage data on a TCP socket should drop the connection ''' garbage = b'\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF' @@ -669,6 +690,7 @@ def eof_received(self): @asyncio.coroutine + @pytest.mark.skipif(not IS_PYTHON3, reason="requires python3.4 or above") def testTcpServerException(self): ''' Sending garbage data on a TCP socket should drop the connection ''' garbage = b'\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF' diff --git a/test/test_server_sync.py b/test/test_server_sync.py index 74ba0cfa5..526da1a5f 100644 --- a/test/test_server_sync.py +++ b/test/test_server_sync.py @@ -365,9 +365,9 @@ def testSerialServerServeForever(self): with patch('pymodbus.server.sync.CustomSingleRequestHandler') as mock_handler: server = ModbusSerialServer(None) instance = mock_handler.return_value - instance.handle.side_effect = server.server_close + instance.response_manipulator.side_effect = server.server_close server.serve_forever() - instance.handle.assert_any_call() + instance.response_manipulator.assert_any_call() def testSerialServerClose(self): ''' test that the synchronous serial server closes correctly '''