Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pymodbus v2.4.0 #520

Merged
merged 61 commits into from
Sep 11, 2020
Merged

Pymodbus v2.4.0 #520

merged 61 commits into from
Sep 11, 2020

Conversation

dhoomakethu
Copy link
Contributor

Version 2.4.0

  • Support async moduls tls server/client
  • Add local echo option
  • Add exponential backoffs on retries.
  • Fix error with rtu client for local_echo
  • Fix asyncio client not working with already running loop
  • Fix passing serial arguments to async clients
  • Support timeouts to break out of responspe await when server goes offline
  • Misc updates and bugfixes.

dices and others added 30 commits September 28, 2018 02:30
…names

Updated requirements-docs.txt to include missing modules
Commenting out python3.7 from Travis while waiting for support. You can run teh 3.7 tests with tox without issues
Fixing endianness in Payload example.
…ev3.7

# Conflicts:
#	CHANGELOG.rst
#	doc/conf.py
#	pymodbus/framer/ascii_framer.py
#	pymodbus/framer/binary_framer.py
#	pymodbus/framer/rtu_framer.py
#	pymodbus/framer/socket_framer.py
#	pymodbus/repl/client.py
#	pymodbus/repl/completer.py
#	pymodbus/server/asynchronous.py
Dev3.7 async keyword changes
* check self.socket

self.socket might be None at this point

* Update pymodbus/client/sync.py

Co-Authored-By: mpf82 <[email protected]>
* #357 Support registration of custom requests

* #368 Fixes write to broadcast address

When writing to broadcast address (unit_id=0) there should be no response according to the Modbus spec. This fix changes expected_response_length to 0 when writing to unit_id=0. This will break any existing code that is improperly using unit_id 0 for a slave address.

* Bump version to 2.2.0

Fix #366 Update failures in sql context

Update Changelog

Fix major minor version in example codes

* Fix #371 pymodbus repl on python3

* 1. Fix tornado async serial client `TypeError` while processing incoming packet.
2. Fix asyncio examples.
3. Minor update in factory.py, now server logs prints received request instead of only function cod

* [fix v3] poprawa sprawdzania timeout

* Release candidate for pymodbus 2.2.0

*  Fix #377 when invalid port is supplied and minor updates in logging

* #368 adds broadcast support for sync client and server

Adds broadcast_enable parameter to client and server, default value is False. When true it will treat unit_id 0 as broadcast and execute requests on all server slave contexts and not send a response and on the client side will send the request and not try to receive a response.

* #368 Fixes minor bug in broadcast support code

* Fixed erronous CRC handling

If the CRC recieved is not correct in my case my slave got caught in a deadlock, not taking any new requests. This addition fixed that.

* Update Changelog

* Fix test coverage

* Fix #387 Transactions failing on 2.2.0rc2.

* Task Cancellation and CRC Errors

Alternate solution for #356 and #360.

Changes the RTU to make the transaction ID as the unit ID instead of an ever incrementing number.

Previously this transaction ID was always 0 on the receiving end but was the unique transaction ID on sending.

As such the FIFO buffer made the most sense. By tying it to the unit ID, we can recover from failure modes such as: -
- Asyncio task cancellations (eg. timeouts) #360
- Skipped responses from slaves. (hangs on master #360)
- CRC Errors #356
- Busy response

* Cherry pick commit from PR #367 , Update changelog , bump version to 2.2.0rc4

* #389 Support passing all serial port parameters to asynchronous server

* Fix BinaryPayloadDecoder and Builder wrt to coils

* Misc updates, bump version to 2.2.0

* ReportSlaveIdResponse now tries to get slave id based on server identity for pymodbus servers

* Update missing bcrypt requirement for testing
* Fix document generation

* Formatting updates in Changelog
It has not been needed by Twisted for a long time, and has been unmaintained
for a long time.
* Fix document generation

* Formatting updates in Changelog

* Fix --upgrade option in install dependencies
If the response is odd size the buffer needs to be padded with an additional byte.
Python 3.4 is EoL and has an easy upgrade path to 3.5+. Support was
dropped in Twisted 19.7.0, which is causing Travis to fail pymodbus
tests for 3.4.
sanjay and others added 26 commits October 7, 2019 15:43
* client/sync.py: Fix missing serial module dependency

The serial.connect failed in PR riptideio#400 with "NameError: name
'serial' is not defined" [1]:

self = <ModbusSerialClient at 0x7fcda4009b00 socket=None, method=ascii, timeout=3>

    def connect(self):
        """ Connect to the modbus serial server

        :returns: True if connection succeeded, False otherwise
        """
        if self.socket:
            return True
        try:
>           self.socket = serial.Serial(port=self.port,
                                        timeout=self.timeout,
                                        bytesize=self.bytesize,
                                        stopbits=self.stopbits,
                                        baudrate=self.baudrate,
                                        parity=self.parity)
E                                       NameError: name 'serial' is not defined

pymodbus/client/sync.py:476: NameError

This patch moves the serial import back to the head.

[1] https://travis-ci.org/riptideio/pymodbus/jobs/566009109

Fixes: commit e6da559 asyncio server implementation (#400)

* server/asyncio.py: Create server with appropriate args and environment

If Python is older than 3.7, the create_server will fail like PR
riptideio#400 with "unexpected keyword argument 'start_serving'" [1]
which is new in Python version 3.7:

self.server_factory = self.loop.create_server(lambda :self.handler(self),
                                              *self.address,
                                              reuse_address=allow_reuse_address,
                                              reuse_port=allow_reuse_port,
                                              backlog=backlog,
>                                             start_serving=not defer_start)

E       TypeError: create_server() got an unexpected keyword argument 'start_serving'

pymodbus/server/asyncio.py:400: TypeError

This patch creates server according to Python environment.

[1] https://travis-ci.org/starnight/pymodbus/jobs/584178484

Fixes: commit e6da559 asyncio server implementation (#400)

* Create asyncio task with appropriate method and environment

If Python is older than 3.7, the asyncio.create_task will fail like PR
riptideio#400 with "AttributeError: module 'asyncio' has no attribute
'create_task'" [1] which is new in Python version 3.7 [2]:

@asyncio.coroutine

def testTcpServerCloseActiveConnection(self):
    ''' Test server_close() while there are active TCP connections'''
    data = b"\x01\x00\x00\x00\x00\x06\x01\x01\x00\x00\x00\x01"
    server = yield from StartTcpServer(context=self.context,address=("127.0.0.1", 0),loop=self.loop)
>   server_task = asyncio.create_task(server.serve_forever())
E   AttributeError: module 'asyncio' has no attribute 'create_task'

test/test_server_asyncio.py:205: AttributeError

This patch creates task according to Python environment.

[1] https://travis-ci.org/starnight/pymodbus/jobs/584193587
[2] https://docs.python.org/3/library/asyncio-task.html#creating-tasks

Fixes: commit e6da559 asyncio server implementation (#400)

* server/asyncio.py: Fix format string for older Python

If Python is older than 3.6, f-Strings will fail like PR riptideio#400
with "SyntaxError: invalid syntax" [1] which is new in Python version
3.6 with PEP 498 -- Literal String Interpolation [2]:

test/test_server_asyncio.py:14: in <module>
    from pymodbus.server.asyncio import StartTcpServer, StartUdpServer, StartSerialServer, StopServer, ModbusServerFactory
E     File "/home/travis/build/starnight/pymodbus/pymodbus/server/asyncio.py", line 424
E       _logger.warning(f"aborting active session {k}")
E                                                    ^
E   SyntaxError: invalid syntax

This patch fixes the format string with traditional format string
syntax.

[1] https://travis-ci.org/starnight/pymodbus/jobs/584427976
[2] https://www.python.org/dev/peps/pep-0498/

Fixes: commit e6da559 asyncio server implementation (#400)

* test: Make assert_called_once() test only with Python 3.6+

If Python is older than 3.6, unittest.mock.assert_called_once() will
fail like PR riptideio#400 with "AttributeError: assert_called_once" [1]
which is new in Python version 3.6 [2]:

>       self.loop.create_server.assert_called_once()

test/test_server_asyncio.py:76:

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <CoroutineMock name='mock.create_server' id='139638313234784'>
name = 'assert_called_once'

    def __getattr__(self, name):
        if name in {'_mock_methods', '_mock_unsafe'}:
            raise AttributeError(name)
        elif self._mock_methods is not None:
            if name not in self._mock_methods or name in _all_magics:
                raise AttributeError("Mock object has no attribute %r" % name)
        elif _is_magic(name):
            raise AttributeError(name)
        if not self._mock_unsafe:
            if name.startswith(('assert', 'assret')):
>               raise AttributeError(name)
E               AttributeError: assert_called_once

/opt/python/3.5.6/lib/python3.5/unittest/mock.py:585: AttributeError

This patch skips the tests if they are not in Python 3.6+.

[1] https://travis-ci.org/starnight/pymodbus/jobs/584431003
[2] https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called_once

Fixes: commit e6da559 asyncio server implementation (#400)

* test: Make serve_forever() test only with Python 3.7+

If Python is older than 3.7, asyncio.base_events.Server.serve_forever
will fail like PR riptideio#400 with "AttributeError: <class
'asyncio.base_events.Server'> does not have the attribute
'serve_forever'" [1] which is new in Python version 3.7 [2]:

@asyncio.coroutine
def testTcpServerServeNoDefer(self):
    ''' Test StartTcpServer without deferred start (immediate execution of server) '''
>   with patch('asyncio.base_events.Server.serve_forever', new_callable=asynctest.CoroutineMock) as serve:

test/test_server_asyncio.py:81:

...

if not self.create and original is DEFAULT:
    raise AttributeError(
>       "%s does not have the attribute %r" % (target, name)
    )
E   AttributeError: <class 'asyncio.base_events.Server'> does not have the attribute 'serve_forever'

This patch skips the tests if they are not in Python 3.7+.

[1] https://travis-ci.org/starnight/pymodbus/jobs/584212511
[2] https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.Server.serve_forever

Fixes: commit e6da559 asyncio server implementation (#400)
* Add TLS feature for Modbus synchronous

Modbus.org released MODBUS/TCP Security Protocol Specification [1],
which focuses variant of the Mobdbus/TCP protocol utilizing Transport
Layer Security (TLS). This patch enables the Modbus over TLS feature as
ModbusTlsClient with the Python builtin module ssl - TLS/SSL wrapper for
socket objects.

[1]: http://modbus.org/docs/MB-TCP-Security-v21_2018-07-24.pdf

* Implement MODBUS TLS synchronous server

Since we have the MODBUS TLS synchronous client, we can also have the
MODBUS TLS synchronous server.
* rtu_framer: fix processing of incomplete frames

* rtu_framer: fix test case
Added RetryOnInvalid flag and Backoff delay.
Added retry on invalid data received and exponetial backoff delay between retries.
* Add TLS feature for Modbus asynchronous client

Since we have Modbus TLS client in synchronous mode, we can also
implement Modbus TLS client in asynchronous mode with ASYNC_IO.

* Add TLS feature for Modbus asynchronous server

Since we have Modbus TLS server in synchronous mode, we can also
implement Modbus TLS server in asynchronous mode with ASYNC_IO.
Previously only generic "Python" support (without a version) was announced.
* Closes  #491

* 1. update requirements
2. Fix examples
3. Fix #494 - handle_local_echo
4. Fix #500 -- asyncio serial client with already running loop
5. Fix #486 - Pass serial args for asyncio serial client
6. Fix #490 - Typo in decode_data for socker_framer
7. Fix #385 - Support timeouts to break out of responspe await when server goes offline
8. Misc updates

* #516 custom data block fix
* 1. update requirements
2. Fix examples
3. Fix #494 - handle_local_echo
4. Fix #500 -- asyncio serial client with already running loop
5. Fix #486 - Pass serial args for asyncio serial client
6. Fix #490 - Typo in decode_data for socker_framer
7. Fix #385 - Support timeouts to break out of responspe await when server goes offline
8. Misc updates

* #516 custom data block fix

* Fix broadcast error  with REPL client #515

* Fix #509 Wrong unit ID referenced in framers

* Update documentation for serial forwarder example. Fixes #525

* Fix unit tests, support python 3.8 for tests, renamed:    pymodbus/server/asyncio.py -> pymodbus/server/async_io.py and pymodbus/client/asynchronous/asyncio -> pymodbus/client/asynchronous/async_io

* Ignore python3 code syntax while reporting coverage

* Fix tests failing on python 3.6 and osx

* Fix typo in makefile

* Fix test execution errors specific to python3.6

* Osx travis issue - Fix trial 1

* Travis reverting xcode to 8.x for mac osx
@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.

Bug E 13 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 3 Security Hotspots to review)
Code Smell A 62 Code Smells

No Coverage information No Coverage information
7.0% 7.0% Duplication

@dhoomakethu dhoomakethu merged commit 8ef3299 into master Sep 11, 2020
dhoomakethu added a commit that referenced this pull request Sep 11, 2020
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.