Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fake redis server to properly test connection handshake #2947

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

kristjanvalur
Copy link
Contributor

Pull Request check-list

Please make sure to review and check all of these items:

  • Do tests and lints pass with this change?
  • Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Is there an example added to the examples folder (if applicable)?
  • Was the change added to CHANGES file?

Description of change

This PR provides a transport agnostic RESP parser and encoder for the unit tests.
It also provides a simple, transport agnostic, RESP server. This allows us to share code and logic
between sync and async code.
A parametrized test is added to test the connection handshake for different client settings against different fake REDIS servers,
both sync and async.

  • the RESP parser / encoder is simple, assumes utf8 encoding, and always does decoding (from bytes to string.)
  • Decoding uses the 'surrogateescape' convention, so that if binary data was intended (a blob) it can always be recovered by re-encoding.
  • Decoding supports most RESP 3 data types
  • Encoding assumes utf8, and chooses RESP3 types if protocol 3 is enabled, when appropriate.
  • RESP server receives resp command arrays and has handlers to respond. Can be expanded to perform more extensive Redis emulation
  • RESP server can be plugged into a sync or async request handler to simplify testing.

@codecov-commenter
Copy link

codecov-commenter commented Sep 16, 2023

Codecov Report

Attention: 35 lines in your changes are missing coverage. Please review.

Comparison is base (0113034) 91.32% compared to head (5a79252) 91.56%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2947      +/-   ##
==========================================
+ Coverage   91.32%   91.56%   +0.24%     
==========================================
  Files         126      128       +2     
  Lines       32719    33258     +539     
==========================================
+ Hits        29881    30454     +573     
+ Misses       2838     2804      -34     
Files Coverage Δ
tests/test_resp.py 100.00% <100.00%> (ø)
tests/test_asyncio/test_connect.py 96.03% <91.66%> (-2.07%) ⬇️
tests/test_connect.py 95.45% <91.66%> (-1.54%) ⬇️
tests/resp.py 92.46% <92.46%> (ø)

... and 6 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

tests/resp.py Outdated
"""

def __init__(
self, protocol: int = 2, encoding: str = "utf-8", errorhander="strict"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why no type hint for errorhander here?

tests/resp.py Outdated
if self.generator is not None:
self.generator.close()
self.generator = None
del self.consumed[:]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe do self.consumed.clear() here to make it more readable? same for all other places

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair point

if isinstance(data, dict):
if self.protocol > 2:
code = "|" if isinstance(data, Attribute) else "%"
result = f"{code}{len(data)}\r\n".encode()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe use CRNL variable here and in all other lines below since its already defined?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, CRNL is a bytes object, just a shorthand for parsing, not emitting where we use str.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, i commented on wrong line. there are still a few insurances of b"\r\n" in code in addition to strings

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I like it like that since it makes the code more consistent and explicit where we are writing RESP responses.

@kristjanvalur kristjanvalur marked this pull request as ready for review September 17, 2023 12:48
@chayim chayim added the maintenance Maintenance (CI, Releases, etc) label Sep 19, 2023
@kristjanvalur kristjanvalur force-pushed the kristjan/resp-tests branch 2 times, most recently from 6f55c63 to 78a5f92 Compare October 17, 2023 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Maintenance (CI, Releases, etc)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants