Support box.error #245

DifferentialOrange · 2022-10-13T14:36:23Z

setup: clarify Python versions

We run tests with Python 3.6 and newer. Python 3.5 has reached its end of life two years ago [1]. Our code already doesn't work on Python 3.5 [2].

Part of #232

deps: backport dataclasses for Python 3.6

Dataclasses would be a really convenient approach to provide box.error support. They are supported in Python since 3.7, but this package make it able to introduce their support in Python 3.6 which we still support.

Part of #232

api: ConnectionPool support for Python 3.6

After backporting dataclasses, it became possible to use ConnectionPool on Python 3.6.

Follows #196

msgpack: improve unknown ext type error message

Part of #232

iproto: support errors extra information

Since Tarantool 2.4.1, iproto error responses contain extra info with backtrace. After this patch, DatabaseError would contain extra_info property, if it was provided.

Error extra information is parsed based on common encoder/decoder rules. String fields are converted to either str or bytes based on encoding mode.

https://www.tarantool.io/en/doc/latest/dev_guide/internals/box_protocol/#responses-for-errors

Part of #232

msgpack: support error extension type

Tarantool supports error extension type since version 2.4.1 [1], encoding was introduced in Tarantool 2.10.0 [2]. This patch introduces the support of Tarantool error extension type in msgpack decoders and encoders.

Tarantool error extension type objects are decoded to tarantool.BoxError type. tarantool.BoxError may be encoded to Tarantool error extension type objects.

Error extension type internals are the same as errors extra information: the only difference is that extra information is encoded as a separate error dictionary field and error extension type objects is encoded as MessagePack extension type objects.

Error extension type objects are parsed based on common encoder/decoder rules. String fields are converted to either str or bytes based on encoding mode.

The only way to receive an error extension type object from Tarantool is to receive an explicitly built box.error object: either from
return box.error.new(...) or a tuple with it. All errors raised within Tarantool (including those raised with box.error(...)) are encoded based on the same rules as simple errors due to backward compatibility.

It is possible to create error extension type objects with Python code, but it not likely to be really useful since most of their fields is
computed on error initialization on the server side (even for custom error types):

tarantool.BoxError(
    type='ClientError',
    file='[string "                     local err = box.error.ne..."]',
    line=1,
    message='Unknown error',
    errno=0,
    errcode=0,
)

Closes #232

DifferentialOrange · 2022-10-21T13:47:33Z

There is some minor CI issue after rebase, I think it is still possible to review current version.

tarantool/msgpack_ext/error.py

tarantool/types.py

DifferentialOrange · 2022-10-24T09:37:06Z

There is some minor CI issue after rebase, I think it is still possible to review current version.

Fixed in separate PR #251 (this PR could be reviewed independently).

We run tests with Python 3.6 and newer. Python 3.5 has reached its end of life two years ago [1]. Our code already doesn't work on Python 3.5 [2]. 1. https://endoflife.date/python 2. https://github.com/tarantool/tarantool-python/actions/runs/3243224481/jobs/5317632662 Part of #232

Dataclasses would be a really convenient approach to provide box.error support. They are supported in Python since 3.7, but this package make it able to introduce their support in Python 3.6 which we still support. Part of #232

After backporting dataclasses, it became possible to use ConnectionPool on Python 3.6. Follows #196

Part of #232

Since Tarantool 2.4.1, iproto error responses contain extra info with backtrace. After this patch, DatabaseError would contain `extra_info` property, if it was provided. Error extra information is parsed based on common encoder/decoder rules. String fields are converted to either `str` or `bytes` based on `encoding` mode. 1. https://www.tarantool.io/en/doc/latest/dev_guide/internals/box_protocol/#responses-for-errors Part of #232

Tarantool supports error extension type since version 2.4.1 [1], encoding was introduced in Tarantool 2.10.0 [2]. This patch introduces the support of Tarantool error extension type in msgpack decoders and encoders. Tarantool error extension type objects are decoded to `tarantool.BoxError` type. `tarantool.BoxError` may be encoded to Tarantool error extension type objects. Error extension type internals are the same as errors extra information: the only difference is that extra information is encoded as a separate error dictionary field and error extension type objects is encoded as MessagePack extension type objects. Error extension type objects are parsed based on common encoder/decoder rules. String fields are converted to either `str` or `bytes` based on `encoding` mode. The only way to receive an error extension type object from Tarantool is to receive an explicitly built `box.error` object: either from `return box.error.new(...)` or a tuple with it. All errors raised within Tarantool (including those raised with `box.error(...)`) are encoded based on the same rules as simple errors due to backward compatibility. It is possible to create error extension type objects with Python code, but it not likely to be really useful since most of their fields is computed on error initialization on the server side (even for custom error types): ``` tarantool.BoxError( type='ClientError', file='[string " local err = box.error.ne..."]', line=1, message='Unknown error', errno=0, errcode=0, ) ``` 1. tarantool/tarantool#4398 2. tarantool/tarantool#6433 Closes #232

oleg-jukovec

Thank you!

Overview This release introduces the support of extention types (decimal, uuid, error, datetime, interval) in MessagePack, various IProto features support (feature discovery and push protocol) and major infrastructure updates (scm version computation, full documentation for external and internal API both as code docstrings and readthedocs HTML, deb and RPM packages, and everything is processed with CI/CD pipelines). Breaking changes This release should not break any existing behavior. New features - Backport ConnectionPool support for Python 3.6 (PR #245). - Support iproto feature discovery (#206). - Decimal type support (#203). - UUID type support (#202). - Support extra information for iproto errors (#232). - Error extension type support (#232). - Datetime type support and tarantool.Datetime type (#204, PR #252). Tarantool datetime objects are decoded to `tarantool.Datetime` type. `tarantool.Datetime` may be encoded to Tarantool datetime objects. You can create `tarantool.Datetime` objects either from MessagePack data or by using the same API as in Tarantool: ```python dt1 = tarantool.Datetime(year=2022, month=8, day=31, hour=18, minute=7, sec=54, nsec=308543321) dt2 = tarantool.Datetime(timestamp=1661969274) dt3 = tarantool.Datetime(timestamp=1661969274, nsec=308543321) ``` `tarantool.Datetime` exposes `year`, `month`, `day`, `hour`, `minute`, `sec`, `nsec`, `timestamp` and `value` (integer epoch time with nanoseconds precision) properties if you need to convert `tarantool.Datetime` to any other kind of datetime object: ```python pdt = pandas.Timestamp(year=dt.year, month=dt.month, day=dt.day, hour=dt.hour, minute=dt.minute, second=dt.sec, microsecond=(dt.nsec // 1000), nanosecond=(dt.nsec % 1000)) ``` Use `tzoffset` parameter to set up offset timezone: ```python dt = tarantool.Datetime(year=2022, month=8, day=31, hour=18, minute=7, sec=54, nsec=308543321, tzoffset=180) ``` You may use `tzoffset` property to get timezone offset of a datetime object. Use `tz` parameter to set up timezone name: ```python dt = tarantool.Datetime(year=2022, month=8, day=31, hour=18, minute=7, sec=54, nsec=308543321, tz='Europe/Moscow') ``` If both `tz` and `tzoffset` is specified, `tz` is used. You may use `tz` property to get timezone name of a datetime object. `timestamp_since_utc_epoch` is a parameter to set timestamp convertion behavior for timezone-aware datetimes. If ``False`` (default), behaves similar to Tarantool `datetime.new()`: ```python >>> dt = tarantool.Datetime(timestamp=1640995200, timestamp_since_utc_epoch=False) >>> dt datetime: Timestamp('2022-01-01 00:00:00'), tz: "" >>> dt.timestamp 1640995200.0 >>> dt = tarantool.Datetime(timestamp=1640995200, tz='Europe/Moscow', ... timestamp_since_utc_epoch=False) >>> dt datetime: Timestamp('2022-01-01 00:00:00+0300', tz='Europe/Moscow'), tz: "Europe/Moscow" >>> dt.timestamp 1640984400.0 ``` Thus, if ``False``, datetime is computed from timestamp since epoch and then timezone is applied without any convertion. In that case, `dt.timestamp` won't be equal to initialization `timestamp` for all timezones with non-zero offset. If ``True``, behaves similar to `pandas.Timestamp`: ```python >>> dt = tarantool.Datetime(timestamp=1640995200, timestamp_since_utc_epoch=True) >>> dt datetime: Timestamp('2022-01-01 00:00:00'), tz: "" >>> dt.timestamp 1640995200.0 >>> dt = tarantool.Datetime(timestamp=1640995200, tz='Europe/Moscow', ... timestamp_since_utc_epoch=True) >>> dt datetime: Timestamp('2022-01-01 03:00:00+0300', tz='Europe/Moscow'), tz: "Europe/Moscow" >>> dt.timestamp 1640995200.0 ``` Thus, if ``True``, datetime is computed in a way that `dt.timestamp` will always be equal to initialization `timestamp`. - Datetime interval type support and tarantool.Interval type (#229). Tarantool datetime interval objects are decoded to `tarantool.Interval` type. `tarantool.Interval` may be encoded to Tarantool interval objects. You can create `tarantool.Interval` objects either from MessagePack data or by using the same API as in Tarantool: ```python di = tarantool.Interval(year=-1, month=2, day=3, hour=4, minute=-5, sec=6, nsec=308543321, adjust=tarantool.IntervalAdjust.NONE) ``` Its attributes (same as in init API) are exposed, so you can use them if needed. - Datetime interval arithmetic support (#229). Valid operations: - `tarantool.Datetime` + `tarantool.Interval` = `tarantool.Datetime` - `tarantool.Datetime` - `tarantool.Interval` = `tarantool.Datetime` - `tarantool.Datetime` - `tarantool.Datetime` = `tarantool.Interval` - `tarantool.Interval` + `tarantool.Interval` = `tarantool.Interval` - `tarantool.Interval` - `tarantool.Interval` = `tarantool.Interval` Since `tarantool.Interval` could contain `month` and `year` fields and such operations could be ambiguous, you can use `adjust` field to tune the logic. The behavior is the same as in Tarantool, see [Interval arithmetic RFC](https://github.com/tarantool/tarantool/wiki/Datetime-Internals#interval-arithmetic). - `tarantool.IntervalAdjust.NONE` -- only truncation toward the end of month performed (default mode). ```python >>> dt = tarantool.Datetime(year=2022, month=3, day=31) datetime: Timestamp('2022-03-31 00:00:00'), tz: "" >>> di = tarantool.Interval(month=1, adjust=tarantool.IntervalAdjust.NONE) >>> dt + di datetime: Timestamp('2022-04-30 00:00:00'), tz: "" ``` - `tarantool.IntervalAdjust.EXCESS` -- overflow mode, without any snap or truncation to the end of month, straight addition of days in month, stopping over month boundaries if there is less number of days. ```python >>> dt = tarantool.Datetime(year=2022, month=1, day=31) datetime: Timestamp('2022-01-31 00:00:00'), tz: "" >>> di = tarantool.Interval(month=1, adjust=tarantool.IntervalAdjust.EXCESS) >>> dt + di datetime: Timestamp('2022-03-02 00:00:00'), tz: "" ``` - `tarantool.IntervalAdjust.LAST` -- mode when day snaps to the end of month, if happens. ```python >>> dt = tarantool.Datetime(year=2022, month=2, day=28) datetime: Timestamp('2022-02-28 00:00:00'), tz: "" >>> di = tarantool.Interval(month=1, adjust=tarantool.IntervalAdjust.LAST) >>> dt + di datetime: Timestamp('2022-03-31 00:00:00'), tz: "" ``` - Full documentation of internal and external API (#67). Bugfixes - Allow any MessagePack supported type as a request key (#240). - Make connection close idempotent (#250). Infrastructure - Use git version to set package version (#238). - Test pip install from branch (PR #241). - Pack and publish pip, RPM and deb packages with GitHub Actions (#164, #198). - Publish on readthedocs with CI/CD (including PRs) (#67).

DifferentialOrange force-pushed the DifferentialOrange/gh-232-error-type branch 4 times, most recently from 4b35e4c to 45819a1 Compare October 14, 2022 09:06

DifferentialOrange changed the base branch from master to DifferentialOrange/gh-206-feature-discovery October 14, 2022 10:47

Base automatically changed from DifferentialOrange/gh-206-feature-discovery to master October 17, 2022 13:10

DifferentialOrange force-pushed the DifferentialOrange/gh-232-error-type branch 24 times, most recently from 7354bc2 to 7a25db9 Compare October 19, 2022 09:10

DifferentialOrange marked this pull request as ready for review October 19, 2022 09:53

DifferentialOrange force-pushed the DifferentialOrange/gh-232-error-type branch 2 times, most recently from 5eba28d to 042765c Compare October 19, 2022 10:04

DifferentialOrange requested a review from oleg-jukovec October 19, 2022 10:06

DifferentialOrange force-pushed the DifferentialOrange/gh-232-error-type branch from 042765c to 9691e66 Compare October 21, 2022 12:34

oleg-jukovec reviewed Oct 24, 2022

View reviewed changes

tarantool/msgpack_ext/error.py Show resolved Hide resolved

tarantool/types.py Show resolved Hide resolved

DifferentialOrange force-pushed the DifferentialOrange/gh-232-error-type branch from 9691e66 to ee004a0 Compare October 24, 2022 09:32

DifferentialOrange changed the base branch from master to DifferentialOrange/gh-250-unittest-warnings October 24, 2022 09:36

DifferentialOrange force-pushed the DifferentialOrange/gh-232-error-type branch 2 times, most recently from da1c4f6 to 00977a6 Compare October 24, 2022 14:35

DifferentialOrange force-pushed the DifferentialOrange/gh-250-unittest-warnings branch from cb12b7b to 7baa481 Compare October 27, 2022 12:18

Base automatically changed from DifferentialOrange/gh-250-unittest-warnings to master October 27, 2022 12:22

DifferentialOrange added 6 commits October 27, 2022 15:41

deps: backport dataclasses for Python 3.6

249e1e8

Dataclasses would be a really convenient approach to provide box.error support. They are supported in Python since 3.7, but this package make it able to introduce their support in Python 3.6 which we still support. Part of #232

api: ConnectionPool support for Python 3.6

cd6a62a

After backporting dataclasses, it became possible to use ConnectionPool on Python 3.6. Follows #196

msgpack: improve unknown ext type error message

40d6ce8

Part of #232

DifferentialOrange force-pushed the DifferentialOrange/gh-232-error-type branch from 00977a6 to 951aa1a Compare October 27, 2022 12:42

DifferentialOrange requested a review from oleg-jukovec October 27, 2022 12:43

DifferentialOrange mentioned this pull request Oct 31, 2022

iproto: support feature push #247

Merged

oleg-jukovec approved these changes Oct 31, 2022

View reviewed changes

DifferentialOrange merged commit bdd4a45 into master Oct 31, 2022

DifferentialOrange deleted the DifferentialOrange/gh-232-error-type branch October 31, 2022 08:53

DifferentialOrange mentioned this pull request Nov 9, 2022

Release 0.10.0 #258

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support box.error #245

Support box.error #245

DifferentialOrange commented Oct 13, 2022 •

edited

Loading

DifferentialOrange commented Oct 21, 2022

DifferentialOrange commented Oct 24, 2022

oleg-jukovec left a comment

Support box.error #245

Support box.error #245

Conversation

DifferentialOrange commented Oct 13, 2022 • edited Loading

setup: clarify Python versions

deps: backport dataclasses for Python 3.6

api: ConnectionPool support for Python 3.6

msgpack: improve unknown ext type error message

iproto: support errors extra information

msgpack: support error extension type

DifferentialOrange commented Oct 21, 2022

DifferentialOrange commented Oct 24, 2022

oleg-jukovec left a comment

Choose a reason for hiding this comment

DifferentialOrange commented Oct 13, 2022 •

edited

Loading