Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update various Python dependencies to latest stable versions #4767

Merged
merged 7 commits into from
Aug 19, 2019

Conversation

Kami
Copy link
Member

@Kami Kami commented Aug 16, 2019

This pull request updates various dependencies to latest stable versions.

I went over the changeslogs and changed things mostly include bug fixes and small improvements.

mongoengine v0.18.x includes change / performance improvement which should be of a particular interest to us - MongoEngine/mongoengine#2049.

I tested / verified it locally and confirmed it increases "save" performance for very large objects (that's very large executions in our case). It's not really drastic, but around 5-30% performance increase depending on the execution size.

It's not as fast as directly inserting execution into database using pymongo and avoiding mongoengine layer all together (something we discussed in the past), but it comes mostly "for free".

@Kami Kami added this to the 3.2.0 milestone Aug 16, 2019
@Kami Kami requested a review from arm4b August 16, 2019 08:24
@Kami
Copy link
Member Author

Kami commented Aug 16, 2019

There seems to be regression in the latest version of pymongo and mongoengine (and potentially combination of both) related to unicode keys which causes test failure :/

I need to dig in and look into a fix.

@Kami
Copy link
Member Author

Kami commented Aug 16, 2019

It turns out the issue is indeed related to pymongo and mongoengine - MongoEngine/mongoengine#2147

I wasted quite a lot of time tracking it down (since it depends on Python version and default encoding used by Python).

In the mean time, I will revert pymongo upgrade and in the future, I will try to get that fix finished and merged into upstream mongoengine library.

CHANGELOG.rst Outdated
@@ -17,6 +17,8 @@ Changed
* Install pack with the latest tag version if it exists when branch is not specialized.
(improvement) #4743
* Implement "continue" engine command to orquesta workflow. (improvement) #4740
* Update various internal dependencies to latest stable versions (apscheduler, eventlet,
kombu, amqp, pyyaml, mongoengine, python-gnupg, paramiko, tooz, webob, bcrypt). #4767
Copy link
Member

@arm4b arm4b Aug 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to include a small note that mongo upgrade led to perf improvements, considering you've tested it.

Copy link
Member

@arm4b arm4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@Kami Kami merged commit 26ffc2f into master Aug 19, 2019
@arm4b arm4b deleted the deps_update_320 branch August 19, 2019 15:41
@blag blag mentioned this pull request Aug 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants