Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

♻️ Changing == by is_ in sqlalchemy queries #6654

Merged

Conversation

pcrespov
Copy link
Member

@pcrespov pcrespov commented Nov 1, 2024

What do these changes do?

I've encountered an issue several times where ruff automatically converts expressions like x == True to x is True. This becomes problematic in SQLAlchemy queries when expressions like table.c.x == True are involved, as ruff will transform it to table.c.x is True. While this is syntactically valid, it produces a different SQL query

import sqlalchemy as sa
from simcore_postgres_database.models.file_meta_data import *
from simcore_postgres_database.utils import *

wrong = sa.select(file_meta_data.c.file_id).where(file_meta_data.c.is_soft_link is True)  # WRONG
right = sa.select(file_meta_data.c.file_id).where(file_meta_data.c.is_soft_link.is_(True) ) # RIGHT

print( as_postgres_sql_query_str(wrong) )
print( as_postgres_sql_query_str(right) )

creates

SELECT file_meta_data.file_id 
FROM file_meta_data 
WHERE false

which will never return anything (wrong!) and

SELECT file_meta_data.file_id 
FROM file_meta_data 
WHERE file_meta_data.is_soft_link IS true

which is the correct filter

I've noticed this bug on multiple occasions and, to prevent it from reoccurring, I've created this PR to update all such occurrences in our repository. This PR also serves as a reminder.

Suggestion

To avoid issues in your SQLAlchemy queries,

  • use table.c.x.is_(True) instead of table.c.x == True.
  • use table.c.x.is_not(True) instead of table.c.x != True.
  • analogously with False and None (when nullable).

Related issue/s

Maintenance.

How to test

None

Dev-ops checklist

None

@pcrespov pcrespov added the t:maintenance Some planned maintenance work label Nov 1, 2024
@pcrespov pcrespov added this to the Event Horizon milestone Nov 1, 2024
@pcrespov pcrespov self-assigned this Nov 1, 2024
Copy link

sonarqubecloud bot commented Nov 1, 2024

@pcrespov pcrespov changed the title ♻️ Changing == by is_ in sqlalchemy ♻️ Changing == by is_ in sqlalchemy queries Nov 1, 2024
Copy link

codecov bot commented Nov 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.48%. Comparing base (0bc2185) to head (28ff9f8).
Report is 1 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (0bc2185) and HEAD (28ff9f8). Click for more details.

HEAD has 27 uploads less than BASE
Flag BASE (0bc2185) HEAD (28ff9f8)
unittests 30 3
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6654      +/-   ##
==========================================
- Coverage   87.87%   82.48%   -5.40%     
==========================================
  Files        1563      609     -954     
  Lines       62881    30889   -31992     
  Branches     2088      265    -1823     
==========================================
- Hits        55258    25478   -29780     
+ Misses       7305     5350    -1955     
+ Partials      318       61     -257     
Flag Coverage Δ
integrationtests 64.84% <ø> (+0.01%) ⬆️
unittests 87.78% <ø> (+1.95%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library ∅ <ø> (∅)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 77.44% <ø> (-7.84%) ⬇️
agent ∅ <ø> (∅)
api_server ∅ <ø> (∅)
autoscaling ∅ <ø> (∅)
catalog ∅ <ø> (∅)
clusters_keeper ∅ <ø> (∅)
dask_sidecar ∅ <ø> (∅)
datcore_adapter ∅ <ø> (∅)
director ∅ <ø> (∅)
director_v2 76.39% <ø> (-14.46%) ⬇️
dynamic_scheduler ∅ <ø> (∅)
dynamic_sidecar 59.72% <ø> (-30.00%) ⬇️
efs_guardian ∅ <ø> (∅)
invitations ∅ <ø> (∅)
osparc_gateway_server 79.68% <ø> (-5.47%) ⬇️
payments ∅ <ø> (∅)
resource_usage_tracker ∅ <ø> (∅)
storage ∅ <ø> (∅)
webclient ∅ <ø> (∅)
webserver 89.30% <ø> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0bc2185...28ff9f8. Read the comment docs.

Copy link
Contributor

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wau, cool thanks!

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct

@pcrespov pcrespov merged commit b7814bb into ITISFoundation:master Nov 4, 2024
84 of 88 checks passed
@pcrespov pcrespov deleted the mai/sqlachemy_and_rust_issue branch November 4, 2024 10:45
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request Nov 6, 2024
57 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
t:maintenance Some planned maintenance work
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants