AIP-81 Add Insert Multiple Pools API #44121
Conversation
jason810496
requested review from
bbovenzi,
pierrejeambrun,
ryanahamilton,
jscheffl and
ephraimbuddy
as code owners
jason810496
force-pushed
the
feature/AIP-81/add-bulk-insert-pools-api
branch
from
9fa560e
to
42e8a51
Compare
|
Looks good! Thanks for the changes @jason810496! I believe the CI failure is likely a transient error caused by a DNS issue that couldn’t resolve AWS S3 at that moment. |
just reran. seems to work fine 👀 |
- handle exception from db level instead of application level
jason810496
force-pushed
the
feature/AIP-81/add-bulk-insert-pools-api
branch
from
42e8a51
to
1ec682f
Compare
|
Just resolved the issue with inserting duplicate pools by handling the database exception rather than adding extra logic to the |
| response = test_client.post("/public/pools/", json=body) | ||
| assert response.status_code == expected_status_code | ||
|
|
||
| assert response.json() == expected_response | ||
| if check_count: | ||
| assert session.query(Pool).count() == n_pools + 1 | ||
|
|
There was a problem hiding this comment.
Maybe don't factorize this part. It looks like this is an Arrange function (sense of Arrange/Act/Assert), like others similar function in tests, to prepare some objects for the test. But then it actually is a 'full' test, with assertions etc...
We only re-use this at 2 places, we can leave that inlined, especially the code won't be exacly the same and simpler for the 409 case
That would be great. You can create an issue to track that if you're not going to work on it directly just so we do not forget about it. If you plan on tackling that right away, no need to create an issue opening a PR is perfectly fine. |
| session.commit() | ||
| except IntegrityError as e: | ||
| session.rollback() | ||
| raise HTTPException( | ||
| status.HTTP_409_CONFLICT, | ||
| detail=f"One or more pools already exists. Error: {e}", | ||
| ) |
There was a problem hiding this comment.
Actually I am second guessing this.
The session is already commited on request exit.
Maybe what we need is simply an additional exception handler for IngegrityError, global.
Because here the context manager will try to commit again an empty session on exiting the endpoint. Which is a little bit weird and unecessary. Also this would remove the need to try / commit/rollback manually.
There was a problem hiding this comment.
Maybe something like this:
https://fastapi.tiangolo.com/tutorial/handling-errors/#install-custom-exception-handlers
And we check that in the error there is Unique and then we return 409 response ?
So we can just remove all the try / catch / etc... specific code, and the double call to commit()
closes: #43896
related: #43657
Fix: Add
max_length=256forPoolPostBody.poolThe model defined in https://github.com/apache/airflow/blob/main/airflow/models/pool.py#L54 enforces a string length constraint on
Pool.pool. To maintain consistency, this constraint should also be validated at the router level.Refactor: Handle Duplicate Cases in
Insert PoolThe
Insert Pool(single insert) functionality should account for cases where a duplicate pool name is provided, asPool.poolis defined with auniqueconstraint.Feat: Add Insert Multiple Pools API
A new API endpoint for inserting multiple pools has been introduced as
/pools/bulk. Alternative names such as/pools/batchor/pools/multiplewere considered. Feedback on which name best describes the endpoint is welcome.