Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

156 1mb error handling #179

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from
Open

156 1mb error handling #179

wants to merge 3 commits into from

Conversation

jounaidr
Copy link
Contributor

@jounaidr jounaidr commented Sep 13, 2021 •
edited
Loading

Resolves #156
Resolves #155

If AMS throws the specific 'message too large' exception, catch said exception and move the message to the rejected queue, and log this as a warning.

Logs when testing in dirq mode:


...
2021-09-13 15:28:27,909 - ssm.ssm2 - INFO - Found 1 messages.
2021-09-13 15:28:27,911 - ssm.ssm2 - INFO - Sending message: 12345678/1234567890abcd
2021-09-13 15:28:28,727 - ssm.ssm2 - WARNING - Message 12345678/1234567890abcd could not be sent as its larger than 1MB
2021-09-13 15:28:28,731 - ssm.ssm2 - INFO - Message 12345678/1234567890abcd saved to reject queue as 613f6e00/613f6e1cb290ed
2021-09-13 15:28:28,733 - ssm.ssm2 - INFO - Tidying message directory.
...

Logs when testing in directory mode (with valid messages before and after 'too large' message):


...
2021-09-13 15:39:38,857 - ssm.ssm2 - INFO - Found 3 messages.
2021-09-13 15:39:38,858 - ssm.ssm2 - INFO - Sending message: messagebeforebigone
2021-09-13 15:39:39,208 - ssm.ssm2 - INFO - Sent messagebeforebigone, Argo ID: 80
2021-09-13 15:39:39,213 - ssm.ssm2 - INFO - Sending message: apel_message
2021-09-13 15:39:40,016 - ssm.ssm2 - WARNING - Message apel_message could not be sent as its larger than 1MB
2021-09-13 15:39:40,018 - ssm.ssm2 - INFO - Message apel_message saved to reject queue as 59d5ce95-55a6-4ee2-8cd5-2cfbc8323871
2021-09-13 15:39:40,019 - ssm.ssm2 - INFO - Sending message: messageafterbigone
2021-09-13 15:39:40,426 - ssm.ssm2 - INFO - Sent messageafterbigone, Argo ID: 81
2021-09-13 15:39:40,427 - ssm.ssm2 - INFO - Tidying message directory.
...

@jounaidr jounaidr requested a review from a team as a code owner September 13, 2021 15:31
@tofu-rocketry tofu-rocketry self-assigned this Sep 14, 2021
@tofu-rocketry tofu-rocketry added this to the 3.3.0 milestone Jun 22, 2022
@tofu-rocketry
Copy link
Member

Think I'll extract the basic changes for #155 for the next release and save the bits for #156 for later.

@gregcorbett
Copy link
Member

Think I'll extract the basic changes for #155 for the next release and save the bits for #156 for later.

Do the commits left in this PR represent just the changes needed for #156 currently?

@tofu-rocketry tofu-rocketry force-pushed the 156-1mb-error-handling branch 2 times, most recently from 4f9c095 to e779fb4 Compare December 12, 2022 17:06
@tofu-rocketry
Copy link
Member

Do the commits left in this PR represent just the changes needed for #156 currently?

No, I've not split it out yet.

jounaidr and others added 3 commits June 12, 2023 13:24
@jounaidr @tofu-rocketry
Messages larger than 1 MB caught
3052e3d 
Implement code to stop send_all from clearing the message if it's too
large.
@jounaidr @tofu-rocketry
Place too large message in rejected queue
bda9e02 
Signed-off-by: jounaidr <[email protected]>
@tofu-rocketry
Reorder imports, move comment, use logging.warning
e266757 
- Reorder argo_ams_library imports alphabetically.
- Move comment to more logical position.
- Change logging.warn to .warning as former is deprecating.
@tofu-rocketry tofu-rocketry removed this from the 3.3.0 milestone Jun 12, 2023
@tofu-rocketry tofu-rocketry added this to the 3.5.0 milestone Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@tofu-rocketry tofu-rocketry

Labels
None yet
Milestone
4.1.0
Development

Successfully merging this pull request may close these issues.

Warn and handle if AMS messages are > 1 MB Handle AMS message too large error gracefully
3 participants
@jounaidr @tofu-rocketry @gregcorbett