Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: split c8y JSON over MQTT messages using newline delimiter #3302

Conversation

Bravo555
Copy link
Contributor

@Bravo555 Bravo555 commented Dec 16, 2024

Proposed changes

#3301 was wrong: using a function meant for splitting Smartrest will mess up JSON over MQTT since that function will unescape " sequences which we want to do in Smartrest but avoid in JSON over MQTT. This PR undoes this function's rename and uses regular line splitting for JSON over MQTT.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
  • Documentation Update (if none of the other choices apply)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA (in all commits with git commit -s)
  • I ran cargo fmt as mentioned in CODING_GUIDELINES
  • I used cargo clippy as mentioned in CODING_GUIDELINES
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

@Bravo555 Bravo555 temporarily deployed to Test Pull Request December 16, 2024 21:26 — with GitHub Actions Inactive
@Bravo555 Bravo555 changed the title Fix/3297/c8y devicecontrol split on newlines fix(#3297) Simply split JSON over MQTT over \n Dec 16, 2024
Instead of using a function made for splitting smartrest, just split on
\n since that function will unescape \" sequences which we want to do in
smartrest but avoid in JSON over MQTT.

Signed-off-by: Marcel Guzik <[email protected]>
@Bravo555 Bravo555 force-pushed the fix/3297/c8y-devicecontrol-split-on-newlines branch from b92b458 to 9eee7ce Compare December 16, 2024 21:41
@reubenmiller reubenmiller changed the title fix(#3297) Simply split JSON over MQTT over \n fix: split JSON over MQTT messages using newline delimiter Dec 16, 2024
@Bravo555 Bravo555 temporarily deployed to Test Pull Request December 16, 2024 21:41 — with GitHub Actions Inactive
@reubenmiller reubenmiller changed the title fix: split JSON over MQTT messages using newline delimiter fix: split c8y JSON over MQTT messages using newline delimiter Dec 16, 2024
@reubenmiller reubenmiller added theme:c8y Theme: Cumulocity related topics bug Something isn't working labels Dec 16, 2024
Copy link
Contributor

github-actions bot commented Dec 16, 2024

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
549 0 2 549 100 1h28m6.986741999s

Copy link

codecov bot commented Dec 16, 2024

Codecov Report

Attention: Patch coverage is 85.18519% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/extensions/c8y_mapper_ext/src/converter.rs 0.00% 0 Missing and 3 partials ⚠️
...rates/extensions/c8y_firmware_manager/src/actor.rs 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files

📢 Thoughts on this report? Let us know!

@reubenmiller reubenmiller added this to the 1.4.0 milestone Dec 16, 2024
Copy link
Contributor

@albinsuresh albinsuresh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@reubenmiller
Copy link
Contributor

Given the nature of this PR I also verified it end-to-end just to be sure it works as expected.

Copy link
Contributor

@didier-wenzek didier-wenzek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

Comment on lines 635 to +637
// JSON over MQTT messages on c8y/devicecontrol/notifications can contain multiple operations in a single MQTT
// message, so split them
let operation_payloads = collect_c8y_messages(message.payload_str()?);
let operation_payloads = message.payload_str()?.lines();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay - assuming that Cumulocity emits a batch of messages using JSON lines. A point I fail to find in Cumulocity docs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately I don't think this is publicly documented.

@reubenmiller reubenmiller added this pull request to the merge queue Dec 17, 2024
Merged via the queue into thin-edge:main with commit cdaf948 Dec 17, 2024
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working theme:c8y Theme: Cumulocity related topics
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants