Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encode/decode event data as a Variant #2815

Merged
merged 24 commits into from
Sep 20, 2024

Conversation

sergeuz
Copy link
Member

@sergeuz sergeuz commented Aug 26, 2024

Note: This PR is targeting typed-publish/sc-129495 for review purposes.

Description

This PR introduces the following changes:

  • Promote the internal Buffer class to a public API.
  • Extend Variant to support Buffer as one of its alternative types.
  • Extend Particle.publish() to support publishing event data as a variant.
  • Extend Particle.subscribe() to support a new callback type that takes event data as a variant.

Buffer is a general purpose class for storing arbitrary data in a dynamically allocated buffer. Note that one implication of supporting it as one of the Variant's alternative types is that it makes it possible to store binary data in a ledger so this PR enables that as well.

When an event is published as a variant, it's serialized as CBOR on the wire. The server then converts it to JSON and forwards it to the subscribers. In the opposite direction, from the server to device, a conversion to CBOR is performed if necessary (see the examples below).

Examples

Sending event data as a variant:

Variant data;
data["a"] = 123;
data["b"] = 456;
Particle.publish("devout", data);

Subscribing via the CLI:

particle subscribe devout

{"name":"devout","data":"{\"a\":123,\"b\":456}",...}

Receiving event data as a variant:

void eventHandler(const char* name, Variant data) {
    Log.info("Received event: %s", name);
    Log.print(data.toJSON());
    Log.print("\r\n");
}

void setup() {
    Particle.subscribe("devin", eventHandler);
}

When sending an event to the device, the content type of the event data can be arbitrary. On the wire, the data is converted to a CBOR which is then parsed by Device OS as a variant:

particle publish devin abc
particle publish devin 'data:application/json,[1,2,3,"abc"]'
particle publish devin 'data:application/cbor;base64,oWNhYmMYew=='

Device output:

0000368773 [app] INFO: Received event: devin
"abc"
0000377012 [app] INFO: Received event: devin
[1,2,3,"abc"]
0000398583 [app] INFO: Received event: devin
{"abc":123}

@sergeuz sergeuz force-pushed the typed-publish-variant/sc-130203 branch from 8c218ac to f3f9e48 Compare August 30, 2024 09:28
*/
char* data() {
return d_.data();
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original Buffer implementation that we used internally exposed its data as char* so I did the same here to reduce changes in unrelated parts of Device OS that use this class. The Wiring library tends to use uint8_t* for binary data though so I'm wondering if it would be better to use that type instead.

@sergeuz sergeuz marked this pull request as ready for review August 30, 2024 16:16
@sergeuz sergeuz force-pushed the typed-publish-variant/sc-130203 branch from 863993c to 5288bca Compare September 18, 2024 14:47
@sergeuz sergeuz force-pushed the typed-publish-variant/sc-130203 branch from 53057dc to ac3ea97 Compare September 19, 2024 14:45
Copy link
Member

@monkbroc monkbroc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

user/tests/integration/communication/events/events.cpp Outdated Show resolved Hide resolved
user/tests/integration/communication/events/events.cpp Outdated Show resolved Hide resolved
user/tests/wiring/api/cloud.cpp Outdated Show resolved Hide resolved
@Kategrode Kategrode added this to the 6.2.0 milestone Sep 20, 2024
@sergeuz sergeuz merged commit 81479be into typed-publish/sc-129495 Sep 20, 2024
13 checks passed
@sergeuz sergeuz deleted the typed-publish-variant/sc-130203 branch September 20, 2024 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants