Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Enable Batch to optionally marshal data as Events #977

Merged
merged 3 commits into from
Oct 20, 2021

Conversation

lenny-goodell
Copy link
Member

@lenny-goodell lenny-goodell commented Oct 19, 2021

Added the IsEventData bool setting so Batch can know to marshal batched data
as []Events before passing to next pipeline function.

In future 3.0 release this flag will be added to the factory methods.

closes #962

If your build fails due to your commit message not passing the build checks, please review the guidelines here: https://github.com/edgexfoundry/app-functions-sdk-go/blob/main/.github/CONTRIBUTING.md

PR Checklist

Please check if your PR fulfills the following requirements:

  • I am not introducing a breaking change (if you are, flag in conventional commit message with BREAKING CHANGE: describing the break)
  • I am not introducing a new dependency (add notes below if you are)
  • I have added unit tests for the new feature or bug fix (if not, why?)
  • I have fully tested (add details below) this the new feature or bug fix (if not, why?)
  • I have opened a PR for the related docs change (if not, why?)
    feat: Document new IsEventData for Batch function edgex-docs#610

Testing Instructions

  1. Run no secure EdgeX stack
  2. Clone branch for this PR
  3. Clone app-service-configurable and modify go.mod as follows:
replace github.com/edgexfoundry/app-functions-sdk-go/v2 => ../app-functions-sdk-go
  1. run make build
  2. Modify ./res/functional-tests/configuration.toml as follows:
  ExecutionOrder = "Batch, SetResponseData"

add:
    [Writable.Pipeline.Functions.Batch]
      [Writable.Pipeline.Functions.Batch.Parameters]
      Mode = "bycount" 
      BatchThreshold= "2"
      IsEventData = "true"
  1. run ./app-service-configurable -p=functional-tests
  2. Using Postman or curl send the the following to localhost:59705/api/v2/trigger
{
    "requestId": "82eb2e26-0f24-48ba-ae4c-de9dac3fb9bc",
    "apiVersion": "v2",
    "event": {
        "apiVersion": "v2",
        "deviceName": "Random-Float-Device",
        "profileName": "Random-Float-Device",
        "sourceName" : "Float32",
        "origin": 1540855006456,
        "id": "94eb2e26-0f24-5555-2222-de9dac3fb228",
        "readings": [
            {
                "apiVersion": "v2",
                "resourceName": "Float32",
                "profileName": "Random-Float-Device",
                "deviceName": "Random-Float-Device",
                "value": "666",
                "origin": 1540855006469,
                "ValueType": "Float32",
                "id": "82eb2e36-0f24-48aa-ae4c-de9dac3fb920"
            }
        ]
    }
}
  1. Verify response is 200 w/o any data.
  2. Repeat step 7 and validate the response data is as follows:
[
    {
        "apiVersion": "v2",
        "id": "94eb2e26-0f24-5555-2222-de9dac3fb228",
        "deviceName": "Random-Float-Device",
        "profileName": "Random-Float-Device",
        "sourceName": "Float32",
        "origin": 1540855006456,
        "readings": [
            {
                "id": "82eb2e36-0f24-48aa-ae4c-de9dac3fb920",
                "origin": 1540855006469,
                "deviceName": "Random-Float-Device",
                "resourceName": "Float32",
                "profileName": "Random-Float-Device",
                "valueType": "Float32",
                "value": "666"
            }
        ]
    },
    {
        "apiVersion": "v2",
        "id": "94eb2e26-0f24-5555-2222-de9dac3fb228",
        "deviceName": "Random-Float-Device",
        "profileName": "Random-Float-Device",
        "sourceName": "Float32",
        "origin": 1540855006456,
        "readings": [
            {
                "id": "82eb2e36-0f24-48aa-ae4c-de9dac3fb920",
                "origin": 1540855006469,
                "deviceName": "Random-Float-Device",
                "resourceName": "Float32",
                "profileName": "Random-Float-Device",
                "valueType": "Float32",
                "value": "666"
            }
        ]
    }
]
  1. Now stop ASC and change config so that IsEventData = "false" and rerun step 7 twice and validate that 2nd response contains the following:
[
    "eyJhcGlWZXJzaW9uIjoidjIiLCJpZCI6Ijk0ZWIyZTI2LTBmMjQtNTU1NS0yMjIyLWRlOWRhYzNmYjIyOCIsImRldmljZU5hbWUiOiJSYW5kb20tRmxvYXQtRGV2aWNlIiwicHJvZmlsZU5hbWUiOiJSYW5kb20tRmxvYXQtRGV2aWNlIiwic291cmNlTmFtZSI6IkZsb2F0MzIiLCJvcmlnaW4iOjE1NDA4NTUwMDY0NTYsInJlYWRpbmdzIjpbeyJpZCI6IjgyZWIyZTM2LTBmMjQtNDhhYS1hZTRjLWRlOWRhYzNmYjkyMCIsIm9yaWdpbiI6MTU0MDg1NTAwNjQ2OSwiZGV2aWNlTmFtZSI6IlJhbmRvbS1GbG9hdC1EZXZpY2UiLCJyZXNvdXJjZU5hbWUiOiJGbG9hdDMyIiwicHJvZmlsZU5hbWUiOiJSYW5kb20tRmxvYXQtRGV2aWNlIiwidmFsdWVUeXBlIjoiRmxvYXQzMiIsInZhbHVlIjoiNjY2In1dfQ==",
    "eyJhcGlWZXJzaW9uIjoidjIiLCJpZCI6Ijk0ZWIyZTI2LTBmMjQtNTU1NS0yMjIyLWRlOWRhYzNmYjIyOCIsImRldmljZU5hbWUiOiJSYW5kb20tRmxvYXQtRGV2aWNlIiwicHJvZmlsZU5hbWUiOiJSYW5kb20tRmxvYXQtRGV2aWNlIiwic291cmNlTmFtZSI6IkZsb2F0MzIiLCJvcmlnaW4iOjE1NDA4NTUwMDY0NTYsInJlYWRpbmdzIjpbeyJpZCI6IjgyZWIyZTM2LTBmMjQtNDhhYS1hZTRjLWRlOWRhYzNmYjkyMCIsIm9yaWdpbiI6MTU0MDg1NTAwNjQ2OSwiZGV2aWNlTmFtZSI6IlJhbmRvbS1GbG9hdC1EZXZpY2UiLCJyZXNvdXJjZU5hbWUiOiJGbG9hdDMyIiwicHJvZmlsZU5hbWUiOiJSYW5kb20tRmxvYXQtRGV2aWNlIiwidmFsdWVUeXBlIjoiRmxvYXQzMiIsInZhbHVlIjoiNjY2In1dfQ=="
]

New Dependency Instructions (If applicable)

N/A

one section should not be a comment

Signed-off-by: Leonard Goodell <[email protected]>
@lenny-goodell lenny-goodell force-pushed the batch-events branch 2 times, most recently from 9fb3307 to a892bec Compare October 19, 2021 19:41
@lenny-goodell lenny-goodell marked this pull request as ready for review October 19, 2021 19:43
AlexCuse
AlexCuse previously approved these changes Oct 19, 2021
Add the IsEventData bool setting so Batch can know to marshal batched data
as []Events before passing to next pipeline function.

closes edgexfoundry#962

Signed-off-by: Leonard Goodell <[email protected]>
Signed-off-by: Leonard Goodell <[email protected]>
@lenny-goodell lenny-goodell merged commit b877746 into edgexfoundry:main Oct 20, 2021
@lenny-goodell lenny-goodell deleted the batch-events branch October 20, 2021 01:11
FelixTing pushed a commit to FelixTing/app-functions-sdk-go that referenced this pull request Mar 3, 2022
…#977)

* feat: Enable Batch to optionally marshal data as Events

Add the IsEventData bool setting so Batch can know to marshal batched data
as []Events before passing to next pipeline function.

closes edgexfoundry#962

Signed-off-by: Leonard Goodell <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enhance Batch to have option to un-marshal list of object when using Events
2 participants