Importing a specific invalid YAML DeviceProfile causes all subsequent device profile queries to fail

[weichou1229]

🐞 Bug Report

Affected Services

The issue is located in: Metadata

Is this a regression?

No

Description and Minimal Reproduction

Add yaml format DeviceProfile should invoke Validate function to prevent other API's JSON unmarshal error

1.Startup core-data and core-metadata
2.Add device profile with API {{baseUrl}}/api/v1/deviceprofile/upload or {{baseUrl}}/api/v1/deviceprofile/uploadfile

# test-device-profile.yml
name: "Random-Integer-Device"
manufacturer: "IOTech"
model: "Device-Virtual-01"
labels:
- "device-virtual-example"
description: "Example of Device-Virtual"

deviceResources:
-
  name: "EnableRandomization_Int8"
  description: "used to decide whether to re-generate a random value"
  properties:
    value:
      { type: "Bool", readWrite: "W", defaultValue: "true" }
    units:
      { type: "String", readWrite: "R", defaultValue: "Random" }

deviceCommands:
-
  name: "Int8"
  get:
  - { operation: "get", deviceResource: "" }

coreCommands:
-
  name: "Int8"
  get:
    path: "/api/v1/device/{deviceId}/Int8"
    responses:
    -
      code: "200"
      description: ""
      expectedValues: ["Int8"]
    -
      code: "503"
      description: "service unavailable"
      expectedValues: []

3.Quey device profile

% curl http://localhost:48081/api/v1/deviceprofile
Object and DeviceResource are both blank

% curl http://localhost:48081/api/v1/deviceprofile/name/Random-Integer-Device
Object and DeviceResource are both blank

🔥 Exception or Error

% curl http://localhost:48081/api/v1/deviceprofile/name/Random-Integer-Device
Object and DeviceResource are both blank

🌍 Your Environment

Deployment Environment:
macOS

EdgeX Version:
edgex-go master branch

Anything else relevant?

[tonyespy]

While reviewing @weichou1229's PR for this bug, I ran a few tests. First, I was able to reproduce it against geneva, so I've added a geneva label. Second, I edited the title of the issue to better highlight the fact that when the bug occurs, it causes any subsequent device profile queries made to core-metadata to fail. It doesn't seem to affect importing new device profiles, only queries. I also didn't test update or delete.

[jpwhitemn]

Not able to reproduce this with Geneva 1.2.1 metadata (using Docker Compose for redis, no secty) and two bad YAML files attached.
Tried with Nexus images and also no problem. Has to be some additional extenuating circumstances.

Files renamed from .yaml extension to .txt for use with Github
Myprofile3 is the same as the profile above.
myprofile2.txt
myprofile3.txt

[lenny-goodell]

@jpwhitemn , do you get a validation error with these files?

[jpwhitemn]

OK - able to recreate after getting sample from @tonyespy ... the issue is not on all bad YAML. Only YAML that seems to add correctly but then creates problems internally. No error messages are printed to metadata logs when add occurs, but all gets/deletes break after this profile is added.

On the other broken profiles above - you do get appropriate validation error and error messages in the metadata logs. So those files, with simple YAML issues or device profiles on add are not the issue.

Uploading the attached YAML will cause the metadata service to act as if the add is successful (it will give you an UUID and no error), but then the the gets fail. Delete or any get of deviceProfile(s) will not work. Adds will still work. To solve this, I had to use Redis Desktop Manager to remove offending deviceProfile by the ID provided by the add.

Again, yaml extension changed to .txt for Github
tonyprofile.txt

[tonyespy]

Should we create a new issue to continue to track this as an "open" bug in Geneva?

Our bug process says:

If multiple release_affected tags exist and the issue is only fixed in one of these releases resulting in the issue being closed, if any of affected releases is either an active development release or an LTS release, then a new issue should be opened with release_affected set accordingly.

I would think we'd optionally want to also follow this procedure (with bug severity taken into consideration) for the release preceding the current development release (i.e. we'd do this for Geneva until the point at which Hanoi is released), as there's still the possibility that we might have to release additional dot releases for Geneva.

@bnevis-i @lenny-intel @jpwhitemn thoughts?

[cloudxxx8]

@tonyespy issue opened
@weichou1229 please help fix in the Geneva branch