-
Notifications
You must be signed in to change notification settings - Fork 265
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Querying data with query strings results unexpected data #3657
Comments
To complete the picture we would need some extra information:
Thanks! |
Here the registration of the orion-openiot database:
> use orion-openiot switched to db orion-openiot > db["registrations"].find() { "_id" : ObjectId("5eddfc03e7e028b0e09312bb"), "expiration" : NumberLong("9223372036854775807"), "servicePath" : "/", "contextRegistration" : [ { "entities" : [ { "id" : "urn:ngsi-ld:Bell:001", "type" : "Bell" } ], "attrs" : [ { "name" : "ring", "type" : "" } ], "providingApplication" : "http://iot-agent:4041" } ], "format" : "normalized" }
I didn't understand how to get the traces, but I retrieve the orion container's logs. curl "http://localhost:1026/v2/entities/?q=ring_status=='UNKNOWN'&options=keyValues&type=Bell" -H 'fiware-service: openiot' -H 'fiware-servicepath: /' ------------------------- time=Monday 08 Jun 08:51:33 2020.141Z | lvl=INFO | corr=40c10dc8-a965-11ea-a71d-0242ac120104 | trans=1591606151-662-00000000017 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=logMsg.h[1844]:lmTransactionStart | msg=Starting transaction from 172.18.1.1:44940/v2 /entities/ time=Monday 08 Jun 08:51:33 2020.141Z | lvl=INFO | corr=40c10dc8-a965-11ea-a71d-0242ac120104 | trans=1591606151-662-00000000017 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=rest.cpp[874]:servicePathSplit | msg=Service Path 0: '/' time=Monday 08 Jun 08:51:33 2020.141Z | lvl=INFO | corr=40c10dc8-a965-11ea-a71d-0242ac120104 | trans=1591606151-662-00000000017 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=connectionOperations.cpp[182]:collectionRangedQuery | msg=Database Operation Successf ul (query: { query: { _id.type: "Bell", _id.servicePath: "/", attrs.ring_status.value: { $eq: "UNKNOWN" } }, orderby: { creDate: 1 } }) time=Monday 08 Jun 08:51:33 2020.142Z | lvl=INFO | corr=40c10dc8-a965-11ea-a71d-0242ac120104 | trans=1591606151-662-00000000017 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=connectionOperations.cpp[182]:collectionRangedQuery | msg=Database Operation Successf ul (query: { query: { $or: [ { contextRegistration.entities.id: /.*/, contextRegistration.entities.type: "Bell" }, { contextRegistration.entities.id: ".*", contextRegistration.entities.isPattern: "true", contextRegistration.entities.type: { $in: [ "Bell" ] } }, { contextReg istration.entities.id: ".*", contextRegistration.entities.isPattern: "true", contextRegistration.entities.type: { $exists: false } } ], expiration: { $gt: 1591606293 }, servicePath: "/" }, orderby: { _id: 1 } }) time=Monday 08 Jun 08:51:33 2020.142Z | lvl=INFO | corr=40c10dc8-a965-11ea-a71d-0242ac120104 | trans=1591606151-662-00000000018 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=logMsg.h[1844]:lmTransactionStart | msg=Starting transaction to http://iot-agent:4041 //op/query time=Monday 08 Jun 08:51:33 2020.142Z | lvl=INFO | corr=40c10dc8-a965-11ea-a71d-0242ac120104 | trans=1591606151-662-00000000018 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=httpRequestSend.cpp[550]:httpRequestSendWithCurl | msg=Sending message 1 to HTTP serv er: sending message of 333 bytes to HTTP server time=Monday 08 Jun 08:51:33 2020.154Z | lvl=INFO | corr=40c10dc8-a965-11ea-a71d-0242ac120104 | trans=1591606151-662-00000000018 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=httpRequestSend.cpp[570]:httpRequestSendWithCurl | msg=Notification Successfully Sent to http://iot-agent:4041//op/query time=Monday 08 Jun 08:51:33 2020.155Z | lvl=INFO | corr=40c10dc8-a965-11ea-a71d-0242ac120104 | trans=1591606151-662-00000000018 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=httpRequestSend.cpp[579]:httpRequestSendWithCurl | msg=Notification response OK, http code: 200 time=Monday 08 Jun 08:51:33 2020.155Z | lvl=INFO | corr=40c10dc8-a965-11ea-a71d-0242ac120104 | trans=1591606151-662-00000000018 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=logMsg.h[1874]:lmTransactionEnd | msg=Transaction ended And the logs with an invalid query : curl "http://localhost:1026/v2/entities/?q=ring_status=='UNKNsdsdN'&options=keyValues&type=Bell" -H 'fiware-service: openiot' -H 'fiware-servicepath: /' ----------------------------- time=Monday 08 Jun 08:52:33 2020.981Z | lvl=INFO | corr=65048476-a965-11ea-8c7c-0242ac120104 | trans=1591606151-662-00000000025 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=logMsg.h[1844]:lmTransactionStart | msg=Starting transaction from 172.18.1.1:44970/v2 /entities/ time=Monday 08 Jun 08:52:33 2020.981Z | lvl=INFO | corr=65048476-a965-11ea-8c7c-0242ac120104 | trans=1591606151-662-00000000025 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=rest.cpp[874]:servicePathSplit | msg=Service Path 0: '/' time=Monday 08 Jun 08:52:33 2020.981Z | lvl=INFO | corr=65048476-a965-11ea-8c7c-0242ac120104 | trans=1591606151-662-00000000025 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=connectionOperations.cpp[182]:collectionRangedQuery | msg=Database Operation Successf ul (query: { query: { _id.type: "Bell", _id.servicePath: "/", attrs.ring_status.value: { $eq: "UNKNsdsdN" } }, orderby: { creDate: 1 } }) time=Monday 08 Jun 08:52:33 2020.982Z | lvl=INFO | corr=65048476-a965-11ea-8c7c-0242ac120104 | trans=1591606151-662-00000000025 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=connectionOperations.cpp[182]:collectionRangedQuery | msg=Database Operation Successf ul (query: { query: { $or: [ { contextRegistration.entities.id: /.*/, contextRegistration.entities.type: "Bell" }, { contextRegistration.entities.id: ".*", contextRegistration.entities.isPattern: "true", contextRegistration.entities.type: { $in: [ "Bell" ] } }, { contextReg istration.entities.id: ".*", contextRegistration.entities.isPattern: "true", contextRegistration.entities.type: { $exists: false } } ], expiration: { $gt: 1591606353 }, servicePath: "/" }, orderby: { _id: 1 } }) time=Monday 08 Jun 08:52:33 2020.982Z | lvl=INFO | corr=65048476-a965-11ea-8c7c-0242ac120104 | trans=1591606151-662-00000000025 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=connectionOperations.cpp[182]:collectionRangedQuery | msg=Database Operation Successf ul (query: { query: { $or: [ { contextRegistration.entities.id: /.*/, contextRegistration.entities.type: "Bell" }, { contextRegistration.entities.id: ".*", contextRegistration.entities.isPattern: "true", contextRegistration.entities.type: { $in: [ "Bell" ] } }, { contextReg istration.entities.id: ".*", contextRegistration.entities.isPattern: "true", contextRegistration.entities.type: { $exists: false } } ], expiration: { $gt: 1591606353 }, servicePath: "/" }, orderby: { _id: 1 } }) time=Monday 08 Jun 08:52:33 2020.982Z | lvl=INFO | corr=65048476-a965-11ea-8c7c-0242ac120104 | trans=1591606151-662-00000000026 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=logMsg.h[1844]:lmTransactionStart | msg=Starting transaction to http://iot-agent:4041 //op/query time=Monday 08 Jun 08:52:33 2020.982Z | lvl=INFO | corr=65048476-a965-11ea-8c7c-0242ac120104 | trans=1591606151-662-00000000026 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=httpRequestSend.cpp[550]:httpRequestSendWithCurl | msg=Sending message 4 to HTTP serv er: sending message of 333 bytes to HTTP server time=Monday 08 Jun 08:52:33 2020.992Z | lvl=INFO | corr=65048476-a965-11ea-8c7c-0242ac120104 | trans=1591606151-662-00000000026 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=httpRequestSend.cpp[570]:httpRequestSendWithCurl | msg=Notification Successfully Sent to http://iot-agent:4041//op/query time=Monday 08 Jun 08:52:33 2020.992Z | lvl=INFO | corr=65048476-a965-11ea-8c7c-0242ac120104 | trans=1591606151-662-00000000026 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=httpRequestSend.cpp[579]:httpRequestSendWithCurl | msg=Notification response OK, http code: 200 time=Monday 08 Jun 08:52:33 2020.993Z | lvl=INFO | corr=65048476-a965-11ea-8c7c-0242ac120104 | trans=1591606151-662-00000000026 | from=172.18.1.1 | srv=openiot | subsrv=/ | comp=Orion | op=logMsg.h[1874]:lmTransactionEnd | msg=Transaction ended It looks like, when the service and service-path are set, orion will retrieve all registered entity. |
The problem occurs with all registrations with To reproduce the problem you juste have to create a registration without legacyFormat set. example:
with a nonexistent term it gives :
If I create a registration with legacyFormat to true or if I modify the format attribute to JSON in registrations collection in mongo, querying with an invalid term will gives a valid result:
|
Thank you for so detailed report! I'll have a look when I some time and provide feedback |
Do you only modify the format (either directly in MongoDB collection or indirectly by re-creating the registration)? Or do you also modify the provider URL (shown as |
I'd like to share the results of some testing I have been doing. I have two instances of Orion, one running on port 1026 acting as Context Broker, the other running in port 1027 acting as context provider (CPr). I use the following to run them:
I create an entity in the context provider with the following request:
So, setup ready, let's start to test... First test is using a registration with legacy format. I create the registration using the following request. Note the URL uses
Next, I do a request on Context Broker with valid name (OK):
Next, I do a request on Context Broker with an invalid name (NOK):
After deleting the registration of the past test, I create a new one without legacy format, using the following request. Note the URL now uses
Next, I do a request on Context Broker with valid name (OK):
Next, I do a request on Context Broker with an invalid name (NOK)
Conclusion: I think the problem is not related to the usage of legacyForwarding format, as it happens in both cases (either using legacyFormat true or not). This is weird, as report at #3657 (comment) tells otherwise. Some of us (@floufen or me) has some flaw in his testing process :) |
Thank you for the report !
I just modified the format. The URL stays the same. My bad, I didn't notice the "not service found" error was thrown. It is not related to the usage of legacyForwarding. The problem still exist. Can you tell me why the registrations database is searched while querying context broker data ? :) |
Let me start for the end :)
It is searched in order to find Context Providers that could have information about the entity/attribute being queried. Orion does a "merge" on local information and Context Providers registrations. For instance, you could have this situation:
A client does a query for entity E (e.g. There are different kinds of queries and some ugly corner cases, but, basically, this is the way it is works. |
Thank you for the explanation. |
Sorry, I don't understand what you mean. Could you elaborate a little bit, please?
Thanks for sharing your code modification. As far as I understand, you are applying filtering to the DB query done in registration collection in order to find candidate context providers to forward the client NGSI query. However, note that in that registrations collection you only have entities ids and types and attribute names and types, you don't have attribute values so filters like But if you say is solving your issue maybe I'm missing something :) What would be useful at this point is to create a functional test case (i.e. a .test file to be used with testHarness.sh) covering your case. That .test should pass with your code and fail with current master. Could you provide that .test, please? |
Hahah I am sorry. I did the modifications without understanding how the code really works and how the datas are stored. It seemed to work because all the the registrations were filtered. |
Hi,
When provisioning an actuator using iotagent-ul, retrieving data with query string results to unexpected entities.
Here is how to reproduce this behavour :
Add service:
and provision command :
from now if retreive an entity with valid attribute it results to good result :
but if the attributes doesn't exist or not equal it results this entity :
Why this behaviour ?
The text was updated successfully, but these errors were encountered: