Problem to notify Quantunleap
alvaroo278 opened this issue · 15 comments
Description Issue
I am trying to send a notification from Orion-LD to Quantumleap, but nothing arrives in the QL service. To check that Orion is notifying correctly, I am using a duplicate subscription that has as an endpoint a service echo (Webhook). I am using Kubernetes (ArgoCD) to configure and control the service.
Orion-LD configuration
apiVersion: v1
kind: Service
metadata:
labels:
app: orion
name: orion
spec:
ports:
- name: http
port: 1026
protocol: TCP
targetPort: 1026
selector:
app: orion
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: orion
spec:
replicas: 1
selector:
matchLabels:
app: orion
template:
metadata:
labels:
app: orion
spec:
containers:
- command: ["orionld"]
args: ["-fg", "-multiservice", "-ngsiv1Autocast", "-dbhost", "mongodb",
"-logLevel", "DEBUG"]
image: "fiware/orion-ld:1.0.1"
imagePullPolicy: IfNotPresent
name: orion
ports:
- containerPort: 1026
name: http
Quantumleap configuration
apiVersion: v1
kind: Service
metadata:
labels:
app: quantumleap
name: quantumleap
spec:
ports:
- name: http
port: 8668
protocol: TCP
targetPort: 8668
selector:
app: quantumleap
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: quantumleap
labels:
app: quantumleap
annotations:
# Tell Reloader to bounce the Argo CD server whenever the SSO config
# and built-in admin credentials change.
secret.reloader.stakater.com/reload: "postgres-users"
spec:
replicas: 1
selector:
matchLabels:
app: quantumleap
template:
metadata:
labels:
app: quantumleap
spec:
initContainers:
- name: quantumleap-pg-init
image: "smartsdk/quantumleap-pg-init:0.7.6"
imagePullPolicy: IfNotPresent
env:
- name: PG_HOST
value: postgres
- name: PG_PASS
valueFrom:
secretKeyRef:
name: postgres-users
key: postgres.password
- name: QL_DB_PASS
valueFrom:
secretKeyRef:
name: postgres-users
key: quantumleap.password
- name: QL_DB_INIT_DIR
value: /db-init
volumeMounts:
- name: db-init
mountPath: /db-init
containers:
- image: "orchestracities/quantumleap:0.8.3"
imagePullPolicy: IfNotPresent
name: quantumleap
ports:
- containerPort: 8668
name: http
env:
- name: POSTGRES_HOST
value: "postgres"
- name: POSTGRES_PORT
value: "5432"
- name: POSTGRES_USE_SSL
value: "False"
- name: POSTGRES_DB_NAME
value: "quantumleap"
- name: POSTGRES_DB_USER
value: "quantumleap"
- name: POSTGRES_DB_PASS
valueFrom:
secretKeyRef:
name: postgres-users
key: quantumleap.password
- name: CRATE_HOST
value: "crate"
- name: CRATE_PORT
value: "4200"
- name: QL_CONFIG
value: /config/db-routing.yaml
- name: DEBUG
value: "true"
volumeMounts:
- name: db-routing
mountPath: /config
volumes:
- name: db-routing
configMap:
name: db-routing
- name: db-init
emptyDir: {}
Regarding the quantumleap images I use, if I use as initializer the orchestracities image (whatever version I use), the pod doesn't start.
Orion-LD subscription
{
"id": "urn:ngsi-ld:Subscription:63dbab3998657c37179501a9",
"type": "Subscription",
"description": "Notify alert",
"entities": [
{
"type": "AirQualityAlert"
}
],
"notification": {
"attributes": [
"analysis",
"dateIssued",
"prediction"
],
"format": "normalized",
"endpoint": {
"uri": "http://192.168.1.18/quantumleap/v2/notify",
"accept": "application/json"
}
},
"@context": "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld"
}
On the other hand, if I enter the logs of the quantumleap pod, it appears empty:
``` kubectl logs -f {id-pod}````
I have also tried to manually notify Quantumleap and I get a timeout error when making the request
Post Request
curl --location --request POST 'http://192.168.1.18/quantumleap/v2/notify' \
--header 'ngsild-tenant: airqualityalert' \
--header 'fiware-servicepath: /' \
--header 'Content-Type: application/json' \
--data-raw '{
"id": "urn:ngsi-ld:Notification:63e274e0cb0b200efafa99ee",
"type": "Notification",
"subscriptionId": "urn:ngsi-ld:Subscription:63dbaa4998657c37179501a3",
"notifiedAt": "2023-02-07T15:57:20.549Z",
"data": [
{
"id": "urn:ngsi-ld:AirQualityAlert:Out.3857499894.1",
"type": "AirQualityAlert",
"analysis": {
"type": "Property",
"value": "good"
},
"prediction": {
"type": "Property",
"value": "good"
},
"dateIssued": {
"type": "Property",
"value": {
"type": "DateTime",
"value": "2023-02-07T15:57:20.519Z"
}
}
}
]
}'
All pods are running well.
Regarding the quantumleap images I use, if I use as initializer the orchestracities image (whatever version I use), the pod doesn't start.
that's something @c0c0n3 needs to look into.
Orion-LD subscription
{ "id": "urn:ngsi-ld:Subscription:63dbab3998657c37179501a9", "type": "Subscription", "description": "Notify alert", "entities": [ { "type": "AirQualityAlert" } ], "notification": { "attributes": [ "analysis", "dateIssued", "prediction" ], "format": "normalized", "endpoint": { "uri": "http://192.168.1.18/quantumleap/v2/notify", "accept": "application/json" } }, "@context": "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld" }
what's this url "http://192.168.1.18/quantumleap/v2/notify ?
unless this is something you installed, you won't find quantum leap there. 1) the path it's just /v2/notify 2) you better use hostnames and not ips.
hi @alvaroo278 :-)
It looks like you're using QL inside the Kitt4sme demo cluster? In that case, the URL should be
http://quantumleap/v2/notify
Can you give it a try?
/quantumleap it's equal :8668. I'm checked
I checked if Orion-LD can connect with QL. If I execute a Get request inside of pod Orion-LD and I receive version of QL.
[root@orion-66b4c55bc5-gndw9 /]# curl --location --request GET 'http://192.168.1.18/quantumleap/version'
{
"version": "0.8.3-dev"
}
keep in mind you're getting out of the cluster and then routed back in through the Istio gateway, so it's best to make an internal call w/ the url I gave you earlier. The reason that doesn't work is that probably orion gets a 403 from the Istio gateway---security rules if memory serves. If you try any other URL from the Orion pod you should get a 403 too. The reason why the version endpoint works is that that url is excluded from the security check
I changed uri in the subscription, but nothing happened.
My stream is FogFlow (ngsild entity) -> Orion-LD -> Quantumleap. So, it's necessary to put any header in the subscription or something? Because if I remember, I filled in this file:
With a personal token and push in to my repo.
If I update the entity, I receive this in my echo service:
But If I check the entity in QL with the next curl:
curl --location --request GET 'http://192.168.1.18:8668/v2/entities/urn:ngsi-ld:AirQualityAlert:Out.3857499894.1' \
--header 'ngsild-tenant: airqualityalert'
I don't receive errors code but the answer is empty. All request were find in:
But it's weird because I can't see anything inside logs quantumleap. I may make a mistake checking entities in QL.
New subscription
curl --location --request POST 'http://192.168.1.18/orion/ngsi-ld/v1/subscriptions/' \
--header 'Content-Type: application/json' \
--header 'ngsild-tenant: airqualityalert' \
--data-raw '{
"description": "Notify alert",
"type": "Subscription",
"entities": [
{
"type": "AirQualityAlert"
}
],
"notification": {
"attributes": [
"analysis",
"dateIssued",
"prediction"
],
"format": "normalized",
"endpoint": {
"uri": "http://quantumleap/v2/notify",
"accept": "application/json"
}
}
}'
Request subscriptions
curl --location --request GET 'http://192.168.1.18/orion/ngsi-ld/v1/subscriptions/' \
--header 'fiware-service: airqualityalert' \
--header 'fiware-servicepath: /airqualityalert'
Response
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 814 100 814 0 0 262k 0 --:--:-- --:--:-- --:--:-- 397k
[
{
"id": "urn:ngsi-ld:Subscription:63dbaa4998657c37179501a3",
"type": "Subscription",
"description": "Notify alert",
"entities": [
{
"type": "AirQualityAlert"
}
],
"notification": {
"attributes": [
"analysis",
"dateIssued",
"prediction"
],
"format": "normalized",
"endpoint": {
"uri": "https://webhook.site/d18d4b5d-d40a-4a71-a714-c26acf7aa7be",
"accept": "application/json"
}
},
"@context": "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld"
},
{
"id": "urn:ngsi-ld:Subscription:63e36541cb0b200efafa9a04",
"type": "Subscription",
"description": "Notify alert",
"entities": [
{
"type": "AirQualityAlert"
}
],
"notification": {
"attributes": [
"analysis",
"dateIssued",
"prediction"
],
"format": "normalized",
"endpoint": {
"uri": "http://quantumleap/v2/notify",
"accept": "application/json"
}
},
"@context": "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld"
}
]
- All requests are sent inside the computer where the service is hosted.
If it's necessary, you can check all code of deployment inside my repo.
hi @alvaroo278 :-) It looks like you're using QL inside the Kitt4sme demo cluster?
Hi, yes, I'm working on project KITT4SME and deploying the connexion FogFlow -> KITT4SME instance.
@alvaroo278 thanks so much for all the info! I'll try reproducing the scenario and see what breaks...
I just check how to create a subscription and then wrote:
http://quantumleap:8668/v2/notify (https://www.postman.com/fiware/workspace/fiware-foundation-ev-s-public-workspace/request/513743-b355261f-7dee-44db-aba3-b6b5499975de)
thanks so much for all the info! I'll try reproducing the scenario and see what breaks...
In my service (FogFlow), I introduce directly entity to json valid to Orion-LD, so I jumped Agents and sent my entity directly to Orion:
'http://192.168.1.18/orion/ngsi-ld/v1/entityOperations/upsert'
Hi @alvaroo278,
I think I finally got to the bottom of this. It looks like it's a configuration issue rather than a bug---thank goodness for that :-)
Try reconfiguring your system like this
- QuantumLeap's URL in the Orion subscription should be:
http://quantumleap:8668/v2/notify. That's b/c inside your cluster you have a K8s service calledquantumleaprunning on port8668which sits in the same namespace as Orion. (Yes, the URL I suggested earlier was wrong b/c it didn't have the port in it, sorry about that!) - You should tell QuantumLeap which DB backend to use for your tenant. If you want all your tenants to store data in Postgres, then add this env to your K8s deployment for QuantumLeap:
QL_DEFAULT_DB=timescale. On the other hand, if you'd only like theairqualityalerttenant to have its data in Postgres, configure per-tenant routing. - Add an additional
fiware-service: airqualityalertheader to your subscription and upsert Orion calls. This is b/c at the moment Quantum Leap still doesn't support thengsild-tenantheader. But this feature should be merged into the mainline soon, see #695.
I couldn't manage to reproduce the container init issue. The container init image starts correctly on my side, it creates the Quantum Leap DB, and then exits.
@alvaroo278 here's a Docker Compose file you can use to test the points I mentioned above.
version: '3'
services:
mongodb:
image: mongo:4.4
networks:
- qltest
orion:
image: fiware/orion-ld:0.8.0
entrypoint: orionld -fg -multiservice -ngsiv1Autocast -dbhost mongodb -logLevel DEBUG
networks:
- qltest
ports:
- "1026:1026"
depends_on:
- mongodb
postgres:
image: timescale/timescaledb-postgis:2.3.0-pg13
ports:
- "5432:5432"
networks:
- qltest
environment:
- POSTGRES_PASSWORD=*
quantumleap-db-setup:
image: smartsdk/quantumleap-pg-init:0.7.6
depends_on:
- postgres
networks:
- qltest
environment:
- QL_DB_PASS=*
- QL_DB_INIT_DIR=/ql-db-init
- PG_HOST=postgres
- PG_PASS=*
quantumleap:
image: orchestracities/quantumleap
depends_on:
- postgres
networks:
- qltest
ports:
- "8668:8668"
environment:
- QL_DEFAULT_DB=timescale
- POSTGRES_HOST=postgres
- POSTGRES_PORT=5432
- POSTGRES_DB_NAME=quantumleap
- POSTGRES_DB_USER=quantumleap
- POSTGRES_DB_PASS=*
- POSTGRES_USE_SSL=False
- USE_GEOCODING=False
- CACHE_QUERIES=False
- LOGLEVEL=ERROR
networks:
qltest:
driver: bridge
Start Docker Compose with the file above
$ docker compose up -dThen when all services are up and running and the QL container init has exited, you should be able to get into the QL DB
$ psql postgres://quantumleap:*@localhost
psql (13.4, server 13.3)
Type "help" for help.
quantumleap=> \qAt this point you can set up your subscription with Orion like this
curl 'http://localhost:1026/ngsi-ld/v1/subscriptions/' \
-H 'Content-Type: application/json' \
-H 'ngsild-tenant: airqualityalert' \
-H 'fiware-service: airqualityalert' \
-d @sub.jsonNotice the extra header of fiware-service: airqualityalert. The sub.json file:
{
"id": "urn:ngsi-ld:Subscription:63dbab3998657c37179501a9",
"type": "Subscription",
"description": "Notify alert",
"entities": [
{
"type": "AirQualityAlert"
}
],
"notification": {
"attributes": [
"analysis",
"dateIssued",
"prediction"
],
"format": "normalized",
"endpoint": {
"uri": "http://quantumleap:8668/v2/notify",
"accept": "application/json"
}
}
}Now with the subscription in place, you can start sending your entities to Orion
$ curl 'http://localhost:1026/ngsi-ld/v1/entityOperations/upsert' \
-H 'Content-Type: application/json' \
-H 'ngsild-tenant: airqualityalert' \
-H 'fiware-service: airqualityalert' \
-H 'fiware-servicepath: /' \
-d @data.jsonAgain, notice the extra header of fiware-service: airqualityalert. The data.json file:
[
{
"id": "urn:ngsi-ld:AirQualityAlert:Out.3857499894.1",
"type": "AirQualityAlert",
"analysis": {
"type": "Property",
"value": "good"
},
"prediction": {
"type": "Property",
"value": "good"
},
"dateIssued": {
"type": "Property",
"value": {
"type": "DateTime",
"value": "2023-02-07T15:57:20.519Z"
}
}
}
]Finally get back into Postgres to check QL stored your data
$ psql postgres://quantumleap:*@localhostselect * from mtairqualityalert.etairqualityalert;
entity_id | entity_type | time_index | fiware_servicepath | __original_ngsi_entity__ | instanceid | analysis | prediction | dateissued
----------------------------------------------+-----------------+-------------------------------+--------------------+--------------------------+--------------------------------------------------+----------+------------+-----------------------------------------------------------
urn:ngsi-ld:AirQualityAlert:Out.3857499894.1 | AirQualityAlert | 2023-02-08 14:28:08.825083+00 | / | | urn:ngsi-ld:3a196511-cea1-4ab1-992e-398c9c62f41c | good | good | {"type": "DateTime", "value": "2023-02-07T15:57:20.519Z"}
urn:ngsi-ld:AirQualityAlert:Out.3857499894.1 | AirQualityAlert | 2023-02-08 14:28:12.170136+00 | / | | urn:ngsi-ld:44de05ac-138c-4f77-af87-d04972fcc18f | good | good | {"type": "DateTime", "value": "2023-02-07T15:57:20.519Z"}
urn:ngsi-ld:AirQualityAlert:Out.3857499894.1 | AirQualityAlert | 2023-02-08 14:28:15.281166+00 | / | | urn:ngsi-ld:ae6e6a91-27d0-40d2-b8aa-ae2db73154a3 | good | good | {"type": "DateTime", "value": "2023-02-07T15:57:20.519Z"}
(3 rows)Hope this helps!
3. Add an additional
fiware-service: airqualityalertheader to your subscription and upsert Orion calls. This is b/c at the moment Quantum Leap still doesn't support thengsild-tenantheader. But this feature should be merged into the mainline soon, see [ngsild-tenant header support | Fix for issue #669 & #664 #695].
I discuss this with Ken Zangelin, and he said me that now NGSI-LD don't support header 'fiware-service':
For backwards compatibility, I accept FIWARE-Service (and save it as NGSILD-Tenant) while the service-path is hardcoded to "/".
@alvaroo278 yea, I agree w/ you and Ken. The extra fiware-service header I suggested to put in is not a good idea in general. I was just suggesting a stop gap solution until we merge #695.