For NGSI-v2, ensure GeoJSON is correctly encode in the request. #854

jason-fox · 2020-04-06T07:48:40Z

I've noticed that the NGSI-v2 entity processor does not encode geo:json properly - this PR alters the attribute from a string to a geo:json as shown:

 {
   "id": "TheFirstLight",
   "type": "TheLightType",
   "location": {
-    "type": "geo:point",
-    "value": "0, 0"    
+    "type": "geo:json",
+    "value": {
+        "coordinates": [
+            0,
+            0
+        ],
+        "type": "Point"
+    }   
   }
 }

This work has already been done for GeoProperties for NGSI-LD, and it is merely a matter of back-porting to v2 and ensuring consistency between the provisioning of both formats.

This PR does the following:

Move Lat/Lng processing to a common location
For NGIS-v2, iterate attributes and convert common GeoJSON types
Ensure that if a GeoJSON object is parsed, it remains as GeoJSON (v2 and LD)
Amend test expectations to ensure geo:json is properly formated.
Add new NGSI-v2 tests for GeoJSON
Update documentation

It is small, so I assume it makes sense to merge into the ongoing NGSI-LD PR, but it could also be pushed to master after feature/842_ngsi_ld has landed.

fgalan · 2020-04-06T16:33:10Z

I've noticed that the NGSI-v2 entity processor does not encode geo:json properly - this PR alters the attribute from a string to a geo:json as shown:

 {
   "id": "TheFirstLight",
   "type": "TheLightType",
   "location": {
-    "type": "geo:point",
-    "value": "0, 0"    
+    "type": "geo:json",
+    "value": {
+        "coordinates": [
+            0,
+            0
+        ],
+        "type": "Point"
+    }   
   }
 }

Not sure of unerstanding this PR... "geo:point" attribute type (using as value a string with comma separate values) is a legal value en NGSIv2.

The provision of the IOTA devices should specify the type used in the CB update. I mean:

If geo:point is used as attribute type in the device provision, then a string with comma separate coordinates is used as attribute value
If geo:json is used as attribute type in the device provision, then a valid GeoJSON object is used as attribute value

Is this the way it works after the PR? In that case, which particular aspect has fixed the PR?

jason-fox · 2020-04-06T17:24:45Z

Not sure of understanding this PR... "geo:point" attribute type (using as value a string with comma separated values) is a legal value en NGSIv2.

It is legal NGSIv2 but meaningless - an attribute defined as: "type": "geo:point", "value": "0, 0" or "type": "geo:json", "value": "0, 0" or geo:whatever currently has no special processing and is basically just a string, and it shouldn't be - it should be GeoJSON.

However, the type geo:json is defined with a special meaning and format in NGSIv2, and this is what the PR fixes. The example could have been written like this:

 {
   "id": "TheFirstLight",
   "type": "TheLightType",
   "location": {
-    "type": "geo:json",
-    "value": "0, 0"    
+    "type": "geo:json",
+    "value": {
+        "coordinates": [
+            0,
+            0
+        ],
+        "type": "Point"
+    }   
   }
 }

Only the updated version is a valid location GeoProperty in NGSIv2 and can be accessed using geo-queries. "type": "geo:json", "value": "0, 0" is invalid NGSIv2.

In addition, for equivalence with the NGSI-LD changes, the following types are also converted to proper GeoJSON types

geo:json, geo:point, Point, GeoProperty - Point
geo:linestring, LineString - LineString
geo:polygon, Polygon - Polygon
geo:multipoint', MultiPoint - MultiPoint
geo:multilinestring, MultiLineString - MultiLineString

https://github.com/jason-fox/iotagent-node-lib/blob/feature/ngsi-ld/lib/services/ngsi/entities-NGSI-v2.js#L50-L85

Note that GeoProperty, Point, LineString, Polygon, MultiPoint and MultiLineString are all reserved words in the NGSI-LD core context, so you cannot legally set the values to comma separated strings.

fgalan · 2020-06-09T12:20:42Z

So the fragment in the PR body

 {
   "id": "TheFirstLight",
   "type": "TheLightType",
   "location": {
-    "type": "geo:point",
-    "value": "0, 0"    
+    "type": "geo:json",
+    "value": {
+        "coordinates": [
+            0,
+            0
+        ],
+        "type": "Point"
+    }   
   }
 }

has a typo and really it is

 {
   "id": "TheFirstLight",
   "type": "TheLightType",
   "location": {
-    "type": "geo:json",
-    "value": "0, 0"    
+    "type": "geo:json",
+    "value": {
+        "coordinates": [
+            0,
+            0
+        ],
+        "type": "Point"
+    }   
   }
 }

Is my understanding correct, pls?

fgalan · 2020-06-09T12:23:27Z

In addition, with regards, to:

In addition, for equivalence with the NGSI-LD changes, the following types are also converted to > proper GeoJSON types

geo:json, geo:point, Point, GeoProperty - Point

geo:linestring, LineString - LineString

geo:polygon, Polygon - Polygon

geo:multipoint, MultiPoint - MultiPoint

geo:multilinestring, MultiLineString - MultiLineString

Question: geo:linestring, geo:multipoint' and geo:multilinestring comes from the NGSI-LD specification?

jason-fox · 2020-06-09T20:02:34Z

Currently geo:json, geo:point, Point and GeoProperty are all treated as the GeoJSON Point type.

1)geo:json is specified in the NGSI v2 spec.
2) Point is defined by rfc7946
3) geo:point has been added as a synonym for consistency - internally this is just a switch statement after all.
4) GeoProperty is defined in the NGSI-LD spec

Within the NGSI-LD IoT Agent, the default functionality is to treat each attribute as a property and the value as the native JSON type. but NGSI-LD treats GeoJSON type as native as well. Therefore
GeoProperty has to map to some GeoJSON type- Point is the obvious default.

Yes there was a typo in the intial diff example. The type doesn't change in NGSI-v2, only the value does.

NGSI v2

The idea for NGSI-v2 is to provision the attribute with geo:json or geo:point and for the value to be a valid GeoJSON Point. So a GPS device for example is free to pass in an array of numbers or a string of numbers and the result is valid NGSi-v2 geo:json type..

NGSI LD

The equivalent for NGSI-LD is to provision the attribute with Point or GeoProperty and for the value to be a valid GeoJSON Point. So a GPS device for example is free to pass in an array of numbers or a string of numbers and the result is valid GeoProperty.

jason-fox · 2020-06-09T20:06:00Z

geo:point, geo:linestring, geo:multipoint and geo:multilinestring are merely synonyms, they are not defined in rfc7946 (Point, LineString, MultiPoint and MultiLineString are though)

jason-fox · 2020-06-09T20:09:26Z

geo:point had been (Mis?)used in one of the existing NGSI v2 tests:

test/unit/ngsiv2/examples/contextRequests/updateContextStaticAttributes.json

So I assumed that it was supposed to be a synonym for geo:json (which is well defined in NGSI-v2)

- Move Lat/Lng processing to a common location - For NGIS-v2, iterate attributes and convert common GeoJSON types - Ensure that if a GeoJSON object is parsed, it remains as GeoJSON (v2 and LD) - Amend test expectations to ensure geo:json is properly formated. - Add new NGSI-v2 tests for GeoJSON - Basic GeoJSON documentation. - Ensure consistent GeoProvisioning … - NGSI-LD supports geo:json as alias for GeoProperty - Add expression tests - Amend expectations.

fgalan · 2020-11-26T15:11:42Z

Once PR #849 the old base brach feature/842_ngsi_ld disappears. Base branch of this PR changed to master.

jason-fox · 2020-11-26T16:45:09Z

This PR is ready for Review, but I'd suggest actioning #943 first so the builds aren't queuing on Travis for hours before each suggested change.

fgalan · 2020-11-26T19:06:32Z

This PR is ready for Review, but I'd suggest actioning #943 first so the builds aren't queuing on Travis for hours before each suggested change.

For some weeks now, I have observed the long queuing delay you mention. Time ago, the queuing delay was not so high. Do you know if it is due to some change in Travis policy, usage terms or so?

(Just curiosity...)

jason-fox · 2020-11-27T07:40:57Z

Back in October Travis started throttling FOSS builds to a maximum of 500 concurrent Linux Machines across their infrastructure, this causes a backlog of unfulfilled builds to occur:

Since then, as you can see the backlog now spikes during the North American Workday. This is to encourage the transition to travis's commercial product travis-ci.com

fgalan · 2020-11-27T13:02:07Z

This PR is ready for Review, but I'd suggest actioning #943 first so the builds aren't queuing on Travis for hours before each suggested change.

Done. Please sync the PR with master to get the changes.

jason-fox · 2020-12-08T10:01:10Z

Please sync the PR with master to get the changes.

Done.

fgalan · 2020-12-17T09:44:18Z

doc/advanced-topics.md

@@ -46,6 +46,64 @@ curl http://${KEYSTONE_HOST}/v3/OS-TRUST/trusts \
 Apart from the generation of the trust, the use of secured Context Brokers should be transparent to the user of the IoT


Should this PR include a CHANGES_NEXT_RELEASE entry? If I understand correctly the discussion on #961, this PR would be fixing that issue, so it could be mentioned in the CHANGES_NEXT_RELEASE entry.

Fixed 51b8f10 + f4ebb69 - CNR updated

fgalan

LGTM

Consequence of merging telefonicaid/iotagent-node-lib#854

fgalan mentioned this pull request Jul 14, 2020

Should every IoT Agent align to use eslint #830

Closed

jason-fox force-pushed the feature/ngsi-ld branch from 00b1445 to 20fb934 Compare July 21, 2020 11:39

jason-fox mentioned this pull request Sep 7, 2020

Add common NGSI-LD constants #897

Merged

fgalan mentioned this pull request Oct 6, 2020

Feature/eslint #831

Merged

jason-fox force-pushed the feature/ngsi-ld branch from 20fb934 to 17e7a22 Compare November 18, 2020 15:37

manucarrace closed this Nov 26, 2020

manucarrace deleted the branch telefonicaid:master November 26, 2020 14:57

fgalan reopened this Nov 26, 2020

fgalan changed the base branch from feature/842_ngsi_ld to master November 26, 2020 15:11

jason-fox added 2 commits November 27, 2020 14:20

Merge branch 'master' into feature/ngsi-ld

46534ae

Merge branch 'master' into feature/ngsi-ld

bd53f5f

jason-fox mentioned this pull request Dec 15, 2020

Provisioning an Individual Device with GeoJson Attribute #961

Open

fgalan reviewed Dec 17, 2020

View reviewed changes

jason-fox and others added 3 commits December 17, 2020 11:33

Merge branch 'master' into feature/ngsi-ld

2f01c2d

Update CNR

51b8f10

Merge branch 'master' into feature/ngsi-ld

f4ebb69

fgalan approved these changes Dec 18, 2020

View reviewed changes

fgalan merged commit a7eb260 into telefonicaid:master Dec 18, 2020

jason-fox added a commit to jason-fox/iotagent-ul that referenced this pull request Jan 4, 2021

Update NGSI-v2 expectation to offer proper GeoJSON.

3ba1069

Consequence of merging telefonicaid/iotagent-node-lib#854

jason-fox deleted the feature/ngsi-ld branch February 5, 2021 13:45

AlvaroVega mentioned this pull request Apr 6, 2021

device attribute of type "Point" is translated to entity as geo:json #1012

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

For NGSI-v2, ensure GeoJSON is correctly encode in the request. #854

For NGSI-v2, ensure GeoJSON is correctly encode in the request. #854

jason-fox commented Apr 6, 2020

fgalan commented Apr 6, 2020

jason-fox commented Apr 6, 2020 •

edited

Loading

fgalan commented Jun 9, 2020

fgalan commented Jun 9, 2020

jason-fox commented Jun 9, 2020 •

edited

Loading

jason-fox commented Jun 9, 2020

jason-fox commented Jun 9, 2020

fgalan commented Nov 26, 2020

jason-fox commented Nov 26, 2020

fgalan commented Nov 26, 2020

jason-fox commented Nov 27, 2020 •

edited

Loading

fgalan commented Nov 27, 2020

jason-fox commented Dec 8, 2020

fgalan Dec 17, 2020

jason-fox Dec 17, 2020

fgalan left a comment

		@@ -46,6 +46,64 @@ curl http://${KEYSTONE_HOST}/v3/OS-TRUST/trusts \
		Apart from the generation of the trust, the use of secured Context Brokers should be transparent to the user of the IoT

For NGSI-v2, ensure GeoJSON is correctly encode in the request. #854

For NGSI-v2, ensure GeoJSON is correctly encode in the request. #854

Conversation

jason-fox commented Apr 6, 2020

fgalan commented Apr 6, 2020

jason-fox commented Apr 6, 2020 • edited Loading

fgalan commented Jun 9, 2020

fgalan commented Jun 9, 2020

jason-fox commented Jun 9, 2020 • edited Loading

NGSI v2

NGSI LD

jason-fox commented Jun 9, 2020

jason-fox commented Jun 9, 2020

fgalan commented Nov 26, 2020

jason-fox commented Nov 26, 2020

fgalan commented Nov 26, 2020

jason-fox commented Nov 27, 2020 • edited Loading

fgalan commented Nov 27, 2020

jason-fox commented Dec 8, 2020

fgalan Dec 17, 2020

Choose a reason for hiding this comment

jason-fox Dec 17, 2020

Choose a reason for hiding this comment

fgalan left a comment

Choose a reason for hiding this comment

jason-fox commented Apr 6, 2020 •

edited

Loading

jason-fox commented Jun 9, 2020 •

edited

Loading

jason-fox commented Nov 27, 2020 •

edited

Loading