View source for Sensor's network

== How it works ==
[[File:topicHowItWorks.jpg|850px|thumbnail|center|Sensor network]]

== Observations and observation types ==

Observations are the kind of objects that are used by sensors, devices, pieces of software (things in general) to notify about state change, sensor's readings, etc. Like thing's instances, or any other object in our architecture, they have a type that defines its attributes. Usually we differentiate between an observation instance and an observation type but the former is usually called just observation.

An observation type, as any other type, is a JSON-LD that defines the attributes of an observation.

To be able to create an observation you need to first define its type. When you create a new observation type, you will find there are a number of core attributes that are present in all observations. In addition to those, like <code>name</code> and <code>description</code>, that you can find in any object, you will find:
*(See [[IoT Semantics#Entity and Observations semantic|Observations types]])

; topic : This is a String. It contains the destination path where this observation will be stored. We will explain later in this document the characteristics of a [[#Topics|topic]]. Like paths in a filesystem you can specify a list of tokens separated by the character '/' to express an specific position in topic's tree. For instance, you can have <code>/amtech</code> as a topic but you can also express something more specific like <code>/us/nevada/las_vegas/hotel_x/main_parking/parking_lot_1/movement_sensor</code> as the topic of an observation generated by the movement sensor that is located in the parking lot 1 of the main parking of Hotel X in Las Vegas. A topic specification can have [[Observation_enrichment#Placeholders_substitution|placeholders]]. Those will be explained later in this document. Example:

<syntaxhighlight lang="jsonld">
 {
   ...
   "topic": "/#{tenantId}/#{country_code}/#{city}",
   ...
 }
</syntaxhighlight>
(See [[Sensor's network#Topics|Topics]])

; producer : A String used to specify the producer of this observation.

; location : If known, this is the location where this observation took place. It's expressed as a JSON String that contains a WKT specification. The structure of this JSON is explained in the [[Location]]'s page.

; ocurrence_time : This is the date and time an observation occurred. It's specified by the producer of the observation.

; detection_time : This is the date and time an observation is detected in the system. The producer can specify it but it will always be overridden by our system.

        properties topic, producer, targetthings and proximityarea can be [[Observation_enrichment|enriched]] with [[Observation_enrichment#Placeholders_substitution|placeholders]]

== targetthings == 
JSON String that specifies the things that are related in some way to an observation with the objectives of:
# Defines a criterion in the [[Observers#Target_things|Observer]].
# Parses it in a reasoner (javascript [[Binding|binding]])
# Associates thing's proximity area value to the proximity area assigned to the field proximityarea (see [[Observation_enrichment#Proximity_updates|proximity updates]]).
# Associates an observation sent as a command with the instance thing that consumes the observation to execute a [[Actions#Send_command|command]]. 

The value consist on a list of JSONs having 
* thingType: a thing type 
* thingsId: list of thing ids or thing names of the given type
* proximityarea (optional): id for automatic enrichment of the thing's proximity area 

The following example illustrates an observation type gpsGeolocation, that will be use in an observer to:
* create a new thing type truck with id truck888 at the position specified in the observation
* update the position of the thing id truck888 to the position specified by the observation 
<syntaxhighlight lang="jsonld"> 
{
    "topic": "/amtech/#{country_code}/#{city}",
    //targetthings a thing type truck with id truck888
    "targetthings": "[{\"thingType\":\"/amtech/linkeddata/types/composite/entity/truck\",\"thingsId\":[\"truck888\"]}]",
    "location": "{\"sContext\":\"geo\",\"wkt\":\"POINT(-99.16495264 19.41553490)\"}",
    "speed": 0,
    "@type": "/amtech/linkeddata/types/composite/observation/gpsGeolocation",
    "creationDate": "2016-03-25T11:30:53.135Z",
    "guesttenants": [],
    "description": "Simulates a gpsGeolocation inside a customer area",
    "producer": "888",
    "detectiontime": "2015-11-02T23:51:29.000Z",
    "longitude": -99.16495264,
    "@id": "/amtech/things/observations/gpsGeolocationInCustomer",
    "occurrencetime": "2015-11-02T23:51:29.000Z",
    "latitude": 19.4155349,
    "accuracy": 0,
    "altitudeAccuracy": 0,
    "heading": 0
}
</syntaxhighlight>

== Topics ==
A topic is a UTF-8 string, which is used by the broker to filter messages for each connected client. A topic consists of one or more topic levels. Each topic level is separated by a forward slash (topic level separator).
    /sensor_network/observation_errors/graiEPC
In contrast to a message queue a topic is very lightweight. There is no need for a client to create the desired topic before publishing or subscribing to it, because a broker accepts any valid topic without any prior initialization.
* Each topic maintains the observation types '''(semantic)''' that have been sent to it. (See [[Sensor's network#Observations and observation types|Observations and observation types]])
* There is a RESTful API to send and get observations type (See [[IoT Restful API#Observation instances|Observation instances]]) 
* Observations from topics can be asynchronously consumed through Websockets (See [[IoT Restful API#Observations|Asynchronous Observation instances]]) 
* By default and unless configured otherwise, the observations are maintained for 1 hour
* Each observation is served to a user once per topic, assuming the user has the access permissions to access it

Topics are automatically discovered when observations get into the sensor's network. It means that they are automatically created if they don't exist and the user sending the observation has the right to do it. When an observation is about to be inserted into a topic, two different things may happen:
* '''The destination topic already exist'''. This is the simplest case and for the observation to be delivered, the user sending it must meet one of these requirements:
** His active tenant at the moment of sending the observation matches the tenant of the destination topic.
** His active tenant at the moment of sending the observation must be in the collection of guesttenants for the destination topic.
** The destination topic must include the special member <code>_ALL</code> in the guesttenants collection.
: In any other case the operation is forbidden.
* '''The destination topic doesn't exist'''. In this case the destination topic is created only if the user sending the observation has the right to do it. Because the destination topic may consist of one or more hierarchical levels, where each topic level is separated by a forward slash, we analyze the existence of each level starting by the one in the top (the root of the hierarchy). Then we analyze one by one each descendant until we get to the final destination topic. For instance, if the destination topic is <code>/sensor_network/observation_errors/graiEPC</code> then we analyze the topics existence in the following order:
*: <code>/sensor_network</code>
*: <code>/sensor_network/observation_errors</code>
*: <code>/sensor_network/observation_errors/graiEPC</code>
: For each one of these topics we do the following:
:* If the topic exists we skip it.
:* If the topic doesn't exist and it's the root topic we can freely create it.
:* If the topic doesn't exist and it's not the root topic then the active tenant of the user sending the observation must match the tenant of the parent topic.
: Each time a topic is created it will be assigned the user and tenant of the user sending the observation.
: Remarks: If the topic cannot be created then the operation of sending the observation is rejected.
;Ownership
*Root topics are always created by a Creator and he owns them
*topics are published with Activity publication 
*follower get access control to root services by subscribing to services
*Root can not have #{placeholder}
*new topic can be created in root topics with access by services 
**topics and descendants are owned by the creator of the new topic

== Observation Enrichment ==

The enrichment of observations is a group of actions and/or replacements that take place over an observation in the process of delivering it to its final destination or topic. They are placeholder substitutions, geolocation substitutions, and proximity updates.

=== Placeholders substitution ===

A placeholder is a string with format #{placeholder_name} than can be used in certain fields of an observation where the placeholders substitution takes place. Actually there are three fields where this kind of substitution is considered:

* <code>topic</code>
* <code>producer</code>
* <code>targetthings</code>

Some substitutions occur in the process of building the observation in applications in the edge like the [[Intelligence_at_the_Edge|M2M Bridge]] and others occur after the observation is sent to the AMTech's platform. The latter are mostly geolocation fields substitutions which is a special kind of placeholder substitution.

For instance, these substitutions occur in the bridge:
* <code>#{tenantId}</code>
* <code>#{userId}</code>

For a full list of placeholders replaced at the edge see [[Intelligence at the Edge]].

=== Geolocation substitution ===

This substitution provides values for placeholders used in several geolocation attributes based on the <code>[[location]]</code> field of an observation. Based on the [[location|WKT]] that is given in the <code>location</code> field, a process of reverse geolocation is run and the address is determined when possible. These fields are then replaced if they appear in the <code>topic</code>, <code>producer</code> or <code>targetthings</code> fields of the original observation JSON.

* <code>#{country}</code> or <code>#{country_code}</code>
* <code>#{state}</code> or <code>#{province}</code>
* <code>#{postcode}</code> or <code>#{postal code}</code> or <code>#{zip code}</code>
* <code>#{city}</code> or <code>#{town}</code>
* <code>#{suburb}</code> or <code>#{residential}</code> or <code>#{city_district}</code> or <code>#{municipality}</code> or <code>#{neighbourhood}</code>

=== Proximity updates ===

A proximity update is executed when an observation arrives to the system containing information about proximity in its <code>targetthings</code> field. For instance:

<syntaxhighlight lang="jsonld">
 {
   ...
   "targetthings": "[{
                       \"thingType\": \"/amtech/linkeddata/types/composite/entity/smartphone\", 
                       \"thingsId\": [\"main_phone_1\"],
                       \"proximityarea\": \"/some/uri/for/proximity/thing\"
                     }]",
   ...
 }
</syntaxhighlight>

Using this information, the field <code>proximityarea</code> of the thing <code>/amtech/things/entities/smartphone/main_phone_1</code> is updated with the given <code>proximityarea</code> in the observation. If the proximity update fails during the enrichment process, a notification is sent to the sensors_network topic but the enrichment process continues unless the notification of the error also fails.

==CRUD observation==
When a create (new), update, or delete operation is executed on a thing instance, and the type of the instance has been configured to send crud observations, an observation instance of type observationresourcecrud is sent to the configured topic (See [[Sensor's network#Topics|Topics]])

To configure a thing type to produce crud observations:
*Include the observation type observationresourcecrud in the thing type observation production configuration.(See [[Activities#Thing types|Thing types]])
*Include the thing type, with the observation observationresourcecrud, in the observation production configuration of an activity that the user is subscribed to (the user that will create/update/delete things and wants crud observations to be sent).(See [[Activities#Observation production configuration|Observation production configuration]])
The following example illustrates a '''crudoperation''' PUT on a resource '''@id''' /amtech/things/observations/llrpAntennaInRoom5586417595299841, the '''proId''' changed from '''oldvalue''' false to '''newvalue''' true (See [[Sensor's network#Observations and observation types|Observations and observation types]])
 <syntaxhighlight lang="jsonld">
    {
        "topic": "/thingsInBoardroom/ccAsync/",
        "_tenant": "amtechdemocreator",
        "targetthings": "[]",
        "@type": "/amtech/linkeddata/types/composite/observation/observationresourcecrud",
        "propId": "sendSmoothingLost",
        "resourceuri": "/amtech/things/entities/LLRPAntenna/llrpAntennaInRoom",
        "resourcetype": "/amtech/linkeddata/types/composite/entity/LLRPAntenna",
        "newvalue": "true",
        "guesttenants": [],
        "oldvalue": "false",
        "crudoperation": "PUT",
        "producer": "",
        "detectiontime": "Tue Apr 26 22:35:57 UTC 2016",
        "occurrencetime": "Tue Apr 26 22:35:56 UTC 2016",
        "@id": "/amtech/things/observations/llrpAntennaInRoom5586417595299841",
        "_user": "amtechdemocreator@amtech.mx"
    }
</syntaxhighlight>
;This observation allows reasoners to focus just on monitoring the changes to specific thing properties, independent of what originated the transformation of the Thing.

;It supports the following [[Sensor%27s_network#Placeholders_substitution|placeholders]] :
* #{thingId}
* #{tenantId}
* #{userId}
* #{typeId}
* #{crudOperation}

==Sensor network error==
Occurs when there is an error in the process of delivering an observation to its destination topic.
Usually occurs in the process of [[observation enrichment]]
*topic "/sensor_network/"
**An observation type observationoutputmsg get send to the sensor network topic /sensor_network/observation_errors/[observationTypeName] in the following example is an observation type observationresourcecrud sent with a wrong topic value "message": "Failed registering topic for observation"
<syntaxhighlight lang="jsonld">
     {
        "topic": "/sensor_network/observation_errors/observationresourcecrud",
        "_tenant": "m2mcreator",
        "@type": "/amtech/linkeddata/types/composite/observation/observationoutputmsg",
        "resource": "{\"topic\":\" /helloWorld/asynch\",\"_tenant\":\"m2mcreator\",
        \"targetthings\":\"[]\",
        \"@type\":\"/amtech/linkeddata/types/composite/observation/observationresourcecrud\",
        \"propId\":\"hwgreetingLabel\",\"resourceuri\":\"/amtech/things/entities/HelloWorld/helloWorld\",
        \"resourcetype\":\"/amtech/linkeddata/types/composite/entity/HelloWorld\",
        \"newvalue\":\"heyWithError\",\"guesttenants\":[],
        \"oldvalue\":\"hey\",\"crudoperation\":\"PUT\",
        \"producer\":\"helloWorld\",\"detectiontime\":\"2016-04-22T18:33:11.481Z\",
        \"@id\":\"/amtech/things/observations/helloWorld5226521372857087\",
        \"occurrencetime\":\"2016-04-22T18:33:10.948Z\",\"_user\":\"m2mcreator@amtech.mx\"}",
        "_name": "idbf740e7d83064ba492c9ca33d20617a0",
        "resourcetype": "/amtech/linkeddata/types/composite/observation/observationresourcecrud",
        "message": "Failed registering topic for observation",
        "guesttenants": [],
        "producer": "enrichment-topology",
        "detectiontime": "Fri Apr 22 18:33:11 UTC 2016",
        "@id": "/amtech/things/observations/6de7bf42-2ea6-4140-a8f4-a427bc119701",
        "success": false,
        "messagedetail": 
        "Error registering topic '/amtech/things/eventsources/ ': Resource /amtech/things/eventsources/  is not valid for its type definition. Property @idcannot contain blank spaces. ",
        "_user": "m2mcreator@amtech.mx"
    }
</syntaxhighlight>
==Simulator==
The simulator is the page in the UI that allows the user to create observation instances to be used later to:
* Debug activity observer execution. See [[Observers#Observers_for_activities|activity observers]]
* Debug a reasoner at the reasoner debugger sandbox or in the cloud. See [[Debug_%26_check_a_reasoner|debugging activity reasoners]].

Once the observation instances are created, it will be possible for users with '''follower''' role to simulate them, this is to send them to the queue system as if they were generated by a thing in the [[AMTech...#IoT|IoT]], by clicking on the Send button [[File:simulatorSendObservation.png]]. Observations are actually sent to the sensor's network and you can see the result by consuming then from the corresponding topic. 
{{Note|Activity creators are not allowed to send observations. They can use this button to generate and setup (if needed) the topics configured in the observation instance.}}

=== Creating/Editing/Removing observation instance ===
Like every other resource, click on [[File:Ue-create.png]] to create a new observation and select the observation type on the newly opened dialog. The following example illustrates creating an observation instance from gpsGeolocation. (See [[Sensor's network#Observations and observation types|Observations and observation types]]) 
[[File:Simulator1.png|850px|thumbnail|center|Select observation type]]
Edit the observation fields as usual by ensuring to set the ''topic'' and ''producer'' fields which are required.
In order to delete an observation, click on [[File:Ue-delete.png]].

=== Simulate sending an Observation ===
Select the observation to be sent or navigate to one of them. Click on the ''Send observation'' button [[File:simulatorSendObservation.png]] to open the dialog box where you can set
* order to send the observations. Drag and drop the selected observation instances 
* Number of copies to be sent, per selected observations 
* Sequence, the message will be send in the selected sequence.
* It happened now, the observation occurrence time property is set with current time.
**[[File:Simulator3.png|850px|thumbnail|center|Send observation]]
* Observations are actually sent to the sensor's network and you can see the result by consuming them from the corresponding topic.
{{Note|these observations will also trigger the currently deployed activities.}}