1. Introduction

2. Overview

2.1. What is Live Objects?

Live Objects is a Software as a Service belonging to Orange Datavenue service suite.

Live Objects is a software suite for IoT / M2M solution integrators offering a set of tools to facilitate the interconnection between devices (or connected « things ») and business applications:

  • Connectivity Interfaces (public and private) to collect data, send command or notification from/to IoT/M2M devices,

  • Device Management (supervision, configuration, resources, firmware, campaign devices management, etc.),

  • Message Routing between devices and business applications,

  • Data Management: Data Storage with Advanced Search features, Message enrichment process and Encoding service.

Live Objects overview

landing

The public interfaces are reachable from internet. The private interfaces provide interfaces with a selection of devices (MyPlug) or specific network (LORA).

The SaaS allows multiple tenants on the same instance without possible interactions across tenant accounts (i.e. isolation, for example a device belonging to one tenant could not communicate with a device belonging to an other tenant).

A web portal provides a UI to administration functions like manage the messages and events, supervise your devices and control access to the tenant.

2.2. Architecture

Live Objects SaaS architecture is composed of two complementary layers:

  • Connectivity layer: manages the communications with the client devices and applications,

  • Service layer: various modules supporting the high level functions (device management, data processing and storage, etc.).

Live Objects architecture

landing

2.3. Connectivity layer

Live Objects exposes a set of standard and unified public interfaces allowing to connect any programmable devices, gateway or functional IoT backend.

The existing public interfaces are:

MQTT is an industry standard protocol which is designed for efficient exchange of data from and to devices in real-time. It is a binary protocol and MQTT libraries have a small footprint.

HTTPS could be rather used for scarcely connected devices. It does not provide an efficient way to communicate from the SaaS to the devices (requiring periodic polling for example).

For more info, see "Message encodings" section

The public interfaces share a common security scheme based on API keys that you can manage from Live Objects APIs and web portal.

Live Objects is fully integrated with a selection of devices and network. It handles communications from specific families of devices, and translate them as standardized messages available on Live Objects message bus.

The existing private interfaces:

  • LORA interface connected with LORA network server,

    • to provision LORA devices

    • to receive and send data from/to LORA devices

2.4. Service layer

Live Objects connectivity interfaces are connecting to a message bus that could exchange messages with external business application or custom external backend. In this case the MQTT application protocol is used in MQTT broker like a bus service layer.

The message bus offers three distinct modes:

  • FIFO : the solution to prevent from message loss in the case of consumer unavailability. Messages are stored in a queue on disk until consumed and acknowledged. When multiple consumers are subscribed to the same queue concurrently, messages are load-balanced between available consumers. More info: FIFO mode,

  • Application mode : a good fit for real-time exchanges between Live Objects backend and your business application. Data messages exchanged are queued in the FIFO queuing bus mechanism mode "Application".

  • Connector mode : a good fit for connect your custom backend who manage your devices park and when your devices not supported by Live objects connectivity. This mode has an MQTT API described in mode "Connector".

2.4.1. Device management

Live Objects offers various functions dedicated to device operators:

  • supervise devices and manage connectivity,

  • manage devices configuration parameters,

  • send command to devices and monitor the status of these commands,

  • send firmwares (any binary file) to devices and monitor the status of this operation.

Live Objects attempts to send command, firmwares or update the parameters on the device as soon as the device is connected and available.

For more info about device and connectivity inventory, see "Device and connectivity inventory" chapter.
For more info about device connectors, see "Device connectors" section.
For more info about device management, see "Device Management" chapter.

2.4.2. Data management

Live Objects allows to store the collected data from any connectivity interfaces. These data could be then retrieved by using HTTPS REST interface.

A full-text search engine based on Elastic search is provided in order to analyze the data stored. This service is accessible through HTTPS REST interface.

For more info, see "Data Management" chapter.

2.4.3. Messages and Events Processing

2.4.3.1. SEP, SP and AP

Simple event processing service is aimed at detecting notable single event from the flow of data messages.

Based on processing rules that you define, it generates fired events that your business application can consume to initiate downstream action(s) like alarming, execute a business process, etc.

For more info, see "Simple Event Processing" section.

In the other hand, Live Objects allows to process the Activity processing (AP) service which aims at detecting devices that have no activity.

For more info, see "•Activity processing" section.

The State processing (SP) service aims at detecting changes in "device state" computed from data messages.

For more info, see "•State processing" section.

2.4.3.2. Routing and Alarming

The FIFO mode communication is based on the usage of topics to publish a messages with ensuring no message loss. It can forward or route messages to recipients without loss using FIFO Publication and FIFO subscription. You can also use the HTTP push forwarding after the message is processed by event processing service or message routing service. Those services give a notifications or give an alarm to the recipients on purpose.

2.5. Access control

2.5.1. API keys

API keys are used to control the access to the SaaS for devices/application and users to authenticate. You must create an API key to use the API.

For more info, see API key and Role chapters.

2.5.2. Users management

When an account is created, a user with administration priviledges is also created on the account. This administrator can add other users to the account and set their priviledges. These priviledges are defined by a set of roles. The users can connect to the Live Objects web portal.

3. Getting started

This chapter is a step-by-step manual for beginners with Live Objects giving instructions covering the basic use cases of the service. Here is a small process to begining with Live objects and help you to easy building a use case and end-to-end test.

  • If you have device you can starting to use it.

  • If you do not have a device, browse the instructions in this guide to build a usage that uses only simulators to replace a device. You can use your own device simulator and use MQTT or COAP protocol to connect your virtual device to Live Objects.

Requirements : the data message sent by device (or simulator) must respect the Live Objects data model.

Connect your device and retrieve data with Live Objects (steps by device category and connectivity)

landing

3.1. Log in

To log in to Live Objects web portal, connect to liveobjects.orange-business.com using your web-browser:

landing

  • Fill the Sign in form with your credentials:

    • your login,

    • the password set during the activation phase,

  • then click on the Sign in button.

You are redirected to your “Dashboard” page:

landing

3.2. Connect and test your LoRa® device

Lora devices need to be registered on Live Objects to send messages.

To quick register your LoRa® device, use the Live Objects portal, liveobjects.orange-business.com.

3.2.1. Register your device

To register a Lora device, go to the Devices menu, select LoRa in the list and click on Add device.

In the form, select the profile that corresponds to your device. If you do not know, select Generic_classA_RX2SF12. Enter your device’s DevEUI, AppEUI and AppKey. Depending on the LoRa device you are using, these parameters are pre configured and they should come with the device, or you can configure the device firmware with your own parameters. Then click on Register.

You can see your device in the Devices menu.

3.2.2. Connect your LoRa® device

When the device is registred then join LoRa® network and it would automatically connected to Live Objects. The device is now ready to send uplink data and begining to publish the data messages.

When you prefer using an API, see Register a LoRa® device

3.3. Simulating an MQTT device

To make a device or an application communicate with Live Objects using MQTT, you will need an API key.

3.3.1. Create an API key

To create an API key, go to the Configuration menu, click on Api keys in the left menu and create a new API key. Select at least the role "DEVICE_ACCESS".

landing

As a security measure, you can not retrieve the API key again after you have closed the API key creation results page. So, note it down to work with the MQTT client, during the scope of this getting started.

landing

3.3.2. Download your simulator tool

It is up to you to choose your favorite MQTT client or library. We will use here MQTT.Fx client. This client is available on Win/MacOSX/Linux and is free. Download and install the last version of MQTT.fx.

3.3.3. Connect your simulator

We will start by creating a new Connection profile and configure it based on the device mode set up.

General panel:

You will configure here the endpoints of Live Objects including authentication information. In this panel, you can set :

landing

Credentials panel:

  • username: json+device : for device mode MQTT connection

  • password: the API key that you just created with the "DEVICE_ACCESS" role

landing

3.3.4. Verify status

We can simulate a device connection to Live Objects with MQTT.fx client by clicking on Connect button of MQTT.fx client.

In Live Objects portal, go to the Devices page, the connected device has "green" status.

landing

3.3.5. Publishing data messages

We will use MQTT.fx client with device mode to send data message as a device would do.

Data message must be published on this topic: dev/data.

Message:

{
"s" : "urn:lo:nsid:dongle:00-14-22-01-23-45!temperature",
"ts" : "2016-07-10T10:02:44.907Z",
"loc" : [44.1, -1.5],
"m" : "temperatureDevice_v0",
"v" : {
  "status" : "online",
  "tempreature rise frequency" : 10,
  "temp" : 17.25
 },
"t" : [ "City.NYC", "Model.Prototype" ]
}

With:

  • s: the streamId of the message, the message will be forwarded to a topic containing the streamId so an application connected to Live Objects can retrieve messages

  • ts: the timestamp of the message, this field is optionnal, if not set by the device, Live Objects will set it with the time of arrival,

  • loc: the location of the device, this field is optionnal,

  • m: the model of data contained in the v field, (more about data model)

  • v: the values, this is a free field to store device specific data such as the device sensor values,

  • t: a list of tags, this field is optionnal.

landing

3.4. Simulating a CoAP\LWM2M device (Beta)

If you want simulate devices that use CoAP\LWM2M2 protocol, you can use a simulator and this section help you to send data using this protocol and to the Live Objects interface (to see more about CoAP\LWM2M).

3.5. Device Management basics

3.5.1. Sending a command to a Lora device

Messages sent by Lora devices to Live Objects are called uplinks. Messages sent by Live Objects to Lora devices are called downlinks.

To send a command, go to Devices then select your Lora device in the list and go to Downlink tab. Click on Add command, enter a port number between 1 and 223 and enter data in hexadecimal, 0A30F5 for example. The downlink will be sent to your device after the next uplink.

3.5.2. Sending a command to an MQTT device

You must first subscribe to the topic waiting for command "dev/cmd". (Subscribe tab of MQTT.fx)

Go to Devices then select your device in the list and go to Commands tab.

Click on add command then fill the event field with "reboot" then click on Register. The command will appear in MQTT.fx client subscribe tab.

{
   "req":"reboot",
   "arg":{},
   "cid":94514847
 }

landing

A response can be sent to acknowledge the command received.

To send this response, you can publish a message to this topic "dev/cmd/res". (correlation identifier) must be set with the value received previously.

{
  "res": {
     "done": true
  },
  "cid": 94514847
}

Once published, the status of the command will change to "processed" in the portal commands history tab.

A correlation ID uniquely identifies each ‘request’, or the equivalent and used in your intercation with Live Objects API.

3.6. Accessing the stored data

There are several ways to access you device’s data:

  • Going back to Live Objects portal, you can consult the data message that was just stored. Go to Data then search for stream "urn:lo:nsid:dongle:00-14-22-01-23-45!temperature". The data message sent will appear.

landing

  • You can access data from the HTTPS interface.

  • You can perform complex search queries like aggregation using elasticsearch DSL HTTPS interface. See example in See example in Data API chapter

  • Through the MQTT interface, either your configured FIFO queue or your connected MQTT client will receive data messages.

3.7. Connect an application to collect your devices' data

3.7.1. Using HTTPS to retrieve your devices' data

Data from you device can be retrived using HTTPS API.

You will need an API key with the DATA_R role. You can create it as you did in the previous part.

The following endpoint enables you to retrieve data from a stream: https://liveobjects.orange-business.com/api/v0/data/streams/{streamId}

Example with a MQTT device:
GET /api/v0/data/streams/urn:lo:nsid:dongle:00-14-22-01-23-45!temperature
Host: https://liveobjects.orange-business.com
Header: X-API-KEY: <a valid API key with DATA_R role>
Example with a Lora device:
GET /api/v0/data/streams/urn:lo:nsid:lora:<DevEUI>
Host: https://liveobjects.orange-business.com
Header: X-API-KEY: <a valid API key with DATA_R role>

3.7.2. Using MQTT to retrieve your devices' data

Data published by your devices can be consumed using the MQTT protocol.

You will need an API key with the BUS_R role. You can create it as you did in a previous part.

Then configure a FIFO queues to persist your device’s data until you consume them. To do this, you will have to create a FIFO queue and then route your device’s data to the queue.

In the Data menu, go to the FIFO tab and click on the add a FIFO queue button In the pop-in enter a name myFifo for your FIFO queue, then press Register button: the newly created FIFO queue myFifo is now listed.

landing

The FIFO queue is ready to persist data. To make your device’s data enter the queue, you have to route them to the queue.

In the Data menu, go to the Routing key tab and click on the add a routing key button.

  • enter _~event.v1.data.new.typ.\*.dev.*.con.\*.evt.*.grp.#_ in the "_Routing key filter_" input field, [deprecated]

  • enter ~event.v1.data.new.urn.lora.# in the Routing key filter input field, [deprecated]

  • select myFifo in the Target FIFO select box,

  • press the *Create Binding* button. [deprecated]

Now the binding rules mechanism is deprecated, more information about bindings format in the legacy section Routing Key Filters.
The deprecated topic will be decommissioned in december 2018. Please use the new topic to receive data messages. Please avoid the use of topic event.v1.data.new.#

Future messages sent by your device will be stored in the FIFO queue until an MQTT client subscribes to the queue.

To this end, the following endpoints are in place:

  • mqtt://liveobjects.orange-business.com:1883 for non SSL connection

  • mqtts://liveobjects.orange-business.com:8883 for SSL connection

The websockets may alternatively be used:

  • ws://liveobjects.orange-business.com:80/mqtt

  • wss://liveobjects.orange-business.com:443/mqtt

Beforehand, a connection should be established using sending an MQTT Connect packet. This package must be written as follow:

  • clientId: ignored

  • username: "payload"

  • password: a valid LO API key with a BUS_R role

  • willRetain, willQoS, willFlag, willTopic, willMessage: ignored

  • keepAlive: as desired, in seconds (recommended: 30)

Then a subscription to the following topic to consume data from your FIFO : "fifo/myFifo".

4. Account Management and Access Control

4.1. Tenant account

A tenant account is the isolated space on Live Objects dedicated to a specific customer: every interaction between Live Objects and an external actor (user, device, client application, etc.) or registered entities (user accounts, API keys, etc.) is associated with a tenant account.

Live Objects ensures isolation between those accounts: you can’t access the data and entities managed in another tenant account.

Each tenant account is identified by a unique identifier: the tenant ID.

A tenant account also has a name, that should be unique.

4.2. User account

A User Account represents a user identity, that can access the Live Objects web portal.

A user account is identified by a login. A user account is associated with one or many roles. A user can authenticate on the Live Objects web portal using a login and password.

When user authentication request succeeds, a temporary API key is generated and returned, with same roles as the User account.

In case of too many invalid login attempts, the user account is locked out for a while.

For security purpose, a password must be at least 8 characters including at least 1 uppercase letter, 1 lowercase letter, 1 special character, 1 digit and 1 special characters.

4.3. API key

landing

A Live Objects API key is a secret that can be used by a device, an application, or a user to authenticate when accessing to Live Objects on the MQTT or HTTPS/REST interfaces. At least one API key must be generated. As a security measure, an API key can not be retrieved after creation (As describe in this section), .

An API key belongs to a tenant account: after authentication, all interactions will be associated only to this account (and thus isolated from other tenant accounts).

An API key can have zero, one or many Roles. These roles allows to restrict the operations that could be performed with the key.

An API key validity can be limited in time.

Creating a tenant account automatically attribute a "master" API key.

That API key is special: it can’t be deleted.

An API key can generate child-API keys that inherit (a subset of) the parent roles and validity period.

An API key can be restrained to one or more message queues:

  • If one or more message queues are selected, the API key can access only these queues.

  • If no message queue is selected, the API key has no restriction and can access any queue.

Thus, the API key can only be used in MQTT access limited to these selected message queues. This for example, makes it possible, after having oriented the right device to the right queue, to restrict access to the data of specific devices.

Usage:

  • In MQTT, clients must connect to Live Objects by using a valid API key value in the password field of the (first) MQTT « CONNECT » packet,

    • In case of unknown API key value, or invalid, the connection is refused.

    • On success, all messages published on this connection will be enriched with the API key id and roles.

  • In HTTPS, clients must specify a valid API key value as HTTP header X-API-Key for every request,

    • In case of unknown API key value, request is refused (HTTP status 403).

    • In case of invalid API key value, request is refused (HTTP status 401).

    • On success, all messages published due to this request will be enriched with the API key id and roles.

4.4. Role

A Role is attributed to an API key or User Account. It defines the privileges on Live Objects. A Role is attributed to an API key or User Account.

Important Notice : Some features are only available if you have subscribed to the corresponding offer, so you may have the proper roles set on your user but no access to some features because these features are not activated on your tenant account (check the tenant offer).

The currently available roles and their inclusion in Admin or User profiles:

Role Name Technical value Admin profile User profile Priviledges

API key

API_KEY_R

X

X

Read parameters and status of an API key.

API key

API_KEY_W

X

X

Create, modify, disable an API key.

User

USER_R

X

X

Read parameters and status of a user.

User

USER_W

X

Create, modify, disable a user.

Settings

SETTINGS_R

X

X

Read the tenant account custom settings.

Settings

SETTINGS_W

X

X

Create, modify tenant account custom settings.

Device

DEVICE_R

X

X

Read parameters and status of a Device mangement.

Device

DEVICE_W

X

Create, modify, disable a Device mangement, send command, modify config, update firmware of a Device.

Device Campaign

CAMPAIGN_R

X

X

Read parameters and status of a massive deployment campaign on your Device Fleet.

Device Campaign

CAMPAIGN_W

X

Create, modify a campaign on your Device Fleet.

Data

DATA_R

X

X

Read the data collected by the Store Service or search into this data using the Search Service.

Data

DATA_W

X

Insert a data record to the Store Service. Minimum permission required for the API key of a device pushing data to Live Objects in HTTPS.

Data Processing

DATA_PROCESSING_R

X

X

Read parameters and status of an event processing rule or a Data decoder.

Data Processing

DATA_PROCESSING_W

X

Create, modify, disable an event processing rule or a Data decoder.

Kibana (Beta feature, limited access)

KIBANA_R

X

Access to a Kibana instance running on the Search Service. Requires also DATA_R role to be effective.

Bus Config

BUS_CONFIG_R

X

X

Read config parameters of a FIFO queue or a binding rule.

Bus Config

BUS_CONFIG_W

X

Create, modify a FIFO queue or a binding rule.

Bus Access

BUS_R

X

X

Read data on the Live Objects bus. Minimum permission for the API key of an application collecting data on Live Objects in MQTT(s).

Bus Access

BUS_W

X

X

Publish data on the Live Objects bus.

Bus Access

DEVICE_ACCESS

X

X

Role to set on a Device API key to allow only MQTT Device mode

5. Device and connectivity

The version 1 of the device APIs (/api/v1/deviceMgt) comes up with a brand new model of devices in Live Objects.

We’ve worked on providing unified representations whatever the technology that is used to connect the device to Live Objects : Internet (MQTT, HTTP), LoRa® or SMS, while maintaining the access to each specifics technology.

5.1. Principles

A device is a generic term that can designate an equipment (sensor, gateway) or an entity observed by equipments (ex: a building, a car).

5.2. Device description

A device description is a set of Json documents that store a device identity information, a device state information, a metadata (definition, activity, alias) a device connectivty information for each device that you will connect to Live objects.

Each device description has 3 sections to describe the device representation :

  • Device identity.

  • Device interface representation.

  • Device connector node representation.

5.2.1. Devices representation

5.2.1.1. Device identifier format

A device identity is represented by a unique identifier. This identifier must respect the following format: urn:lo:nsid:{ns}:{id}

Where:

  • ns: your device identifier "namespace", used to avoid conflicts between various families of identifier

  • id: your device id
    Should only contain alphanumeric characters (a-z, A-Z, 0-9) and/or any special characters amongst ' ' : - _ | + and must avoid # / !.

Device POJO:

JSON Params Description

id

device unique identifier (Cf. device identifier)

description

Optional. detailed description of the device

name

Optional. name of the device

defaultDataStreamId

default data stream id. Specify the streamId where the data will be store (Cf. "Data management" section).

tags

Optional. list of additional information used to tag device messages

properties

Optional. map of key/value string pairs detailing device properties

group

group to which the device belongs. The group is defined by its id and its path

interfaces

Optional. list of device network interfaces (Cf. interface POJO)

created

creation date of the device

updated

last update date of the device

config

Optional. device configuration

firmwares

Optional. device firmware versions

activityState

Optional. device activity state aggregated from the activity processing service, the special state NOT_MONITORED means that the device is not targeted by any activity rule

5.2.1.2. Device interface representation

An interface is a platform access. A device can have no, one or several interfaces, which represent different connectivities that the device could use to communicate with Live Objects. Each interface is associated to a protocol managed by Live Objects connector: LoRa®, SMS or MQTT.

5.2.1.3. Device connector node representation

A connector node is the copy of a device interface in the Live objects connectors databases. In other words: connector node is an interface of a device , with additional network parameters and status. Besides connector nodes are sorted by connector in the API (LoRa®, MQTT, SMS) and not by device.

multi connectivity device

landing

To resume end-to-end communication between the device and Live Objects over Lora network, mobile network and IP network. The device need a dedicated network interface for each connectivity supported. One or more connectivity can be allowed (when the device supports more than one) for data trafic betwen Live Objects and the device, for example, the main connectivity is reserved for standard trafic and the other(s) to backup when the main connectivity is down.

Interface VS connector node

landing

Interface POJO:

JSON Params Description

connector

connector identifier

nodeId

interface unique identifier

enabled

define if the interface is enabled or disabled

status

interface status

lastContact

Optional. last contact date of the connectivity

Connector Node POJO:

JSON Params Description

connector

connector identifier

nodeId

interface unique identifier

deviceId

Optional. device unique identifier

enabled

define if the interface is enabled or disabled

status

interface status

definition

interface definition. The definition depends on connector.

activity

interface activity. The activity depends on connector.

created

registration date of the interface

updated

last update date of the interface

For more information on each connector definition, activity and status, see the appropriate section:

You can register a device with interface.

Currently, you can associate to a device an SMS interface (Cf. register device with an SMS interface example) or an LoRa® interface (Cf. register device with a LoRa® interface example).
Until the other interfaces, the MQTT and External connector interfaces will be automatically register at the first MQTT connection.

Each connector node has a status field which shows the state of the corresponding interface. The values are the same for all connectors, but each connector sets the status differently. The following table shows which statuses are supported, or will soon be supported by connectors.

5.2.1.4. Node status

Status \ Connector

LoRa®

MQTT

SMS

External connector

REGISTERED

non applicable

ONLINE

OFFLINE

CONNECTIVITY_ERROR

INITIALIZING

INITIALIZED

REACTIVATED

ACTIVATED

DEACTIVATED

Each connector node has an enabled flag which allows or forbids a node from connecting and communicating with Live Objects. The enabled flag does not change the node’s status.

This flag can be set when creating the interface or updating the connector node.

5.3. Device management basic

5.3.1. Register a device

5.3.1.1. Request

Endpoint:

POST /api/v1/deviceMgt/devices

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Body:

JSON Params Description

id

device unique identifier (Cf. device pojo)

tags

Optional. (Cf. device pojo)

name

Optional. (Cf. device pojo)

description

Optional. (Cf. device pojo)

defaultDataStreamId

Optional. (Cf. device pojo)

properties

Optional. (Cf. device pojo)

group

Optional. (Cf. device pojo)

interfaces

Optional. (Cf. device pojo)

You can register a device with interface.

Currently, you can associate to a device an SMS interface (Cf. register device with an SMS interface example) or an LoRa® interface (Cf. register device with a LoRa® interface example).
Until the other interfaces, the MQTT interface will be automatically register at the first MQTT connection

Example: Register a device without interface

POST /api/v1/deviceMgt/devices
{
  "id": "urn:lo:nsid:sensor:temp001",
  "tags": ["Lyon", "Test"],
  "name": "mySensor001",
  "description": "moisture sensor",
  "properties" : {
      "manufacturer": "Orange",
      "model": "MoistureSensorV3"
  },
  "group": {
      "path": "/france/lyon"
  }
}
5.3.1.2. Response

HTTP Code:

200 OK

Body:

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

403

GENERIC_OFFER_DISABLED_ERROR

The requested service is disabled in your offer settings. Please contact a sales representative.

403

GENERIC_ACTION_FORBIDDEN_ERROR

You do not have the required permissions to execute this action.

404

DM_GROUP_NOT_FOUND

Group not found

409

DM_CONNECTOR_NODE_DUPLICATE

Conflict on (connector/nodeId)

409

DM_DEVICE_DUPLICATE

Conflict on device id

Example: Register a device without interface

{
    "id": "urn:lo:nsid:sensor:temp001",
    "description": "moisture sensor",
    "name": "mySensor001",
    "defaultDataStreamId": "urn:lo:nsid:sensor:temp001",
    "tags": ["Lyon", "Test"],
    "properties": {
        "manufacturer": "Orange",
        "model": "MoistureSensorV3"
    },
    "group": {
        "id": "sWyaL2",
        "path": "/france/lyon"
    },
    "created": "2018-02-12T13:29:52.442Z",
    "updated": "2018-02-12T13:29:52.442Z"
}

5.3.2. List devices

5.3.2.1. Request

Endpoint:

GET /api/v1/deviceMgt/devices

Query parameters:

Name Description

limit

Optional. maximum number of devices in response. 20 by default.

offset

Optional. the number of entries to skip in the results list. 0 by default.

sort

Optional. sorting selection. Prefix with '-' for descending order. Supported value: id, name, group, created, updated,interfaces.status, interfaces.enabled, interfaces.lastContact. Example: ["urn","-creationTs"]"..

id

Optional. regexp on device id

groupPath

Optional. regexp on groupPath

groupId

Optional. requested groupId

name

Optional. regexp on device name

tags

Optional. filter list by device tags

connectors

Optional. filter list by interface connector

fields

Optional. fields to return for each device. By default, information returned are id, name, group and tags. Supported value: name, description, group, tags, properties, interfaces, config, firmwares, defaultDataStreamId, activityState, created and updated.

interfaces.nodeId

Optional. Filter list by nodeId.

interfaces.status

Optional. Filter list by interface status. Supported values: REGISTERED, ONLINE, OFFLINE, CONNECTIVITY_ERROR, DELETED.

interfaces.enabled

Optional. Filter list by interface enabled state.

property.{filterName}

Optional. Multiple filters, Example: devices?property.temperature=25&property.humidity=58…​

filterQuery

Optional. Device filter expression using RSQL notation. Supported device properties are 'groupPath', 'groupId', 'tags', 'properties'. Supported RSQL operators are '==', '!=', '=in=', '=out=', '=re=', '=lt=', '=le=', '=gt=', '=ge=', 'and', 'or'.

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json
X-Total-Count: <boolean>

For more info about X-Total-Count, see "API v1 paging" section.

Example:

GET /api/v1/deviceMgt/devices?name=mySensor*&sort=-id&fields=name,created,group
5.3.2.2. Response

HTTP Code:

200 OK

Body: List of device POJO

JSON Params Description

id

device unique identifier

description

Optional. detailed description of the device

name

Optional. name of the device

defaultDataStreamId

Optional. default data stream Id of the device

tags

Optional. list of device tags

properties

Optional. properties of the device

group

Optional. group to which the device belongs

interfaces

Optional. list of device’s network interfaces

created

Optional. registration date of the device

updated

Optional. last update date of the device

config

Optional. device configuration

firmwares

Optional. current version of all device firmwares

activityState

Optional. device activity state aggregated from the activity processing service, the special state NOT_MONITORED means that the device is not targeted by any activity rule

Example:

[
    {
        "id": "urn:lo:nsid:sensor:temp002",
        "name": "mySensor002",
        "group": {
            "id": "root",
            "path": "/"
        },
        "created": "2018-01-08T16:08:39.964Z"
    },
    {
        "id": "urn:lo:nsid:sensor:temp001",
        "name": "mySensor001",
        "group": {
            "id": "sWyaL2",
            "path": "/france/lyon"
        },
        "created": "2018-01-08T16:08:27.259Z"
    }
]

5.3.3. Get a device

5.3.3.1. Request

Endpoint:

GET /api/v1/deviceMgt/devices/<deviceId>

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

GET /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001
5.3.3.2. Response

HTTP Code:

200 OK

Body:

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

401

UNAUTHORIZED

Authentication failure.

404

DM_DEVICE_NOT_FOUND

Device not found.

Example:

{
    "id": "urn:lo:nsid:sensor:temp001",
    "description": "moisture sensor",
    "name": "mySensor001",
    "defaultDataStreamId": "urn:lo:nsid:sensor:temp001",
    "tags": [
        "france",
        "lyon"
    ],
    "properties": {
        "manufacturer": "Orange",
        "model": "MoistureSensorV3"
    },
    "group": {
        "id": "sWyaL2",
        "path": "/france/lyon"
    },
    "created": "2018-01-08T16:08:27.259Z",
    "updated": "2018-01-08T16:08:27.259Z"
}

5.3.4. Delete device

5.3.4.1. Request

Endpoint:

DELETE /api/v1/deviceMgt/devices/<deviceId>

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

DELETE /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001
5.3.4.2. Response

HTTP Code:

204 NO CONTENT

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

404

DM_DEVICE_NOT_FOUND

Device not found

5.4. Device interface management

5.4.1. Add an interface to a registered device

5.4.1.1. Request

Endpoint:

POST /api/v1/deviceMgt/devices/<deviceId>/interfaces

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Body:

JSON Params Description

connector

connector id

enabled

define if the interface is enabled or disabled

definition

interface definition. The definition depends on connector (Cf. SMS interface definition or LoRa® interface definition).

Currently, you can only create an SMS or an LoRa® interface, MQTT interface will be auto-provisionned at the first connection.

Example: Create an SMS interface

POST /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/interfaces
{
    "connector": "sms",
    "enabled": true,
    "definition": {
        "msisdn": "33601201201"
    }
}
5.4.1.2. Response

HTTP Code:

200 OK

Body:

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

403

GENERIC_OFFER_DISABLED_ERROR

The requested service is disabled in your offer settings. Please contact a sales representative.

404

DM_DEVICE_NOT_FOUND

Device not found

404

DM_CONNECTOR_UNAVAILABLE

Connector not found or unavailable

409

DM_CONNECTOR_NODE_DUPLICATE

Conflict on (connector/nodeId)

Example:

{
    "connector": "sms",
    "nodeId": "33601201201",
    "deviceId": "urn:lo:nsid:sensor:temp001",
    "enabled": true,
    "status": "ONLINE",
    "definition": {
        "msisdn": "33601201201"
    },
    "activity": {},
    "created": "2018-03-02T15:54:33.943Z",
    "updated": "2018-03-02T15:54:33.943Z"
}

5.4.2. List device interfaces

5.4.2.1. Request

Endpoint:

GET /api/v1/deviceMgt/devices/<deviceId>/interfaces

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

GET /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/interfaces
5.4.2.2. Response

HTTP Code:

200 OK

Body:

List of interface POJO

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

404

DM_DEVICE_NOT_FOUND

Device not found.

Example:

[
  {
    "connector": "sms",
    "nodeId": "33601201201",
    "enabled": true,
    "status": "ONLINE"
  },
  {
    "connector": "mqtt",
    "nodeId": "urn:lo:nsid:sensor:temp001",
    "enabled": true,
    "status": "OFFLINE",
    "lastContact": "2018-03-02T15:57:23.772Z"
  }
]

5.4.3. List all nodes of a connector

5.4.3.1. Request

Endpoint:

GET /api/v1/deviceMgt/connectors/<connector>/nodes

Query parameters:

Name Description

limit

Optional. maximum number of devices in response. 20 by default.

offset

Optional. the number of entries to skip in the results list. 0 by default.

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json
X-Total-Count: <boolean>

For more info about X-Total-Count, see "API v1 paging" section.

Example:

GET /api/v1/deviceMgt/connectors/sms/nodes
5.4.3.2. Response

HTTP Code:

200 OK

Body:

Error case:

HTTP Code Error code message

403

GENERIC_OFFER_DISABLED_ERROR

The requested service is disabled in your offer settings. Please contact a sales representative.

404

DM_CONNECTOR_UNAVAILABLE

Connector not found or unavailable

Example:

[
    {
        "connector": "sms",
        "nodeId": "33602020202",
        "enabled": true,
        "status": "ONLINE",
        "definition": {
            "msisdn": "33602020202"
            "serverPhoneNumber": "20259",
            "encoding": "myDecoder"
        },
        "activity": {},
        "created": "2018-02-02T14:04:35.049Z",
        "updated": "2018-02-02T14:04:35.049Z"
    },
    {
        "connector": "sms",
        "nodeId": "33601201201",
        "enabled": true,
        "status": "ONLINE",
        "definition": {
            "msisdn": "33601201201"
        },
        "activity": {
            "lastUplink": {
                "timestamp": "2018-03-05T10:43:46.268Z",
                "serverPhoneNumber": "20259"
            }
        },
        "created": "2018-03-05T10:20:06.404Z",
        "updated": "2018-03-05T10:20:06.408Z"
    }
]

5.4.4. Get connector node details

5.4.4.1. Request

Endpoint:

GET /api/v1/deviceMgt/connectors/<connector>/nodes/<nodeId>

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

GET /api/v1/deviceMgt/connectors/sms/nodes/33601201201
5.4.4.2. Response

HTTP Code:

200 OK

Body:

Error case:

HTTP Code Error code message

403

GENERIC_OFFER_DISABLED_ERROR

The requested service is disabled in your offer settings. Please contact a sales representative.

404

DM_CONNECTOR_UNAVAILABLE

Connector not found or unavailable

404

DM_CONNECTOR_NODE_NOT_FOUND

Connector node not found

Example:

{
    "connector": "sms",
    "nodeId": "33601201201",
    "enabled": true,
    "status": "ONLINE",
    "definition": {
        "msisdn": "33601201201"
    },
    "activity": {
        "lastUplink": {
            "timestamp": "2018-03-05T10:43:46.268Z",
            "serverPhoneNumber": "20259"
        }
    },
    "created": "2018-03-05T10:20:06.404Z",
    "updated": "2018-03-05T10:20:06.408Z"
}

5.4.5. Update a connector node

5.4.5.1. Request

Endpoint:

PATCH /api/v1/deviceMgt/connectors/<connector>/nodes/<nodeId>

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Body:

JSON Params Description

deviceId

Optional. new device identifier

enabled

Optional. define if the interface is enabled or disabled

definition

Optional. new interface definition

Example:

PATCH /api/v1/deviceMgt/connectors/sms/nodes/33601201201
{
    "deviceId": "urn:lo:nsid:sensor:temp002",
    "enabled": "false",
    "definition": {
        "encoding": "myDecoder"
    }
}
5.4.5.2. Response

HTTP Code:

200 OK

Body:

Error case:

HTTP Code Error code message

404

DM_CONNECTOR_NODE_NOT_FOUND

Connector node not found

Example:

{
    "connector": "sms",
    "nodeId": "33601201201",
    "deviceId": "urn:lo:nsid:sensor:temp002",
    "enabled": false,
    "status": "ONLINE",
    "definition": {
        "msisdn": "33601201201"
        "encoding": "myDecoder"
    },
    "activity": {
        "lastUplink": {
            "timestamp": "2018-03-05T10:43:46.268Z",
            "serverPhoneNumber": "20259"
        }
    },
    "created": "2018-03-05T10:20:06.404Z",
    "updated": "2018-03-05T13:51:09.312Z"
}

5.4.6. Delete a connector node

5.4.6.1. Request

Endpoint:

DELETE /api/v1/deviceMgt/connectors/<connector>/nodes/<nodeId>

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

DELETE /api/v1/deviceMgt/connectors/sms/nodes/33601201201
5.4.6.2. Response

HTTP Code:

204 NO CONTENT

Error case:

HTTP Code Error code message

403

GENERIC_OFFER_DISABLED_ERROR

The requested service is disabled in your offer settings. Please contact a sales representative.

404

DM_CONNECTOR_UNAVAILABLE

Connector not found or unavailable

404

DM_CONNECTOR_NODE_NOT_FOUND

Connector node not found

5.5. Device connectors

5.5.1. LoRa® connector

For information about legacy API (v0), please see API LoRa® legacy section.

5.5.1.1. Definitions

devEUI

device EUI (cf. LoRa®).

appEUI

appEUI of the device (cf. LoRa®).

appKey

appKey of the device (cf. LoRa®).

activationType

OTAA: Over The Air Activation.

profile

profile of the Device which represents the Device Class (A or C). Can be specific for a Device (ex. LoRaMote devices) or generic (ex. LoRaWAN/DemonstratorClasseA or LoRaWAN/DemonstratorClasseC). Please refer to Orange LoRa® device reference. The API GET Profile allows to have the list of available profiles.

encoding

Optional. encoding type of the binary payload sent by the device, the decoder must be registered first (Cf. "Decoding service" section).

connectivityOptions

connectivity options used for the device. Supported options: ackUl and tdoa.

customConnectivityPlan

Optional. connectivity plan to use for the device. Allowed only if a specific option of Lora offer settings is enabled.

Example:

{
    "devEUI": "0101010210101010",
    "profile": "Generic_classA_RX2SF9",
    "activationType": "OTAA",
    "appEUI": "9879876546543211",
    "appKey": "11223344556677889988776655443322",
    "connectivityOptions" : {
        "ackUl" : true,
        "tdoa" : false
    }
}

Important Notice : Some features are only available if you have subscribed to the corresponding offer, so you may have the rights set on your tenant but no access to some features because these features are not activated on your tenant account (check the tenant offer).

If an option is applied to a device, it will be effective only if the option is allowed for the tenant.

5.5.1.2. Register a device with LoRa interface

For more explanation on device creation, please see Register a device section.

Request:

POST /api/v1/deviceMgt/devices
{
    "id": "urn:lo:nsid:lora:0101010210101010",
    "tags": ["Lyon", "Test"],
    "name": "myLoraSensor",
    "description": "device with LoRa interface",
    "properties" : {
        "manufacturer": "Orange",
        "model": "LoraSensor"
    },
    "interfaces": [
        {
            "connector": "lora",
            "enabled": true,
            "definition": {
                "devEUI": "0101010210101010",
                "profile": "Generic_classA_RX2SF9",
                "activationType": "OTAA",
                "appEUI": "9879876546543211",
                "appKey": "11223344556677889988776655443322",
                "connectivityOptions" : {
                    "ackUl" : true,
                    "tdoa" : false
                }
            }
        }
    ],
    "group": {
        "id": "sWyaL2",
        "path": "/france/lyon"
    }
}

Response:

200 OK
{
    "id": "urn:lo:nsid:lora:0101010210101010",
    "name": "myLoraSensor",
    "description": "device with LoRa interface",
    "tags": [
        "Test",
        "Lyon"
    ],
    "properties": {
        "manufacturer": "Orange",
        "model": "LoraSensor"
    },
    "group": {
        "id": "sWyaL2",
        "path": "/france/lyon"
    },
    "interfaces": [
        {
            "connector": "lora",
            "nodeId": "0101010210101010",
            "enabled": true,
            "status": "REGISTERED"
        }
    ],
    "defaultDataStreamId": "urn:lo:nsid:lora:0101010210101010",
    "created": "2018-03-06T13:23:37.712Z",
    "updated": "2018-03-06T13:23:37.945Z"
}
5.5.1.3. Add a LoRa interface to a registered device

For more explanation on interface addition, please see Add a interface section.

Request:

POST /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/interfaces
{
    "connector": "lora",
    "enabled": true,
    "definition": {
        "devEUI": "0202020220202020",
        "profile": "Generic_classA_RX2SF9",
        "activationType": "OTAA",
        "appEUI": "4573876546543211",
        "appKey": "11113344556677889988776655443322",
        "connectivityOptions" : {
            "ackUl" : true,
            "tdoa" : false
        }
    }
}

Response:

201 CREATED
{
    "connector": "lora",
    "nodeId": "0202020220202020",
    "deviceId": "urn:lo:nsid:sensor:temp001",
    "enabled": true,
    "status": "REGISTERED",
    "definition": {
        "devEUI": "0202020220202020",
        "activationType": "OTAA",
        "profile": "Generic_classA_RX2SF9",
        "appEUI": "4573876546543211"
    },
    "activity": {},
    "created": "2018-03-06T13:37:31.397Z",
    "updated": "2018-03-06T13:37:31.397Z"
}
5.5.1.4. Activity
JSON Params Description

lastActivationTs

Optional. last activation date of the connector node

lastDeactivationTs

Optional. last deactivation date of the connector node

lastSignalLevel

Optional. last signal level

lastBatteryLevel

Optional. last battery level

lastDlFcnt

Optional. last downlink frame counter

lastUlFcnt

Optional. last uplink frame counter

Example:

{
    "activity": {
        "lastActivationTs": "2018-03-19T13:02:13.482Z",
        "lastActivationTs": "2018-03-20T14:10:26.231Z",
        "lastSignalLevel": 5,
        "lastBatteryLevel": 54,
        "lastDlFcnt": 7,
        "lastUlFcnt": 1
    }
}
5.5.1.5. Status

LoRa® supports the following statuses: REGISTERED, INITIALIZING, INITIALIZED, ACTIVATED, DEACTIVATED, REACTIVATED, CONNECTIVITY_ERROR. The following diagram shows how LoRa® connector sets nodes' status. The consistency check referred in the diagram is an automatic periodic check that verifies the consistency of information between Live Objets and the LoRa® network provider, it can detect potential problems on an LoRa® node.

Status state machine in LoRa® connector

landing

5.5.1.6. Get LORA interfaces profiles

A device-profile defines the device capabilities and boot parameters that are needed by the network for access service. These information elements shall be provided by the manufacturer.

The profile of the device which represents the Device Class (A or C). Can be specific for a Device (ex. LoRaMote devices) or generic (ex. LoRaWAN/DemonstratorClasseA or LoRaWAN/DemonstratorClasseC). Please refer to Orange LoRa® device reference. The API GET profiles allows to have the list of available profiles.

5.5.1.6.1. Request

Endpoint:

GET /api/v1/deviceMgt/connectors/lora/profiles

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json
5.5.1.6.2. Response

HTTP Code:

200 OK

Body:

List of LoRa device profiles.

Error case:

HTTP Code Error code message

403

4030

Service is disabled. Please contact your sales entry point.

5.5.2. SMS connector

5.5.2.1. Definitions

msisdn

device msisdn

serverPhoneNumber

Optional. server phone number. Must be defined in the offer settings.

encoding

Optional. name of the decoder that will be used to decode received SMSs, the decoder must be registered first (Cf. "Decoding service" section).

Example of data message sended to an SMS device :

{
    "msisdn": "33601201201",
    "serverPhoneNumber": "21259",
    "encoding": "myDecoder"
}
5.5.2.2. Activity
JSON Params Description

lastUplink

Optional. last uplink date of the connector node

lastDownlink

Optional. last downlink date of the connector node

lastUplink and lastDownlink have the following format:

JSON Params Description

timestamp

date of the activity

serverPhoneNumber

server phone number used

Example:

{
    "activity": {
        "lastUplink": {
            "timestamp": "2018-03-21T16:17:35.172Z",
            "serverPhoneNumber": "20259"
        }
}
5.5.2.3. Status

The SMS connector supports the following statuses describes in node status. When an SMS connector node is created, its status is set to ONLINE and it wil not change.

5.5.2.4. Register a device with an SMS interface

For more explanation on device creation, please see Register a device section.

Request:

POST /api/v1/deviceMgt/devices
{
    "id": "urn:lo:nsid:sensor:temp002",
    "tags": ["Lyon", "Test"],
    "name": "mySensor002",
    "description": "moisture sensor",
    "properties" : {
        "manufacturer": "Orange",
        "model": "MoistureSensorV3"
    },
    "interfaces": [
        {
            "connector": "sms",
            "enabled": true,
            "definition": {
                "msisdn": "33600000001"
            }
        }
    ],
    "group": {
        "id": "sWyaL2",
        "path": "/france/lyon"
    }
}

Response:

200 OK
{
    "id": "urn:lo:nsid:sensor:temp002",
    "name": "mySensor002",
    "description": "moisture sensor",
    "tags": [
        "Test",
        "Lyon"
    ],
    "properties": {
        "manufacturer": "Orange",
        "model": "MoistureSensorV3"
    },
    "group": {
        "id": "sWyaL2",
        "path": "/france/lyon"
    },
    "interfaces": [
        {
            "connector": "sms",
            "nodeId": "33600000001",
            "enabled": true,
            "status": "ONLINE"
        }
    ],
    "defaultDataStreamId": "urn:lo:nsid:sensor:temp002",
    "created": "2018-03-06T11:30:42.777Z",
    "updated": "2018-03-06T11:30:42.819Z"
}
5.5.2.5. Add an SMS interface to a registered device

For more explanation on interface addition and example with SMS interface, please see Add a interface section.

5.5.2.6. Send an SMS for a list of MSISDNs

Endpoint:

POST /api/v0/sms-connector/sms

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Body:

JSON Params Description

serverPhoneNumber

server phone number. Must be defined in the offer settings.

msisdns

list of MSISDN. Each msisdn must be associated to a device (Cf. register a device with SMS interface or add an interface)

textPayload

Optional. message. If set, hexPayload and base64Payload should not be set.

hexPayload

Optional. message. If set, textPayload and base64Payload should not be set.

base64Payload

Optional. message. If set, hexPayload and textPayload should not be set.

Example:

POST /api/v0/sms-connector/sms
{
  "serverPhoneNumber": "20259",
  "msisdns": [
    "33601201201", "33602202202"
  ],
  "textPayload": "Hello"
}
5.5.2.7. Response

HTTP Code:

207 MULTI STATUS

Example:

{
  "statusPerMsisdn": {
    "33601201201": "OK",
    "33602202202": "msisdn  [33602202202] not found"
  }
}

5.5.3. MQTT connector

For information about MQTT interface and messages that your device can send or receive, please refer to the MQTT device mode.

5.5.3.1. Register a device with an MQTT interface

Contrary to SMS and LORA connector, you can’t create explicitly an MQTT interface to a device.

Live Objects will automatically register new MQTT interface in the inventory at the first MQTT connection.

If the clientId of this new MQTT connection matches with the deviceId of a registered device, a new MQTT interface will be provisioned to this device. Otherwise, a new device will be auto-provisioned with the related MQTT interface.

5.5.3.2. Status

MQTT connector supports the ONLINE and OFFLINE and REGISTERED statuses. The following diagram shows how the MQTT connector sets nodes' status.

Status state machine in MQTT connector

landing

6. MQTT interface for device

Live Objects supports the MQTT protocol to enable bi-directional (publish/subscribe) communications between devices or applications and the platform. Communications between devices or external applications and the Live Objects interfaces are translated into interactions with the Live Objects message bus.

MQTT can be used with or without encryption (TLS/SSL layer).

Live Objects also supports MQTT over WebSocket.

The Live Objects MQTT interface offers multiples "modes":

  • mode "Device": dedicated only to device connection use-cases, based on simple JSON messages,

  • mode "Connector": dedicated to application that will act as Cloud Gateway and transfer Status and Data from Devices toward Live Objects

6.1. Endpoints

6.1.1. Available MQTT endpoints

MQTT endpoints:

  • mqtt://liveobjects.orange-business.com:1883 for plain MQTT connection

  • mqtts://liveobjects.orange-business.com:8883 for secure TLS connection (supports both one-way and two-way authentications).

MQTT over Websocket endpoints:

  • ws://liveobjects.orange-business.com:80/mqtt for plain MQTT connection

  • wss://liveobjects.orange-business.com:443/mqtt for secure TLS connection (supports both one-way and two-way authentications).

It is recommended to use the MQTTS or WSS endpoint for your production environment, otherwise your communication with Live Objects will not be secured.

6.1.2. Server certificates

A connection to the MQTT server secure endpoints requires that the MQTT clients maintain an up-to-date list of trusted root Certificate Authorities (root CA) in order to seamlessly handle periodic server certificate updates. This trust store is typically provided with the runtime libraries (Java, .NET) or the operating system. Live Objects will renew the server certificates periodically (typically on a yearly basis). The root CA issuing the server certificates should stay the same between periodic updates but MQTT clients must be aware that it may change.

Constrained devices may not have enough storage space to maintain a complete root certificate list. Nevertheless, they must be able to store at least two root CA to properly deal with a root CA transition. Devices must also be able to update this root CA list during their lifetime.

The root CAs required to be present in the client trust store are listed below:

The public root certificate to import after Sep 23 23:59:59 2018 GMT (due to VeriSign acquisition by DigiCert)

-----BEGIN CERTIFICATE-----
MIIDrzCCApegAwIBAgIQCDvgVpBCRrGhdWrJWZHHSjANBgkqhkiG9w0BAQUFADBh
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBD
QTAeFw0wNjExMTAwMDAwMDBaFw0zMTExMTAwMDAwMDBaMGExCzAJBgNVBAYTAlVT
MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5j
b20xIDAeBgNVBAMTF0RpZ2lDZXJ0IEdsb2JhbCBSb290IENBMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4jvhEXLeqKTTo1eqUKKPC3eQyaKl7hLOllsB
CSDMAZOnTjC3U/dDxGkAV53ijSLdhwZAAIEJzs4bg7/fzTtxRuLWZscFs3YnFo97
nh6Vfe63SKMI2tavegw5BmV/Sl0fvBf4q77uKNd0f3p4mVmFaG5cIzJLv07A6Fpt
43C/dxC//AH2hdmoRBBYMql1GNXRor5H4idq9Joz+EkIYIvUX7Q6hL+hqkpMfT7P
T19sdl6gSzeRntwi5m3OFBqOasv+zbMUZBfHWymeMr/y7vrTC0LUq7dBMtoM1O/4
gdW7jVg/tRvoSSiicNoxBN33shbyTApOB6jtSj1etX+jkMOvJwIDAQABo2MwYTAO
BgNVHQ8BAf8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUA95QNVbR
TLtm8KPiGxvDl7I90VUwHwYDVR0jBBgwFoAUA95QNVbRTLtm8KPiGxvDl7I90VUw
DQYJKoZIhvcNAQEFBQADggEBAMucN6pIExIK+t1EnE9SsPTfrgT1eXkIoyQY/Esr
hMAtudXH/vTBH1jLuG2cenTnmCmrEbXjcKChzUyImZOMkXDiqw8cvpOp/2PV5Adg
06O/nVsJ8dWO41P0jmP6P6fbtGbfYmbW0W5BjfIttep3Sp+dWOIrWcBAI+0tKIJF
PnlUkiaY4IBIqDfv8NZ5YBberOgOzW6sRBc4L0na4UU+Krk2U886UAb3LujEV0ls
YSEY1QSteDwsOoBrp+uvFRTp2InBuThs4pFsiv9kuXclVzDAGySj4dzp30d8tbQk
CAUw7C29C79Fv1C5qfPrmAESrciIxpg0X40KPMbp1ZWVbd4=
-----END CERTIFICATE-----

The public root certificate to import before Sep 23 23:59:59 2018 GMT

-----BEGIN CERTIFICATE-----
MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADCB
yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJp
U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW
ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0
aG9yaXR5IC0gRzUwHhcNMDYxMTA4MDAwMDAwWhcNMzYwNzE2MjM1OTU5WjCByjEL
MAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZW
ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2ln
biwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJp
U2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9y
aXR5IC0gRzUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCvJAgIKXo1
nmAMqudLO07cfLw8RRy7K+D+KQL5VwijZIUVJ/XxrcgxiV0i6CqqpkKzj/i5Vbex
t0uz/o9+B1fs70PbZmIVYc9gDaTY3vjgw2IIPVQT60nKWVSFJuUrjxuf6/WhkcIz
SdhDY2pSS9KP6HBRTdGJaXvHcPaz3BJ023tdS1bTlr8Vd6Gw9KIl8q8ckmcY5fQG
BO+QueQA5N06tRn/Arr0PO7gi+s3i+z016zy9vA9r911kTMZHRxAy3QkGSGT2RT+
rCpSx4/VBEnkjWNHiDxpg8v+R70rfk/Fla4OndTRQ8Bnc+MUCH7lP59zuDMKz10/
NIeWiu5T6CUVAgMBAAGjgbIwga8wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8E
BAMCAQYwbQYIKwYBBQUHAQwEYTBfoV2gWzBZMFcwVRYJaW1hZ2UvZ2lmMCEwHzAH
BgUrDgMCGgQUj+XTGoasjY5rw8+AatRIGCx7GS4wJRYjaHR0cDovL2xvZ28udmVy
aXNpZ24uY29tL3ZzbG9nby5naWYwHQYDVR0OBBYEFH/TZafC3ey78DAJ80M5+gKv
MzEzMA0GCSqGSIb3DQEBBQUAA4IBAQCTJEowX2LP2BqYLz3q3JktvXf2pXkiOOzE
p6B4Eq1iDkVwZMXnl2YtmAl+X6/WzChl8gGqCBpH3vn5fJJaCGkgDdk+bW48DW7Y
5gaRQBi5+MHt39tBquCWIMnNZBU4gcmU7qKEKQsTb47bDN0lAtukixlE0kF6BWlK
WE9gyn6CagsCqiUXObXbf+eEZSqVir2G3l6BFoMtEMze/aiCKm0oHw0LxOXnGiYZ
4fQRbxC1lfznQgUy286dUV4otp6F01vvpX1FQHKOtw5rDgb7MzVIcbidJ4vEZV8N
hnacRHr2lVz2XTIIM6RUthg/aFzyQkqFOFSDX9HoLPKsEdao7WNq
-----END CERTIFICATE-----

6.2. MQTT support

MQTT protocol level supported is 3.1.1 (cf. MQTT Protocol Specification 3.1.1)

LiveObjects acts as a standard MQTT message broker with some limitations:

  • the "will" functionality is not implemented, all "willXXX" flags and headers are not taken into account,

  • the "retain" functionality is not implemented,

  • the "duplicate" flag is not used,

  • the "clean session" flag is not used, the server always starts a new session.

Although LiveObjects supports all control packets, QoS levels beyond the exchanges are not all fully guaranteed.

QoS definition guaranteed limitation

0

at most once delivery

yes

no

1

at least once delivery

yes

"duplicate" flag is not used,

2

exactly once delivery

no

same as QoS1


LiveObjects has several applicative behaviors according to connection mode (username) described in next paragraph.

6.2.1. Connecting

The first packet exchanged must be a MQTT CONNECT packet, sent from the client to the MQTT endpoint.

This packet must contain:

  • clientId: usage depends on the "mode",

  • username: used to select a mode and encoding:

  • password: a tenant API key that can be restrained to consume from / publish into only in specified FIFO queues

  • willRetain, willQoS, willFlag, willTopic, willMessage: !!! Not taken into account !!!,

  • keepAlive: any value including 0, will be correctly interpreted by LiveObjects (recommended: 30 seconds).

On reception, LiveObjects validates the API key provided.

  • If the API key is valid, then LiveObjects returns a MQTT CONNACK message with return code 0x00 Connection Accepted.

  • If the API key is not valid, then LiveObjects returns a MQTT CONNACK message with return code 0x04 Connection Refused: bad user name or password, and closes the TCP connection.

6.2.2. MQTT Ping Req/Res

LiveObjects answers to MQTT PINGREQ packets with MQTT PINGRES packets: this is a way for the MQTT client to avoid connection timeouts.

6.2.3. MQTT Subscribe

Once connected, the client can at any time subscribe and unsubscribe to/from topics.

LiveObjects answers with a MQTT SUBACK packet only once all subscriptions could be resolved:

MQTT specification enforce that a MQTT SUBACK is returned even if actual subscription is impossible / forbidden. In Protocol 3.1 the MQTT client cannot be informed that it subscribed to an non existing Topic. In Protocol 3.1.1 the MQTT client will receive MQTT SUBACK with corresponding QoS of 0x80 (Failure).

LiveObjects answers to MQTT UNSUBSCRIBE packet with a MQTT UNSUBACK packet only once existing subscriptions have been properly closed.

6.2.4. MQTT Disconnect

LiveObjects closes the MQTT / TCP connection when receiving a MQTT DISCONNECT message.

6.2.5. TCP Disconnect

When the TCP connection closes (by client or broker), the broker will close the currently active subscriptions, etc.

6.3. "Device" mode

In the "Device" mode, a single MQTT connection is associated with a specific device, and JSON messages can be exchanged to support various Device Management and Data features:

  • notifying of the device connectivity status,

  • notifying of the current device configuration and receiving configuration updates,

  • notifying of the list of current device "resources" (i.e. binary contents) versions, and receiving resource update requests,

  • receiving commands and responding to them,

  • sending data messages that will be stored.

Device management features

landing

6.3.1. Connection

When initiating the MQTT connection, to select the "Device" mode you must use the following credentials:

  • clientId : your device unique identifier (cf. Device Identifier (URN)). It must be between 1 and 128 characters long, containing no spaces or special characters except :, - or _ .

  • username : json+device, (where "json" indicates the encoding, "device" the mode),

  • password : a valid API key value.

As soon as the MQTT connection has been accepted by Live Objects, your device will appear as "ONLINE" in Live Objects, with various information regarding the MQTT connection.

Once you close the connection (or if the connection timeouts), your device will appear as "OFFLINE" in Live Objects.

6.3.2. Device Identifier (URN)

The "device id" used as MQTT client Id must be a valid Live Objects URN of the following format:

urn:lo:nsid:{namespace}:{id}

Where:

  • namespace:
    your device identifier "namespace", used to avoid conflicts between various families of identifier (ex: device model, identifier class "imei", msisdn", "mac", etc.).
    Should preferably only contain alphanumeric characters (a-z, A-Z, 0-9).

  • id:
    your device id (ex: IMEI, serial number, MAC address, etc.)
    Should only contain alphanumeric characters (a-z, A-Z, 0-9) and/or any special characters amongst : - _ | + and must avoid # / !.

Examples:

urn:lo:nsid:tempSensor:17872800001W
urn:lo:nsid:gtw_M50:7891001

6.3.3. Provisioning

By default, any device with a valid API Key will be able to connect to Live Objects using MQTT protocol.

You can explicitly define the MQTT capability for a device (see 'Device management - Interfaces' in Swagger documentation).

A device with an interface provisioned as 'connector' : 'mqtt' and 'enabled' : false will not be able to connect to Live Objects through MQTT protocol.

6.3.4. Summary

Authorized MQTT actions from the device:

publish to

dev/cfg

to announce the current configuration or respond to a config update request

subscribe to

dev/cfg/upd

to receive configuration update requests

publish to

dev/data

to forward collected data

subscribe to

dev/cmd

to receive commands

publish to

dev/cmd/res

to return command responses

publish to

dev/rsc

to announce the current resource versions

subscribe to

dev/rsc/upd

to receive resource update requests

publish to

dev/rsc/upd/res

to respond to resource update requests

publish to

dev/rsc/upd/err

to announce resource update error

6.3.5. Current Config

To notify Live Objects of its current configuration, your device must publish a message to the MQTT topic dev/cfg with the following JSON structure:

{
   "cfg": {
      "<<param1Key>>": {
         "t": "<<param1Type>>",
         "v": <<param1Value>>
      },
      ...
   }
}

Message fields:

  • param{X}Key: the identifier for the device configuration parameters,

  • param{X}Type : indicates the config parameter type between

    • "i32": the value must be an integer from -2,147,483,647 to 2,147,483,647,

    • "u32": the value must a positive integer from 0 to 4,294,967,296,

    • "str": the value is a UTF-8 string,

    • "bin": the value is a base64 encoded binary content,

    • "f64": the value is float (64 bits) value,

  • param{X}Value : the config parameter value.

Example:

{
   "cfg": {
      "log_level": {
         "t": "str",
         "v": "DEBUG"
      },
      "secret_key": {
         "t": "bin",
         "v": "Nzg3ODY4Ng=="
      },
      "conn_freq": {
         "t": "i32",
         "v": 80000
      }
   }
}

6.3.6. Config update

When your device is ready to receive configuration updates, it can subscribe to the MQTT topic dev/cfg/upd from where it will receive messages of the following format:

{
   "cfg": {
      "<<param1Key>>": {
         "t": "<<param1Type>>",
         "v": <<param1Value>>
      },
      ...
   },
   "cid": <<correlationId>>
}

Message fields:

  • param{X}Key : The identifier of a device configuration parameter that must be updated,

  • param{X}Type, param{X}Value : the new type and value to apply to the parameter,

  • correlationId : an identifier that your device must set when publishing your new configuration, so that Live Objects updates the status of your configuration parameters.

Example:

{
   "cfg": {
      "logLevel": {
         "t": "bin",
         "v": "DEBUG"
      },
      "connPeriod": {
         "t": "i32",
         "v": 80000
      }
   },
   "cid": 907237823
}

6.3.7. Config update response

Once your device has processed a configuration update request, it must return a response to Live Objects by publishing on topic dev/cfg the current value for the parameters that were updated:

{
   "cfg": {
      "<<param1Key>>": {
         "t": "<<param1Type>>",
         "v": <<param1Value>>,
      },
      ...
   },
   "cid": <<correlationId>>
}

Message fields:

  • config : the new configuration of your device (complete or at least all parameters that were in the configuration update request),

  • correlationId : the correlationId of the configuration update request.

Example:

{
   "cfg": {
      "logLevel": {
         "t": "bin",
         "v": "DEBUG"
      },
      "connPeriod": {
         "t": "i32",
         "v": 80000
      }
   },
   "cid": 907237823
}

If the new value for a parameter is the one that was requested in the configuration update request, the parameter will be considered as successfully updated by Live Objects.

If the new value for a parameter is not the one request, the parameter update will be considered as "failed" by Live Objects.

6.3.8. Data push

Data published by devices through MQTT will be processed by the Data enrichment logic .

To publish JSON data into Live Objects, your device must publish on the MQTT topic dev/data the following messages:

{
   "s":  "<<defaultDataStreamId>>",
   "ts": "<<timestamp>>",
   "m":  "<<model>>",
   "v": {
          ... <<value>> JSON object ...
   },
   "t" : [<<tag1>>,<<tag2>>,...]
   "loc": [<<latitude>>, <<longitude>>]
}

Message fields:

  • defaultDataStreamId : identifier of the timeseries this message belongs to,

  • timestamp : (optional) data/time associated with the message, is ISO 8601 format,

  • model : a string identifying the schema used for the "value" part of the message, to avoid conflict at data indexing,

  • value : a free JSON object describing the collected information,

  • tags : list of strings associated to the message to convey extra-information,

  • latitude, longitude : details of the geo location associated with the message (in degrees),

Example:

{
   "s":   "mydevice!temp",
   "ts":  "2016-01-01T12:15:02Z",
   "m":   "tempV1",
   "loc": [45.4535, 4.5032],
   "v": {
      "temp":     12.75,
      "humidity": 62.1,
      "gpsFix":   true,
      "gpsSats":   [12, 14, 21]
   },
   "t" : [ "City.NYC", "Model.Prototype" ]
}

6.3.9. Binary data push

To publish binary data into Live Objects, the device can publish on the following MQTT topics:

  • dev/v1/data/binary/{encoding} where {encoding} is the encoding name used in your decoder

  • dev/v1/data/binary to publish binary data without invoking a decoder

When pushing binary data, a dataMessage will be generated with the following values:

  • defaultDataStreamId: device urn.

  • timestamp: timeStamp of MQTT publication.

  • value.payload: binary content of MQTT publication as a string in hexBinary www.datypic.com/sc/xsd/t-xsd_hexBinary.html

  • model: if a model is set in the decoder then it will be added. if not, this field stays empty.

For decoder management: decoders.

Example:

with the following binary decoder :
{
    "encoding": "my_encoding_v0",
    "format": "float sent_value;",
    "enabled": true,
    "model":"my_encoding_model_v0"
}

Sending a MQTT message with following bytes 0x41200000 as payload on topic dev/v1/data/binary/my_encoding_v0, it will generate and store the following message:

{
    "timestamp" : "2019-05-31T16:15:21.288Z,"
    "streamId" : "urn:lo:nsid:mqtt:{deviceId},"
    "value": {
        "payload": "41200000",
        "sent_value": 10.0
    },
    "model": "my_encoding_model_v0"
}

More info on decoding and model.

6.3.10. Commands

When your device is ready to receive commands, it can subscribe to the MQTT topic dev/cmd from where it can receive the following messages:

{
   "req":  "<<request>>",
   "arg": {
      "<<arg1>>": <<arg1Value>>,
      "<<arg2>>": <<arg2Value>>,
      ...
   },
   "cid":  <<correlationId>>
}

Message fields:

  • request : string identifying the method called on the device,

  • arg{X}, arg{X}Value : name and value (any valid JSON value) of an argument passed to the request call,

  • correlationId : an identifier that must be returned in the command response to help Live Objects match the response and request.

Example:

{
   "req":  "buzz",
   "arg": {
      "durationSec": 100,
      "freqHz":     800.0
   },
   "cid": 12238987
}

6.3.11. Commands response

To respond to a command, your device must publish the response to the MQTT topic dev/cmd/res with a message of the following format:

{
   "res": {
      "<<res1>>": "<<res1Value>>",
      "<<res2>>": "<<res2Value>>",
      ...
   },
   "cid":  <<correlationId>>
}

Message fields:

  • res{X}, res{X}Value : optional information returned by the command execution,

  • correlationId : a copy of the command correlationId value.

Example #1:

{
   "res": {
      "done": true
   },
   "cid": 12238987
}

Example #2:

{
   "res": {
      "error": "unknown method 'buzz'"
   },
   "cid": 12238987
}

Any response of the device will set the command state to PROCESSED and the optional information will be available in the command result.

6.3.12. Current Resources

Once connected, your device can announce the currently deployed versions of its resources by publishing a message on MQTT topic dev/rsc with the following format:

{
   "rsc": {
      "<<resource1Id>>": {
         "v": "<<resource1Version>>",
         "m": <<resource1Metadata>>
      },
      "<<resource2Id>>": {
         "v": "<<resource2Version>>",
         "m": <<resource2Metadata>>
      },
      ...
   }
}

Message fields:

  • resource{X}Id : resource identifier,

  • resource{X}Version : currently deployed version of this resource,

  • resource{X}Metadata : (JSON object) (optional) metadata associated with this resource, useful to resource update.

Example:

{
   "rsc": {
      "X11_firmware": {
         "v": "1.2",
         "m": {
            "username": "78723-672-1232"
         }
      },
      "X11_modem_driver": {
         "v": "4.0.M2"
      }
   }
}

6.3.13. Resources update

When your device is ready to receive resource update request, it just needs to subscribe to MQTT topic dev/rsc/upd. From then on it will receive such request as message with the following JSON format:

{
   "id": "<<resourceId>>",
   "old": "<<resourceCurrentVersion>>",
   "new": "<<resourceNewVersion>>",
   "m": {
      // ... <<metadata>> JSON object ...,
   },
   "cid": "<<correlationId>>"
}

Message fields:

  • resourceId : identifier of resource to update,

  • resourceCurrentVersion : current resource version,

  • resourceNewVersion : new resource version, to download an apply,

  • correlationId : an identifier that must be returned in the resource update response to help Live Objects match the response and request.

Example:

{
   "id": "X11_firmware",
   "old": "1.1",
   "new": "1.2",
   "m": {
      "uri": "http://.../firmware/1.2.bin",
      "md5": "098f6bcd4621d373cade4e832627b4f6"
   },
   "cid": 3378454
}

6.3.14. Resources update response

Once your device receives a "Resource update request", it needs to respond to indicate if it accepts or not the new resource version, by publishing a message on topic dev/rsc/upd/res with the following JSON format:

{
   "res": "<<responseStatus>>",
   "cid": "<<correlationId>>"
}

Message fields:

  • responseStatus : indicates the response status to the resource update request:

    • "OK" : the update is accepted and will start,

    • "UNKNOWN_RESOURCE" : the update is refused, because the resource (identifier) is unsupported by the device,

    • "WRONG_SOURCE_VERSION" : the device is no longer in the "current" (old) resource version specified in the resource update request,

    • "WRONG_TARGET_VERSION" : the device doesn’t support the "target" (new) resource version specified in the resource update request,

    • "INVALID_RESOURCE" : the requested new resource version has incorrect version format or metadata,

    • "NOT_AUTHORIZED" : the device refuses to update the targeted resource (ex: bad timing, "read-only" resource, etc.),

    • "INTERNAL_ERROR" : an error occurred on the device, preventing for the requested resource update,

  • correlationId : copy of the correlationId field from the resource update request.

Example #1:

{
   "res": "OK",
   "cid": 3378454
}

Example #2:

{
   "res": "UNKNOWN_RESOURCE",
   "cid": 778794
}

6.3.15. Resources update response error

Device can report a custom resource update error by publishing a message on MQTT topic dev/rsc/upd/err with the following format:

{
   "errorCode":"ERROR_CODE",
   "errorDetails":"error details"
}

Message fields:

  • errorCode : (optional) device error code,

  • errorDetails : device error details

This fields are limited to 256 characters. Characters outside of this limit will be ignored.

Example:
{
   "errorCode":"DEV123.Z_FIRMW_CRC",
   "errorDetails":"error while loading firmware, bad CRC"
}

6.4. "Connector" mode

The connector mode purpose is to publish data and devices devices' status

Typically, use this mode when you want to connect your back-end service (where your devices are connected to) to Live Objects.

When connected through this connector mode, this is the x-connector interface of the device which is used. By default, the auto-provisioned deviceId for nodeId will be:

urn:lo:nsid:x-connector:{nodeId}

"Connector" mode usage

landing

6.4.1. Connection

When initiating the MQTT connection, to select the "connector" mode you must use the following credentials:

  • clientId : any string between 1 and 128 characters long, containing no spaces or special characters except :, - or _ . The MQTT protocol indicates it must be unique per connection, but LiveObjects does not enforce it.

  • username : connector

  • password : a valid API Key with CONNECTOR_ACCESS role

Also, to securize your connection, MQTTS should be used.

6.4.2. Summary

In "connector" mode, you can publish :

  • NodeStatus : publish in Live Objects a NodeStatus to set the ONLINE/OFFLINE status of the device (i.e. the node)

  • DataMessage : publish in Live Objects a DataMessage sent by the device

Direction Topic Description

Publish

connector/v1/nodes/{nodeId}/status

To publish a NodeStatus for a device

Publish

connector/v1/nodes/{nodeId}/data

To publish a DataMessage in Live Objects received from a device

6.4.3. NodeStatus publication

A NodeStatus publication allows to set the ONLINE/OFFLINE state of the device.

The topic to publish to is : connector/v1/nodes/{nodeId}/status

If the nodeId in the publication topic is unknown by Live Objects, a device with this NodeStatus information will be auto-provisioned in the Device inventory.

If the nodeId is already declared for a device in the Device inventory, then this NodeSatus information will update the corresponding device state in the Device inventory.

The link between a DeviceId and a NodeId is described in the Device and connectivity chapter.

The payload of this publication should follow :

NodeStatus publication
{
    "status": "ONLINE",
    "lastContact": "2019-05-20T16:01:47Z",
    "sessionSequenceId": 1,
    "eventSequenceId": 3
}

Only the 'status' field is mandatory.

Field Description

status

(Mandatory). Either 'ONLINE' or 'OFFLINE'.

lastContact

(Optional). Should follow ISO-8601 format. Set the lastContact information that will be updated in DataManager.

sessionSequenceId

(Optional). If present, the event will be taken into account only if no previous event had a higher sessionSequenceId.

eventSequenceId

(Optional). If present, the event will be taken into account only if no previous event had a higher sessionSequenceId or eventSequenceId.

As messages are asynchronously handled by Live Objects, sessionSequenceId and eventSequenceId mechanism allows to enforce message ordering (only the last status message will be taken into account). If this ordering is mandatory for your application, you should set them. For a unique MQTT session, sessionSequenceId should remain identical; and you can use epoch milliseconds from your application server as eventSequenceId.

If the publication succeeds, Live Objects will acknowledge the message according to the QoS level.

If the publication fails (for ex. because the JSON is badly formatted), Live Objects will nevertheless acknowledge the message according to the QoS level, and an AuditLog message will be sent with anomaly details.

6.4.4. DataMessage publication on behalf of a device

A DataMessage publication in the "connector" mode allows to send a DataMessage on behalf of a specific device.

The topic to publish to is : connector/v1/nodes/{nodeId}/data

The payload of this publication should follow :

DataMessage publication
{
    "streamId": "urn:lo:nsid:detector_A8:12435355",
    "timestamp": "2019-05-20T16:01:47Z",
    "model": "data_v0",
    "value": {
        "temperature" : 14.6,
                "battery" : 53,
                "messageAlert":"low battery"
    },
    "location": {
        "lat": 48.86667,
        "lon": 2.33333,
        "alt": 35.2,
        "accuracy": 12.3,
        "provider": "GPS"
    },
    "tags": [ "production", "london" ]
}
Field Description

streamId

(Optional). The streamId where the data will be stored in; and retrieved using the HTTP store API. If not set, the default streamId set for this device in the Device inventory will be used. Should not contain any of the following character : ' " \ ; { } ( )

timestamp

(Optional). Should follow ISO-8601 format. If not set, current timestamp will be used.

model

(Optional). Can not contains ' ' (space) or '.' (dot) character. Model is needed to be able to use the search APIs on fields inside the value object.

value

(Optional). Json compliant object. No inner field name should contains '.' (dot) character.

location

(Optional). If set, geo-query will be available through search APIs.

tags

(Optional) List of additional information used to tag the DataMessage

If the publication succeeds, Live Objects will acknowledge the message according to the QoS level.

If the publication fails (for ex. because the JSON is badly formatted), Live Objects will nevertheless acknowledge the message according to the QoS level, and an AuditLog message will be sent with anomaly details.

6.5. Secure your MQTT connection to Live Objects

6.5.1. Why use Secure MQTT protocol?

Using MQTT through TLS encryption enables secured and encrypted communication between a device (or an external connector) and Live Objects. It also guarantees that the device is communicating with the actual Live Objects platform and not a potential man in the middle that could spy on or modify the data stream of a client. Using secure TLS communication is becoming a de facto standard when communicating over internet as more and more software / operating systems / Hardware refuse to communicate over non-encrypted protocols.

6.5.2. Go further and use 2-way authentication

Live Objects goes beyond server authentication and implements client authentication based on your own Certification Authority (CA). When enabled for a given Api Key, Live Objects will:

  • make sure that only TLS with 2-way authentication is made with this API key when used for MQTT connections

  • make sure that the client certificate has been signed by one of the CA certificates associated to the API key used by the device,

  • verify the identity of the device, by checking that client certificate’s Common Name (CN) matches the device id (in the MQTT client id)

Live Objects does not provide PKI services, but you can configure your Live Objects account to use your own PKI’s certificates. For the time being, a maximum limit of 10 certificates is set for a tenant.

6.5.3. How to setup 2-way authentication

The 2-way authentication setup is done in 2 steps:

  • Make Live Objects aware of your Certification Authority (CA) by configuring the CA certificate which signed the devices certificates.

  • Associate that CA certificate with the API key used by your devices/external connector. This will force all communications to Live Objects using MQTT using that API key to be secured with a client certificate (or it will be rejected).

  • Generate a client certificate, signed with your CA certificate, with the MQTT Client ID as common name (CN).

2-way authentication is only available for MQTTS protocol. MQTT over secured websocket connection does not support 2-way authentication, but supports one-way authentication.
1. Configure your Certification Authority’s intermediate certificate on Live Objects

Generate a key pair.

openssl genrsa -out rootCA.key 2048

Use the key pair to generate your CA certificate.

openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.pem

Configure your certificate on Live Objects:

POST /api/v0/certificates/ca
{
  "certificate": "your pem formatted certificate",
  "comment": "my awesome intermediate CA certificate"
}

This call will return an id for your certificate.

Only provision the CA’s intermediate certificate that directly signed the device Certificate Signing Request (CSR).
{
  "id": "your_certificate_id"
}
To get a json compliant string of your pem certificate you can use the command line tool jq : cat rootCA.pem | jq -R --slurp
2. Associate this certificate with an Api Key
POST /api/v0/apiKeys/{your-api-key-id}
{
  "clientCert": {
    "caCertIds": [
      "your-certificate-id1",
      "your-certificate-id2"
    ],
    "required": true
  }
}
Once a certificate is associated with an Api Key and clientCert.required=true 2-way authenticated MQTTs connection will be mandatory, otherwise the MQTT(s)/websocket connection will be closed with a Bad username or password error.
3. Generate a client certificate for a device with id your-device-id

Create a key pair for the device.

openssl genrsa -out deviceCert.key 2048

Create a certificate signing request from your key pair

openssl req -new -key deviceCert.key -out deviceCert.csr

Enter the information when prompted

Country Name (2 letter code) []:
State or Province Name (full name) []:
Locality Name (for example, city) []:
Organization Name (for example, company) []:
Organizational Unit Name (for example, section) []:
Common Name (e.g. server FQDN or YOUR name) []: <--------- ENTER YOUR DEVICE URN HERE
Email Address []:

Generate the device certificate by signing the CSR with your Certification Authority intermediate key.

openssl x509 -req -in deviceCert.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out deviceCert.pem -days 365 -sha256
4. Try it

Try your newly configured MQTTs 2-way configuration by sending a new data on the dev/data topic. We will be using the node mqtt.js client:

mqtt publish -h "liveobjects.orange-business.com" --port 8883 -i your-device-id -q 0 -u json+device -C mqtts -t dev/data -m '{"s": "myStreamId", "v": {"temp": 12}}' --key your-cert-private-key --cert your-cert.pem --ca liveObjects.ca.cert.pem -P your-api-key

7. CoAP & LWM2M interface (Beta)

7.1. CoAP protocol

Constrained Application Protocol (CoAP) is an Internet Application Protocol for constrained devices (defined in RFC 7228) for M2M/Internet of Things. CoAP is a service layer protocol that is intended for use in resource-constrained internet devices, such as wireless low-power device. CoAP use HTTP for simplified integration on Rest architecture. CoAP can run on most devices that support UDP.

CoAP design is similar to HTTP principles but more effective on bandwidth and using UDP transport.

7.2. LWM2M protocol

OMA Lightweight M2M (LWM2M) is a protocol from the Open Mobile Alliance for M2M or IoT device management.

7.3. Principles

Live Objects exposes CoAP/LWM2M interface with these features:

  • Basic CoAP connection using DTLS (udp) with PSK KEY identity

  • Data collect to upload a LO JSON data message that will be stored

CoAP protocol is also used in LWM2M application protocol which is described in another section, see LWM2M protocol interface.

In order to provision a CoAP identity to connect your device, you will have to use the dedicated REST API as described in a following section.

The access to this interface is an option, as for now you need to contact our support team to get it enabled in your tenant.

7.4. Endpoints

For the LWM2M protocol, only secured CoAP binding is supported using DTLS Pre-Shared Key (PSK).

    coaps://84.39.42.29:5684

7.5. Content

Currently only LiveObjects JSON data message format is supported to be uploaded using the CoAP\LWM2M Interface.

The data decoding is not yet available on this interface.

7.6. CoAP\LWM2M device Provisionning

7.6.1. Using Live objects API

The DTLS authentication for CoAP\LWM2M adds new concepts not compatible with API key of Live Objects. So you have to provision this new type of credentials (see the dedicated swagger):

    POST /api/v0/vendors/lwm2m/identities
    {
        "device": "URN",
        "ep": "the device's endpoint",
        "pubKeyOrIdentity": "pubKeyOrIdentity",
        "secretKey": "\"something base 64\"",
        "securityMode": "PSK",
        "tags": [
        "tag1",
        "tags2"
        ],
    }

With the given details:

  • device : your device unique identifier (cf. Device Identifier (URN)),

  • ep : the device’s endpoint, the same value as the device URN,

  • pubKeyOrIdentity : the identity used for PSK identity,

  • secretKey : the PSK passwork associated to the identity, encoded in base64,

  • securityMode : must be set to PSK string value.

You can do these steps using the portal GUI, located in menu Devices > CoAP \ LWM2M

7.6.2. Using Live Objects portal

LWM2MProv.png

With the following sample values:

  • Endpoint, Identity and Park with value "urn:lwm2m:device4"

  • Security Mode with value "Pre-Shared Key"

  • Secret Key with a base64 value of the password

7.7. CoAP device simulation

7.7.1. Posting CoAP\LWM2M message

Once the identity provisioned, the CoAP device can send a valid Live Objects Live Objects data message.

The following sample will provide a simulator in Java:

    git clone https://github.com/eclipse/californium.tools
  • and run the following commands (Java JDK and Maven are required)

    cd californium.tools/cf-client
    mvn clean install
    cd ..\run

To test your device, execute the following steps using the credentials previously created:

  • Choose your own password for PSK key (for ex: “password”) then encode it in B64. (See here www.base64decode.org )

  • Create your CoAP device (if not yet) with Live Objects portal (LWM2M menu) and enter the previous key encoded in B64. Do not use the generate button for that

  • Run the CoAP DTLS client Californium with the following command line

java -jar cf-client-1.1.0-SNAPSHOT.jar -psk POST coaps://84.39.42.29:5684/data "{\"streamId\": \"urn:lwm2m:device4!uplink\", \"model\": \"testmodel\", \"value\":{\"testproperty\": \"testvalue\" }}"
  • beware: use the IP indicated in this command line instead of FQDN as in the doc

  • Then you will be prompted to give the PSK Identity (previously provisionned as pubKeyOrIdentity) and the Secrey Key (previously provisionned as secretKey) in ascii string, not encoded in B64 → “password” in our example

  • You should see the data published in the Data menu of your portal

In a more readable manner, the json Live Objects data message in this sample is:

    {
    "streamId": "urn:lwm2m:device4!uplink",
    "model": "testmodel",
    "value":
        {
            "testproperty": "testvalue"
        }
    }

7.7.2. Californium CoAP result view

COAPResult.png

7.8. Using LWM2M

7.8.1. LWM2M device parameters view

LWM2MDeviceInfo.png

Json document model :

{
  "page": 0,
  "size": 20,
  "totalCount": 2,
  "data": [
    {
      "ep": "urn:lo:nsid:fred:device1:2caf4e244430430e",
      "securityMode": "PSK",
      "pubKeyOrIdentity": "a1ebc03a72cf0671",
      "device": "urn:lo:nsid:fred:device1",
      "creationTs": 1544019251440,
      "updateTs": 1544019251440
    },
    {
      "ep": "urn:lo:nsid:tareklw:device2",
      "securityMode": "PSK",
      "pubKeyOrIdentity": "32202c4215c03f76",
      "device": "urn:lo:nsid:my_lwm2m:device2",
      "creationTs": 1544020350136,
      "updateTs": 1544020350136
    }
  ]
}

LWM2M Notify messages are converted to Live Objects JSON data message. The screenshot below shows a data message from an observe action on an LWM2M resource named "/3" (which is the standardized Device definition):

7.8.2. LWM2M data message model

LWM2MIm6.png

Or as text:

    {
        "metadata": {
        "source": "urn:lwm2m:device4",
        "connector": "coap",
        "network": {
        "lwm2m": {
        "ep": "urn:lwm2m:device4"
            }
        }
    },
    "streamId": "urn:lwm2m:device4_3_0!uplink",
    "created": "2017-08-24T13:29:54.087Z",
    "location": null,
    "model": "urn:oma:lwm2m:oma:3",
    "id": "599ed4d2a1a7e86004e2a4b8",
    "value": {
        "Device": {
        "0": {
        "17": "RGVtbw==",
        "18": "MS4wLjE=",
        "19": "MS4wLjI=",
        "20": "Ag==",
        "21": "AAHsAA==",
        "Battery Level": 50,
        "Timezone": "Europe/Paris",
        "Firmware Version": "1.0.0",
        "Memory Free": 93674,
        "Current Time": "2017-08-24T13:29:51.000+0000",
        "UTC Offset": "+02",
        "Model Number": "Model 500",
        "Serial Number": "LT-500-000-0001",
        "Error Code": {
            "0": 0
            },
            "Manufacturer": "Leshan Demo Device",
            "Supported Binding and Modes": "U"
            }
        }
    },
        "timestamp": "2017-08-24T13:29:53.766Z",
        "tags": []
    }

Where:

  • the streamId and the model are generated from the observed resource

8. Device management

8.1. Device group

8.1.1. Principle

A group enables to bring devices together. Groups can be used to :

  • ease browsing and sorting on Live Objects web portal

  • apply different forwarding rules on the data collected from the devices (Cf. "Notifications" section)

  • apply different event processing and alarming rules (Cf. "Alarming" chapter)

Groups behave mostly like directories in a filesystem (considering devices are files): a Group can include other groups and devices. For instance a Device can only belong to one group at a time. There are however some exceptions to this analogy (see group deletion rules).

Each tenant has a default root group. A group, except root group, must have a parent group and can have subgroups. In the same tenant, the complete path of each group must be unique but several groups can have the same local identifier (pathNode). The complete path enables to have the tree view of your group.

Group POJO:

JSON Params Description

id

group unique identifier

pathNode

Optional. local group id in path

path

complete group path

parentId

Optional. parent group identifier

description

Optional. detailed description of the group

created

creation date of the device

updated

last update date of the device

Example:

{
    "id": "u1z1k8",
    "pathNode": "lyon",
    "path": "/france/lyon",
    "parentId": "P2112f",
    "description": "City of Lyon",
    "created": "2018-02-14T16:53:54.515Z",
    "updated": "2018-02-14T16:53:54.515Z"
}
Groups tree view

landing

8.1.2. Create a new group

8.1.2.1. Request

Endpoint:

POST /api/v1/deviceMgt/groups

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Body:

JSON Params Description

pathNode

group local identifier in path (unique for groups with the same parent)

parentId

Optional. reference to group parent (id). Root group by default.

description

Optional. detailed description of the group

Example:

POST /api/v1/deviceMgt/groups
{
    "pathNode":"lyon",
    "parentId":"P2112f",
    "description":"City of Lyon"
}
8.1.2.2. Response

HTTP Code:

201 CREATED

Body:

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

401

UNAUTHORIZED

Authentication failure.

403

GENERIC_ACTION_FORBIDDEN_ERROR

Request forbidden.

404

DM_GROUP_NOT_FOUND

Group not found

409

DM_GROUP_DUPLICATE

Conflict on group path

Example:

{
    "id": "u1z1k8",
    "pathNode": "lyon",
    "path": "/france/lyon",
    "parentId": "P2112f",
    "description": "City of Lyon",
    "created": "2018-02-14T16:53:54.515Z",
    "updated": "2018-02-14T16:53:54.515Z"
}

8.1.3. Update a group

The path change of a group does not change its content (Devices, Groups).

8.1.3.1. Request

Endpoint:

PUT /api/v1/deviceMgt/groups/<id>

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Body:

JSON Params Description

pathNode

Optional. group local id in path (unique for groups with the same parent)

parentId

Optional. reference to group parent (id). Root group by default

description

Optional. detailed description of the group

Example:

PUT /api/v1/deviceMgt/groups/u1z1k8
{
    "pathNode":"paris",
    "parentId":"P2112f",
    "description":"City of Paris"
}
8.1.3.2. Response

HTTP Code:

200 OK

Body:

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

401

UNAUTHORIZED

Authentication failure.

403

GENERIC_ACTION_FORBIDDEN_ERROR

Request forbidden.

404

DM_GROUP_NOT_FOUND

Group not found

Example:

{
    "id": "u1z1k8",
    "pathNode": "paris",
    "path": "/france/paris",
    "parentId": "P2112f",
    "description": "City of Paris",
    "created": "2018-02-14T16:53:54.515Z",
    "updated": "2018-02-15T10:31:26.667Z"
}

8.1.4. List groups

8.1.4.1. Request

Endpoint:

GET /api/v1/deviceMgt/groups

Query parameters:

Name Description

limit

Optional. maximum number of devices in response. 20 by default.

offset

Optional. the number of entries to skip in the results list. 0 by default.

parentId

Optional. filter list by group’s parent

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json
X-Total-Count: <boolean>

For more info about X-Total-Count, see "API v1 paging" section.

Example:

GET /api/v1/deviceMgt/groups?limit=20&offset=0
8.1.4.2. Response

HTTP Code:

200 OK

Body:

List of group POJO

Example:

[
    {
        "id": "root",
        "path": "/",
        "created": "2017-11-03T15:23:22.771Z",
        "updated": "2017-11-03T15:23:22.771Z"
    },
    {
        "id": "P2112f",
        "pathNode": "france",
        "path": "/france",
        "parentId": "root",
        "description": "",
        "created": "2018-02-14T16:47:37.318Z",
        "updated": "2018-02-14T16:47:37.318Z"
    },
    {
        "id": "u1z1k8",
        "pathNode": "paris",
        "path": "/france/paris",
        "parentId": "P2112f",
        "description": "City of Paris",
        "created": "2018-02-14T16:53:54.515Z",
        "updated": "2018-02-15T10:31:26.667Z"
    }
]

8.1.5. Get a group

8.1.5.1. Request

Endpoint:

GET /api/v1/deviceMgt/groups/<id>

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

GET /api/v1/deviceMgt/groups/u1z1k8
8.1.5.2. Response

HTTP Code:

200 OK

Body:

Error case:

HTTP Code Error code message

404

DM_GROUP_NOT_FOUND

Group not found

Example:

{
    "id": "u1z1k8",
    "pathNode": "paris",
    "path": "/france/paris",
    "parentId": "P2112f",
    "description": "City of Paris",
    "created": "2018-02-14T16:53:54.515Z",
    "updated": "2018-02-15T10:31:26.667Z"
}

8.1.6. Delete a group

You can’t delete a group which has devices.

8.1.6.1. Request

Endpoint:

DELETE /api/v1/deviceMgt/groups/<id>

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

DELETE /api/v1/deviceMgt/groups/u1z1k8
8.1.6.2. Response

HTTP Code:

204 NO CONTENT

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

401

UNAUTHORIZED

Authentication failure.

403

DM_GROUP_UNDELETABLE

Group can’t be deleted: contains devices or sub-groups

404

DM_GROUP_NOT_FOUND

Group not found

8.2. Configuration

Currently, APIs allow to manage configuration only for MQTT connector.

8.2.1. Principle

A device can declare one or many parameters: a parameter is identified by a string "key" and can take a typed value (binary, int32, uint32, timestamp).

Live Objects can track the changes of the current value of a device parameters, and allow users to set different target values for those parameters. Live Objects will then try to update the parameters on the device once it’s connected and available.

Device configuration sync steps

landing

  • (before) :

    • device initiates MQTT connection with Live Objects,

    • device subscribes in MQTT to a private topic, where it will receive later the configuration update requests,

  • step 0 : device notifies Live Objects that it is connected and available for configuration updates on a specific topic,

  • step 1 : device notifies Live Objects of its current configuration,

  • step2 : Live Objects compares the current and target configuration for this device. If they differ:

    • step 3 : Live Objects sends to the device, on the topic indicated at step 0, the list of parameters to update, with their target value,

    • step 4 : device handles the request, and tries to apply the change(s),

    • step 5 : device respond to the change request with the new configuration,

    • step 6 : Live Objects saves the new configuration. Parameters that have been successfully updated now have the status "OK" and the others the status "FAILED".

Device configuration sync states

landing

If your device is connected in Device mode, please refer to the MQTT device mode part for messages that your device can send or receive.

8.2.2. Get device configuration parameters

To retrieve the details of the configuration map of the device we can use the GET API.

8.2.2.1. Request

Endpoint:

GET /api/v1/deviceMgt/devices/<deviceId>/config/parameters

Response

{   "parameters": {
       "MyParamINT32" : {
          "type" : "INT32",
       "value" : -333
       },
       "MyParamUINT32" : {
          "type" : "UINT32",
          "value" : 1245
       },
       "MyParamFLOAT" : {
          "type" : "FLOAT",
          "value" : 1245.0
       },
       "MyParamSTRING" : {
          "type" : "STRING",
          "value" : "My sentense"
       },
       "MyParamBINARY" : {
         "type" : "BINARY",
         "value" : "Nzg3ODY4Ng=="
       }
   },
   }

8.2.3. Set device configuration parameters

8.2.3.1. Request

Endpoint:

POST /api/v1/deviceMgt/devices/<deviceId>/config

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Body:

JSON Params Description

parameters

list of device configuration update

notifyTo

Optional. topic where configuration change/sync events must be published to

Each parameter to update must have the following structure:

"{paramKey}": {
    "type": {paramType},
    "value": {paramValue}
}

With:

paramKey

a string uniquely identifying the device configuration parameter

paramType

indicates the config parameter type between

"INT32"

the value must be an integer from -2,147,483,648 to 2,147,483,647,

"UINT32"

the value must a positive integer from 0 to 4,294,967,295,

"BINARY"

the value is a base64 encoded binary content,

"STRING"

the value is a UTF-8 string,

"FLOAT"

the value is float (64 bits) value.

Example:

POST /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/config
{
    "parameters": {
        "MyParamINT32" : {
            "type" : "INT32",
            "value" : -333
        },
        "MyParamSTRING" : {
            "type" : "STRING",
            "value" : "My sentense"
        }
    },
    "notifyTo": "fifo/configUpdateFifo"
}
8.2.3.2. Response

HTTP Code:

200 OK

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

If the device does not exist, it will be auto-provisionned.

8.2.4. Get a description of the device configuration

8.2.4.1. Request

Endpoint:

POST /api/v1/deviceMgt/devices/<deviceId>/config

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Example:

GET /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/config
8.2.4.2. Response

HTTP Code:

200 OK

Body:

JSON Params Description

parameters

list of device configuration update

notifyTo

Optional. topic where configuration change/sync events must be published to

For each parameter, the following information is available :

"{paramKey}": {
    "reported": {
        "type": {paramType},
        "value": {paramValue},
        "timestamp": {timestampsValue}
    },
    "requested": {
        "type": {paramType},
        "value": {paramValue},
        "timestamp": {timestampsValue}
    },
    "syncStatus": {paramStatus},
    "updateReqCorrelationId": {cid}
}

With:

paramKey

string uniquely identifying the device configuration parameter

paramType

indicates the config parameter type between: INT32, UINT32, BINARY, STRING or FLOAT,

paramValue

value requested for the parameter,

timestampsValue

update date of the parameter,

paramStatus

parameter update status; NONE, PENDING, SENT, OK or FAILED.

cid

correlation id

The section updateRepCorrelationId is present only when the paramStatus is SENT. The section reported is present when the device has answered and corresponds to parameter configuration return by the device, the paramStatus could be OK or FAILED.

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

404

DM_DEVICE_NOT_FOUND

Device not found

Example:

{
    "parameters": {
        "MyParamFLOAT": {
            "requested": {
                "type": "FLOAT",
                "value": 1245,
                "timestamp": "2018-03-07T10:55:21Z"
            },
            "syncStatus": "PENDING"
        },
        "MyParamBINARY": {
            "requested": {
                "type": "BINARY",
                "value": "Nzg3ODY4Ng==",
                "timestamp": "2018-03-07T10:54:55.948Z"
            },
            "syncStatus": "SENT",
            "updateReqCorrelationId": -1662696938
        },
        "MyParamINT32": {
            "reported": {
                "type": "INT32",
                "value": -333,
                "timestamp": "2018-03-07T10:53:21.934Z"
            },
            "requested": {
                "type": "INT32",
                "value": -333,
                "timestamp": "2018-03-07T10:53:21.937Z"
            },
            "syncStatus": "OK"
        }
    },
    "notifyTo": "fifo/configUpdateFifo"
}

8.2.5. Get state of a specific device configuration parameter

8.2.5.1. Request

Endpoint:

POST /api/v1/deviceMgt/devices/<deviceId>/config/parameters/<paramKey>

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Example:

GET /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/config/parameters/MyParamFLOAT
8.2.5.2. Response

HTTP Code:

200 OK

Body:

JSON Params Description

reported

Optional. reported configuration parameter value

requested

requested configuration parameter value

syncStatus

parameter update status; NONE, PENDING, SENT, OK or FAILED

updateReqCorrelationId

Optional. correlation id

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

404

DM_DEVICE_CONFIG_PARAM_NOT_FOUND

Device configuration parameter not found

Example:

{
    "requested": {
        "type": "FLOAT",
        "value": 1245,
        "timestamp": "2018-03-07T10:55:21Z"
    },
    "syncStatus": "PENDING"
}

8.3. Commands

The API described in this chapter allows to manage only commands for MQTT devices. If you want to register a command for LoRa® connector, please refer to LoRa® commands section.

8.3.1. Introduction

A command request is a downlink message that Live Objects sends to the device, with acknowledgement mechanism. Depending on the protocol used, a command response can be sent by the device to Live Objects.

You can register commands targeting a specific device: as soon as an interface for this device is available for commands, Live Objects will send them one by one, waiting for an acknowledgment for each command before sending the next one.

Live Objects keeps a record of every registered command with its status, and possible response.

8.3.2. MQTT example

Let’s take the example of a smart lock, connected to Live Objects with an MQTT interface.

From our smartphone, we want to unlock our door for a family member who forgot his keys. The application on our phone will create a command on Live Objects, with a request like the one below:

POST /api/v1/deviceMgt/devices/<deviceId>/commands

{
    "request": {
        "connector": "mqtt",
        "value": {
            "req": "unlock",
            "arg": {
                "delay": 1000
            }
        }
    },
    "policy": {
        "expirationInSeconds": 120,
        "ackMode": "APPLICATIVE"
    }
}

When the smart lock is available to receive commands (online and subscribed), Live Objects sends the first PENDING command (in FIFO order).

Then, depending on the acknowledgement level, Live Objects will wait for a response from the smart lock. In this case, we choose an APPLICATIVE acknowldegment level, so a reponse from the device is mandatory.

Here is the sequence of events expected to happen during the processing of the command, as well as the evolution of the statuses :

dm command mqtt sequence

The application can check the status of the command with the following query:

GET /api/v1/deviceMgt/commands/<commandId>

{
    "id": "ae49129f-9ce4-4782-82c4c6a2",
    "targetDeviceId": "urn:lo:nsid:smartlock:123456",
    "request": {
        "connector": "mqtt",
        "value": {
            "req": "unlock",
            "arg": {
                "delay": 1000
            }
        }
    },
    "response": {
        "done": true
    },
    "status": "PROCESSED",
    "deliveryStatus": "REPLIED",
    "policy": {
        "expirationInSeconds": 120,
        "ackMode": "APPLICATIVE"
    },
    "history": [
        {
            "timestamp": "2017-12-06T11:32:25.055Z",
            "status" : "PENDING"
        }, {
            "timestamp": "2017-12-06T11:38:22.481Z",
            "status": "PROCESSING",
            "deliveryStatus": "SENT"
            "nodeId": "abcd123456"
        }, {
            "timestamp": "2017-12-06T11:38:24.124Z",
            "status": "PROCESSED",
            "deliveryStatus": "REPLIED",
            "nodeId": "abcd123456",
        }
    ],
    "created": "2017-12-06T11:32:25.055Z",
    "updated": "2017-12-06T11:38:24.124Z"
}

Please refer to the swagger for details of REST API.

Please refer to the MQTT device mode part for messages that your device can send or receive.

8.3.3. Command status

The commands can have the following states:

Status Description

PENDING

The command is recorded and waiting for processing

PROCESSING

The command is being processed by the dedicated interface (waiting for an acknowledge)

PROCESSED

The command has reached its final acknowledgement level (FINAL STATE)

ERROR

An error occurred during the processing of the command (FINAL STATE)

CANCELED

The command was canceled before reaching PROCESSED state (FINAL STATE)

EXPIRED

The command could not be processed within the time limit (expirationTimeoutInSecond) (FINAL STATE)

SENT (deprecated since 2.7)

Replaced by PROCESSING

Here is a status diagram with possible transitions:

Device commands states

dm command status

8.3.4. Delivery status

During the PROCESSING state, the device manager exposes an additional information: the delivery status. The delivery status provides more detailed information on the processing step of the command, based on the acknowledgement information available, depending on the connectivity used.

Device commands delivery states

dm command delivery status

8.3.5. Acknowledgement level

The acknowledgement level determines the transition from the PROCESSING state to the PROCESSED state of a command.

The device manager offers two different levels of acknowledgement:

  • NONE : The device manager only listens to the internal acknowledgement that notifies the sending of the command. The command is output from Live Objects. Equivalent to the delivery status SENT

  • APPLICATIVE : The device manager waits for an applicative acknowledgement, with potentially a response. Equivalent to the delivery status REPLIED

Here is a summary table of the acknowledgement levels, as well as their relationship with the connectivity used:

Ack level Delivery State linked MQTT

NONE

SENT

MQTT connector sent through open connection

APPLICATIVE

REPLIED

Applicative reponse from device (default value)

AUTO (deprecated since 2.7)

replaced by NONE

DEVICE (deprecated since 2.7)

replaced by APPLICATIVE

8.3.6. Expiration Timeout

In most cases, we cannot predict exactly the availability of the device, for connectivity reason for example. And in some cases, a command must be executed in near future, or not executed at all.

Let’s return to the example of our connected lock: when we send a "unlock" command, we want that the command will be executed within the next two minutes, not in three hour due to a connectivity issues.

The command API proposes the expiration timeout. This is the maximal amount of time allowed to reach the status PROCESSING. If this value is exceeded, the status of the command goes to the status EXPIRED

8.3.7. The command object

We will detail the command object of the device manager, as used in API v1.

All possible operations on this object are detailed in the swagger.

Here is an unrealistic example, using a blank interface :

{
    "id": "ae49129f-9ce4-4782-82c4c6a2",
    "targetDeviceId": "urn:lo:nsid:sensor:2327398",
    "request": {
        "connector": "...",
        "value": {
            [...]
        }
    },
    "response": {
      [...]
    },
    "status": "PROCESSED",
    "deliveryStatus": "REPLIED",
    "errorCode": "INVALID_COMMAND_REQUEST",
    "policy": {
        "expirationInSeconds": 120,
        "ackMode": "APPLICATIVE"
    },
    "history": [
        {
            "timestamp": "2017-12-06T11:32:25.055Z",
            "status" : "PENDING"
        }, {
            "timestamp": "2017-12-06T11:38:22.481Z",
            "status": "PROCESSING",
            "deliveryStatus": "SENT",
            "errorCode": "INVALID_COMMAND_REQUEST",
            "nodeId": "2327398"
        }, {
            "timestamp": "2017-12-06T11:38:24.124Z",
            "status": "PROCESSED",
            "deliveryStatus": "REPLIED",
            "nodeId": "2327398",
        }
    ],
    "created": "2017-12-06T11:32:25.055Z",
    "updated": "2017-12-06T11:38:24.124Z"
}

The root fields are as follows:

JSON Params Description

id

Unique id of the command

targetDeviceId

Targeted device identifier

request

Command request (Cf. command request format)

response

Optional. Command response from device

status

Status of the command. Please refer to Command status for more details.

deliveryStatus

Optional. Delivery status of the command. Please refer to Delivery status for more details.

errorCode

Optional. Error code encountered during command processing.

policy

Policy for the command (Cf. Policy format)

history

Contains the history of changes in the status of the command. (Cf. Command history)

created

Registration date of the command

updated

Last "status" update date of the command.


The format of a command request is the following:

JSON Params Description

connector

connector/protocol to use to forward the command. Allowable values: mqtt.

value

command value (protocol/connector-dependant)


The format of the policy for a command is the following:

JSON Params Description

expirationInSeconds

Optional. expiration in seconds since command creation date. Default is no expiry. Min value is 5 seconds. (Cf. Expiration Timeout)

ackMode

Optional. Ack mode for this command. Please refer to Acknowledgement level for more details.


The format of the history for a command is an array of this:

JSON Params Description

timestamp

Timestamp of this event

status

Status of the command at that moment

deliveryStatus

Optional. Delivery status of the command at that moment

errorCode

Optional. Error code encountered during command processing.

nodeId

Optional. NodeId of the interface used to process the command


8.3.8. LORA Commands (downlinks)

8.3.8.1. Register command
8.3.8.1.1. Request

Endpoint:

POST /api/v1/deviceMgt/connectors/lora/nodes/<devEUI>/downlinks

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Body:

JSON Params Description

data

hexadecimal raw data of the command

port

port of the device on which the command will be sent (cf. LoRa® downlinks messages)

confirmed

network ack confirmation

Example:

POST /api/v1/deviceMgt/connectors/lora/nodes/0101010210101010/downlinks
{
  "data": "1A1A1A",
  "port": 1,
  "confirmed": true
}
8.3.8.1.2. Response

HTTP Code:

200 OK

Body:

JSON Params Description.

id

unique id of the command.

data

hexadecimal raw data of the command.

port

port of the device on which the command was sent (cf. LoRa® downlinks messages).

confirmed

network ack confirmation.

commandStatus

status of the command. SENT: The command was sent to the Device. ERROR: The command could not be sent to the Device.

creationTs

registration date of the command.

Error case:

HTTP Code Error code message

400

4001

The device EUI validation has failed (must be an hexadecimal string of size 16)

400

4002

The command validation has failed

403

4030

Service is disabled. Please contact your sales entry point.

404

40411

The device with DevEui has not been found

Example:

{
    "id": "5a9eab22ff240b343c5e54e5",
    "data": "1A1A1A",
    "port": 1,
    "confirmed": true,
    "commandStatus": "SENT",
    "creationTs": "2018-03-06T14:52:18.059Z"
}
8.3.8.2. List commands
8.3.8.2.1. Request

Endpoint:

GET /api/v1/deviceMgt/connectors/lora/nodes/<devEUI>/downlinks

Query parameters:

Name Description

page

Optional. page selection. 0 by default.

size

Optional. size selection. 20 by default.

status

Optional. status of the command: SENT or ERROR

timeRange

Optional. filter data where timestamp is in timeRange "from,to"

sort

Optional. sorting selection. Prefix with '-' for descending order.

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

GET /api/v1/deviceMgt/connectors/lora/nodes/0101010210101010/downlinks?sort=port&timeRange=2018-03-06T14:46:01.000Z,2018-03-07T14:00:01.000Z
8.3.8.2.2. Response

HTTP Code:

200 OK

Body:

JSON Params Description

page

page selection.

size

size selection.

totalCount

list size

data

contains the command list

id

unique id of the command

data

hexadecimal raw data of the command

port

port of the device on which the command was sent (cf. LoRa® downlinks messages)

confirmed

network ack confirmation

commandStatus

status of the command. SENT: The command was injected into LoRa® network core. ERROR: The command could not be injected into LoRa® network core.

creationTs

registration date of the command

updateTs

last "commandStatus" update date of the command.

Error case:

HTTP Code Error code message

400

4001

The device EUI validation has failed (must be an hexadecimal string of size 16)

400

4002

The parameter validation has failed for the specified field

403

4030

Service is disabled. Please contact your sales entry point.

404

40411

The device with DevEui has not been found

Example:

{
    "page": 0,
    "size": 20,
    "totalCount": 2,
    "data": [
        {
            "id": "5a9eab22ff240b343c5e54e5",
            "data": "1A1A1A",
            "port": 1,
            "confirmed": true,
            "commandStatus": "SENT",
            "creationTs": "2018-03-06T14:52:18.059Z"
        },
        {
            "id": "5a9ead86ff240b343c5e54e6",
            "data": "1B1B1B",
            "port": 2,
            "confirmed": false,
            "commandStatus": "SENT",
            "creationTs": "2018-03-06T15:02:30.809Z"
        }
    ]
}

8.4. Firmwares update

Currently, APIs allow to manage firmware update only for MQTT connector.

8.4.1. Principle

A firmware is a versioned binary content (for example a device firmware).

You can manage a repository of firmwares in your tenant account.

Live Objects can track the current versions of firmwares on a specific device.

You can set the target version of firmwares for a specific device in Live Objects that will then try to update the firmwares on the device as soon as the device is available for firmware update.

Device firmware update steps

landing

  • step 1: the device (or the codec communicating on its behalf) notifies the Firmware Manager module of the currently deployed firmware versions,

  • step 2: the Firmware Manager module update the current state of device/thing firmware versions in database, and compares it to the "target" firmwares versions for this device. For each firmware on the device that is not in the "target" version :

    • step 3: the Firmware Manager module send a "prepare firmware update" request to the Updated module in charge of the new firmware version,

    • step 4: the Updater module prepares the update (for example by retrieving the binary content of this firmware version, by creating a temporary access for the device on this resource, etc.);

    • step 5: the Updater module replies to the Firmware Manager module with a status (update possible or not) and extra information to transmit to the device (for example a URN where the new resource can be downloaded, a security token to use to access the new resource, etc.);

    • step 6: the Firmware Manager receives the Updater reply, and if update is possible builds a firmware update request to the device, with the extra info provided by the Updater module;

    • step7: the Firmware Manager send the firmware update request to the device;

    • step 8: if its context permits, the device proceeds to retrieve the new resource version (ex: HTTP/FTP download…​) from the Updater module, using if needed the extra info that was specified in the firmware update request;

    • step 9: during transfer the Updater module or the device notifies the Firmware Manager module of the transfer progress;

    • step 10: once the new firmware has been completely transferred to the device, the device can verify the binary content (for example checking crypto signature, comparing content hash, etc.) and applies the update;

    • step 11: the device notifies the Firmware Manager of the firmware update result (success or failure).

Device firmware update states

landing

If your device is connected in Device mode, please refer to the MQTT device mode part for messages that your device can send or receive.

8.4.2. Set device firmware version

8.4.2.1. Request

Endpoint:

POST /api/v1/deviceMgt/devices/<deviceId>/firmwares/<firmwareId>

HTTP Headers:

X-API-Key: <your API key>
Content-Type: application/json
Accept: application/json

Body:

JSON Params Description

version

requested firmware version

notifyTo

Optional. topic where firmware update status change events must be published

Example:

POST /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/firmwares/MyFirmware001
{
    "version": "1.0",
    "notifyTo": "fifo/fwUpdateEvents"
}
8.4.2.2. Response

HTTP Code:

200 OK

Body:

JSON Params Description

reported

current device firmware status

requested

requested firmware status

version

firmware version

timestamp

date of firmware version association

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

404

GENERIC_NOT_FOUND_ERROR

The requested resource is unknown.

Example:

{
    "reported": {
        "version": "0.1",
        "timestamp": "2018-02-15T14:27:31.491Z"
    },
    "requested": {
        "version": "1.0",
        "timestamp": "2018-02-15T14:39:56.420Z"
    }
}

8.4.3. List device firmwares

8.4.3.1. Request

Endpoint:

GET /api/v1/deviceMgt/devices/<deviceId>/firmwares

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

GET /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/firmwares
8.4.3.2. Response

HTTP Code:

200 OK

Body:

JSON Params Description

reported

current device firmware status

requested

requested firmware status

version

firmware version

timestamp

date of firmware version association

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

Example:

{
    "MyFirmware002": {
        "reported": {
            "version": "1.0",
            "timestamp": "2018-02-15T14:43:12.640Z"
        }
    },
    "MyFirmware001": {
        "reported": {
            "version": "1.0",
            "timestamp": "2018-02-15T14:40:33.504Z"
        },
        "requested": {
            "version": "1.0",
            "timestamp": "2018-02-15T14:39:56.420Z"
        }
    }
}

8.4.4. Get a specific device firmware

8.4.4.1. Request

Endpoint:

GET /api/v1/deviceMgt/devices/<deviceId>/firmwares/<firmwareId>

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

GET /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/firmwares/MyFirmware001
8.4.4.2. Response

HTTP Code:

200 OK

Body:

JSON Params Description

reported

current device firmware status

requested

requested firmware status

version

firmware version

timestamp

date of firmware version association

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

404

GENERIC_NOT_FOUND_ERROR

The requested resource is unknown.

Example:

{
    "reported": {
        "version": "1.0",
        "timestamp": "2018-02-15T14:40:33.504Z"
    },
    "requested": {
        "version": "1.0",
        "timestamp": "2018-02-15T14:39:56.420Z"
    }
}

8.4.5. Get information about last update of the device firmware

8.4.5.1. Request

Endpoint:

GET /api/v1/deviceMgt/devices/<deviceId>/firmwares/<firmwareId>/lastUpdate

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json

Example:

GET /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/firmwares/MyFirmware001/lastUpdate
8.4.5.2. Response

HTTP Code:

200 OK

Body:

JSON Params Description

id

firmware update identifier

created

date of firmware version association

updated

date of last firmware update status

initialVersion

initial firmware version

requestedVersion

requested firmware version

status

firmware update status

errorOrigin:

error origin

errorCode

error code

deviceErrorCode

Optional. device error code

errorDetails

Optional. error details

progress

ratio of progress (%)

Error case:

HTTP Code Error code message

400

GENERIC_INVALID_PARAMETER_ERROR

The submitted parameter is invalid.

404

GENERIC_NOT_FOUND_ERROR

The requested resource is unknown.

Example:

{
    "id": "5a859bbcff240b18ecf55a8d",
    "created": "2018-02-15T14:39:56.437Z",
    "updated": "2018-02-15T14:40:33.513Z",
    "initialVersion": "0.1",
    "requestedVersion": "1.0",
    "status": "DONE",
    "errorOrigin": "INTERNAL",
    "errorCode": "OK",
    "progress": 100
}

8.4.6. Update history

8.4.6.1. Request

Endpoint:

GET /api/v1/deviceMgt/devices/<deviceId>/firmwareUpdates

Query parameters:

Name Description

size

Optional. maximum number of devices per page. 20 by default.

page

Optional. requested page number. 0 by default.

HTTP Headers:

X-API-Key: <your API key>
Accept: application/json
X-Total-Count: <boolean>

For more info about X-Total-Count, see "API v1 paging" section.

Example:

GET /api/v1/deviceMgt/devices/urn:lo:nsid:sensor:temp001/firmwareUpdates
8.4.6.2. Response

HTTP Code:

200 OK

Body:

JSON Params Description

id

firmware update identifier

created

date of firmware version association

updated

date of last firmware update status

initialVersion

initial firmware version

requestedVersion

requested firmware version

status

firmware update status

errorOrigin:

error origin

errorCode

error code

deviceErrorCode

Optional. device error code

errorDetails

Optional. error details

progress

ratio of progress (%)

Example:

[
    {
        "id": "5ad74d28ff07c60e4083c085",
        "created": "2018-02-16T13:50:32.719Z",
        "updated": "2018-02-16T13:50:33.473Z",
        "initialVersion": "1.0",
        "requestedVersion": "2.0",
        "status": "ERROR",
        "errorOrigin": "DEVICE",
        "errorCode": "ASSET_ORIGINATED_ERROR",
        "deviceErrorCode": "DV196639",
        "errorDetails": "Function execution failed: 'startPairing'",
        "progress": 0.0
    },
    {
        "id": "5a85b039ff240b18ecf55a92",
        "created": "2018-02-15T16:07:21.468Z",
        "updated": "2018-02-15T16:07:41.504Z",
        "initialVersion": "1.0",
        "requestedVersion": "2.0",
        "status": "ERROR",
        "errorOrigin": "INTERNAL",
        "errorCode": "ASSET_PREPARATION_ERROR",
        "progress": 0
    },
    {
        "id": "5a859bbcff240b18ecf55a8d",
        "created": "2018-02-15T14:39:56.437Z",
        "updated": "2018-02-15T14:40:33.513Z",
        "initialVersion": "0.1",
        "requestedVersion": "1.0",
        "status": "DONE",
        "errorOrigin": "INTERNAL",
        "errorCode": "OK",
        "progress": 100
    }
]

8.5. Campaign management

Campaign management is a Live Objects feature that allows a fleet manager to schedule execution of device management operations on a large number of devices.

The following operations are supported in a campaign definition:

  • configuration,

  • commands,

  • firmware update.

8.5.1. Campaign creation

When creating a campaign, the user must provide the following information:

name

A short name to identify the campaign

description

(optional) Detailed description of the campaign

options

(optional) Set of campaign options.

planning

The scheduling configuration including the start date and the end date for the campaign

targets

Devices targeted: either idList or filterQuery (exclusively). cf. Campaign targets

operations

A sequence of operations that will be executed on each device of the campaign.

Campaigns are created from the REST API by providing a campaign definition with properties described previously.

8.5.1.1. Campaign options

Campaign options section could be empty or omitted.

Example of campaign options:

"options": {
    "dynamicallyAddEligibleDevice": true
  }

Options are:

  • dynamicallyAddEligibleDevice (default: false).

    Set this option to true to dynamically enroll devices to the campaign. During the campaign planning, new or updated devices could be dynamically enrolled if they match the filterQuery.

    This option requires a filterQuery target definition.

    A campaign with dynamicallyAddEligibleDevice option will always be in running state until end date, whereas others (non-dynamics) campaigns could end as soon as all devices operations ended or end date reached.

8.5.1.2. Campaign operations

Operation types are: Config, Command, Resource.

For each device, operations are executed one after the other, each operation waits for the success of the previous operation.

All operations definitions could embed an optional maxRetry attribute:

default is 0, and max is 5.

In case of operation failure, operation will be retried in the limit of maxRetry.

If an operation fails after maxRetry attempts (or is canceled) for a device, the remaining operations are not executed for this device and the sequence ends in failure (or in canceled status).

8.5.1.3. Config operation

Example of config operation definition:

  {
     "action":"configure",
     "definition":{
        "assetParameters":{
           "param1":{
              "type":"INT32",
              "valueInt32":1234
           }
        },
        maxRetry: 1
     }
  }
action

configure action will send to the device one or many "parameters" to update.

definition

assetParameters has the same format as the corresponding unitary device management operation: Device configuration.

maxRetry (optional) defines how many retries should be executed in case of failure of the current operation.

The following table shows how the config operation statuses are converted into sequence statuses

Config operation status

Resulting status in the sequence

PENDING

PENDING

SENT

PENDING

CANCELED

CANCELED

OK

if it is the last operation: SUCCESS, if not: PENDING and the sequence moves onto the next operation

FAILED

FAILURE

8.5.1.4. Command operation

Example of command operation definition:

  {
    "action": "command",
    "definition": {
        "event": "reset",
        "data": {
            "temp": "12"
        },
        "maxRetry": 0
    }
  }
action

command action will register (and send) a command to the device.

definition

this section uses event, data and payload attributes in the same format as the corresponding unitary device management operation: Commands.

maxRetry (optional) defines how many retries should be executed in case of failure of the current operation.

The following table shows how the command operation statuses are converted into sequence statuses

Command operation status

Resulting status in the sequence

PENDING

PENDING

SENT

PENDING

PROCESSED

if it is the last operation: SUCCESS, if not: PENDING and the sequence moves onto the next operation

CANCELED

CANCELED

UNKNOWN

FAILURE

Note that any response from the device sets the operation status to PROCESSED and the resulting status in the sequence is SUCCESS.

8.5.1.5. Resource operation

Example of resource operation definition:

{
  "action": "resource",
  "definition": {
    "resourceId": "X11_firmware",
    "targetVersion": "2.1",
    "maxRetry": 4
  }
}
action

resource action will send a resource update request to the device.

definition

resourceId identifies the resource to update

targetVersion is the new version of the resource to download and install

maxRetry (optional) defines how many retries should be executed in case of failure of the current operation.

The following table shows how the resource operation statuses are converted into sequence statuses

Resource operation status

Resulting status in the sequence

PENDING

PENDING

PREPARING_CONNECTOR

PENDING

PREPARING_ASSET

PENDING

WAITING_TRANSFER_INFO

PENDING

TRANSFER_PENDING

PENDING

TRANSFER_IN_PROGRESS

PENDING

TRANSFER_IN_PROGRESS

PENDING

TRANSFER_SUCCESS

PENDING

DONE

if it is the last operation: SUCCESS, if not: PENDING and the sequence moves onto the next operation

ERROR

FAILURE

CANCELED

CANCELED

8.5.1.6. Campaign target (idList)

Campaign devices targets: either idList or filterQuery (exclusively).

idList is a flat list of devices identifiers.

Devices are identified using a URN identifier.

The format of this identifier must be urn:lo:nsid:{ns}:{id} with

ns the target device identifier namespace

id the target device identifier

idList is not compatible with dynamicallyAddEligibleDevice option.

Below is an example of campaign targets definition using idList:

   "targets":{
      "idList":[
         "urn:lo:nsid:namespace:device1",
         "urn:lo:nsid:namespace:device2"
      ]
   }
8.5.1.7. Campaign target (filterQuery RSQL)

filterQuery: use RSQL to target devices.

Below is an example of campaign targets definition using filterQuery:

  "targets": {"filterQuery": "groupPath==/"}
Table 1. RSQL Semantic Table
desc syntax

Logical AND

; or and

Logical OR

, or or

Equal to

==

Not equal to

!=

Less than

=lt= or <

Less than or equal to

=le= or

Greater than operator

=gt= or >

Greater than or equal to

=ge= or >=

In

=in=

Not in

=out=

Below are example of filterQuery values.

  • Filter using tags

tags=in=(FUT,TEST1)

For example, devices with at least tags "FUT" and "TEST1" whatever the order or additional tags

  • Filter on properties

properties.mykey=in=(enum1, enum3)
properties.mykey==toto
  • Filter on groups

groupID=in=(1224,1234)
groupID==1234
groupPath=in=(/FR, /EN)
8.5.1.8. Campaign creation examples

Below are examples of campaign definition:

Set parameter "param1" to value 1234 on two devices
POST /api/v0/deviceMgt/campaigns
{
   "name":"campaign1",
   "description":"A campaign that configures parameters",
   "planning":{
      "startDate":"2017-07-01T00:00:00Z",
      "endDate":"2017-07-23T23:59:59Z"
   },
   "targets":{
      "idList":[
         "urn:lo:nsid:namespace:device1",
         "urn:lo:nsid:namespace:device2"
      ]
   },
   "operations":[
      {
         "action":"configure",
         "definition":{
            "assetParameters":{
               "param1":{
                  "type":"INT32",
                  "valueInt32":1234
               }
            }
         }
      }
   ]
}
Send a reset command with a delay parameter for devices with foo tag
POST /api/v0/deviceMgt/campaigns
{
   "name":"campaign2",
   "description":"A campaign that sends a command",
   "planning":{
      "startDate":"2017-07-01T00:00:00Z",
      "endDate":"2017-07-23T23:59:59Z"
   },
   "options": {
    "dynamicallyAddEligibleDevice": true
   },
   "targets": {"filterQuery": "tags=in=(foo)"},
   "operations":[
      {
         "action":"command",
         "definition":{
            "event":"reset",
            "data":{
               "delay":"5000"
            }
         }
      }
   ]
}
Update the resource firmware.bin to version 1.1 (with 2 retries max)
POST /api/v0/deviceMgt/campaigns
{
   "name":"campaign3",
   "description":"A campaign that updates a resource",
   "planning":{
      "startDate":"2017-07-01T00:00:00Z",
      "endDate":"2017-07-23T23:59:59Z"
   },
   "targets":{
      "idList":[
         "urn:lo:nsid:namespace:device1",
         "urn:lo:nsid:namespace:device2"
      ]
   },
   "operations":[
      {
         "action":"resource",
         "definition":{
            "resourceId":"firmware.bin",
            "targetVersion":"1.1",
            "maxRetry": 2
         }
      }
   ]
}

8.5.2. Campaign reporting

Once a campaign is created, a fleet manager can monitor the state of a campaign.

A campaign can have one of the statuses described below:

SCHEDULED

The campaign has not yet started

RUNNING

The campaign is in progress

COMPLETE

The campaign is finished and all devices ended their sequence in success

INCOMPLETE

The campaign is finished but some devices could not be configured

SERVER_ERROR

An internal error occured in the platform and the campaign could not be completed

CANCELING

The campaign is waiting for running sequences to end, sequences that have not started yet will not start

CANCELED

The campaign was canceled and some devices might not have been configured

The possible statuses for a device are presented below:

notStarted

No operation executed on the device

pending

At least one operation of the sequence is still in progress

success

All operations were successufully executed on the device

failure

One operation of the sequence failed, so the remaining operations (if any) were not executed

canceled

The sequence was canceled before the end of all operations

8.5.2.1. Campaign cancelation

A campaign can be canceled with the following REST API endpoint:

PUT /api/v0/deviceMgt/campaigns/{campaignId}/cancel

If the campaign is already running, canceling it will set its state to CANCELING and the campaign will wait for running sequences to end. Then the campaign state will switch to CANCELED.

To abort running sequences, the force flag can be used.

PUT /api/v0/deviceMgt/campaigns/{campaignId}/cancel
force = true
8.5.2.2. Campaign deletion

A campaign can be deleted with the following REST API endpoint:

DELETE /api/v0/deviceMgt/campaigns/{campaignId}

If the campaign is in RUNNING or CANCELING state, it cannot be deleted. In this case the force flag can be used to execute a forced cancelation and automatically delete the campaign once it is in CANCELED state.

DELETE /api/v0/deviceMgt/campaigns/{campaignId}
force = true

8.5.3. Global report

The global report indicates the campaign definition, the current status of a campaign and statistics about the number of devices with a given status.

Get global status of a speficied campaign
GET /api/v0/deviceMgt/campaigns/{campaignId}
{
   "name":"campaign1",
   "description":"A campaign that configures parameters",
   "planning":{
      "startDate":"2017-07-01T00:00:00Z",
      "endDate":"2017-07-23T23:59:59Z"
   },
   "targets":{
      "idList":[
         "urn:lo:nsid:namespace:device1",
         "urn:lo:nsid:namespace:device2"
      ]
   },
   "operations":[
      {
         "action":"configure",
         "definition":{
            "assetParameters":{
               "param1":{
                  "type":"INT32",
                  "valueInt32":1234
               }
            }
         }
      }
   ],
   "numberOfTargets":2,
   "totalTargetsPerStatus":{
      "notStarted":0,
      "pending":1,
      "failed":0,
      "success":1,
      "canceled":0
   },
   "campaignStatus":"RUNNING",
   "created":"2017-06-01T00:00:00Z",
   "updated":"2017-07-01T00:00:00Z"
}

8.5.4. Detailed report

The detailed report gives the status of each device in a campaign. The status property gives the status for the whole sequence of operations. The detailed report also indicates the status of each operation (operation reports are ordered just like in the campaign definition).

operationStatus

Exact status reported by the device manager (the list of possible values depends on the type of operation). A special value notStarted is used when the operation is not yet started.

operationId

Identifier returned by the device manager when the campaign manager created the operation

started

Date when the operation was started

updated

Last time the operation report was updated

ended

Date when the operation was finished

currentRetry

(Option) Retry attempt count of the latest operation executed.

For example, an operationStatus equals to OK (or DONE) and currentRetry equals to 1 means that operation first failed, but the first retry attempt was a success.

Get global status of a speficied campaign
GET /api/v0/deviceMgt/campaigns/{campaignId}/targets
{
   "page":0,
   "size":10,
   "totalCount":2,
   "data":[
      {
         "device":"urn:lo:nsid:namespace:id1",
         "status":"pending",
         "created":"2017-07-01T16:12:21.000Z",
         "updated":"2017-07-01T16:12:21.000Z",
         "operations":[
            {
               "action":"configure",
               "operationStatus":"OK",
               "started":"2017-07-01T16:20:21.000Z",
               "updated":"2017-07-01T16:25:21.000Z",
               "ended":"2017-07-01T16:25:21.000Z"
            },
            {
               "action":"command",
               "operationStatus":"SENT",
               "operationId":"12345",
               "started":"2017-07-01T16:30:21.000Z",
               "updated":"2017-07-01T16:31:21.000Z"
            },
            {
               "action":"resource",
               "operationStatus":"notStarted"
            }
         ]
      },
      {
         "device":"urn:lo:nsid:namespace:id2",
         "status":"success",
         "created":"2017-07-01T16:12:21.000Z",
         "updated":"2017-07-01T16:12:21.000Z",
         "operations":[
            {
               "action":"configure",
               "operationStatus":"OK"
               "started":"2017-07-01T16:20:21.000Z",
               "updated":"2017-07-01T16:25:21.000Z",
               "ended":"2017-07-01T16:25:21.000Z"
            },
            {
               "action":"command",
               "operationStatus":"PROCESSED",
               "operationId":"6789",
               "started":"2017-07-01T16:30:21.000Z",
               "updated":"2017-07-01T16:31:21.000Z"
            },
            {
                "action": "resource",
                "operationStatus": "DONE",
                "operationId": "X11_firmware",
                "started": "2017-07-01T16:30:21.000Z",
                "updated": "2017-07-01T16:38:21.000Z",
                "ended": "2017-07-01T16:38:21.000Z",
                "currentRetry": 1
            }
         ]
      }
   ]
}

9. Data management

9.1. Data managment components and concepts

You can send data to Live Objects using two sources, the device sent data using connectivity or by using http data push. Each source is processed by one or many composants. Data management is then relies upon:

  • the data enrichment process which makes the link between Live Objects Data Message and Device Management. Data enrichment is a merge of third-party data model from device source with a Live Objects common data model.

  • the decoding service which process Live Objects message for handling encoded data collected from various networks.

  • the store service which is aimed to store data messages from IoT things (devices, gateway, IoT app collecting data, etc.) as time-series data streams,

  • the search service based on the popular open-source Elasticsearch product.

Data journey

landing

9.2. Model field for indexing

After the data collected it may be associated to a model. The model is a fundamental concept for the search service, it specifies the schema of the JSON "value" object. The model is dynamically updated based on the data injected.

The model concept is necessary to avoid mapping conflicts in the underlying elasticsearch system.

9.2.1. Model field in data message

The model allows to give pattern for the value object, A model can be seen as a "mapping space" in which data types must not conflict. Some precautions must be taken when using model field :

  • If the model is not provided, "value" object will be not indexed by the search service. Nevertheless, the data will be stored in the store service and all information except value object will be indexed in search service.

  • If the value JSON object does not comply with the provided model (for example, a field change from Integer type to String type), the data will be not inserted in the search service. The data message will be only stored in the store service.

9.2.2. Set model field to index your data

To make the messages sent by devices indexed in data store you must setting your data model with the following steps :

  • when you provision your decoder set "model" field in your data model.

  • when the "model" field is set and elasticsearch (ES) is enable, you can make elasticsearch queries on value fields only after the messages are decoded by your decoder and indexing service done.

  • each field of value collection is indexed by search service. In order to be able to search the message, copy this 'value' in a @model.value field which is analyzed by Elasticsearch. The message is consistent with others messages having the same model.

Max total field number which can be indexed in ES : the limit allowed per tenant is max 1000 fields.

exemple : data message sent by device and shown the indexed fields with values.

{
    "timestamp" : "2018-05-31T16:15:21.288Z,"
    "streamId" : "urn:lo:nsid:mqtt:{deviceId},"
    "model": "my_encoding_model_v0"
    "value": {
      "temp":     12.75,
      "humidity": 62.1,
      "gpsFix":   true,
      "gpsSats":   [12, 14, 21]
    }
}

Indexed field :

Field

type

temp

float

humiditity

float

gpsFix

boolean

gpsSats

integer

9.3. Data Message Enrichment process

9.3.1. Overview

The Live Objects Data Message pipeline goes through an enrichment process that applies the folowing logic. This pipeline concerns messages coming from LORA and MQTT protocols.

9.3.1.1. Enrichment logic

The enrichment logic uses Device Management repository to enrich a DataMessage with Live Objects device information .

  • when using LORA protocol a device is retreived by using devEUI.

  • when using MQTT protocol a device is retreived by using clientId.

Table 2. Data Message fields modified by Enricher Logic

streamId

if not provided, will be set to Live Objects urn

tags

will be enriched by merging this fields with the device tags

extra

will be enriched by merging with the device properties

group

will be set with the device group

The enrichment process is executed before the decoding process

9.4. Decoding service

9.4.1. Overview

The data messages sent to the Live Objects platform can be encoded in a customer specific format. For instance, the payload may be a string containing an hexadecimal value or a csv value.

The data decoding feature enables you to provision your own decoding grammar. On receiving the encoded message, the Live Objects platform will use the grammar to decode the payload into plain text json fields and record the json in the Store service. The stored message will then be searchable with the Advanced Search service.

A "template" option allows to perform mathematical operations on the decoded fields or to define an output format.

A "model" option allows to override original data 'model' field.

The decoding feature is not activated by default.

9.4.2. Binary decoding

9.4.2.1. Decoder provisioning

The custom decoder describes the grammar to be used to decode the message payload. The Live Objects API to manage the decoders are described in the swagger documentation: https://liveobjects.orange-business.com/swagger-ui/index.html.

The binary decoding module uses the Java Binary Block Parser (JBBP) library.
You must use the JBBP DSL language to describe the binary payload format for your decoder.

Additional types

the float and utf8 are additional types that can be used in the grammar (see examples).

9.4.2.1.1. Basic decoder

Example: create a binary decoder with the REST API:

POST /api/v0/decoders/binary
X-API-Key: <your API key>
Accept: application/json
{
"encoding":"twointegers",  (1)
"enabled":true,  (2)
"format":"int pressure;int temperature;", (3)
"template":"{\"pressure\":{{pressure}},
  \"temperature\" : \"{{#math}}{{temperature}}/10{{/math}} celsius\"}", (4)
"model":"model_twointegers"  (5)
}
1 identifies the decoder. This name will be associated to the devices during the provisioning and will be present in the data message.
2 activation/deactivation of the decoder.
3 describes the payload frame (cf. JBBP DSL language). The name of the fields will be found in the resulting decoded payload json.
4 optional parameter describing a post-decoding template format. In this example, the output temperature will be divided by 10 and stored in a string format including its unit. More information on templates.
5 optional parameter that will override the 'model' field in decoded data. If empty, the original value of 'model' field of the encoded data will be used. More information on model field
9.4.2.1.2. When to change the model in the decoder ?
  • The "model" field is set in order to enable elasticsearch (ES) indexing and queries on the decoded value.

  • If the "model" field is not set when provisioning the decoder, the data message will still be stored but it will not be searchable with ES queries.

  • The "model" must be changed when the type of a field already indexed in ES is modified. Example :

    • Suppose you have created a binary decoder with the following payload description "format":"utf8[4] field1; int field2;"

    • You have defined a model named "my_model_v0" associated with this decoder.

    • You start using this decoder with your devices. Data messages are stored and indexed in Live Objects.

    • Then, you can perform searches in your data messages using the model. See search examples.

    • If, after checking your decoded data, you find out that field1 is a float, not a UTF8 string, you may modify your decoder format. "format":"float field1; int field2;"

    • In this case, if you want your new messages to be stored and indexed properly in elasticsearch, you must change the model name in your decoder (for instance "my_model_v1"). Otherwise, "field1" will still be mapped as String and the message will be rejected by elasticsearch.

9.4.2.1.3. Endianness

The decoding service uses the big-endian order (the high bytes come first). If your device uses little-endian architecture, you can use the < character to prefix a type in your format description.

Example: create a binary decoder for a device sending data in little-endian format:

POST /api/v0/decoders/binary
X-API-Key: <your API key>
Accept: application/json
{
"encoding":"my_little_endian_encoding",
"enabled":true,
"format":"<float temperature;" (1)
}
1 : <float means 32-bit float sent in little-endian.
9.4.2.1.4. How to test the binary decoder format?

The Live Objects API provides a "test" endpoint which takes a payload format and a payload value as input and provides the decoded value in the response body, if the decoding is successful. Optionally, you can provide a post-decoding template which will describe the output format.

In the following example, the decoded value for the pressure will remain unchanged, while the decoded value for temperature will be divided by 10.
The test endpoint is described in swagger.

Request:

POST /api/v0/decoders/binary/test
X-API-Key: <your API key>
Accept: application/json
{
"binaryPayloadStructure":"int  pressure; int temperature;",
"binaryPayloadHexString":"000003F5000000DD",
"template":"{\"pressure\":{{pressure}}, \"temperature\" : \"{{temperature}}/10\"}"
}

Response:

{
   "parsingOk": true,
   "decodingResult":    {
      "temperature": 22.1,
      "pressure": 1013
   },
   "descriptionValid": true
}
9.4.2.1.5. How to customize the fields once the payload has been decoded?

The fields resulting of a decoded payload might need to be processed using a template description, in order to change their output format. More information on templates.

9.4.2.2. Referencing a decoder in a LoRa® device

When provisioning a LoRa® device, you may reference the decoder to be used for the device so that Live Objects will automatically decode all the payloads received from this device, using the referenced decoder.

Example: landing

9.4.2.3. Message decoding

The data message is decoded using the decoder previously provisioned and the decoded fields are added to the value. The encoded raw payload is kept in the decoded message. Once the message has been decoded and stored, "Advanced Search" requests can be performed using the newly decoded fields.

Table 3. Examples :
Frame format Payload example Decoded payload (json)
int temperature;

000000DD

{
  "value": {
    "payload" : "000000DD",
    "temperature": 221
  }
}
ubyte temperature;

DD

{
  "value": {
    "payload" : "DD",
    "temperature": 221
  }
}
utf8 [16] myString;

2855332e3632542b323144323235503029

{
  "value": {
    "payload" : "2855332e3632542b323144323235503029",
    "myString": "(U3.62T+21D225P0)"
  }
}
byte is_led_on;
float pressure;
float temperature;
float altitude;
ubyte battery_lvl;
byte[6] raw_gps;
ushort altitude_gps;

00
447CE000
41CEF5C3
45CAB8CD
38
00000000000
0FFFF

{
  "value": {
    "payload" : "00447CE00041CEF5C345CAB8CD38000000000000FFFF",
    "is_led_on": 0,
    "pressure": 1011.5,
    "temperature": 25.87,
    "altitude": 6487.1,
    "battery_lvl": 56,
    "raw_gps_list": [0,0,0,0,0,0],
    "altitude_gps": 65535
  }
}
float pi;
measure[2] {
  int length;
  utf8 [length] name;
  float value;
}

4048F5C3
0000000B
C2A955544638537472696E67
41480000
00000012
C2A9616E6F7468657255544638537472696E67
447D4000

{
  "value": {
    "payload" : "4048F5C3
      0000000BC2A955544638537472696E6741480000
      00000012C2A9616E6F7468657255544638537472696E67447D4000",
    "pi": 3.14,
    "measure_list":[
      {
        "length":11,
        "name":"©UTF8String",
        "value":12.5
      },
      {
        "length":18,
        "name":"©anotherUTF8String",
        "value":1013.0
      }
    ]
  }
}
Table 4. Json fields

value.payload

a string containing the encoded payload in hexadecimal (raw value)

metadata.encoding

contains the decoder name

model

remains unchanged after decoding if model field of decoder is empty; else it will be set with the value of model field in the decoder

additional LoRa® fields (lora port, snr…​) in the value

remain unchanged after decoding.

9.4.3. Csv decoding

9.4.3.1. Decoder provisioning

The custom decoder describes the columns format and options to be used to decode the message csv payload. The Live Objects API to manage the decoders are described in the swagger documentation : https://liveobjects.orange-business.com/swagger-ui/index.html.

When provisioning a csv decoder, you must specify an ordered list of column names and their associated type. Three column types are available : STRING, NUMERIC or BOOLEAN.
Several options (column separator char, quote char, escape char…​) may be set to customize the csv decoding.

A template option enables you to provide a post-decoding output format including mathematical evaluation. More information on templates.

Column types
  • STRING column may contain UTF-8 characters

  • NUMERIC column may contain integer (32 bits), long (64 bits), float or double values. The values may be signed.

  • BOOLEAN column must contain true or false.

Table 5. Available options
name default definition example

quoteChar

double-quote "\""

character used for quoting values that contain column separator characters or linefeed.

"pierre, dupont",25,true will be decoded as 3 fields.

columnSeparator

comma ","

character used to separate values.

lineFeedSeparator

"\n"

character used to separate data rows. If the message payload contains several rows, only the first one will be decoded.

the decoding result for pierre,35,true\nmarie,25,false will be 3 fields containing pierre, 35 and true.

useEscapeChar

false

set to true if you want to use an escape char.

escapeChar

backslash "\\"

character used to escape values.

skipWhiteSpace

false

if set to true, will trim the decoded values (white spaces before and after will be removed).

Example 1: create a simple csv decoder with the REST API:

POST /api/v0/decoders/csv
X-API-Key: <your API key>
Accept: application/json
{
    "encoding":"my csv encoding", (1)
    "enabled":true, (2)
    "columns": [ (3)
        {"name":"column1","jsonType":"STRING"},
        {"name":"column2","jsonType":"NUMERIC"},
        {"name":"column3","jsonType":"BOOLEAN"}
    ],
    "model":"model_csv_decoded"  (4)
}
1 identifies the decoder. This name will be associated to the devices during the provisioning and will be present in the data message.
2 activation/deactivation of the decoder.
3 an ordered list of column descriptions.
4 optional parameter that will override the 'model' field of decoded data. If empty, the original value of 'model' field of the encoded data will be used. More information on model field.

Example 2: create a csv decoder with options, using the REST API:

POST /api/v0/decoders/csv
X-API-Key: <your API key>
Accept: application/json
{
    "encoding":"my csv encoding with options",
    "enabled":true,
    "columns": [
        {"name":"unit","jsonType":"STRING"},
        {"name":"temperature","jsonType":"NUMERIC"},
        {"name":"normal","jsonType":"BOOLEAN"}
    ],
    "options" : {
        "columnSeparator": "|",
        "quoteChar": "\"",
        "lineFeedSeparator": "/r/n"
    }
}
In the POST request, you can provide only the options you wish to modify. The other options will keep the default values.
9.4.3.1.1. How to customize the fields once the payload has been decoded?

The fields resulting of a decoded payload might need to be processed using a template description, in order to change their output format. More information on templates.

9.4.3.1.2. How to test the csv decoder ?

The Live Objects API provides a "test" endpoint which takes a csv format description and a payload value as input and provides the decoded value in the response body, if the decoding is successful. The test endpoint is described in swagger.

Request:

POST /api/v0/decoders/csv/test
X-API-Key: <your API key>
Accept: application/json
{
    "columns": [
        {"name":"unit","jsonType":"STRING"},
        {"name":"temperature","jsonType":"NUMERIC"},
        {"name":"threasholdReached","jsonType":"BOOLEAN"}
    ] ,
    "options":{
        "columnSeparator": ","
    },
    "csvPayload":"celsius,250,true",
    "template":"{\"temperature\" : \"{{temperature}}/10\",
      \"unit\":\"{{unit}}\", \"thresholdReached\":\"{{thresholdReached}}\"} "
}

Response:

{
   "parsingOk": true,
   "decodingResult":    {
      "unit": "celsius",
      "thresholdReached": "true",
      "temperature": 25
   },
   "descriptionValid": true
}
9.4.3.2. Referencing a decoder in a LoRa® device

When provisioning a LoRa® device, you may reference the decoder to be used for the device so that Live Objects will automatically decode all the payloads received from this device, using the referenced decoder.

9.4.3.3. Message decoding

The data message is decoded using the decoder previously provisioned and the decoded fields are added to the value. The csv encoded raw payload is kept in the decoded message. Once the message has been decoded and stored, "Advanced Search" requests can be performed using the newly decoded fields.

Example in https:

Request
POST /api/v0/data/streams/{streamId}
X-API-Key: <your API key>
Accept: application/json
{
  "value": {"payload":"celsius,25,true"},
  "model": "temperature_v0",
  "metadata" : {"encoding" : "my csv encoding"}
 }

The data message will be stored as:

{
      "id": "585aa47de4b019917e342edd",
      "streamId": "stream0",
      "timestamp": "2016-12-21T15:49:17.693Z",
      "model": "temperature_v0",
      "value":       {
         "payload": "celsius,25,true",
         "normal": true,
         "unit": "celsius",
         "temperature": 25
      },
      "metadata": {"encoding": "my csv encoding"},
      "created": "2016-12-21T15:49:17.750Z"
}
Table 6. Json fields

value.payload

a string containing the csv encoded payload (raw value)

metadata.encoding

contains the decoder name

model

remains unchanged after decoding if model field of decoder is empty; else it will be set with the value of model field in the decoder

9.4.4. Templating

The Live Objects provides, for the decoder creation and the decoder test APIs, an optional parameter named "template". This parameter is a string field describing the target output fields in a mustache-like format.

Table 7. Available functions :

{{#math}}{{/math}}

performs mathematical operations on a field

{{#toUpperCase}}{{/toUpperCase}}

converts a string to upper case

{{#toLowerCase}}{{/toLowerCase}}

converts a string to lower case

The following examples shows, for the same raw binary payload, the output if you are not using any template, or if you define a custom template.

Request (WITHOUT the template parameter)
POST /api/v0/decoders/binary/test
X-API-Key: <your API key>
Accept: application/json
{
"binaryPayloadStructure": "byte:1 led; ushort pressure; ushort temperature; ushort altitude; ubyte battery; byte[6] raw_gps; ushort altitude_gps;",
"binaryPayloadHexString":"0027830a1bfd6738000000000000ffff"
}
Response
{
   "parsingOk": true,
   "decodingResult":    {
      "led": 0,
      "pressure": 10115,
      "temperature": 2587,
      "altitude": 64871,
      "battery": 56,
      "raw_gps": [0, 0, 0, 0, 0, 0],
      "altitude_gps": 65535
   },
   "descriptionValid": true
}
Request (WITH the template parameter)
POST /api/v0/decoders/binary/test
X-API-Key: <your API key>
Accept: application/json
{
  "binaryPayloadStructure":"byte:1 led; ushort pressure; ushort temperature;
    ushort altitude; ubyte battery; byte[6] raw_gps; ushort altitude_gps;",
  "binaryPayloadHexString":"0027830a1bfd6738000000000000ffff",
  "template":"{
    \"pressure\": \"{{pressure}} / 10\",
    \"temperature\": \"{{temperature}} / 100\",
    \"altitude\": \"{{altitude}} / 10\",
    \"view\": {
      \"Pressure\": \"{{#math}}{{pressure}}/10{{/math}} hPa\",
      \"Temperature\": \"{{#math}}{{temperature}}/100{{/math}} C\",
      \"Altitude\": \"{{#math}}{{altitude}}/100{{/math}} m\",
      \"GPSAltitude\": \"{{altitude_gps}} m\",
      \"Battery\": \"{{battery}} %\"
    }
  }"
}
Response
{
   "parsingOk": true,
   "decodingResult":    {
      "altitude": 6487.1,
      "view":       {
         "Pressure": "1011.5 hPa",
         "Temperature": "25.87 C",
         "Altitude": "648.71 m",
         "GPSAltitude": "65535 m",
         "Battery": "56 %"
      },
      "temperature": 25.87,
      "pressure": 1011.5,
      "led": 0,
      "battery": 56,
      "raw_gps": [0, 0, 0, 0, 0, 0],
      "altitude_gps": 65535
   },
   "descriptionValid": true
}

Please do not use dot-separated fields in the template.

Example:

`"template":"{\"field.with.dot\": {{temperature}}/10}"` will be rejected.

The {{#math}}{{/math}} template is needed only if you wish to evaluate a mathematical expression within a string.

Example for a template containing:

\"Temperature\": \"{{temperature}}/100 celsius\" (1)
\"Temperature\": \"{{#math}}{{temperature}}/100{{/math}} celsius\" (2)
\"Temperature\": \"{{#math}}{{temperature}}/100{{/math}}\" (3)
\"Temperature\": \"{{temperature}}/100\" (4)
1 the output will be like Temperature": "2587/100 celsius" (the division is not evaluated).
2 the output will be like Temperature": "25.87 celsius" (a string output. the division is evaluated).
3 the output will be like Temperature": 25.87 (a numeric). In this case, the {{#math}} function is not needed.
4 the output will be like Temperature": 25.87 (a numeric).

You need to specify in the template, all the fields you wish to get in the output, even if they are not modified by the template.

Example:

 `"template":"{\"pressure\":{{pressure}}, \"temperature\" : {{temperature}}/10}"`
If you omit the *pressure* field in the template, it will simply not appear in the output.
location

If the decoded value contains a location field with latitude and longitude, it will override the location field provided in Live Objects at the same json level as the value field.

Example:

A loRa message contains an encoded payload. The default location in the message is the location provided by the loRa network.

  "streamId": "urn:lora:xxxxxxxxxxxx1009!uplink",
  "created": "2019-04-05T14:37:12.047Z",
  "location": {
      "provider": "lora",
      "alt": 0,
      "accuracy": 7500,
      "lon": -3.469017,
      "lat": 48.762119
   },
   "model": "model_yyyy",
   "value": {
      "payload": "01005ca767ee00400810",
   },
   "timestamp": "2019-04-05T14:37:08.489Z",

If the device is a tracker, after decoding the payload, the decoded message may have the following format :

  "streamId": "urn:lora:xxxxxxxxxxxx1009!uplink",
  "created": "2019-04-05T14:37:12.047Z",
  "location": {(2)
        "provider": "device", (3)
        "lon": -3.469020,
        "lat": 48.762120
   },
   "model": "model_yyyy",
   "value": {
      "payload": "01005ca767ee00400810",
      "location": {  (1)
            "lon": -3.469020,
            "lat": 48.762120
        },
      "temperature": {
         "unit": "°C",
         "currentTemperature": {
            "values": [
               20.64
            ]
         }
      }
   },
   "timestamp": "2019-04-05T14:37:08.489Z",
1 if the decoded location has the expected "location" format,
2 the decoded location replaces the location provided by the loRa network,
3 and the provider is now "device" instead of "lora"

9.4.5. Javascript decoding

If the payloads are more complex (several payloads, specific calculations/transformations…​), you can develop a scriptable (javascript) decoder. The javascript version must be compliant with ECMA Script 5. We also provide a development framework that can be used with Eclipse or IntelliJ IDE that can ease the development of the decoder. More info in the links below :

9.5. Store service

The REST interface allows to add data to a stream and to retrieve data from a stream. A stream could be for example associated to a unique of device (streamID could be therefore a device Identifier) or one type of data coming from a device (streamID could be therefore in this format deviceIdentifier-typeOfData)

9.5.1. Add a data message to a stream

Request:

POST /api/v0/data/streams/{streamId}
X-API-Key: <your API key>
Accept: application/json

body param

description

data

JSON object conforming to data message structure

Warning: the streamId is provided as the last segment of the url.

Example:

POST /api/v0/data/streams/myDeviceTemperature
{
  "value": {"temp":24.1},
  "model": "data_model_v0"
 }

For this example, the "value.temp" field of model "data_model_v0" will be defined as a double type. If a String type is used in the future for "value.temp", a new model must be defined. In case that "value.temp" is set a String type with model "data_model_v0", the message will be dropped by the search service.

9.5.2. Add a bulk of data messages

Request:

POST "/api/v0/data/bulk
X-API-Key: <your API key>
Accept: application/json

body param

description

data

JSON array conforming to an array of data message structure

A bulk will be be processed if all arrays elements are valid, otherwise the bulk will be rejected. Maximum size of the bulk is 1000.

Warning : the streamId is mandatory for each element of the Bulk. This is a difference with the REST API for adding data to a stream.

Example:

POST /api/v0/data/bulk
[
  {
    "streamId" : "temperature_stream_1"
    "value": {"temp":24.1},
    "model": "data_model_v1"
  },
  {
    "streamId" : "temperature_stream_1"
    "value": {"temp":24.1},
    "model": "data_model_v1"
  },
  {
    "streamId" : "pressure_stream_1"
    "value": {"pressure":1024.0},
    "model": "data_model_v1"
  }
]

9.5.3. Retrieve data from a stream

Request:

GET /api/v0/data/streams/{streamId}
X-API-Key: <your API key>
Accept: application/json

Query params

Description

limit

Optional. max number of data to return, value is limited to 100

timeRange

Optional. filter data where timestamp is in timeRange "from,to"

bookmarkId

Optional. id of document. This id will be used as an offset to access to the data.

Documents are provided in reverse chronological order (newest to oldest).

Example:

GET /api/v0/data/streams/myDeviceTemperature
{
  "id": "57307f6c0cf294ec63848873",
  "streamId": "myDeviceTemperature",
  "timestamp": "2016-05-09T12:15:41.620Z",
  "model": "temperature_v0",
  "value": {
    "temp": 24.1
  },
  "created": "2016-05-09T12:15:40.286Z"
}

The REST request body search API is provided to perform search queries.

To learn more about the search API, read the Exploring your Data section of Elasticsearch: The Definitive Guide.

9.6.1. To perform a search query

9.6.1.1.1. Request
POST /api/v0/data/search
X-API-Key: <your API key>
Accept: application/json

body param

description

dsl request

elasticsearch DSL request

9.6.1.1.2. Example

This query requests statistics from the myDeviceTemperature stream temp field.

Request:

POST /api/v0/data/search
{
  "size": 0,
  "query": {
    "term": {
      "streamId": "myDeviceTemperature"
    }
  },
  "aggs":
  {
    "stats_temperature": {
      "stats": {
        "field": "@temperature_v0.value.temp"
      }
    }
  }
}

If a model has been provided, search query must be prefixed by @<model>: @temperature_v0.value.datapath

Response:

{
  "took": 1,
  "hits": {
    "total": 2
  },
  "aggregations": {
    "stats_temperature": {
      "count": 2,
      "min": 24.1,
      "max": 25.9,
      "avg": 25,
      "sum": 50
    }
  }
}

To perform the same search query; but with the 'hits' part extracted and JSON formated as an array of data messages (to use when you are only interested in the 'hits' part of Elasticsearch answer):

9.6.1.2.1. Request
POST /api/v0/data/search/hits
X-API-Key: <your API key>
Accept: application/json

body param

description

dsl request

elasticsearch DSL request

9.6.1.2.2. Example

This query requests last data for all devices using the model: temperature_v0.

Request:

POST /api/v0/data/search/hits
{
  "size": 10,
  "query": {
    "term": {
      "model": "temperature_v0"
    }
  }
}

Response:

[
  {
    "id": "57308b3b7d84805820b35345",
    "streamId": "myDeviceTemperature",
    "timestamp": "2016-05-09T13:06:03.903Z",
    "model": "temperature_v0",
    "value": {
      "temp": 25.9
    },
    "created": "2016-05-09T13:06:03.907Z"
  },
  {
    "id": "573087777d84805820b35344",
    "streamId": "myDeviceTemperature",
    "timestamp": "2016-05-09T12:49:59.966Z",
    "model": "temperature_v0",
    "value": {
      "temp": 24.1
    },
    "created": "2016-05-09T12:49:59.977Z"
  },
  {
    "id": "5730b1577d84805820b35347",
    "streamId": "myStreamDemo-temperature",
    "timestamp": "2016-05-09T15:48:39.390Z",
    "model": "temperature_v0",
    "value": {
      "temp": 24.1
    },
    "created": "2016-05-09T15:48:39.395Z"
  }
]

9.6.2. Geo Queries

Geo Query can be performed through all fields with name matching location (case insensitive).
In order to geoquery these fields, you must add @geopoint to the location query path: location@geopoint.

Request:

POST /api/v0/data/search/hits
{
  "query": {
            "bool" : {
            "must" : {
                "match_all" : {}
     },
     "filter": {
        "geo_distance": {
          "distance": "10km",
          "location.@geopoint": {
            "lat": 43.848,
            "lon": -3.417
          }
        }
      }
    }
  }
}

Response:

[
  {
    "id": "57308b3b7d84805820b35345",
    "streamId": "myDeviceTemperature",
    "location": {
      "lat": 43.8,
      "lon": -3.3
    }
    "timestamp": "2016-05-09T13:06:03.903Z",
    "model": "temperature_v0",
    "value": {
      "temp": 25.9
    },
    "created": "2016-05-09T13:06:03.907Z"
  }
]

9.6.3. Search Query samples

Here are some query samples that can be used. Aggregations are very usefull to retrieve data grouped by any criteria: list all known tags, get all last value per stream, get mean temperature per tag, get the list of streams that have not send data since a date…​ The aggregations results are stored as 'buckets' in the result.
You can also add filters (geoquery, wildcards, terms…​) to all your aggregations query to target specific 'buckets' or data.

9.6.3.1. Give me all you got !

Request:

{
  "query": {
    "match_all": {}
  }
}
9.6.3.2. Give me the list of all known tags

Request:

{
  "size": 0,
  "aggs": {
    "grouped_by_tags": {
      "terms": {
        "field": "tags",
        "size": 0
      }
    }
  }
}

Response:

{
  "took": 44,
  "hits": {
    "total": 66
  },
  "aggregations": {
    "grouped_by_tags": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "tag_1",
          "doc_count": 53
        },
        {
          "key": "tag_2",
          "doc_count": 13
        }
      ]
    }
  }
}
9.6.3.3. Give me the last value of all my streams

Request:

{
  "size":0,
  "aggs": {
    "tags": {
      "terms": {
        "field": "streamId",
        "size": 0
      },
      "aggs": {
        "last_value": {
          "top_hits": {
            "size": 1,
            "sort": [
              {
                "timestamp": {
                  "order": "desc"
                }
              }
            ]
          }
        }
      }
    }
  }
}

Response:

{
  "took": 19,
  "hits": {
    "total": 11
  },
  "aggregations": {
    "tags": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "device_1",
          "doc_count": 7,
          "last_value": {
            "hits": {
              "total": 7,
              "max_score": null,
              "hits": [
                {
                    ...
                }
              ]
            }
          }
        },
        {
          "key": "device_2",
          "doc_count": 123,
          "last_value": {
            "hits": {
              "total": 123,
              "max_score": null,
              "hits": [
                {
                    ...
                }
              ]
            }
          },
         ...
        }
      ]
    }
  }
}
9.6.3.4. Give me the list of devices that have not send data since 2017/03/23 10:00:00

Request:

{
  "size":0,
  "aggs": {
    "tags": {
      "terms": {
        "field": "streamId",
        "size": 0
      },
      "aggs": {
        "last_date": {
          "max": {
            "field": "timestamp"
          }
        },
        "filter_no_info_since": {
          "bucket_selector": {
            "buckets_path": {
              "lastdate":"last_date"
            },
            "script": {
              "inline": "lastdate<1490263200000",
              "lang":"expression"
            }
          }
        }
      }
    }
  }
}

Response:

{
  "took": 8,
  "hits": {
    "total": 9
  },
  "aggregations": {
    "tags": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "device_12",
          "doc_count": 7,
          "last_date": {
            "value": 1489504105020,
            "value_as_string": "2017-03-14T15:08:25.020Z"
          }
        },
        {
          "key": "device_153",
          "doc_count": 2,
          "last_date": {
            "value": 1489049619254,
            "value_as_string": "2017-03-09T08:53:39.254Z"
          }
        }
      ]
    }
  }
}

9.7. Kibana

Kibana is a tool to visualize all the data injected in Live Objects.

On the first connection, you will be redirected to the index pattern screen.
Keep all options as default and choose 'timestamp' for the 'Time-field name' box. Press Create button, this will create a new index pattern for Kibana.

Do not check 'Do not expand index pattern when searching' or 'Use event times to create index names' checkboxes, this will lead to error messages in later screens. In that case, you should delete the 'index-pattern' created, and recreate a new one without these options.

If you add new fields in your data model, you will need to refresh this index pattern in order for kibana to be able to use these new fields.
Just go to the Settings tab and click on the refresh field list orange button on the top.

Kibana is based on 3 main tabs : Discover, Visualize and Dashboard.

9.7.1. Discover

Here you will access to all your data. The idea is to 'play' with the filters on the left side of the screen to extract usefull data you need to explore.
You can then save this filtered 'search' to visualize it in the next visualize screen.

There is an important time filter on the upper-right corner of the screen. By default, it will display only last 15 minutes of data. You can choose to display last month data for instance.

9.7.2. Visualize

Here you will be able to create histogram, map, charts, table, metrics; based on your previous search. You can then save this visualization to be displayed in the next dashboard screen.

9.7.3. Dashboard

Here you will be able to display the visualization tab you have previously created; and gather them all in a dashboard page. You can create and save several dashboards meant for different users. You can share these dahsboards with the share button.

9.7.4. Example

For this example we will use data of a device that measures temperature and we will create a line chart of the temperature through time.

First, go to the Discover tab. In the search bar tap the identifier of your device surrounded by *, for example: *myDevice*.
You should find your device’s messages in the results. Expand one of your device’s messages, we will observe the following fields:

{
    "metadata":{
        "source": "urn:lo:nsid:test:myDevice" (1)
    },
    "timestamp": "2017-11-28T16:55:56.972Z", (2)
    "model": "temperature_v0" (3)
    "value":{
        "temperature": "25.5" (4)
    }
}
1 the URN of your device
2 the date of the message
3 the data model of the "value" field
4 the temperature value measured by your device

Fields in "value" are indexed by Elasticsearch using the following format: {model}.value.{field}.
For example the "temperature" field in the previous message is indexed as "@temperature_v0.value.temperature".

Go to the Visualize tab, select Line chart then From a new search.

On the left panel, expand the Y-axis and select Average in the Aggregation, and the indexed temperature field in Field, in our example it is "@temperature_v0.value.temperature".
You can write a title such as "Temperature in Celsius" in the CustomLabel text field.

Click on X-axis and select Date Histogram in the Aggregation.
You can write a title such as Date in the CustomLabel text field.

Then click on the green Play button. The chart shows the temperature through time.

10. Message routing

To configure your message selection process routing your data message, you must configure :

  • Your Message selector to set a filter/s wich can filtering your data flow on your data steram.

  • Your action policies (your triggers and associated actions) to select a message when the filter met a selection criteria in your data message.

  • Your Output target, to set where your data can be routed.

10.1. Message selector configuration

Triggers, within an action policy, enable downstream action execution, passing the event or the message to the specified actions, the triggers work also with rule processing.

Depending on the trigger, the action will have to handle different kind of data format.

This is a basic trigger that enables to route the data messages to the desired actions. Currently message can have 1 origin :

data_new

This is the standard source of Data Message that is going through the Live Objects platform.

Representation of a message selector trigger
{
  "messageSelector": {
      "origin": "data_new",
      "filter": {
        "connectors": [
          "mqtt","sms"
        ],

  "groupPaths": [
          {
            "includeSubPath": false,
            "path": "/europe/france"
          }
        ]
      }
    }
}

Message Selector

Message Selector will select messages coming from origin and apply 3 filters with a "AND" logic operators: groupPaths, connectors, deviceIds.

In the previous example, the Data Message will be selected if the message has been emitted though "mqtt" OR "sms", AND belongs to "/europe/france" group.

origin

Select the source of the message stream. This is an enumeration whose values are within the following range [data_new].

filter.groupPaths

Filter the messages coming from the selected origin on group. The message will pass through the filter when at least one group matches ("OR" logic).

filter.connectors

Filter the messages coming from the selected origin on network connector ("http","lora", "mqtt", "sms"). The message will pass through the filter when at least one connector element matches ("OR" logic).

filter.deviceIds

Filter the messages coming from the selected origin on deviceId. The message will pass through the filter when at least one deviceId matches ("OR" logic).

10.2. Action policy used in message routing

The relationship between a trigger and actions is called an action policy. To configure your action policy you must set filter/s on your message selector trigger. The format msut respect a Json logic syntax.

To create a new action policy :

Endpoint:

POST /api/v0/event2action/actionPolicies
An action policy has the following top level data representation :
{
  "id": "Message_selection",
  "name": "on message selected sending email",
  "enabled": true,
  "triggers": {
    "messageSelector": {
      "origin": "data_new",
  },
  "actions": {
    "emails": [{
            "to": [
              "name.last@yourdomain.com"
            ],
            "subjectTemplate": "<text> + <mustache template>",
            "contentTemplate": "<text> + <mustache template>"
          }
    ],
    "sms": [],
    "httpPush": [],
    "fifoPublish": []
  }
}

In this example a simple Message based trigger is used.

To retrieve your action policy :

Endpoint:

GET /api/v0/event2action/actionPolicies/Message_selection
field name is required description

id

optional

Defines the id of the action policy

name

optional

Defines a user friendly name for the action policy

enable

required

Enables or disables the action policy

triggers

required

Defines the type of trigger that will start an action. It can be a List of event rule ids OR a message selector.
Note: triggers object should have exactly one trigger defined.

triggers.eventRuleIds

optional

A list of rule ids as String

triggers.messageSelector

optional

A MessageSelector object (see Message selector configuration section)

actions

required

An Actions object that defines the action that will be started upon a trigger activation
Note: actions object should not be empty. At least one action should be defined.

actions.emails

optional

A collection of Email actions (see Email notification section)

actions.sms

optional

A collection of SMS actions (see SMS notification section)

actions.httpPush

optional

A collection of Http Push actions (see Http Push action section)

An example of an action policy with a message trigger from the data_new source and the output message is routed to the FIFO.
{
  "id": "push-to-fifo",
  "name": "push data fifo",
  "enabled": true,
  "triggers": {
    "eventRuleIds": [],
    "messageSelector": {
      "origin": "data_new",
      "filter": {
        "groupPaths": [],
        "connectors": [],
        "deviceIds": [
          "<deviceId>"
        ]
      }
    }
  },
  "actions": {
    "emails": [],
    "sms": [],
    "httpPush": [],
    "fifoPublish": [
        {
            "fifoName": "<your fifo name>"
        }
    ]
  }
}
The same example of an action policy with a message trigger from the data_new source and the output message is pushed to a web site using HTTP Push action.
{
  "id": "push-to-fifo",
  "name": "push data fifo",
  "enabled": true,
  "triggers": {
    "eventRuleIds": [],
    "messageSelector": {
      "origin": "data_new",
      "filter": {
        "groupPaths": [],
        "connectors": [],
        "deviceIds": [
          "<deviceId>"
        ]
      }
    }
  },
  "actions": {
    "emails": [],
    "sms": [],
    "httpPush": [
    {
        "webhookUrl": "https://webhook.site/....",
        "headers": {
          "authorization": [
            "Bearer 00000000-0000-0000-0000-000000000000"
          ]
        },
        "content": "<text>+<mustache template>",
        "retryOnFailure": true
      }
    ],
    "fifoPublish": []
  }
}

10.3. Output target

10.3.1. FIFO publish action

This action is self explanatory and serve the purpose of publishing to a FIFO when a trigger is activated. FIFO publish is a routing feature without mustach templating.

Representation of a FIFO action
{
        "fifoName": "destinationfifo"
}

fifoName

Name of the FIFO that will receive the triggered message

Example of an action policy sending all new data messages to a FIFO
{
        "id": "push-to-fifo",
    "name": "push data new to fifo",
    "enabled": "true",
    "triggers": {
            "messageSelector": {
                    "origin": "data_new"
            }
    },
    "actions": {
            "fifoPublish": [
                    {
                            "fifoName": "mydata"
                    }]
    }
}

10.3.2. Http Push action

This action enables to do an HTTP POST to a given URL with custom headers and a custom body. The body can be templated using the Mustache templating syntax. Depending on the user needs, a retry policy can be enabled to be partially resilient to listening endpoint downtimes.

Representation of a Http Push action
{
     "url": String,
     "headers": Map of <String,List of String> ({"key": ["value1","value2",...]}),
     "retryOnFailure": boolean,
     "jsonPath": String,
     "content": String
}
field name is required description

webhookUrl

required

defines the location of the webhook (only the ports 80, 443, 8080, 8443 and 9243 are allowed).

headers

optional

defines custom headers that are sent along the HTTP POST request.
This can be used to pass authorization headers or any header required to connect to an endpoint.

retryOnFailure

optional

Enables retry on failure policy (see related section)

jsonPath

optional

Select a part of the triggering data using a jsonPath syntax. This selected data will be used as the root of the template datacontext. This field can be omitted when you what to work with the whole data in template.

content

optional

A string representing a Mustache template. This template will be rendered as the request body.

Example of Http Push action on a message selector trigger
{
    "id": "some_id",
    "name": "some_user_friendly_name",
    "enabled": true,
    "triggers": {
        "messageSelector": {
          "origin": "data_new"
        }
    },
    "actions": {
        "httpPush": [{
            "webhookUrl": "https://hooks.myservice.com/services/SOMEWEBHOOKREFERENCE",
            "headers": {"authorization": ["Bearer 00000000-0000-0000-0000-000000000000"]},
            "retryOnFailure": true,
            "content": "{\"text\": \"data pushed to stream : {{streamId}}, at {{timestamp}}\"}"
        }]
    }
}
10.3.2.1. Http Push generated headers
x-orange-lo-policy-id

Every Http Push request contains a header with the action policy id.

If an action policy is provisioned with the id lo-policy-id. The http push will have the following header :

x-orange-lo-policy-id: lo-policy-id
x-orange-lo-message-id

Every Http Push request contains a header with an id that identifies that push uniquely (this id will be kept accross retries in case of errors). This id can be used to identify if a message has been delivered multiple times to the webhook.

x-orange-lo-message-id: 00000000-0000-0000-0000-000000000000

Live Objects will try to validate the body of the HTTP POST as json and will set the request Content-Type header accordingly :

  • application/json if the body contains valid json

  • test/plain otherwise

10.3.2.2. Http Push retry policy and error management

When an error message is sent back by the destination resource, a retry policy will occur (if enabled when creating the Http Push action). The Http Push action will be retried depending on the HTTP error code sent back from the destination resource to Live Objects.

Table 8. Error handling according to HTTP ERROR
HTTP ERROR CLASS Error handling mode

4xx

Unrecoverable error : discard Http Push action

5xx

Recoverable error : retry Http Push action execution (if retryPolicy is true) otherwise discard the Http Push action execution

When enabled, the retry policy will try to regurlarly post a request to the resource referenced by the Http Push action url parameter. These retries will be attempted over twenty five hours and one minute with the following pattern :

Table 9. Retries and delays (in order)
# of retry delay between retry

12

5 seconds

60

1 minute

24

1 hour

In the case where the retries are unsuccessful, the Http Push action execution is discarded.

10.3.2.3. Message Overflow

If the targeted server fot Http Push (webHook) is down or slow, Http push request will be buffered. Buffer size can contain up to 10 000 requests. If this limit is reached, oldest request will be discarded first and an Http push result message emited on bus.

10.3.2.4. Statistics

Successful deliveries and discarded message counters are displayed in the datazone section of the Live Objects statistics endpoint

10.3.2.5. Security considerations

A secure resource endpoint (HTTPS) should always be used. Using a non secure endpoint (HTTP) is a security issue as the data exchange between Live Objects and the resource endpoint will not be encrypted. This is especially true when leveraging the custom headers mechanisms to pass authentication token.

As a general recommendation, when using a custom authorization headers, consider it unsecure and be sure to give the token/credentials the least permissions. It should only be able to do an HTTP POST on the defined resource location.

10.3.2.6. Rate limiting

Http Push action execution is rate limited according to your current offer. The rate limiting occurs when the rate of outgoing http pushes exceeds its limit over a given time windows (offer dependent).

Exceeding traffic will be discarded.

10.3.2.7. Http Push Results

Each Http Push action will generate reports on bus. This can help analysing traffic problems.

The solution is to:

  • use the Audit Log REST APIs or web portal (only ERROR logs are visible)

{
        "tenantId": "httppush550b5cd6-b8c6-49a4-baa9-d3c21a84202d",
        "httpPushRequest": {
                "url": "http://localhost:14567",
                "retryOnFailure": false,
                "headers": null,
                "requestBody": "{\"x\":0}",
                "messageUUID": "b5540f77-bbc8-42c8-9942-35c0b133a689"
        },
        "success": true,
        "responseStatusCode": 200,
        "responseBody": "ok",
        "errorMessage": ""
}
10.3.2.8. About HTTPS certificate

When using https to push data to a webhook, the certification chain must be valid according to the default Java 8 JVM implementation. Any self signed certificate or certificate issued by an unknown orginization (according to the JAVA 8 JVM) presented by a webhook will be refused and the http push action will not be executed.

10.3.2.9. About HTTP/HTTPS authorized ports by Live Objects

The list of ports authorized by liveobjects during a http push action, are : 443 and 8443 (HTTPS), 80 and 8080 (HTTP), 9243 for ES. Other ports are blocked.

If you have a 403 forbidden error, you need to check if you have using a right port number. In the other hand, the remote server must allow the port number that you are using.

10.4. Examples

These examples aim at giving a wide range of example to enable an easy implementation of common use cases for users of the triggers and action service.

10.4.1. Post all your data streams on elastic cloud

Even though Live Objects offers you the ability to store and explorer your data leveraging the power of Elastic Search and Kibana, you may want to keep your data on your dedicated space on Elastic’s Cloud.
This example will demonstrate how to route all your incoming data into your elastic search instance and store only the value part of the data.

Pushing data to elastic search cloud needs to pass credentials through basic auth mechanism. This example will demonstrate how to leverage custom headers mechanism to pass credentials for authentication.

1. Create the dedicated action policy
Given
  • a dedicated Elastic instance at url myinstance.cloud.es.io:9243

  • an elastic cloud username liveobjects with password rocks

  • an elastic search index my_lo_example

  • an elastic search type my_lo_type

PUT /api/v0/event2action/actionPolicies/lo-elastic-cloud
{
    "name": "push data to elastic cloud",
    "enabled": "true",
    "triggers": {
            "messageSelector": {
                    "origin": "data_new"
            }
    },
    "actions": {
            "httpPush": [
                    {
                            "webhookUrl": "https://myinstance.cloud.es.io:9243/my_lo_example/my_lo_type",
                            "headers": {"Authorization": ["Basic bGl2ZW9iamVjdHM6cm9ja3MK="]},
                "jsonPath": "value"
                    }]
    }
}

Connecting the http push action to a service requiring a JWT token will only need to change the Authorization header to Bearer and pass the token.

2. Push some data
POST /api/v0/data/streams/my-stream
{
    "location": {
      "provider": "lora-macro",
      "accuracy": 10,
      "lon": -122.169846,
      "lat": 37.602902},
   "model": "lora_v0",
   "value": {
      "payload": "ae2109000cf6",
      "customMetadata": { "name": "sensor1" }
   },
   "tags": [
      "San Francisco", "Tracker"
     ]
}
3. Check data on elastic cloud
curl -X POST -u liveobjects:rocks https://myinstance.cloud.es.io:9243/my_lo_example/my_lo_type/_search -H 'Content-Type: application/json' -d '{ "query": {"match_all": {}}}'

10.4.2. Select data streams from a group of devices

To be presented soon

10.4.3. Select a stream and post it on IFTTT webhook

To be presented soon

11. Live Objects messages data model

Live Objects platform relies on message processing to provide services such as storing things datas, detecting events, managing devices. These messages can be injected and listened by various means : Web Portal, HTTPS APIs, MQTT, WebHook. This is why it is important to understand live Objects messaging model in order to build business use cases on top of it.

This section will provide a global overview of message types, please refer to specific section for detailed data model.

11.1. Data Message

Inside Live Objects platform, Data Message carries datas sent by devices through network interfaces ( HTTPS, MQTT, LoRa®, SMS, ..). These messages will be stored and processed by other services.

Data Message format has a key role inside Live Objects as a common input/output format for storing, searching, processing, and routing features.

Note that even if devices can publish their custom data to Live Objects with a specific format depending on the network interface used, Data Message will be used as a common format inside the platform.

Data Message example
{
   "metadata": {
      "connector": "mqtt",
      "source": "urn:lo:nsid:<my_stream>",
      "group": {
         "path": "/",
         "id": "<my_path>"
      },
      "network": {
         "mqtt": {
            "clientId": "urn:lo:nsid:<my_mqtt_client_id>"
         }
      }
   },
   "streamId": "<streamId>",
   "created": "2019-06-26T08:03:09.394Z",
   "extra": {},
   "location": {
      "provider": null,
      "alt": null,
      "accuracy": null,
      "lon": 2.30886,
      "lat": 48.81839
   },
   "model": "demo",
   "id": "<id>",
   "value": {
      "revmin": 9992,
      "CO2": 683,
      "doorOpen": false,
      "hygrometry": 45,
      "temperature": 139,
      "pressure": 1367
   },
   "timestamp": "2019-06-26T08:03:09.390Z",
   "tags": []
}
Lora Data Message example
    "streamId": "urn:lora:xxxxxxxxxxxx1009!uplink",
        "created": "2019-04-05T14:37:12.047Z",
        "location": {
            "provider": "lora",
                "alt": 0,
            "accuracy": 7500,
            "lon": -3.469017,
        "lat": 48.762119
        },
    "model": "model_yyyy",
    "value": {
    "payload": "01005ca767ee00400810",
   },
   "timestamp": "2019-04-05T14:37:08.489Z",

11.2. Json encoded data model

This JSON encoding aims to model the data collected from IoT things (devices, etc.).

streamId

string uniquely identifying a timeseries / "stream",

timestamp

date/time associated with the collected information,

location

geo location (latitude and longitude) associated with the collected info,

model

string used to indicate what schema is used for the value part of the message, more about model field

value

structured representation (JSON object) of the transported info,

tags

list of strings associated with the message to convey extra-information,

metadata

section controlled/enriched by Live Objects

  • source: unique identifier of the source device, usually an URN (urn:lo:nsid:<namespace>:<id>)

  • group: group to which the device belongs, defined by its id and its path

  • connector: entry point of the payload: lora, mqtt, …​

  • network: information from the network, depends on the connector

Message must be published to :

  • with device mode to dev/data

  • with application mode for payload to fifo/<fifo name>(…​),

Example #1 - data collected from MQTT
{
   "streamId": "urn:lo:nsid:mqtt:61d2a520-c153-4ec8-a47e-fee21f4eee82!atmos",
   "timestamp": "2016-03-08T10:02:44.907Z",
   "location": {
      "lat": 44.1,
      "lon": -1.5,
      "alt": 5.2,
      "accuracy": 10.0,
      "provider": "device"
   },
   "model": "atmos_v0",
   "value": {
      "temp": 17.25,
      "humidity": 12.0
   },
   "tags": [ "City.Lyon", "Model.atmosV0" ],
   "metadata": {
      "source": "urn:lo:nsid:mqtt:61d2a520-c153-4ec8-a47e-fee21f4eee82",
      "group": {
         "id": "root",
         "path": "/"
      },
      "connector": "mqtt"
   }
}
Example #2 - data collected from LoRa®
{
   "streamId": "urn:lo:nsid:lora:7A09AEF7E097A7EF",
   "timestamp": "2016-03-08T10:02:43.944Z",
   "location": {
      "lat": 44.1,
      "lon": -1.5,
      "alt": 5.2,
      "accuracy": 10.0,
      "provider": "device"
   },
   "model": "lpwa_v1",
   "value": {
      "payload": "a3e1eff054"
   },
   "tags": [ "City.Lyon", "Model.LoraMoteV1" ],
   "metadata": {
      "source": "urn:lo:nsid:lora:7A09AEF7E097A7EF",
      "group": {
         "id": "root",
         "path": "/"
      },
      "connector": "lora",
      "network": {
         "lora": {
            "devEUI": "7A09AEF7E097A7EF",
            "port": 1,
            "fcnt": 138,
            "rssi": -111,
            "snr": -6,
            "sf": 8,
            "signalLevel": 5,
            "gatewayCnt": 4,
            "location": {
               "lat": 44.1,
               "lon": -1.5,
               "alt": 5.2,
               "accuracy": 10.0,
               "provider": "device"
            }
         }
      }
   }
}

11.3. Data Message constraints

There are some constraints that a Data Message should follow :

Field Constraint

timestamp

should follow ISO-8601 date time format. value allowed by network.

created

value allowed by Live Objects.

streamId

should not contain following characters: ' " \ ; { } ( ) and ' ' (space).

model

can be empty, but can not contains ' ' (space) or '.' (dot) character

value

field can be empty, but if it exists it must be compliant with following constraints :

  • be a JSON object (not a primitive like number or string).

  • do not contains field name with '.' (dot) character.

  • its size in bytes can not exceed 1 MiB (1024*1024 bytes).

When the Lora device send message to Live Objects, the network set timestamp value and when the massage is processed by Live Objects, it set the created field value by his own timer. Liev Objects does not process messages according to the chronological order of transmission (timestamp value set by the network provider timestamp) but according to the order of processing (created value set by Live objects).

11.4. Event message models

11.4.1. Fired Event message model

The Data Message flow goes through the Simple Event Processing service. This service can apply custom binary logics ( written in jsonLogic syntax ) on each message to detect specific events. Theses events will be sent as Live Objects Fired Event Messages according to firing policies.

Fired Event example
{
   "metadata": null,
   "streamId": "event:<streamId>",
   "created": "2019-04-15T07:43:26.491Z",
   "extra": null,
   "location": {
      "provider": null,
      "alt": null,
      "accuracy": null,
      "lon": 2.29413,
      "lat": 48.80121
   },
   "model": "event:demo",
   "id": "<message_id>",
   "value": {
      "matchingContext": {
         "matchingRule": {
            "dataPredicate": "{\">\":[{\"var\":\"value.temperature\"},30]}",
            "name": "Temparature threshold",
            "id": "<rule_id>",
            "enabled": true
         },
         "data": {
            "metadata": {
               "connector": "mqtt",
               "source": "urn:lo:nsid:<streamId>",
               "group": {
                  "path": "/",
                  "id": "<my_path>"
               },
               "network": {
                  "mqtt": {
                     "clientId": " urn:lo:nsid:<my_device_id>"
                  }
               }
            },
            "streamId": "<streamId>",
            "extra": {},
            "location": {
               "lon": 2.29413,
               "lat": 48.80121
            },
            "model": "demo",
            "value": {
               "revmin": 4831,
               "CO2": 488,
               "doorOpen": false,
               "hygrometry": 54,
               "temperature": 77,
               "pressure": 976
            },
            "timestamp": "2019-04-15T07:43:26.475Z",
            "tags": []
         },
         "tenantId": "0000000000000000",
         "timestamp": "2019-04-15T07:43:26.492Z"
      },
      "tenantId": "5b115b4891fd9904ab33e478",
      "timestamp": "2019-04-15T07:43:26.493Z",
      "firingRule": {
         "name": "Trigger of temperature threshold",
         "matchingRuleIds": [
            "<matching_rule_id>"
         ],
         "id": "<firing_rule_id>",
         "aggregationKeys": [
            "value.temperature"
         ],
         "firingType": "ALWAYS",
         "enabled": true
      }
   },
   "timestamp": "2019-04-15T07:43:26.493Z",
   "tags": [
      "event"
   ]
}

11.4.2. State Change Event message model

The Data Message flow goes through the State change Processing service. This service can detect changes in devices states, for example geozone location changes. Theses events will be sent as Live Objects State Change event Messages.

Reference, see State Processing section

State Change Event example
{
   "metadata": null,
   "streamId": "event:<streamId>",
   "created": "2019-06-26T08:03:05.412Z",
   "extra": null,
   "location": {
      "provider": null,
      "alt": null,
      "accuracy": null,
      "lon": 2.30866,
      "lat": 48.81822
   },
   "model": "event:demo",
   "id": "<message_id>",
   "value": {
      "stateProcessingRuleId": "<state_processing_rule_id>",
      "data": {
         "metadata": {
            "connector": "mqtt",
            "source": "urn:lo:nsid:<stream_id>",
            "group": {
               "path": "/",
               "id": "root"
            },
            "network": {
               "mqtt": {
                  "clientId": "urn:lo:nsid:<mqtt_client_id>"
               }
            }
         },
         "streamId": "<stream_id>",
         "extra": {},
         "location": {
            "lon": 2.30866,
            "lat": 48.81822
         },
         "model": "demo",
         "value": {
            "revmin": 9992,
            "CO2": 683,
            "doorOpen": false,
            "hygrometry": 60,
            "temperature": 139,
            "pressure": 1367
         },
         "timestamp": "2019-06-26T08:03:05.398Z",
         "tags": []
      },
      "tenantId": "0000000000000000000",
      "newState": "Level of Hygrometry normal",
      "stateKey": "<state_key>",
      "previousState": "Level of Hygrometry high",
      "timestamp": "2019-06-26T08:03:05.408Z"
   },
   "timestamp": "2019-06-26T08:03:05.408Z",
   "tags": [
      "event"
   ]
}

11.4.3. Activity Event message model

The Data Message flow goes through the Activity Processing service. This service can detect device inactivity. Theses events will be sent as Live Objects Activity Event Messages.

Activity Event example
{
  "deviceId": "urn:lo:nsid:dongle:00-14-22-01-23-45",
  "state":"SILENT",
  "timestamp": "2018-04-24T08:29:49.029Z",
  "activityRule": {
    "id": "<my_activity_rule_id>",
    "name": "devices in Lyon",
    "enabled": true,
    "silentPolicy": {
      "duration" : "P1D",
      "repeatInterval" : "PT12H"
    },
    "targets": {
      "groupPaths" : [
        {
          "path" : "<my_path>",
          "includeSubPath" : true
        }
      ]
    }
  }
}

11.5. Https Push Result

This result refer to HttpPush actions. This will generate HttpPushResult message containing webHook push result.

Https Push Result example
{
        "tenantId": "httppush550b5cd6-b8c6-49a4-baa9-d3c21a84202d",
        "httpPushRequest": {
                "url": "http://localhost:14567/webHook",
                "retryOnFailure": false,
                "headers": null,
                "requestBody": "{\"x\":0}",
                "messageUUID": "b5540f77-bbc8-42c8-9942-35c0b133a689"
        },
        "success": true,
        "responseStatusCode": 200,
        "responseBody": "ok",
        "errorMessage": ""
}

12. RESTful and MQTT API’s for business applications

When your device publish data’s to Live Objects, you have two way to retrieve them :

  • mode RESTful API: dedicated only to application that will retrieve Data and Events using HTTPS protocol and Restful API architecture design.

  • mode "Application": dedicated only to application that will retrieve Data and Events from FIFO queues (pushed by Notifications rules)

The restriction about the first "mode" is that it’s statless. In the second one MQTT clients can create a stateful session with the queues and without loss data (persitent sessions).

12.1. REST API

12.1.1. Endpoints

https://liveobjects.orange-business.com/api/

Currently, two version coexist "v0" and "v1". As a consequence all methods described in this document are available on URLs starting by:

https://liveobjects.orange-business.com/api/v0/

or

https://liveobjects.orange-business.com/api/v1/

Some v0 methods have been deprecated and replaced by v1 methods.

12.1.2. Principles

All the functionalities of Live Objects are available on the REST APIs. It is the same API that is used by the Live Objects web portal.

12.1.3. Content

By default all methods that consume or return content only accept one format: JSON (cf. http://json.org ).

As a consequence, for those methods the use of HTTP headers Content-Type or Accept with value application/json is optional.

12.1.4. API-key authentication

Clients of the Live Objects Rest API are authenticated, based on an API key that must be provided with any request made to the API.

This API key must be added has a HTTP header named X-API-Key to the request.

Example (HTTP request to the API):

GET /api/v1/deviceMgt/devices HTTP/1.1
Host: <base URL>
X-API-Key: <API key>

If you don’t provide such an API key, or if you use an invalid API key, Live Objects responds with the standard HTTP Status code 403 Forbidden.

12.1.5. Paging

Some methods that return a list of entities allow paging: the method doesn’t return the full list of entities, but only a subset of the complete list matching your request.

Two standard query parameters are used in the list method (i.e. that must be added at the end of the URL, after a ?, separated by a &, and defined like this: <param>=<value>). These parameters depend on API version.

12.1.5.1. API v0
  • size: maximum number of items to return (i.e. number of items per page),

  • page: number of the page to display (starts at 0).

Those parameters are not mandatory: by default page will be set to 0 and size to 20.

Example:

  • If size=10 and page=0 then item number 0 to 9 (at most) will be returned.

  • If size=20 and page=1, then items number 20 to 39 (at most) will be returned.

Example (HTTP request to the API):

GET /api/v0/vendors/lora/devices?page=100&size=20 HTTP/1.1
Host: <base URL>
X-API-Key: <API key>

The responses of such methods are a “page” of items - a JSON object with the following attributes:

  • totalCount: total number of entities matching request in service (only part of them are returned),

  • size: the value for “size” taken into account (can be different of the one in request if the value was invalid),

  • page: the value for “page” taken into account (can be different of the one in request if the value was invalid),

  • data: list of returned entities.

12.1.5.2. API v1
  • limit: maximum number of items to return (i.e. number of items per page),

  • offset: number of items to skip (starts at 0).

Those parameters are not mandatory: by default offset will be set to 0 and limit to 20.

Example:

  • If limit=10 and offset=0 then item number 0 to 9 (at most) will be returned.

  • If limit=20 and offset=1, then items number 1 to 20 (at most) will be returned.

Example (HTTP request to the API):

GET /api/v1/deviceMgt/devices?limit=100&offset=20 HTTP/1.1
Host: <base URL>
X-API-Key: <API key>
X-Total-Count: <boolean>

The responses of such methods are a list of items. If request header X-Total-Count is true, the total count of items will be present in the HTTP response header.

Response HTTP headers:

  • X-Result-Count: number of items return in the response

  • X-Result-Limit: the value for “limit” taken into account

  • X-Total-Count: Optional. total number of entities matching request in service

12.1.6. HTTPS REST API documentation

All HTTPS REST methods (device management, data management and bus management, etc.) are described in the HTTPS REST API documentation available here : https://liveobjects.orange-business.com/swagger-ui/index.html.

12.2. Using FIFO

Live Objects "message bus" is the central layer between the Connectivity layer and Service layer.

This message bus offers various modes:

  • FIFO : the solution to set up a point to point messaging and guarantee that the messages are delivery to the consumer. Messages are stored in a queue on disk until consumed and acknowledged. Each message is delivered to only one consumer, when multiple consumers are subscribed to the same queue concurrently, messages are load-balanced between available consumers. More info: FIFO mode,

  • Router : adapted to situations where publisher don’t know the destination of the messages. Messages can be either consumed with transient subscriptions or Static Bindings can be declared to route messages into FIFO queues. Support of "router mode" ended.

  • PubSub : a good fit for real-time exchanges. Message are broadcast to all available subscribers or dropped. Support of "PubSub mode" ended..

For example, on the Live Objects MQTT interface, a publication to MQTT topic "fifo/<your fifo name>" is translated into a message publication on the Live Objects message bus on "fifo/<your fifo name>" topic.

Tenants are free to use Fifo topics to achieve the communication patterns they need between their devices and applications.

12.2.1. FIFO mode / queues

Communications in FIFO mode is based on the usage of topics.

There are no conflict between the naming of PubSub topics and FIFO topics: the PubSub topic "test" is different from FIFO topic "test".

Messages published on a FIFO topic are persisted until a subscriber is available and acknowledges the handling of the message. If multiple subscribers consume from the same FIFO topic, messages are load balanced between them. Publication to and consumption from a FIFO topic use acknowledgement, ensuring no message loss. Before being used, a FIFO topic must be created from the Live Objects web portal.

FIFO mode

landing

  • On the left, a client publishes in FIFO topic/queue "fifo01" while no consumer is subscribed. The message is stored into the queue, on disk. When later a consumers subscribes to the FIFO topic/queue, the message will be delivered. The message will only disappear from disk once a subscriber acknowledges the reception of the message.

  • On the right, a client publishes on FIFO topic/queue "fifo02" while a consumer is subscribed: the message is stored on disk and immediately delivered to the consumer. The message will only disappear from disk once a subscriber acknowledges the reception of the message. When a consumer that received the message but didn’t acknowledged the message unsubscribes from the topic/queue, the message is put back into the "fifo02" queue and will be delivered to the next available consumer.

FIFO are size-limited. The maximum size is given in bytes. Messages will be dropped from the front of the queue to make room for new messages once the limit is reached meaning that the older messages will be dropped first.

The total number of FIFO and the sum of the size of the FIFO is limited depending on your offer.

For more info about limition, see "Limitations" chapter.

12.3. "Application" mode

The "application" mode purpose is to retrieve messages available in your FIFOs.

Typically, use this mode when you want to consume flows of data collected by Live Objects.

The FIFO you connect to must be declared using the FIFO APIs.

To push data toward these FIFOs, you must use the Fifo publish action rules.

12.3.1. Connection

When initiating the MQTT connection, to select the "application" mode you must use the following credentials:

  • clientId : any string between 1 and 128 characters long, containing no spaces or special characters except :, - or _ . The MQTT protocol indicates it must be unique per connection, but LiveObjects does not enforce it.

  • username : application

  • password : a valid API Key with BUS_R role

12.3.2. Summary

In "application" mode, you can subscribe:

  • fifo/{fifoName}, directly consume from a specific FIFO queue. This works only if the used API key has no restriction or if the FIFO queue is specified in the API key’s restriction list.

Direction Topic Description

Subscribe

fifo/{fifoName}

Consume this fifo messages

12.3.3. FIFO subscription

To subscribe to a FIFO queue, a MQTT client connected in "application" mode must subscribe to the following MQTT topic:

fifo/{fifoName}

where fifoName is the identifier of the targeted FIFO queue.

This works only if the used API key has no restriction or if the FIFO queue is specified in the API key’s restriction list.

If the subscription succeeds, Live Objects only returns a MQTT SUBACK packet once the subscription is active.

If the subscription fails (for ex. because the FIFO does not exist), a MQTT SUBACK packet is returned based on the MQTT protocol version used. In MQTT 3.1.1, the Failure return code 0x80 will be used. In MQTT 3.1, 0x01 will be returned.

You can refer to your Audit Logs where additional information regarding errors during MQTT connection will be displayed.

13. Alarming

13.1. Concepts

13.1.1. Alarms rules concepts

3 independent features are available for event processing :

  • Simple Event Processing (SEP) : a stateless service aimed at detecting notable single event from the flow of data messages. Example: raise an event when the device temperature is above 25°C.

  • State processing (SP) : a service aimed at detecting changes in "device state" computed from data messages. Example: raise an event when the device temperature status changes from "cold" to "hot". A state is computed by applying a state function to a data Message. A notification is sent by Live Objects each time a state value change.

  • Activity processing (AP) : detect 'silent devices', i.e. the lack of activity of a device. Example: raise an event when the device is not connected or did not send data message since 1 day.

Event processing and State processing services share common concepts :

  • apply rules when receiving a message : matching rules for SEP and state processing rules for SP. The rules are defined using the JsonLogic format.

  • have a common Context repository where you can store useful information for rule definition If your system is observable and periodically send measurement data, you can observe the state changes and notify them by using context repository (ex your device send periodically ambient temperature measurment : "cold" threshold is when data is below 20°C).

  • have a common Geozone repository where you can store the geographical references (polygons) you may use in your rules.

  • generate output events that your business application can consume to initiate downstream action(s) like alarming, execute a business process, etc.

  • limit the number of created rules (matching, firing and state processing) - order of idea is a thousand - in order to ensure high performances. The idea is to create generic rules and if needed, apply device-specific threshold using the Context repository.

Differences between Simple Event processing and State processing :

  • Simple Event Processing provides a stateless detection function (matching rule) while State Processing provides a stateful (the current state of the device is known by the system) detection function which is useful for uses case more complex than normal/alert status. State processing can be seen as a basic state machine where transitions between states are managed by the state function result and events are transition notifications.

  • Simple Event Processing has a frequency function (firing rule) which defines when "fired events" must be generated : ONCE, ALWAYS, SLEEP.

Event Processing E2E overview

lom_ep_architecture

13.1.2. Notifications concepts

The triggers and actions service purpose is to execute an action on a given trigger. This enables to create notifications easily on an event detection or to enable new external downstream action on new data.

Upon trigger activation, the following actions can be taken :
Email

An email can be sent to one or multiple recipient. The email content and subject can be specified and templated using the event or message as template data context.

SMS

An SMS can be sent to one or multiple recipient. The SMS content can be specified and templated using the event or message as template data context.

Http PUSH

An http push can be triggered to a particular web hook identified by its url. The data can be forwarded as-is or templated to match a particular format. A retry policy can be enabled if needed.

13.1.3. Rationale

The role of this service is to provide ability to trigger an action upon a Live Objects message. See message data model Event 2 Action can enable interesting use case for the user such as tighter integration of Live Objects to existing communication tools (such as but not limited to Slack, mattermost, HipChat, IFTTT etc.), enable downstream actions on its toolchains (trigger builds or various tasks on jenkins etc.) or enable more complex backend/cloud connectivity.

13.2. Alarm configuration

To configure your event rules and process data message of your streams, you must configure :

13.2.1. Context repository

13.2.1.1. Definition

The context repository is a database that allows storing user data that could be useful in the event rules definition and not present in the data messages. The context may include, for instance, thresholds definition, geographical zones, a list of device identifiers, a user preference, a group of contexts. The context has a key-value format. The key is a string and the value can be a primitive (string, numeric…​), a json object or an array. Optional tags are available to ease the search among the tenant contexts.

for geographical zones, a dedicated geozone database is provided. Once the user has provisioned his geozones, they are automatically available in the user context.
13.2.1.2. Context provisioning

The Live Objects API to manage context provisioning are described in the swagger documentation (Event processing - Context section) : https://liveobjects.orange-business.com/swagger-ui/index.html.

13.2.1.3. Context groups

A context value may reference other context keys. Instead of referencing each context individually, the rule can then reference the context group.

Example: See a context groups example.

extract context key

A context key is not necessarily hard coded in your rule. For instance, it can be extracted from your data message (using tags or device identifier).

Here, the context key is generated with the concatenation of the value.streamId field and a string.
 {"ctx" : {"cat":[{"var" : "value.streamId"},"alertingzone"]}}
Here, the context key is extracted from the value.tags field.
"ctx": { "get": [{"filter": [{"var": "value.tags"},"zone"]},0]}

13.2.2. Geozone repository

13.2.2.1. Definition

The Geozone repository is a database that allows the user to save his geographical sites/zones of interest. The geozones are stored as polygons (array of geopoints coordinates in decimal degrees). Meta information like a description and tags can be stored with the geozone.

Format:
  • coordinate order for polygon definition : use longitude as the first coordinate and latitude as the second coordinate.

  • the polygons are closed linestrings. Closed LineStrings have at least four coordinate pairs and specify the same position as the first and last coordinates.

Example of polygon :

[[[1.780892, 48.091452], [2.301382, 48.000565], [2.281961, 47.509630], [1.252634, 47.729556], [1.780892, 48.091452]]]]
13.2.2.2. Provisioning

The Live Objects API to manage geozone provisioning are described in the swagger documentation (Event processing - Geozone section) : https://liveobjects.orange-business.com/swagger-ui/index.html.

Example:

PUT liveobjects.orange-business.com/api/v0/eventprocessing/geozones/grand-orleans
{
  "description": "my geozone grand Orleans",
  "geometry": {
    "coordinates": [[[1.780892, 48.091452],[2.301382, 48.000565],
    [2.281961, 47.509630],[1.252634, 47.729556],[1.780892, 48.091452]]]],
    "type": "Polygon"
  },
  "tags": ["zone-nord"]
}
  • Once a geozone is provisioned, it is available in the user context. Hence, it can be referenced in event processing rules or in groups of context.

  • When a geozone is updated, the modifications are immediately taken into account by the contexts or rules referencing the geozone.

13.2.3. Rules and JsonLogic syntax

A rule is a function applied on a data message in order to detect any significant change in the data (exceeding threshold, state modification, change of location). The rules in Simple Event Processsing and State Processing are defined within Live Objects plateform with the JsonLogic syntax.

the JsonLogic log operator has been deactivated.

13.2.4. Additional operators

In addition to the existing JsonLogic operators (logic and boolean operators, numeric operators, string operators, array operators), Live Objects provides geographic operators (distance, inside, insideindex, closeto, closetoindex), context operator (ctx) and miscellaneous operators (get, currentstate).

Table 10. distance
Name

distance

Description

Geographical operator. Returns the distance in meters between two points, given their latitude and longitude in decimal degrees.

Parameters

lon1, lat1, lon2, lat2 in decimal degrees

Logic

{
  ">" :
    {
      "distance" : [
        { "var" : "location.lon"},
        {"var" : "location.lat"},
        2.296565,
        48.800206
      ]
    },
    6000
  ]
}

Data

Eiffel Tower

{
  "location":{
     "lon" : 2.2945,
     "lat" : 48.8584
  }
}

inside

Result

true

Table 11. ctx
Name

ctx

Description

Retrieve, from the context repository, one or several values using a key or an array of keys. Several ctx operators can be nested (group of contexts).

Parameters

key or array of keys

Context

In the following example, "freezingThreshold" and "liquidThreshold" must have been provisioned in the tenant context before being used.

PUT liveobjects.orange-business.com/api/v0/eventprocessing/context/freezingThreshold
{
  "contextData": 0
}

PUT liveobjects.orange-business.com/api/v0/eventprocessing/context/liquidThreshold
{
  "contextData": 100
}

Logic

{
  "if": [
    {"<": [
      {"var":"value.temp"},
      {"ctx": "freezingThreshold"}
    ]},
    "ice",
    {"<": [
      {"ctx": "freezingThreshold"},
      {"var":"value.temp"},
      {"ctx": "liquidThreshold"}
    ]},
    "liquid",
    "gas"
  ]
}

Pre-requisite

Data

{
  "value":{"temp":55}
}

Result

"liquid"

Table 12. currentstate
Name

currentstate

Description

Retrieve the current state of a device when applying a stateProcessing function. For state processing rules only. In the folowing example, current state can be "cold", "normal" or "hot".

The example folowing logic function is an hysteresis :

if current state is not hot, transition to hot if value.temp > 100

if current state is hot, transition to normal if value.temp < 80

if value.temp < 0 transition to cold

Logic

{"if" : [
  {"and": [
    { "!==": [
      { "currentstate": [] },
      "hot"
    ]},
    {"<": [
      80,
      {"var": "value.temp"},100
    ]}
  ]},
  "normal",
  {"<": [
    {"var":"value.temp"},
    0
  ]},
  "cold",
  {"<": [
    {"var":"value.temp"},
    80
  ]},
  "normal",
  "hot"
]}

Data

{
  "value":{"temp":20.0}
}

Result

"normal"

Table 13. get
Name

get

Description

Returns the element at the specified position in an array.

Parameters

array, index in the array

Context

In the following example, an array containing latitude and longitude values must have been provisioned in the tenant context :

PUT liveobjects.orange-business.com/api/v0/eventprocessing/context/2geopoints
{
  "contextData": [48.800206, 2.296565, 48.800206, 2.296565]
}

Logic

{
  "distance": [
    {
      "get": [
        {"ctx": "2geopoints"},
        0
      ]
    },
    {
      "get": [
        {"ctx": "2geopoints"},
        1
      ]
    },
    {
      "get": [
        {"ctx": "2geopoints"},
        2
      ]
    },
    {
      "get": [
        {"ctx": "2geopoints"},
        3
      ]
    }
  ]
}

Data

 {}

Result

0

Table 14. inside
Name

inside

Description

Checks if a point defined by its latitude and longitude is inside a polygon (or at least one polygon if an array of polygons is provided as input parameter).

Parameters

longitude, latitude in decimal degrees for the point to be tested, polygon(s) defined by the coordinates of their vertices (lon, lat in decimal degrees).

Logic

{
  "inside": [
    {"var": "location.lon"},
    {"var": "location.lat"},
    [[
      [2.381121,48.627973],
      [2.129376,48.629499],
      [2.099351,48.768217],
      [2.116302,48.955198],
      [2.317994,48.927845],
      [2.455176,48.913357],
      [2.489472,48.841933],
      [2.392301,48.762871],
      [2.381121,48.627973]
    ]]
  ]
}

Data

{
  "location":{
    "lon":2.350350,
    "lat":48.854064
  }
}

inside

Result

true

Table 15. insideindex
Name

insideindex

Description

Checks if a point is inside an array of polygons. Returns the index of the first matching polygon. Returns -1 if no matching was found. This operator is usually in conjunction with the "get" operator which will return the matching polygon.

Parameters

longitude, latitude in decimal degrees for the point to be tested, array of polygons defined by the coordinates of their vertices (lon, lat in decimal degrees).

Context

In the example, an array containing latitude and longitude values must have been provisioned in the tenant context :

PUT liveobjects.orange-business.com/api/v0/eventprocessing/context/zone-nord
{
  "contextData": ["zone-grandparis", "zone-grandorleans"]
}

PUT liveobjects.orange-business.com/api/v0/eventprocessing/context/zone-grandparis
{
  "contextData": [[
    [2.381121, 48.627973],
    [2.129376, 48.629499],
    [2.099351, 48.768217],
    [2.116302, 48.955198],
    [2.317994, 48.927845],
    [2.455176, 48.913357],
    [2.489472, 48.841933],
    [2.392301, 48.762871],
    [2.381121, 48.627973]
  ]]
}

PUT liveobjects.orange-business.com/api/v0/eventprocessing/context/zone-grandorleans
{
  "contextData": [[
    [1.780892, 48.091452],
    [2.301382, 48.000565],
    [2.281961, 47.509630],
    [1.252634, 47.729556],
    [1.780892, 48.091452]
  ]]
}

Logic

{
  "get": [
    {
      "ctx": {
        "get": [
          {
            "filter": [
              {"var": "tags"},
              "zone"
            ]
          },
          0
        ]
      }
    },
    {
      "insideindex": [
        {"var": "location.lon"},
        {"var": "location.lat"},
        {
          "ctx": {
            "ctx": {
              "get": [
                {
                  "filter": [
                    {"var": "tags"},
                    "zone"
                  ]
                },
                0
              ]
            }
          }
        }
      ]
    }
  ]
}

}

Data

{
  "location":{
    "lat" : 48.854064,
    "lon" : 2.350350
  },
  "tags" : [
    "otherTag2",
    "zone-nord",
    "otherTag1"
  ]
}

Result

"zone-grandparis"

Table 16. closeto
Name

closeto

Description

Checks if a circle is close to a polygon or at least one of the polygons (polygon array).

Parameters

longitude, latitude (in decimal degrees for the circle center), circle radius, polygon or array of polygons

Logic

{
  "closeto": [
    { "var": "location.lon" },
    { "var": "location.lat" },
    { "var": "location.accuracy" },
    [
      [[
        [1.780892,48.091452],
        [2.301382,48.000565],
        [2.281961,47.509630],
        [1.252634,47.729556],
        [1.780892,48.091452]
      ]],
      [[
        [2.281961,47.509630],
        [1.252634,47.729556]
      ]],
      [[
        [2.22412,48.85863],
        [2.25219,48.88143],
        [2.28404,48.8785],
        [2.26816,48.86721],
        [2.2588,48.84913],
        [2.22859,48.85004],
        [2.22412,48.85863]
      ]]
    ]
  ]
}

Data1 : circle center outside polygons, the circle does not intersect any polygon.

{
  "location":{
    "lon": 2.263849,
    "lat": 48.855983,
    "accuracy" : 100
  }
}

closeTo1

Result1

false

Data2 : circle center outside polygons, the circle intersects one polygon.

{
  "lon" : 2.263849,
  "lat" : 48.855983,
  "accuracy" : 200
}

closeTo2

Result2

true

Data3 : a point inside one of the polygons.

{
  "lon" : 2.260265350341797,
  "lat" : 48.85693640789798,
  "accuracy" : 0
}

closeTo3

Result3

true

Table 17. closetoindex
Name

closetoindex

Description

Checks if a circle is close to an array of polygons. Returns the index of the first matching polygon (first index in the array is 0). Returns -1 if no matching was found.

Parameters

longitude, latitude (in decimal degrees for the circle center), circle radius, array of polygons

Logic

{ "
  closetoindex" : [
    { "var" : "location.lon"},
    { "var" : "location.lat"},
    { "var" : "location.accuracy"} ,
    [
      [[
        [1.780892, 48.091452],
        [2.301382, 48.000565],
        [2.281961, 47.509630],
        [1.252634, 47.729556],
        [1.780892, 48.091452]
      ]],
      [[
        [2.224120, 48.858630],
        [2.252190, 48.881430],
        [2.284040, 48.878500],
        [2.268160, 48.867210],
        [2.258800, 48.849130],
        [2.228590, 48.850040],
        [2.224120, 48.858630]
      ]]
    ]
  ]
}

N.B.: first polygon in the array is the Orleans area; 2nd polygon is the Paris area.

Data

{
  "location":{
    "lon" : 2.263849,
    "lat" : 48.855983,
    "accuracy" : 500
  }
}

closeToIndex1

Result

1

Table 18. now_utc
Name

now_utc

Description

Returns the processing time as ISO 8601 string.

Similar to javaScript new Date().toISOString()

Parameters

none

Logic

{
   "<=":[
      0,
      {
         "get_utc_hours":[
            {
               "now_utc":[

               ]
            }
         ]
      },
      12
   ]
}

Data

 {}

Result

true if processing hour is between 0 and 12

Table 19. get_utc_hours
Name

get_utc_hours

Description

Returns the hour of the ISO 8601 provided parameter .

Similar to javaScript new Date(param).getUTCHours()

Parameters

ISO 8601 String

Logic

{\"get_utc_hours\" : [\"2018-02-15T13:01:37.290Z\"]}

Data

 {}

Result

13

Table 20. • get_utc_minutes
Name
  • get_utc_minutes

Description

Returns the minutes of the ISO 8601 provided parameter .

Similar to javaScript new Date(param).getUTCMinutes()

Parameters

ISO 8601 String

Logic

{\"get_utc_minutes\" : [\"2018-02-15T13:01:37.290Z\"]}

Data

 {}

Result

37

Table 21. • get_utc_day
Name
  • get_utc_day

Description

Returns the day of the week of the ISO 8601 provided parameter .

Similar to javaScript new Date(param).getUTCDay()

Parameters

ISO 8601 String

Logic

{\"get_utc_day\" : [\"2018-02-15T13:01:37.290Z\"]}

Data

 {}

Result

4

Table 22. • get_utc_date
Name
  • get_utc_date

Description

Returns the day of the month of the ISO 8601 provided parameter .

Similar to javaScript new Date(a).getUTCDate()

Parameters

ISO 8601 String

Logic

{\"get_utc_day\" : [\"2018-02-15T13:01:37.290Z\"]}

Data

 {}

Result

15

Table 23. • get_time
Name
  • get_time

Description

Returns epoch (milliseconds) of the ISO 8601 provided parameter .

Similar to javaScript new Date(a).getTime()

Parameters

ISO 8601 String

Logic

{\"get_time\" : [\"2018-01-15T13:00:37.290Z\"]}

Data

 {}

Result

1516021237290.0

13.2.5. Simple Event Processing

13.2.5.1. Concepts

Simple event processing (SEP) service is aimed at detecting notable single event from the flow of data messages.

Simple event processing combines a stateless boolean detection function (matching rule) with a frequency function (firing rule).

It generates fired events as output that your business application can consume to initiate downstream action(s) like alarming, execute a business process, etc.

Simple Event Processing service E2E overview

lom_sep_architecture

13.2.5.2. Processing rules

You can set up Matching rules and Firing rules to define how data messages are processed by the SEP service and how fired events are triggered:

13.2.5.3. Matching rule

A matching rule is a simple or compound rule that will be applied on each data message to evaluate if a « match » occurs. A matching rule is evaluated as a boolean result. Matching rule supports numeric, string, logic and distance operators and is based on JsonLogic.

Matching context (containing data message and matching rule id, etc.) are processed by the firing rules associated to these matching rules.

13.2.5.4. Firing rule

A firing rule applies to the matches triggered by one or many matching rules and defines when fired events must be generated.

A firing rule specifies:

  • the list of matching rules associated to this firing rule – when these matching rules match, the firing rule is applied,

  • the frequency of firing: once, sleep and always,

  • optionally, a list of aggregation keys identifying fields to extract from the matching context to identify the firing context.  

The firing rule is applied as follow on each matching context:

  • the firing rule generates the firing context from the matching context, by extracting one or multiple fields defined with the aggregation keys,

  • the firing rule then applies the frequency parameter to optionally throttle the triggering of fired events belonging to the same firing context.  

If the frequency of the firing rule is defined as ONCE or SLEEP then firing guards are created in the system to prevent new generation of fired events for a given firing context. You can manage the firing guards, and for example, remove a firing guard to re-activate a firing rule for a specific firing context.

As an example, by setting the metadata.source field as aggregation key, if a fired event is generated for a device “A”, a firing guard will prevent new fired event for this device “A” and this firing rule. By the way, fired events could occur for devices “B”, “C”, etc…​ for this rule.

With SLEEP mode, a duration specifies the minimum time between two fired events. When the duration is elapsed, the firing guards is removed and new fired events could occur. This duration is computed for each element of the tuple composed of firing rule id + aggregation keys + value (firingRuleID:metadata.source:deviceId1 , firingRuleID:metadata.source:deviceId2, …)

inside

The sleepDuration is expressed in a iso8601-duration format.

13.2.5.5. Fired events consumption

Fired events are accessible with the MQTT API. Your business application must connect using MQTT application mode and subscribe to a FIFO. This FIFO should previously be filed with matchingFired events.

Fired Event data representation
{
  "tenantId": String
  "timestamp": String
  "firingRule": {
    // The originating firing rule
  },
  "matchingContext": {
    "tenantId": (String) the id of the tenant,
    "timestamp": (String) iso-8601 timestamp of the originating matching event,
    "matchingRule": {
      // the originating matching rule
    },
    "data": {
      // the data message that triggered the matching rule and the firing rule
    }
  }
}

tenantId

String the tenant that issued the fired event,

timestamp

iso-8601 formatted timestamp, representing the date of emission of the fired event,

firingRule

The firing rule that triggered the event,

matchingContext.tenantId

String the tenant that issued the upstream matching event,

matchingContext.timestamp

ISO-8601 timestamp of the upstream matching event,

matchingContext.matchingRule

The rule that triggered the upstream matching event,

matchingContext.data

The Data Message that triggered the event chain,

Example of a fired event data representation

This example shows a fired event object triggered by one matching rule and filtered by a firing rule (that will block any other event from occuring unless the guard is lifted). The matching rule looks for any field named "string" containing the string "deviceId".

{
  "tenantId": "5a6760fe9566a132eb30f5e7",
  "timestamp": "2018-01-23T16:21:19.404Z",
  "firingRule": {
    "id": "65e6ef26-d9dc-48ce-9ad0-5f75eca7804f",
    "name": "name-025d02d8-6aa4-4614-8c7f-b4a6ad0edb12",
    "enabled": true,
    "matchingRuleIds": ["4d109ebb-fac0-453e-bc28-6a6f3b890606", "a8fed2a7-ea23-4fbb-9a76-bca8a9f94728"],
    "aggregationKeys": ["data.streamId"],
    "firingType": "ONCE"
  },
  "matchingContext": {
    "tenantId": "5a6760fe9566a132eb30f5e7",
    "timestamp": "2018-01-23T16:21:19.397Z",
    "matchingRule": {
      "id": "a8fed2a7-ea23-4fbb-9a76-bca8a9f94728",
      "name": "name-7f2fb1f4-11d4-4d56-8134-9a7a83249692",
      "enabled": true,
      "dataPredicate": {
        "in": ["deviceId", {
          "var": "value.string"
        }]
      }
    },
    "data": {
      "streamId": "e3295119-efa9-4494-94ec-0b6c539d90bd",
      "timestamp": "2018-01-23T16:21:18.903Z",
      "value": {
        "string": "deviceId-123",
        "integer": 0
      }
    }
  }
}
13.2.5.6. Examples

Here are some examples of usage of the simple event processing service.

Data message sent by a device with temperature set to 100 and location set at San Francisco (37.773972,-122.431297)

{
  "streamId":"urn:lo:nsid:dongle:00-14-22-01-23-45!temperature",
  "timestamp":"2016-08-29T08:27:52.874Z",
  "location":{
    "lat":37.773972,
    "lon":-122.431297
  },
  "model":"temperatureDevice_v0",
  "value":{
    "temp":100
  },
  "metadata":{
    "source":"urn:lo:nsid:dongle:00-14-22-01-23-45"
  }
}

Matching rule: numeric (temperature higher than 99) and distance operator (distance between data message and Paris (48.800206, 2.296565) must be higher than 6km)

{
  "name": "compound rule with numeric and distance operators",
  "enabled": true,
  "dataPredicate":
  {
    "and" :[
      { ">" : [
        {
          "distance" : [
            {var" : "location.lat"},
            {"var" : "location.lon"},
            48.800206,
            2.296565
          ]
        },
        6
       ]
      },
      { ">" : [
        { "var" : "value.temp" },
        99
      ]}
    ]
  }
}

Firing rule with frequency ONCE and aggregationKeys based on the source field :

{
    "name": "firing rule test",
    "enabled": true,
    "matchingRuleIds": ["{matchingRuleId}"],
    "aggregationKeys":["metadata.source"],
    "firingType":"ONCE"
}

Fired event will be generated once for each source sending data with temperature higher than 99 and not located within a radius of 6km of Paris.

Example with other operators ">", "if", "in", "cat" :

{
  ">":[
    {
      "var":{
        "cat":[
          "value.", {
            "if" : [
              {"in": [{"var":"model"}, "v0"]},
              "temp",
              {"in": [{"var":"model"}, "v1"]},
              "temperature",
              "t"
            ]
          }
        ]
      }
    },
    100
  ]
}

This rule allows to specify the field to be compared to the value "100” based on the model of the data message.

If the model value is:

  • "v0", the comparison will be made with the field "value.temp”,

  • "v1", the comparison will be made with the field "value.temperature”,

  • else it will be made with the field "value.t”.

13.2.6. State Processing

13.2.6.1. Concepts

State processing (SP) service aims at detecting changes in "device state" computed from data messages.

A state can represent any result computed from Live Objects data messages : geo-zone ("paris-area", "london-area", ..), temperature status ("hot", "cold", ..), availability status ("ok" , "ko"). Each state is identified by a key retrieved from the user-defined json-path in the data message.

stateKeyPath examples : "streamId", "metadata.source"

A state is computed by applying a state function to a data Message. A notification is sent by Live Objects each time a state value change. State processing differs from event processing as it provides statefull rules which is useful for uses case more complex than normal/alert status. State processing can be seen as a basic state machine where transitions between states are managed by the state function result and events are transition notifications.

13.2.6.2. State Processing rules

You can set up StateProcessing rules to define how data messages are processed by the SP service.

A StateProcessing rule applies to all new data messages.

Table 24. StateProcessing rule
field name required ? description

enabled

required

Defines if the rule should be enabled and applied to new data message.

filterPredicate

optional

Filters data on which the state processing logic should be applied. This boolean function is described in JsonLogic syntax.
If no filter predicate is specified, state function is applied to every data message.

stateKeyPath

required

A json path that will be used to retrieve the state key form the triggering data message.
Example: In order to associate a state with a device status and depending on your needs, you can use streamId or metadata.source as stateKey path.

stateFunction

required

This is the core of the state processing logic. This function takes as input a data message and computes a state associated whith the state key.
The State function is written in JsonLogic syntax and can return any primitive value : String, Number, Boolean.

13.2.6.3. State change events

State change events are accessible with the MQTT API. Your business application must connect using MQTT application mode and subscribe to a FIFO. This FIFO should previously be filed with stateChange events.

State Changed Event data structure
{
  "stateKey": String
  "previousState": String,
  "newState":  String,
  "timestamp": ISO-8601 Date String,
  "stateProcessingRuleId": String,
  "data": <<JSON_DATA_ENCODING, Data Message>>
}
Table 25. State Changed event field description

stateKey

The state key value,

previousState

Name of the previous state of the stateKey,

newState

Name the new state of the stateKey,

timestamp

ISO-8601 timestamp of the state change event,

stateProcessingRuleId

id of the rule that generated this state change event,

data

the Data Message that triggered the state change,

State Changed Event data structure
{
  "stateKey": String
  "previousState": String,
  "newState":  String,
  "timestamp": ISO-8601 Date String,
  "stateProcessingRuleId": String,
  "data": <<JSON_DATA_ENCODING, Data Message>>
}
Table 26. State Changed event field description

stateKey

The state key value,

previousState

Name of the previous state of the stateKey,

newState

Name the new state of the stateKey,

timestamp

ISO-8601 timestamp of the state change event,

stateProcessingRuleId

id of the rule that generated this state change event,

data

the Data Message that triggered the state change,

13.2.6.4. State processing initialization

When a state is computed for the first time, it generate a state change event with previous state equals null.

13.2.6.5. Examples

Here are some examples of usage of the state processing.

Temperature monitoring of a device sensor, with 3 temperature range.

Temperature State processing logic:

  • if temperature is below 0 degree Celsius, sensor state is cold.

  • if temperature is between 0 and 100 degrees Celsius sensor state is normal.

  • if temperature is higher than 100 degrees Celsius sensor state is hot.

The sensor is identified by the streamId field within the data message.

{
  "name": "temperature state rule",
  "enabled": true,
  "stateKeyPath": "streamId",
  "stateFunction": {
    "if": [
      {"<": [
        {"var": "value.temp"},
        0
      ]},
      "cold",
      {"<": [
        {"var": "value.temp"},
        100
      ]},
      "normal",
      "hot"
    ]
  }
}

We assume that the current state of the sensor is "normal". The following data message will generate a state change event from "normal" to "hot" for state key : "urn:lo:nsid:dongle:00-14-22-01-23-45!temperature".

{
  "streamId":"urn:lo:nsid:dongle:00-14-22-01-23-45!temperature",
  "timestamp":"2017-05-24T08:29:49.029Z",
  "location":{
    "lat":37.773972,
    "lon":-122.431297
  },
  "model":"temperatureDevice_v0",
  "value":{
    "temp":200
  },
  "metadata":{
    "source":"urn:lo:nsid:dongle:00-14-22-01-23-45"
  }
}

State change event :

{
  "stateKey": "urn:lo:nsid:dongle:00-14-22-01-23-45!temperature",
  "previousState":"normal",
  "newState": "hot",
  "timestamp": "2017-05-24T08:29:49.029Z",
  "stateProcessingRuleId": "266d3b22-70e0-4f28-9df1-5186c6094f5b",
  "data": {
    "streamId": "urn:lo:nsid:dongle:00-14-22-01-23-45!temperature",
    "timestamp":"2017-05-24T08:29:49.029Z",
    "location":{
      "lat":37.773972,
      "lon":-122.431297
    },
    "model":"temperatureDevice_v0",
    "value": {
      "temp": 200
    },
    "metadata":{
      "source":"urn:lo:nsid:dongle:00-14-22-01-23-45"
    }
  }
}

13.2.7. Activity Processing

13.2.7.1. Concepts

Activity processing (AP) service aims at detecting devices that have no activity.

All of the following 'activities' are monitored :

  • 'connected' event sent by the device

  • new data sent with the deviceId as 'metadata.source'

  • command response from this device

  • setParam response from this device

13.2.7.2. Activity rules

You can set up Activity rules to define how devices activity should be monitored.

You can target a list of deviceIds or a list of groups. All devices of these groups will be targeted. You can target a group with its path, including or not the subPath.

If a device is targeted by several rules, then several activity events can be thrown for this device.

Table 27. Activity rule
field name required ? description

enabled

required

Defines if the rule should be activated or not.

name

required

Name of the rule that can be used to ease rule management, must be unique and not empty.

silentPolicy

required

Define the duration of device inactivity before throwing an event. This duration will be reused when the device will be active once again.
You can also define a repeat interval (optional) : the 'silent' event will be re-thrown at this interval as long as the device remains silent.
All the duration are defined using ISO 8601, e.g. 'P1D' for 1 day, 'PT12H' for 12 hours, 'P2DT3H30M' for 2 days 3 hours 30 minutes. Please note that any duration below 10 minutes will be handled by the platform as a 10 minutes delay (this goes for both inactivity delay and repeat interval)

targets

required

List of deviceIds and/or devices groups that targeted by the rule. At least 1 deviceId or 1 group must be set.
Groups are targeted by their path (e.g. '/europe/france'), and an additional marker to specify if all subpath groups must be targeted.
If a new device is added to or removed from a targeted group, it will be taken into account.

tags

optional

A list of tags can be set : they will be recorded in all events triggered by this rule. This can ease event search and management.

When an activityRule is updated, all activity states linked to this rule are reset to a specific 'UNKNOWN' state.

13.2.7.3. Activity States

An activity state is unique for a couple 'activity rule id + device id', it holds :

  • the state 'ACTIVE', 'SILENT' or 'UNKNOWN' of the device for this rule. As a a device can be targeted by several rules with different silent duration, at a specific moment, 1 device can have several activity states, with different states. 'UNKNOWN' is specific to the initial state after activity rule creation or update.

  • the last activity timestamp of the device.

  • the timestamp of the next alarm that will be sent if there is no device activity until then. It can be triggered by the silent duration or repeatInterval duration depending on the current state (respectively ACTIVE or SILENT).

  • the number of times a reminder has been sent for the current state.

13.2.7.4. Activity Events

Activity events are accessible with the MQTT API. Your business application must connect using MQTT application mode and subscribe to a FIFO. This FIFO should previously be filed with deviceActivity events.

Activity Event data structure
{
  "deviceId": String,
  "deviceAdditionalInfo": {
      "deviceName": String,
      "groupPath": String
  },
  "activityRule": ActivityRule,
  "state":  String 'SILENT' or 'ACTIVE',
  "numberOfAlarmReminders": Integer
  "timestamp": ISO-8601 Date String
}
Table 28. Activity event field description

deviceId

Id of the device triggering this activity event

deviceAdditionalInfo

Additional information on the device. Consists of the name of the device and the path of the group it belongs to.

activityRule

ActivityRule triggering this activity event

state

Current state of device, either 'SILENT' or 'ACTIVE'

numberOfAlarmReminders

The number of times a reminder has been sent for the current state.

timestamp

ISO-8601 timestamp of the state change event

13.2.7.5. Example

Here is one examples of usage of the activity processing.

ActivityRule targeting the group path '/france/lyon' an all its sub path. The silent trigger is 1 day and the repeat alarm interval is 12 hours.

{
  "name": "devices in Lyon",
  "enabled": true,
  "silentPolicy": {
    "duration" : "P1D",
    "repeatInterval" : "PT12H"
  },
  "targets": {
    "groupPaths" : [
      {
        "path" : "/france/lyon",
        "includeSubPath" : true
      }
    ]
  }
}

If the device does not connect nor send any data for 1 day, then the following event will be sent:

{
  "deviceId": "urn:lo:nsid:dongle:00-14-22-01-23-45",
  "deviceAdditionalInfo": {
    "deviceName": "myDongle001",
    "groupPath": "/france/lyon"
  },
  "state":"SILENT",
  "numberOfAlarmReminders": 0,
  "timestamp": "2018-04-24T08:29:49.029Z",
  "activityRule": {
    "id": "d16b4319-a486-4cb8-a10e-b4a452cda4be",
    "name": "devices in Lyon",
    "enabled": true,
    "silentPolicy": {
      "duration" : "P1D",
      "repeatInterval" : "PT12H"
    },
    "targets": {
      "groupPaths" : [
        {
          "path" : "/france/lyon",
          "includeSubPath" : true
        }
      ]
    }
  }
}

12 hours later, if the device is still silent, the event will be sent again with 'numberOfAlarmReminders' incremented by 1:

{
  "deviceId": "urn:lo:nsid:dongle:00-14-22-01-23-45",
  "deviceAdditionalInfo": {
    "deviceName": "myDongle001",
    "groupPath": "/france/lyon"
  },
  "state":"SILENT",
  "numberOfAlarmReminders": 1,
  "timestamp": "2018-04-24T20:29:49.029Z",
  "activityRule": {
    "id": "d16b4319-a486-4cb8-a10e-b4a452cda4be",
    "name": "devices in Lyon",
    "enabled": true,
    "silentPolicy": {
      "duration" : "P1D",
      "repeatInterval" : "PT12H"
    },
    "targets": {
      "groupPaths" : [
        {
          "path" : "/france/lyon",
          "includeSubPath" : true
        }
      ]
    }
  }
}

13.3. Action policy used in alarming

Event processing and State processing services use an action policy.

To create a new action policy linked with an event rule :

Endpoint:

POST /api/v0/event2action/actionPolicies
An action policy has the following top level data representation :
{
  "id": "event-notification",
  "name": "on event sending email",
  "enabled": true,
  "triggers": {
    "eventRuleIds": ["01111111-1111-1111-1111-111111111110", "10000000-0000-0000-0000-000000000001"],
  },
  "actions": {
    "emails": [{
            "to": [
              "name.last@yourdomain.com"
            ],
            "subjectTemplate": "<text> + <mustache template>",
            "contentTemplate": "<text> + <mustache template>"
          }
    ],
    "sms": [],
    "httpPush": [],
    "fifoPublish": []
  }
}

To retrieve your action policy :

Endpoint:

GET /api/v0/event2action/actionPolicies/event-notification
field name is required description

id

optional

Defines the id of the action policy

name

optional

Defines a user friendly name for the action policy

enable

required

Enables or disables the action policy

triggers

required

Defines the type of trigger that will start an action. It can be a List of event rule ids OR a message selector.
Note: triggers object should have exactly one trigger defined.

triggers.eventRuleIds

optional

A list of rule ids as String

triggers.messageSelector

optional

A MessageSelector object (see Message Based trigger section)

actions

required

An Actions object that defines the action that will be started upon a trigger activation
Note: actions object should not be empty. At least one action should be defined.

actions.emails

optional

A collection of Email actions (see Email notification section)

actions.sms

optional

A collection of SMS actions (see SMS notification section)

actions.httpPush

optional

A collection of Http Push actions (see Http Push action section)

13.4. Event based Triggers

There is three kinds of event based triggers. You can either trigger an action on a fired event, on a state changed event or on an activity event. See the previous link for events data representation specificities.

An event trigger is represented by the id of the rule that will emit the corresponding event. You can specify multiple rule ids as trigger for one action policy.

Here is an example of an action policy with 2 event triggers triggered by 2 rules.
{
    "id": "some_id",
    "name": "some_user_friendly_name",
    "enabled": true,
    "triggers": {
        "eventRuleIds": ["01111111-1111-1111-1111-111111111110", "10000000-0000-0000-0000-000000000001"]
    },
    "actions": {
        ...
    }
}
You can mix state processing rule ids with event processing rule ids. But a state processing rule and a firing rule will emit two different kind of event with each their own data representation. Depending on your needs, you should consider creating 2 different action policys if you need to trigger an action on a fired event and on a state changed event. .A trigger can be of the following types :
Event

An Event message ( Fired Event, State Change Event, Activity Event ) coming from the event processing, the state processing or the activity processing services. This allows you to write complex matching, filtering, state or activity processing rules on your data and route chosen data to the desired actions.

Data Message

This allows you to route all your data traffic to the desired actions.
message data model

13.5. Notifications

An action, within an action policy, will define what to do upon trigger activation. An action is always passed the data message or the event that triggered it as datacontext for templating purpose or cherry picking a particular field within that data. Some actions gives you the ability to template their output. The templating language used is Mustache. You can use fields of your triggering data (event or message) by leveraging the mustache variable mechanism.

13.5.1. Email notification

This action is self explanatory and serve the purpose of sending email to one or multiple recipients when a trigger is activated.

Representation of an email action
{
    "to": [] of String,
    "cc": [] of String,
    "cci": [] of String,
    "subjectTemplate": String,
    "contentTemplate": String
}

to

A List of String each representing a valid email

cc

A List of String each representing a valid email

cci

A List of String each representing a valid email

subjectTemplate

A string representing a Mustache template. This will be use to render the email subject.

contentTemplate

A string representing a Mustache template. This will to render the email content.

Example of an action policy sending an email on a state change event
{
    "id": "some_id",
    "name": "some_user_friendly_name",
    "enabled": true,
    "triggers": {
        "eventRuleIds": ["00000000-0000-0000-0000-000000000000"]
    },
    "actions":
    {
        "emails": [{
            "to": ["notification@orange.com"],
            "cc": ["cc@orange.com"],
            "cci": ["cci@orange.com"],
            "subjectTemplate": "State change for {{stateKey}}",
            "contentTemplate": "{{stateKey}} change from state {{previousState}} to state {{newState}} at {{timestamp}}"
    }
}

13.5.2. SMS notification

This action is self explanatory and serve the purpose of sending sms to one or multiple recipients (MSISDNs) when a trigger is activated.

Representation of an SMS action
{
    "destinationPhoneNumbers": [] of String,
    "contentTemplate": String
}

destinationPhoneNumbers

a collection of string representing each a recipient msisd

contentTemplate

A string representing a Mustache template. It will be rendered as the sms content.

Example of an action policy sending an sms on a fired event
{
    "id": "some_id",
    "name": "some_user_friendly_name",
    "enabled": true,
    "triggers": {
        "eventRuleIds": ["22222222-2222-2222-2222-222222222222"]
    },
    "actions":
    {
        "sms": [{
                "destinationPhoneNumbers": ["+33123456789"],
                "contentTemplate": "Event fired at {{timestamp}} with value : {{value}}"
            }]
    }
}

13.5.3. FIFO publish notification

13.5.4. Http Push notification

13.5.5. Data contexts for templating

Actions template can have 4 kinds of datacontexts :

These are used as the root object of the mustache templating engine.

13.6. Alarms and notifications exemples

13.6.1. Use case 1: geozone supervision of a tracker

Pre-requisites:

  • the event processing feature is enabled for the tenant.

  • the tenant has a valid Live Objects API key.

13.6.1.1. Use case description: tracking of package between the shipment zone, transportation zone and delivery zone.

The REST requests for this example are available here and can be run in Postman.

A truck leaves San Francisco with its shipment. A tracker is embedded in the shipment. The truck may take Highway 101 or Route 5 to Los Angeles. A state change event will be sent when the tracker changes of zone.

  • Shipment zone (red) = San Francisco GPS polygon (lat, lon): (38.358596, -123.019952) (38.306889, -120.954523) (37.124990, -121.789484)

  • Delivery zone (green) = LA GPS polygon: (34.238622, -118.909873) (34.346562, -117.747086) (33.620728, -117.551111) (33.533648, -118.269687)

  • Transportation zone 1 (yellow) = 101 Highway: (37.561997, -122.05261237) (34.059617, -118.154639) (34.102708, -119.203276) (37.440666, -122.641996)

  • Transportation zone 2 (blue) = Route 5: (37.8705177, -121.3220217) (34.309766, -118.027739) (33.679366, -118.377685) (37.714244, -121.662597)

13.6.1.2. Geographical zones

lom_ep_stateprocessing1

13.6.1.3. Steps

inside

Step1: Geozone provisioning

First you need to create the 4 geozones you would like to monitor.

  • Make sure that you enter the coordinates with the longitude first (lon, lat).

  • The polygon must be closed (last point=first point).

PUT liveobjects.orange-business.com/api/v0/eventprocessing/geozones/san-francisco

In the request body:

{
  "description": "San Francisco zone",
  "geometry": {
    "coordinates": [[
        [-123.019952, 38.358596],[-120.954523, 38.306889],
        [-121.789484, 37.124990],[-123.019952, 38.358596]
    ]],
    "type": "Polygon"
  },
  "tags": [
    "SF-area", "shipment"
  ]
}
PUT liveobjects.orange-business.com/api/v0/eventprocessing/geozones/los-angeles
{
  "description": "Los Angeles zone",
  "geometry": {
    "coordinates": [[
      [-118.909873, 34.238622],[-117.747086, 34.346562],
      [-117.551111, 33.620728],[-118.269687, 33.533648],[-118.909873, 34.238622]
    ]],
    "type": "Polygon"
  },
  "tags": [
    "LA-area", "delivery"
  ]
}
PUT liveobjects.orange-business.com/api/v0/eventprocessing/geozones/transportation1
{
  "description": "Transportation zone Highway 101",
  "geometry": {
    "coordinates": [[
      [-122.05261237, 37.561997],[-118.154639, 34.059617],
      [-119.203276, 34.102708],[-122.641996, 37.440666],[-122.05261237, 37.561997]
    ]],
    "type": "Polygon"
  },
  "tags": [
    "transportation"
  ]
}
PUT liveobjects.orange-business.com/api/v0/eventprocessing/geozones/transportation2
{
  "description": "Transportation zone Route 5",
  "geometry": {
    "coordinates": [[
      [-121.3220217, 37.8705177],[-118.027739, 34.309766],
      [-118.377685, 33.679366],[-121.662597, 37.714244],[-121.3220217, 37.8705177]
    ]],
    "type": "Polygon"
  },
  "tags": [
    "transportation"
  ]
}

Once the geozones are provisioned, they are available in your user context and can be referenced in your rules.

Step2: Context provisioning
  • There are 2 transportation zones. You can group them into a single transportation context which will be used in your rule.

  • If you want to apply the rule only to a specific tracking device (the one present in the truck), you can create a device-group context containing the device identifier.

  • You can use the geozones san-francisco and los-angeles in your rule definition. But you probably do not want to reference directly the city names in the rule in order to be able to change the shipment and delivery zones without modifying the rule. Hence, you create an indirection in the context (san-francisco→shipment; los-angeles→delivery).

    PUT liveobjects.orange-business.com/api/v0/eventprocessing/context/transportation
    {
      "contextData": ["transportation1","transportation2"],
      "tags": [
        "transportation","zone","california"
      ]
    }
    PUT liveobjects.orange-business.com/api/v0/eventprocessing/context/device-group
    {
      "contextData": ["urn:lo:nsid:lora:0020B20000000101"],
      "tags": [
        "device","truck"
      ]
    }
    PUT liveobjects.orange-business.com/api/v0/eventprocessing/context/shipment
    {
      "contextData": "san-francisco",
      "tags": [
        "geozone"
      ]
    }
    PUT liveobjects.orange-business.com/api/v0/eventprocessing/context/delivery
    {
      "contextData": "los-angeles",
      "tags": [
        "geozone"
      ]
    }
Step3: State processing rule provisioning
  • This example is aimed at detecting a change in the device state, so you have to create a state processing rule which will be applied only to the monitored device (in the truck).

  • An event will be raised when the truck moves from one zone to the next one (shipment→transportation or transportation→delivery).

Before provisioning the state processing rule, it is useful to run the state processing function on a test data message.

POST liveobjects.orange-business.com/api/v0/eventprocessing/stateprocessing-rule/test
{
  "currentState": {},
  "data": {
    "metadata": {
      "connector": "http",
      "source": "urn:lo:nsid:lora:0020B20000000101"
   },
   "streamId": "urn:lo:nsid:lora:0020B20000000101",
    "location": {
      "provider": "lora-macro",
      "accuracy": 10,
      "lon": -122.169846,
      "lat": 37.602902
   },
   "model": "lora_v0",
   "value": {
      "payload": "ae2109000cf3"
   }
  },
  "stateProcessingFunction": {
    "if": [
      {
        "inside": [
          {
            "var": "location.lon"
          },
          {
            "var": "location.lat"
          },
          {
            "ctx": {"ctx":"shipment"}
          }
        ]
      },
      "shipment_zone",
      {
        "inside<