Decoding service

Overview

The data messages sent to the Live Objects platform can be encoded in a customer specific format. For instance, the payload may be a string containing an hexadecimal value or a csv value.

The data decoding feature enables you to provision your own decoding grammar. On receiving the encoded message, the Live Objects platform will use the grammar to decode the payload into plain text JSON fields and record the JSON in the Store service. The stored message will then be searchable with the Advanced Search service.

A "template" option allows to perform mathematical operations on the decoded fields or to define an output format.

A "model" option allows to override original data 'model' field.

The decoding feature is not activated by default.

Binary decoding

Decoder provisioning

The custom decoder describes the grammar to be used to decode the message payload. The Live Objects API to manage the decoders are described in the swagger documentation: https://liveobjects.orange-business.com/swagger-ui/index.html.

The binary decoding module uses the Java Binary Block Parser (JBBP) library in version 2.0.6.
You must use the JBBP DSL language to describe the binary payload format for your decoder.

Available types

The following JBBP types are not available: floatj, doublej and stringj. Please use respectively float, double and utf8 or asciichar instead (see decoding examples).
Field names

Each field can have case insensitive name which must not contain '.' (reserved for links to structure field values) and '#'(reserved for internal library use). A field name must not be started with either number or chars '$' and ''. Keep in mind that field names are case insensitive!_

Naming fields to avoid :

myString =/= mystring

Prohibited field names examples :

Not authorized : my.string, my#string, $mystring,_my.string,
Basic decoder

Example: create a binary decoder with the REST API:

POST /api/v0/decoders/binary
X-API-Key: <your API key>
Accept: application/json
{
"encoding":"twointegers",  (1)
"enabled":true,  (2)
"format":"int pressure;int temperature;", (3)
"template":"{\"pressure\":{{pressure}},
  \"temperature\" : \"{{#math}}{{temperature}}/10{{/math}} celsius\"}", (4)
"model":"model_twointegers"  (5)
}
1 identifies the decoder. This name will be associated to the devices during the provisioning and will be present in the data message.
2 activation/deactivation of the decoder.
3 describes the payload frame (cf. JBBP DSL language). The name of the fields will be found in the resulting decoded payload JSON.
4 optional parameter describing a post-decoding template format. In this example, the output temperature will be divided by 10 and stored in a string format including its unit. More information on templates.
5 optional parameter that will override the 'model' field in decoded data. If empty, the original value of 'model' field of the encoded data will be used. More information on model field
When to change the model in the decoder ?

  • The "model" field is set in order to enable elasticsearch (ES) indexing and queries on the decoded value.

  • If the "model" field is not set when provisioning the decoder, the data message will still be stored but it will not be searchable with ES queries.

  • The "model" must be changed when the type of a field already indexed in ES is modified. Example :

    • Suppose you have created a binary decoder with the following payload description "format":"utf8[4] field1; int field2;"

    • You have defined a model named "my_model_v0" associated with this decoder.

    • You start using this decoder with your devices. Data messages are stored and indexed in Live Objects.

    • Then, you can perform searches in your data messages using the model. See search examples.

    • If, after checking your decoded data, you find out that field1 is a float, not a UTF8 string, you may modify your decoder format. "format":"float field1; int field2;"

    • In this case, if you want your new messages to be stored and indexed properly in elasticsearch, you must change the model name in your decoder (for instance "my_model_v1"). Otherwise, "field1" will still be mapped as String and the message will be rejected by elasticsearch.

Endianness

The decoding service uses the big-endian order (the high bytes come first). If your device uses little-endian architecture, you can use the < character to prefix a type in your format description.

Example: create a binary decoder for a device sending data in little-endian format:

POST /api/v0/decoders/binary
X-API-Key: <your API key>
Accept: application/json
{
"encoding":"my_little_endian_encoding",
"enabled":true,
"format":"<float temperature;" (1)
}
1 : float means 32-bit float sent in little-endian.
How to test the binary decoder format?

The Live Objects API provides a "test" endpoint which takes a payload format and a payload value as input and provides the decoded value in the response body, if the decoding is successful. Optionally, you can provide a post-decoding template which will describe the output format.

In the following example, the decoded value for the pressure will remain unchanged, while the decoded value for temperature will be divided by 10.
The test endpoint is described in swagger.

Request:

POST /api/v0/decoders/binary/test
X-API-Key: <your API key>
Accept: application/json
{
"binaryPayloadStructure":"int  pressure; int temperature;",
"binaryPayloadHexString":"000003F5000000DD",
"template":"{\"pressure\":{{pressure}}, \"temperature\" : \"{{temperature}}/10\"}"
}

Response:

{
   "parsingOk": true,
   "decodingResult":    {
      "temperature": 22.1,
      "pressure": 1013
   },
   "descriptionValid": true
}
How to customize the fields once the payload has been decoded?

The fields resulting of a decoded payload might need to be processed using a template description, in order to change their output format. More information on templates.

Message decoding

The data message is decoded using the decoder previously provisioned and the decoded fields are added to the value. The encoded raw payload is kept in the decoded message. Once the message has been decoded and stored, "Advanced Search" requests can be performed using the newly decoded fields.

Table 1. Examples :
Frame format Payload example Decoded payload (JSON) 
int temperature;

000000DD

 
{
  "value": {
    "payload" : "000000DD",
    "temperature": 221
  }
}
 
int signed;
uint unsigned;

80000000
80000000

 
{
  "value": {
    "payload" : "8000000080000000",
    "signed": -2147483648,
    "unsigned": 2147483648
  }
}
 
float max_value;

7F7FFFFF

 
{
  "value": {
    "payload" : "7F7FFFFF",
    "max_value": 3.4028235e+38
  }
}
 
double max_value;

7FEFFFFFFFFFFFFF

 
{
  "value": {
    "payload" : "7FEFFFFFFFFFFFFF",
    "max_value": 1.7976931348623157e+308
  }
}
 
ubyte temperature;

DD

 
{
  "value": {
    "payload" : "DD",
    "temperature": 221
  }
}
 
utf8 [17] mystring;

2855332e3632542b323144323235503029

 
{
  "value": {
    "payload" : "2855332e3632542b323144323235503029",
    "mystring": "(U3.62T+21D225P0)"
  }
}
 
asciichar[13] name;

616E4153434949537472696E67

 
{
  "value": {
    "payload" : "616E4153434949537472696E67",
    "myasciistring": "anASCIIString"
  }
}
 
byte is_led_on;
float pressure;
float temperature;
float altitude;
ubyte battery_lvl;
byte[6] raw_gps;
ushort altitude_gps;

00
447CE000
41CEF5C3
45CAB8CD
38
00000000000
0FFFF

 
{
  "value": {
    "payload" : "00447CE00041CEF5C345CAB8CD38000000000000FFFF",
    "is_led_on": 0,
    "pressure": 1011.5,
    "temperature": 25.87,
    "altitude": 6487.1,
    "battery_lvl": 56,
    "raw_gps_list": [0,0,0,0,0,0],
    "altitude_gps": 65535
  }
}
 
float pi;
measure[2] {
  int length;
  utf8 [length] name;
  float value;
}

4048F5C3
0000000B
C2A955544638537472696E67
41480000
00000012
C2A9616E6F7468657255544638537472696E67
447D4000

 
{
  "value": {
    "payload" : "4048F5C3
      0000000BC2A955544638537472696E6741480000
      00000012C2A9616E6F7468657255544638537472696E67447D4000",
    "pi": 3.14,
    "measure_list":[
      {
        "length":11,
        "name":"©UTF8String",
        "value":12.5
      },
      {
        "length":18,
        "name":"©anotherUTF8String",
        "value":1013.0
      }
    ]
  }
}

The binary decoding feature can handle long type (i.e 64-bit integer). Nevertheless, when displayed in a web portal, the long number might be rounded unaccurately due to javascript limitations (max safe integer = 2^53-1 in javascript).
Table 2. JSON fields

value.payload

a string containing the encoded payload in hexadecimal (raw value)

metadata.encoding

contains the decoder name

model

remains unchanged after decoding if model field of decoder is empty; else it will be set with the value of model field in the decoder

additional LoRa® fields (lora port, snr…​) in the value

remain unchanged after decoding.

Csv decoding

More information on csv format

Decoder provisioning

The custom decoder describes the columns format and options to be used to decode the message csv payload. The Live Objects API to manage the decoders are described in the swagger documentation : https://liveobjects.orange-business.com/swagger-ui/index.html.

When provisioning a csv decoder, you must specify an ordered list of column names and their associated type. Three column types are available : STRING, NUMERIC or BOOLEAN.
Several options (column separator char, quote char, escape char…​) may be set to customize the csv decoding.

A template option enables you to provide a post-decoding output format including mathematical evaluation. More information on templates.

Column types

  • STRING column may contain UTF-8 characters

  • NUMERIC column may contain integer (32 bits), long (64 bits), float or double values. The values may be signed.

  • BOOLEAN column must contain true or false.

Table 3. Available options
name default definition example

quoteChar

double-quote "\""

character used for quoting values that contain column separator characters or linefeed.

"pierre, dupont",25,true will be decoded as 3 fields.

columnSeparator

comma ","

character used to separate values.

lineFeedSeparator

"\n"

character used to separate data rows. If the message payload contains several rows, only the first one will be decoded.

the decoding result for pierre,35,true\nmarie,25,false will be 3 fields containing pierre, 35 and true.

useEscapeChar

false

set to true if you want to use an escape char.

escapeChar

backslash "\\"

character used to escape values.

skipWhiteSpace

false

if set to true, will trim the decoded values (white spaces before and after will be removed).

Example 1: create a simple csv decoder with the REST API:

POST /api/v0/decoders/csv
X-API-Key: <your API key>
Accept: application/json
{
    "encoding":"my csv encoding", (1)
    "enabled":true, (2)
    "columns": [ (3)
        {"name":"column1","jsonType":"STRING"},
        {"name":"column2","jsonType":"NUMERIC"},
        {"name":"column3","jsonType":"BOOLEAN"}
    ],
    "model":"model_csv_decoded"  (4)
}
1 identifies the decoder. This name will be associated to the devices during the provisioning and will be present in the data message.
2 activation/deactivation of the decoder.
3 an ordered list of column descriptions.
4 optional parameter that will override the 'model' field of decoded data. If empty, the original value of 'model' field of the encoded data will be used. More information on model field.

Example 2: create a csv decoder with options, using the REST API:

POST /api/v0/decoders/csv
X-API-Key: <your API key>
Accept: application/json
{
    "encoding":"my csv encoding with options",
    "enabled":true,
    "columns": [
        {"name":"unit","jsonType":"STRING"},
        {"name":"temperature","jsonType":"NUMERIC"},
        {"name":"normal","jsonType":"BOOLEAN"}
    ],
    "options" : {
        "columnSeparator": "|",
        "quoteChar": "\"",
        "lineFeedSeparator": "/r/n"
    }
}
In the POST request, you can provide only the options you wish to modify. The other options will keep the default values.
How to customize the fields once the payload has been decoded?

The fields resulting of a decoded payload might need to be processed using a template description, in order to change their output format. More information on templates.

How to test the csv decoder ?

The Live Objects API provides a "test" endpoint which takes a csv format description and a payload value as input and provides the decoded value in the response body, if the decoding is successful. The test endpoint is described in swagger.

Request:

POST /api/v0/decoders/csv/test
X-API-Key: <your API key>
Accept: application/json
{
    "columns": [
        {"name":"unit","jsonType":"STRING"},
        {"name":"temperature","jsonType":"NUMERIC"},
        {"name":"threasholdReached","jsonType":"BOOLEAN"}
    ] ,
    "options":{
        "columnSeparator": ","
    },
    "csvPayload":"celsius,250,true",
    "template":"{\"temperature\" : \"{{temperature}}/10\",
      \"unit\":\"{{unit}}\", \"thresholdReached\":\"{{thresholdReached}}\"} "
}

Response:

{
   "parsingOk": true,
   "decodingResult":    {
      "unit": "celsius",
      "thresholdReached": "true",
      "temperature": 25
   },
   "descriptionValid": true
}

  • The csv decoding feature can handle long type (i.e 64-bit integer) but numbers above 2^31-1 are displayed in exponential notation (2147483648 will be displayed as 2.147483648E9). The decoding of very large numbers might lead to inaccurate rounding when the precision is above 15 digits.

Example : 9007199254740997 is displayed as 9.007199254740996E15.

  • For large numbers above 16 digits, you should use the STRING type when decoding your payload to avoid any rounding issue. Please note also that using string type for a number has impact on indexing/searching in Elasticsearch (data search feature). Since the data will be indexed and stored as string, it will not be possible to do aggregations or calculations.

Referencing a decoder in a LoRa® device

When provisioning a LoRa® device, you may reference the decoder to be used for the device so that Live Objects will automatically decode all the payloads received from this device, using the referenced decoder.

Message decoding

The data message is decoded using the decoder previously provisioned and the decoded fields are added to the value. The csv encoded raw payload is kept in the decoded message. Once the message has been decoded and stored, "Advanced Search" requests can be performed using the newly decoded fields.

Example in https:

Request
POST /api/v0/data/streams/{streamId}
X-API-Key: <your API key>
Accept: application/json
{
  "value": {"payload":"celsius,25,true"},
  "model": "temperature_v0",
  "metadata" : {"encoding" : "my csv encoding"}
 }

The data message will be stored as:

{
      "id": "585aa47de4b019917e342edd",
      "streamId": "stream0",
      "timestamp": "2016-12-21T15:49:17.693Z",
      "model": "temperature_v0",
      "value":       {
         "payload": "celsius,25,true",
         "normal": true,
         "unit": "celsius",
         "temperature": 25
      },
      "metadata": {"encoding": "my csv encoding"},
      "created": "2016-12-21T15:49:17.750Z"
}
Table 4. JSON fields

value.payload

a string containing the csv encoded payload (raw value)

metadata.encoding

contains the decoder name

model

remains unchanged after decoding if model field of decoder is empty; else it will be set with the value of model field in the decoder

Templating

The Live Objects provides, for the decoder creation and the decoder test APIs, an optional parameter named "template". This parameter is a string field describing the target output fields in a mustache-like format.

Table 5. Available functions :

{{#math}}{{/math}}

performs mathematical operations on a field

{{#toUpperCase}}{{/toUpperCase}}

converts a string to upper case

{{#toLowerCase}}{{/toLowerCase}}

converts a string to lower case

The following examples shows, for the same raw binary payload, the output if you are not using any template, or if you define a custom template.

Request (WITHOUT the template parameter)
POST /api/v0/decoders/binary/test
X-API-Key: <your API key>
Accept: application/json
{
"binaryPayloadStructure": "byte:1 led; ushort pressure; ushort temperature; ushort altitude; ubyte battery; byte[6] raw_gps; ushort altitude_gps;",
"binaryPayloadHexString":"0027830a1bfd6738000000000000ffff"
}
Response
{
   "parsingOk": true,
   "decodingResult":{
      "led": 0,
      "pressure": 10115,
      "temperature": 2587,
      "altitude": 64871,
      "battery": 56,
      "raw_gps": [0, 0, 0, 0, 0, 0],
      "altitude_gps": 65535
   },
   "descriptionValid": true
}
Request (WITH the template parameter)
POST /api/v0/decoders/binary/test
X-API-Key: <your API key>
Accept: application/json
{
  "binaryPayloadStructure":"byte:1 led; ushort pressure; ushort temperature;ushort altitude; ubyte battery; byte[6] raw_gps; ushort altitude_gps;",
  "binaryPayloadHexString":"0027830a1bfd6738000000000000ffff",
  "template":"{\"pressure\": \"{{pressure}} / 10\",\"temperature\": \"{{temperature}} / 100\",\"altitude\": \"{{altitude}} / 10\",\"view\": {\"Pressure\": \"{{#math}}{{pressure}}/10{{/math}} hPa\",\"Temperature\": \"{{#math}}{{temperature}}/100{{/math}} C\",\"Altitude\": \"{{#math}}{{altitude}}/100{{/math}} m\",\"GPSAltitude\": \"{{altitude_gps}} m\",\"Battery\": \"{{battery}} %\"}}"
}
Response
{
   "parsingOk": true,
   "decodingResult":    {
      "altitude": 6487.1,
      "view":       {
         "Pressure": "1011.5 hPa",
         "Temperature": "25.87 C",
         "Altitude": "648.71 m",
         "GPSAltitude": "65535 m",
         "Battery": "56 %"
      },
      "temperature": 25.87,
      "pressure": 1011.5,
      "led": 0,
      "battery": 56,
      "raw_gps": [0, 0, 0, 0, 0, 0],
      "altitude_gps": 65535
   },
   "descriptionValid": true
}

Please do not use dot-separated fields in the template.

Example:

`"template":"{\"field.with.dot\": {{temperature}}/10}"` will be rejected.

The {{#math}}{{/math}} template is needed only if you wish to evaluate a mathematical expression within a string.

Example for a template containing:

\"Temperature\": \"{{temperature}}/100 celsius\" (1)
\"Temperature\": \"{{#math}}{{temperature}}/100{{/math}} celsius\" (2)
\"Temperature\": \"{{#math}}{{temperature}}/100{{/math}}\" (3)
\"Temperature\": \"{{temperature}}/100\" (4)
1 the output will be like Temperature": "2587/100 celsius" (the division is not evaluated).
2 the output will be like Temperature": "25.87 celsius" (a string output. the division is evaluated).
3 the output will be like Temperature": 25.87 (a numeric). In this case, the {{#math}} function is not needed.
4 the output will be like Temperature": 25.87 (a numeric).

You need to specify in the template, all the fields you wish to get in the output, even if they are not modified by the template.

Example:

 `"template":"{\"pressure\":{{pressure}}, \"temperature\" : {{temperature}}/10}"`
If you omit the *pressure* field in the template, it will simply not appear in the output.
location

If the decoded value contains a location field with latitude and longitude, it will override the location field provided in Live Objects at the same JSON level as the value field.

Example:

A LoRa® message contains an encoded payload. The default location in the message is the location provided by the LoRa® network.

  "streamId": "urn:lora:xxxxxxxxxxxx1009!uplink",
  "created": "2019-04-05T14:37:12.047Z",
  "location": {
      "provider": "lora",
      "alt": 0,
      "accuracy": 7500,
      "lon": -3.469017,
      "lat": 48.762119
   },
   "model": "model_yyyy",
   "value": {
      "payload": "01005ca767ee00400810",
   },
   "timestamp": "2019-04-05T14:37:08.489Z",

If the device is a tracker, after decoding the payload, the decoded message may have the following format :

  "streamId": "urn:lora:xxxxxxxxxxxx1009!uplink",
  "created": "2019-04-05T14:37:12.047Z",
  "location": {(2)
        "provider": "device", (3)
        "lon": -3.469020,
        "lat": 48.762120
   },
   "model": "model_yyyy",
   "value": {
      "payload": "01005ca767ee00400810",
      "location": {  (1)
            "lon": -3.469020,
            "lat": 48.762120
        },
      "temperature": {
         "unit": "°C",
         "currentTemperature": {
            "values": [
               20.64
            ]
         }
      }
   },
   "timestamp": "2019-04-05T14:37:08.489Z",
1 if the decoded location has the expected "location" format,
2 the decoded location replaces the location provided by the LoRa® network,
3 and the provider is now "device" instead of "lora"

Javascript decoding

If the payloads are more complex (several payloads, specific calculations/transformations…​), you can develop a scriptable (javascript) decoder. The javascript version must be compliant with ECMA Script 5. We also provide a development framework that can be used with Eclipse or IntelliJ IDE that can ease the development of the decoder. More info in the links below :

Split decoding

If your device sends several measures in a single data message, you can use a « split decoder ». It is a special type of javascript decoder that enables you to « split » the original data message into several data messages (typically one per measure). Each resulting data message will then follow the usual data processing path described here.

You will find an example in the scriptable decoders documentation.

  • metadata.transformation.decoder.split.rank is added in the messages produced by a split decoder (it is not present in the other messages). It can be used to filter out the « split » messages from a search query. This is for instance necessary if you want to count the messages sent by your devices (in that case you only want to count one of the « split » messages).

  • The metadata.network section of the message is only preserved in one of the data messages produced by a split decoder (in the one with a « split rank » equal to 0).
Version 2026.7.0 • Updated date 2026-08-11 12:37:29 UTC