Indexing service
After the data collected it may be associated to a model. The model is a fundamental concept for the search service, it specifies the schema of the JSON "value" object. The model is dynamically updated based on the data injected.
The model concept is necessary to avoid mapping conflicts in the underlying elasticsearch system.
Model field in data message
The model allows to give pattern for the value object, A model can be seen as a "mapping space" in which data types must not conflict. Some precautions must be taken when using model field :
-
If the model is not provided, "value" object will be not indexed by the search service. Nevertheless, the data will be stored in the store service and all information except value object will be indexed in search service.
-
If the value JSON object does not comply with the provided model (for example, a field change from Integer type to String type), the data will be not inserted in the search service. The data message will be only stored in the store service.
| Without a model, you will not be able to search or aggregate on any value.* field. |
Set model field to index your data
To make the messages sent by devices indexed in data store you must setting your data model with the following steps :
-
when you provision your decoder set "model" field in your data model.
-
when the "model" field is set and elasticsearch (ES) is enable, you can make elasticsearch queries on value fields only after the messages are decoded by your decoder and indexing service done.
-
each field of value collection is indexed by search service. In order to be able to search the message, copy this 'value' in a @model.value field which is analyzed by Elasticsearch. The message is consistent with others messages having the same model.
| Max total field number which can be indexed in ES : the limit allowed per tenant is max 1000 fields. |
exemple : data message sent by device and shown the indexed fields with values.
{
"timestamp" : "2018-05-31T16:15:21.288Z,"
"streamId" : "urn:lo:nsid:mqtt:{deviceId},"
"model": "my_encoding_model_v0"
"value": {
"temp": 12.75,
"humidity": 62.1,
"gpsFix": true,
"gpsSats": [12, 14, 21]
}
}Indexed field :
Field |
type |
temp |
float |
humiditity |
float |
gpsFix |
boolean |
gpsSats |
integer |