Search service
The REST request body search API is provided to perform search queries.
To learn more about the search API, read the Exploring your Data section of Elasticsearch: The Definitive Guide.
To perform a search query
Full search
Request
POST /api/v1/data/search X-API-Key: <your API key> Accept: application/json
parameter |
description |
dsl request (body) |
elasticsearch DSL request |
Elasticsearch 7
A major upgrade of our search engine from Elasticsearch 5 to Elasticsearch 7 has been made in 2021. The search request DSL body parameter will follow the Elasticsearch 7 one.
There are a few breaking changes compared to the previous one based on version 5, mainly some deprecated operators or changes in naming.
| You need to ensure you are not currently using these specific operators before the change will apply. |
You will receive a HTTP 400 response in such a case.
Below are the main changes :
-
Queries on boolean fields now strictly parse boolean-like values. This means only the strings "true" and "false" will be parsed into their boolean counterparts.
-
The 'in' query (a synonym for the 'terms' query) has been removed
-
Support for empty query objects ({ }) has been removed from the query DSL
-
The deprecated 'geo_distance_range' query has been removed
-
The 'all_fields' parameter for the 'query_string' and 'simple_query_string' has been removed. Set 'default_field' to '*' instead.
-
The deprecated 'prefix' parameter (a synonym for the 'value' parameter) of the 'prefix' query has been removed.
-
The deprecated 'le' (a synonym for 'lte') and 'ge' (a synonym for 'gte') parameter of the range query have been removed.
The full list can be found here (refer to section 'Search and Query DSL changes') :
The response of /api/v0/data/search request will stay unchanged.
But the new API version /api/v1/data/search uses the new default Elasticsearch 7 behavior : hits.total is limited to 10.000 documents; and the field is now a JsonObject instead of a number :
{
"hits": {
"total": 10123,
...
}
}{
"hits": {
"total": {
"value": 1000,
"relation": "eq"
},
...
}
}
{
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
...
}
}In this new v1 API, you can also use the query parameter trackTotalHits=true to force a full count (i.e. that can exceed 10.000).
Example
This query requests statistics from the myDeviceTemperature stream temp field.
Request:
POST /api/v1/data/search
{
"size": 0,
"query": {
"term": {
"streamId": "myDeviceTemperature"
}
},
"aggs":
{
"stats_temperature": {
"stats": {
"field": "@temperature_v0.value.temp"
}
}
}
}If a model has been provided, search query must be prefixed by @<model>: @temperature_v0.value.datapath
Response:
{
"took": 1,
"hits": {
"total": {
"value": 2,
"relation": "eq"
}
},
"aggregations": {
"stats_temperature": {
"count": 2,
"min": 24.1,
"max": 25.9,
"avg": 25,
"sum": 50
}
}
}Hits search
To perform the same search query; but with the 'hits' part extracted and JSON formated as an array of data messages (to use when you are only interested in the 'hits' part of Elasticsearch answer):
Request
POST /api/v1/data/search/hits X-API-Key: <your API key> Accept: application/json
parameter |
description |
dsl request (body) |
elasticsearch DSL request |
Example
This query requests last data for all devices using the model: temperature_v0.
Request:
POST /api/v0/data/search/hits
{
"size": 10,
"query": {
"term": {
"model": "temperature_v0"
}
}
}Response:
[
{
"id": "57308b3b7d84805820b35345",
"streamId": "myDeviceTemperature",
"timestamp": "2016-05-09T13:06:03.903Z",
"model": "temperature_v0",
"value": {
"temp": 25.9
},
"created": "2016-05-09T13:06:03.907Z"
},
{
"id": "573087777d84805820b35344",
"streamId": "myDeviceTemperature",
"timestamp": "2016-05-09T12:49:59.966Z",
"model": "temperature_v0",
"value": {
"temp": 24.1
},
"created": "2016-05-09T12:49:59.977Z"
},
{
"id": "5730b1577d84805820b35347",
"streamId": "myStreamDemo-temperature",
"timestamp": "2016-05-09T15:48:39.390Z",
"model": "temperature_v0",
"value": {
"temp": 24.1
},
"created": "2016-05-09T15:48:39.395Z"
}
]Geo Queries
Geo Query can be performed through all fields with name matching location (case insensitive).
In order to geoquery these fields, you must add @geopoint to the location query path: location@geopoint.
Request:
POST /api/v1/data/search/hits
{
"query": {
"bool" : {
"must" : {
"match_all" : {}
},
"filter": {
"geo_distance": {
"distance": "10km",
"location.@geopoint": {
"lat": 43.848,
"lon": -3.417
}
}
}
}
}
}Response:
[
{
"id": "57308b3b7d84805820b35345",
"streamId": "myDeviceTemperature",
"location": {
"lat": 43.8,
"lon": -3.3
}
"timestamp": "2016-05-09T13:06:03.903Z",
"model": "temperature_v0",
"value": {
"temp": 25.9
},
"created": "2016-05-09T13:06:03.907Z"
}
]Search Query samples
Here are some query samples that can be used.
Aggregations are very useful to retrieve data grouped by any criteria: list all known tags, get all last value per stream, get mean temperature per tag, get the list of streams that have not send data since a date…
The aggregations results are stored as 'buckets' in the result.
You can also add filters (geoquery, wildcards, terms…) to all your aggregations query to target specific 'buckets' or data.
Give me all you got !
Request:
{
"query": {
"match_all": {}
}
}Give me the list of all known tags
Request:
{
"size": 0,
"aggs": {
"grouped_by_tags": {
"terms": {
"field": "tags"
}
}
}
}Response:
{
"took": 44,
"hits": {
"total": {
"value": 66,
"relation": "eq"
}
},
"aggregations": {
"grouped_by_tags": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "tag_1",
"doc_count": 53
},
{
"key": "tag_2",
"doc_count": 13
}
]
}
}
}Give me the last value of all my streams
Request:
{
"size":0,
"aggs": {
"tags": {
"terms": {
"field": "streamId"
},
"aggs": {
"last_value": {
"top_hits": {
"size": 1,
"sort": [
{
"timestamp": {
"order": "desc"
}
}
]
}
}
}
}
}
}Response:
{
"took": 19,
"hits": {
"total": {
"value": 11,
"relation": "eq"
}
},
"aggregations": {
"tags": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "device_1",
"doc_count": 7,
"last_value": {
"hits": {
"total": 7,
"max_score": null,
"hits": [
{
...
}
]
}
}
},
{
"key": "device_2",
"doc_count": 123,
"last_value": {
"hits": {
"total": 123,
"max_score": null,
"hits": [
{
...
}
]
}
},
...
}
]
}
}
}Give me the list of devices that have not send data since 2017/03/23 10:00:00
Request:
{
"size":0,
"aggs": {
"tags": {
"terms": {
"field": "streamId"
},
"aggs": {
"last_date": {
"max": {
"field": "timestamp"
}
},
"filter_no_info_since": {
"bucket_selector": {
"buckets_path": {
"lastdate":"last_date"
},
"script": {
"inline": "lastdate<1490263200000",
"lang":"expression"
}
}
}
}
}
}
}Response:
{
"took": 8,
"hits": {
"total": {
"value": 9,
"relation": "eq"
}
},
"aggregations": {
"tags": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "device_12",
"doc_count": 7,
"last_date": {
"value": 1489504105020,
"value_as_string": "2017-03-14T15:08:25.020Z"
}
},
{
"key": "device_153",
"doc_count": 2,
"last_date": {
"value": 1489049619254,
"value_as_string": "2017-03-09T08:53:39.254Z"
}
}
]
}
}
}