diff options
Diffstat (limited to 'docs/reference/search/aggregations')
21 files changed, 2138 insertions, 0 deletions
diff --git a/docs/reference/search/aggregations/bucket.asciidoc b/docs/reference/search/aggregations/bucket.asciidoc new file mode 100644 index 0000000..6bdd504 --- /dev/null +++ b/docs/reference/search/aggregations/bucket.asciidoc @@ -0,0 +1,25 @@ +[[search-aggregations-bucket]] + +include::bucket/global-aggregation.asciidoc[] + +include::bucket/filter-aggregation.asciidoc[] + +include::bucket/missing-aggregation.asciidoc[] + +include::bucket/nested-aggregation.asciidoc[] + +include::bucket/terms-aggregation.asciidoc[] + +include::bucket/range-aggregation.asciidoc[] + +include::bucket/daterange-aggregation.asciidoc[] + +include::bucket/iprange-aggregation.asciidoc[] + +include::bucket/histogram-aggregation.asciidoc[] + +include::bucket/datehistogram-aggregation.asciidoc[] + +include::bucket/geodistance-aggregation.asciidoc[] + +include::bucket/geohashgrid-aggregation.asciidoc[]
\ No newline at end of file diff --git a/docs/reference/search/aggregations/bucket/datehistogram-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/datehistogram-aggregation.asciidoc new file mode 100644 index 0000000..f48ce4a --- /dev/null +++ b/docs/reference/search/aggregations/bucket/datehistogram-aggregation.asciidoc @@ -0,0 +1,136 @@ +[[search-aggregations-bucket-datehistogram-aggregation]] +=== Date Histogram + +A multi-bucket aggregation similar to the <<search-aggregations-bucket-histogram-aggregation,histogram>> except it can +only be applied on date values. Since dates are represented in elasticsearch internally as long values, it is possible +to use the normal `histogram` on dates as well, though accuracy will be compromised. The reason for this is in the fact +that time based intervals are not fixed (think of leap years and on the number of days in a month). For this reason, +we need a special support for time based data. From a functionality perspective, this histogram supports the same features +as the normal <<search-aggregations-bucket-histogram-aggregation,histogram>>. The main difference is that the interval can be specified by date/time expressions. + +Requesting bucket intervals of a month. + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "articles_over_time" : { + "date_histogram" : { + "field" : "date", + "interval" : "month" + } + } + } +} +-------------------------------------------------- + +fractional values are allowed, for example 1.5 hours: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "articles_over_time" : { + "date_histogram" : { + "field" : "date", + "interval" : "1.5h" + } + } + } +} +-------------------------------------------------- + +Other available expressions for interval: `year`, `quarter`, `week`, `day`, `hour`, `minute`, `second` + +==== Time Zone + +By default, times are stored as UTC milliseconds since the epoch. Thus, all computation and "bucketing" / "rounding" is +done on UTC. It is possible to provide a time zone (both pre rounding, and post rounding) value, which will cause all +computations to take the relevant zone into account. The time returned for each bucket/entry is milliseconds since the +epoch of the provided time zone. + +The parameters are `pre_zone` (pre rounding based on interval) and `post_zone` (post rounding based on interval). The +`time_zone` parameter simply sets the `pre_zone` parameter. By default, those are set to `UTC`. + +The zone value accepts either a numeric value for the hours offset, for example: `"time_zone" : -2`. It also accepts a +format of hours and minutes, like `"time_zone" : "-02:30"`. Another option is to provide a time zone accepted as one of +the values listed here. + +Lets take an example. For `2012-04-01T04:15:30Z`, with a `pre_zone` of `-08:00`. For day interval, the actual time by +applying the time zone and rounding falls under `2012-03-31`, so the returned value will be (in millis) of +`2012-03-31T00:00:00Z` (UTC). For hour interval, applying the time zone results in `2012-03-31T20:15:30`, rounding it +results in `2012-03-31T20:00:00`, but, we want to return it in UTC (`post_zone` is not set), so we convert it back to +UTC: `2012-04-01T04:00:00Z`. Note, we are consistent in the results, returning the rounded value in UTC. + +`post_zone` simply takes the result, and adds the relevant offset. + +Sometimes, we want to apply the same conversion to UTC we did above for hour also for day (and up) intervals. We can +set `pre_zone_adjust_large_interval` to `true`, which will apply the same conversion done for hour interval in the +example, to day and above intervals (it can be set regardless of the interval, but only kick in when using day and +higher intervals). + +==== Factor + +The date histogram works on numeric values (since time is stored in milliseconds since the epoch in UTC). But, +sometimes, systems will store a different resolution (like seconds since UTC) in a numeric field. The `factor` +parameter can be used to change the value in the field to milliseconds to actual do the relevant rounding, and then +be applied again to get to the original unit. For example, when storing in a numeric field seconds resolution, the +factor can be set to 1000. + +==== Pre/Post Offset + +Specific offsets can be provided for pre rounding and post rounding. The `pre_offset` for pre rounding, and +`post_offset` for post rounding. The format is the date time format (`1h`, `1d`, etc...). + +==== Keys + +Since internally, dates are represented as 64bit numbers, these numbers are returned as the bucket keys (each key +representing a date - milliseconds since the epoch). It is also possible to define a date format, which will result in +returning the dates as formatted strings next to the numeric key values: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "articles_over_time" : { + "date_histogram" : { + "field" : "date", + "interval" : "1M", + "format" : "yyyy-MM-dd" <1> + } + } + } +} +-------------------------------------------------- + +<1> Supports expressive date <<date-format-pattern,format pattern>> + +Response: + +[source,js] +-------------------------------------------------- +{ + "aggregations": { + "articles_over_time": { + "buckets": [ + { + "key_as_string": "2013-02-02", + "key": 1328140800000, + "doc_count": 1 + }, + { + "key_as_string": "2013-03-02", + "key": 1330646400000, + "doc_count": 2 + }, + ... + ] + } + } +} +-------------------------------------------------- + +Like with the normal <<search-aggregations-bucket-histogram-aggregation,histogram>>, both document level scripts and +value level scripts are supported. It is also possilbe to control the order of the returned buckets using the `order` +settings and filter the returned buckets based on a `min_doc_count` setting (by defaults to all buckets with +`min_doc_count > 1` will be returned).
\ No newline at end of file diff --git a/docs/reference/search/aggregations/bucket/daterange-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/daterange-aggregation.asciidoc new file mode 100644 index 0000000..710f3ac --- /dev/null +++ b/docs/reference/search/aggregations/bucket/daterange-aggregation.asciidoc @@ -0,0 +1,110 @@ +[[search-aggregations-bucket-daterange-aggregation]] +=== Date Range + +A range aggregation that is dedicated for date values. The main difference between this aggregation and the normal <<search-aggregations-bucket-range-aggregation,range>> aggregation is that the `from` and `to` values can be expressed in Date Math expressions, and it is also possible to specify a date format by which the `from` and `to` response fields will be returned: + +Example: + +[source,js] +-------------------------------------------------- +{ + "aggs": { + "range": { + "date_range": { + "field": "date", + "format": "MM-yyy", + "ranges": [ + { "to": "now-10M/M" }, + { "from": "now-10M/M" } + ] + } + } + } +} +-------------------------------------------------- + +In the example above, we created two range buckets, the first will "bucket" all documents dated prior to 10 months ago and +the second will "bucket" all documents dated since 10 months ago + +Response: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "range": { + "buckets": [ + { + "to": 1.3437792E+12, + "to_as_string": "08-2012", + "doc_count": 7 + }, + { + "from": 1.3437792E+12, + "from_as_string": "08-2012", + "doc_count": 2 + } + ] + } + } +} +-------------------------------------------------- + +[[date-format-pattern]] +==== Date Format/Pattern + +NOTE: this information was copied from http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html[JodaDate] + +All ASCII letters are reserved as format pattern letters, which are defined as follows: + +[options="header"] +|======= +|Symbol |Meaning |Presentation |Examples +|G |era |text |AD +|C |century of era (>=0) |number |20 +|Y |year of era (>=0) |year |1996 + +|x |weekyear |year |1996 +|w |week of weekyear |number |27 +|e |day of week |number |2 +|E |day of week |text |Tuesday; Tue + +|y |year |year |1996 +|D |day of year |number |189 +|M |month of year |month |July; Jul; 07 +|d |day of month |number |10 + +|a |halfday of day |text |PM +|K |hour of halfday (0~11) |number |0 +|h |clockhour of halfday (1~12) |number |12 + +|H |hour of day (0~23) |number |0 +|k |clockhour of day (1~24) |number |24 +|m |minute of hour |number |30 +|s |second of minute |number |55 +|S |fraction of second |number |978 + +|z |time zone |text |Pacific Standard Time; PST +|Z |time zone offset/id |zone |-0800; -08:00; America/Los_Angeles + +|' |escape for text |delimiter +|'' |single quote |literal |' +|======= + +The count of pattern letters determine the format. + +Text:: If the number of pattern letters is 4 or more, the full form is used; otherwise a short or abbreviated form is used if available. + +Number:: The minimum number of digits. Shorter numbers are zero-padded to this amount. + +Year:: Numeric presentation for year and weekyear fields are handled specially. For example, if the count of 'y' is 2, the year will be displayed as the zero-based year of the century, which is two digits. + +Month:: 3 or over, use text, otherwise use number. + +Zone:: 'Z' outputs offset without a colon, 'ZZ' outputs the offset with a colon, 'ZZZ' or more outputs the zone id. + +Zone names:: Time zone names ('z') cannot be parsed. + +Any characters in the pattern that are not in the ranges of ['a'..'z'] and ['A'..'Z'] will be treated as quoted text. For instance, characters like ':', '.', ' ', '#' and '?' will appear in the resulting time text even they are not embraced within single quotes.
\ No newline at end of file diff --git a/docs/reference/search/aggregations/bucket/filter-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/filter-aggregation.asciidoc new file mode 100644 index 0000000..5166002 --- /dev/null +++ b/docs/reference/search/aggregations/bucket/filter-aggregation.asciidoc @@ -0,0 +1,38 @@ +[[search-aggregations-bucket-filter-aggregation]] +=== Filter + +Defines a single bucket of all the documents in the current document set context that match a specified filter. Often this will be used to narrow down the current aggregation context to a specific set of documents. + +Example: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "in_stock_products" : { + "filter" : { "range" : { "stock" : { "gt" : 0 } } }, + "aggs" : { + "avg_price" : { "avg" : { "field" : "price" } } + } + } + } +} +-------------------------------------------------- + +In the above example, we calculate the average price of all the products that are currently in-stock. + +Response: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggs" : { + "in_stock_products" : { + "doc_count" : 100, + "avg_price" : { "value" : 56.3 } + } + } +} +--------------------------------------------------
\ No newline at end of file diff --git a/docs/reference/search/aggregations/bucket/geodistance-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/geodistance-aggregation.asciidoc new file mode 100644 index 0000000..db4cd4a --- /dev/null +++ b/docs/reference/search/aggregations/bucket/geodistance-aggregation.asciidoc @@ -0,0 +1,105 @@ +[[search-aggregations-bucket-geodistance-aggregation]] +=== Geo Distance + +A multi-bucket aggregation that works on `geo_point` fields and conceptually works very similar to the <<search-aggregations-bucket-range-aggregation,range>> aggregation. The user can define a point of origin and a set of distance range buckets. The aggregation evaluate the distance of each document value from the origin point and determines the buckets it belongs to based on the ranges (a document belongs to a bucket if the distance between the document and the origin falls within the distance range of the bucket). + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "rings_around_amsterdam" : { + "geo_distance" : { + "field" : "location", + "origin" : "52.3760, 4.894", + "ranges" : [ + { "to" : 100 }, + { "from" : 100, "to" : 300 }, + { "from" : 300 } + ] + } + } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + "aggregations": { + "rings" : { + "buckets": [ + { + "unit": "km", + "to": 100.0, + "doc_count": 3 + }, + { + "unit": "km", + "from": 100.0, + "to": 300.0, + "doc_count": 1 + }, + { + "unit": "km", + "from": 300.0, + "doc_count": 7 + } + ] + } + } +} +-------------------------------------------------- + +The specified field must be of type `geo_point` (which can only be set explicitly in the mappings). And it can also hold an array of `geo_point` fields, in which case all will be taken into account during aggregation. The origin point can accept all formats supported by the `geo_point` <<mapping-geo-point-type,type>>: + +* Object format: `{ "lat" : 52.3760, "lon" : 4.894 }` - this is the safest format as it is the most explicit about the `lat` & `lon` values +* String format: `"52.3760, 4.894"` - where the first number is the `lat` and the second is the `lon` +* Array format: `[4.894, 52.3760]` - which is based on the `GeoJson` standard and where the first number is the `lon` and the second one is the `lat` + +By default, the distance unit is `km` but it can also accept: `mi` (miles), `in` (inch), `yd` (yards), `m` (meters), `cm` (centimeters), `mm` (millimeters). + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "rings" : { + "geo_distance" : { + "field" : "location", + "origin" : "52.3760, 4.894", + "unit" : "mi", <1> + "ranges" : [ + { "to" : 100 }, + { "from" : 100, "to" : 300 }, + { "from" : 300 } + ] + } + } + } +} +-------------------------------------------------- + +<1> The distances will be computed as miles + +There are two distance calculation modes: `sloppy_arc` (the default), `arc` (most accurate) and `plane` (fastest). The `arc` calculation is the most accurate one but also the more expensive one in terms of performance. The `sloppy_arc` is faster but less accurate. The `plane` is the fastest but least accurate distance function. Consider using `plane` when your search context is "narrow" and spans smaller geographical areas (like cities or even countries). `plane` may return higher error mergins for searches across very large areas (e.g. cross continent search). The distance calculation type can be set using the `distance_type` parameter: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "rings" : { + "geo_distance" : { + "field" : "location", + "origin" : "52.3760, 4.894", + "distance_type" : "plane", + "ranges" : [ + { "to" : 100 }, + { "from" : 100, "to" : 300 }, + { "from" : 300 } + ] + } + } + } +} +-------------------------------------------------- diff --git a/docs/reference/search/aggregations/bucket/geohashgrid-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/geohashgrid-aggregation.asciidoc new file mode 100644 index 0000000..3ed2896 --- /dev/null +++ b/docs/reference/search/aggregations/bucket/geohashgrid-aggregation.asciidoc @@ -0,0 +1,127 @@ +[[search-aggregations-bucket-geohashgrid-aggregation]] +=== GeoHash grid + +A multi-bucket aggregation that works on `geo_point` fields and groups points into buckets that represent cells in a grid. +The resulting grid can be sparse and only contains cells that have matching data. Each cell is labeled using a http://en.wikipedia.org/wiki/Geohash[geohash] which is of user-definable precision. + +* High precision geohashes have a long string length and represent cells that cover only a small area. +* Low precision geohashes have a short string length and represent cells that each cover a large area. + +Geohashes used in this aggregation can have a choice of precision between 1 and 12. + +WARNING: The highest-precision geohash of length 12 produces cells that cover less than a square metre of land and so high-precision requests can be very costly in terms of RAM and result sizes. +Please see the example below on how to first filter the aggregation to a smaller geographic area before requesting high-levels of detail. + +The specified field must be of type `geo_point` (which can only be set explicitly in the mappings) and it can also hold an array of `geo_point` fields, in which case all points will be taken into account during aggregation. + + +==== Simple low-precision request + +[source,js] +-------------------------------------------------- +{ + "aggregations" : { + "myLarge-GrainGeoHashGrid" : { + "geohash_grid" : { + "field" : "location", + "precision" : 3 + } + } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + "aggregations": { + "myLarge-GrainGeoHashGrid": { + "buckets": [ + { + "key": "svz", + "doc_count": 10964 + }, + { + "key": "sv8", + "doc_count": 3198 + } + ] + } + } +} +-------------------------------------------------- + + + +==== High-precision requests + +When requesting detailed buckets (typically for displaying a "zoomed in" map) a filter like <<query-dsl-geo-bounding-box-filter,geo_bounding_box>> should be applied to narrow the subject area otherwise potentially millions of buckets will be created and returned. + +[source,js] +-------------------------------------------------- +{ + "aggregations" : { + "zoomedInView" : { + "filter" : { + "geo_bounding_box" : { + "location" : { + "top_left" : "51.73, 0.9", + "bottom_right" : "51.55, 1.1" + } + } + }, + "aggregations":{ + "zoom1":{ + "geohash_grid" : { + "field":"location", + "precision":8, + } + } + } + } + } + } +-------------------------------------------------- + +==== Cell dimensions at the equator +The table below shows the metric dimensions for cells covered by various string lengths of geohash. +Cell dimensions vary with latitude and so the table is for the worst-case scenario at the equator. + +[horizontal] +*GeoHash length*:: *Area width x height* +1:: 5,009.4km x 4,992.6km +2:: 1,252.3km x 624.1km +3:: 156.5km x 156km +4:: 39.1km x 19.5km +5:: 4.9km x 4.9km +6:: 1.2km x 609.4m +7:: 152.9m x 152.4m +8:: 38.2m x 19m +9:: 4.8m x 4.8m +10:: 1.2m x 59.5cm +11:: 14.9cm x 14.9cm +12:: 3.7cm x 1.9cm + + + +==== Options + +[horizontal] +field:: Mandatory. The name of the field indexed with GeoPoints. + +precision:: Optional. The string length of the geohashes used to define + cells/buckets in the results. Defaults to 5. + +size:: Optional. The maximum number of geohash buckets to return + (defaults to 10,000). When results are trimmed, buckets are + prioritised based on the volumes of documents they contain. + +shard_size:: Optional. To allow for more accurate counting of the top cells + returned in the final result the aggregation defaults to + returning `max(10,(size x number-of-shards))` buckets from each + shard. If this heuristic is undesirable, the number considered + from each shard can be over-ridden using this parameter. + + diff --git a/docs/reference/search/aggregations/bucket/global-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/global-aggregation.asciidoc new file mode 100644 index 0000000..da6ddc1 --- /dev/null +++ b/docs/reference/search/aggregations/bucket/global-aggregation.asciidoc @@ -0,0 +1,51 @@ +[[search-aggregations-bucket-global-aggregation]] +=== Global + +Defines a single bucket of all the documents within the search execution context. This context is defined by the indices and the document types you're searching on, but is *not* influenced by the search query itself. + +NOTE: Global aggregators can only be placed as top level aggregators (it makes no sense to embed a global aggregator + within another bucket aggregator) + +Example: + +[source,js] +-------------------------------------------------- +{ + "query" : { + "match" : { "title" : "shirt" } + }, + "aggs" : { + "all_products" : { + "global" : {}, <1> + "aggs" : { <2> + "avg_price" : { "avg" : { "field" : "price" } } + } + } + } +} +-------------------------------------------------- + +<1> The `global` aggregation has an empty body +<2> The sub-aggregations that are registered for this `global` aggregation + +The above aggregation demonstrates how one would compute aggregations (`avg_price` in this example) on all the documents in the search context, regardless of the query (in our example, it will compute the average price over all products in our catalog, not just on the "shirts"). + +The response for the above aggreation: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations" : { + "all_products" : { + "doc_count" : 100, <1> + "avg_price" : { + "value" : 56.3 + } + } + } +} +-------------------------------------------------- + +<1> The number of documents that were aggregated (in our case, all documents within the search context) diff --git a/docs/reference/search/aggregations/bucket/histogram-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/histogram-aggregation.asciidoc new file mode 100644 index 0000000..469ea23 --- /dev/null +++ b/docs/reference/search/aggregations/bucket/histogram-aggregation.asciidoc @@ -0,0 +1,283 @@ +[[search-aggregations-bucket-histogram-aggregation]] +=== Histogram + +A multi-bucket values source based aggregation that can be applied on numeric values extracted from the documents. +It dynamically builds fixed size (a.k.a. interval) buckets over the values. For example, if the documents have a field +that holds a price (numeric), we can configure this aggregation to dynamically build buckets with interval `5` +(in case of price it may represent $5). When the aggregation executes, the price field of every document will be +evaluated and will be rounded down to its closest bucket - for example, if the price is `32` and the bucket size is `5` +then the rounding will yield `30` and thus the document will "fall" into the bucket that is associated withe the key `30`. +To make this more formal, here is the rounding function that is used: + +[source,java] +-------------------------------------------------- +rem = value % interval +if (rem < 0) { + rem += interval +} +bucket_key = value - rem +-------------------------------------------------- + +The following snippet "buckets" the products based on their `price` by interval of `50`: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "prices" : { + "histogram" : { + "field" : "price", + "interval" : 50 + } + } + } +} +-------------------------------------------------- + +And the following may be the response: + +[source,js] +-------------------------------------------------- +{ + "aggregations": { + "prices" : { + "buckets": [ + { + "key": 0, + "doc_count": 2 + }, + { + "key": 50, + "doc_count": 4 + }, + { + "key": 150, + "doc_count": 3 + } + ] + } + } +} +-------------------------------------------------- + +The response above shows that none of the aggregated products has a price that falls within the range of `[100 - 150)`. +By default, the response will only contain those buckets with a `doc_count` greater than 0. It is possible change that +and request buckets with either a higher minimum count or even 0 (in which case elasticsearch will "fill in the gaps" +and create buckets with zero documents). This can be configured using the `min_doc_count` setting: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "prices" : { + "histogram" : { + "field" : "price", + "interval" : 50, + "min_doc_count" : 0 + } + } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + "aggregations": { + "prices" : { + "buckets": [ + { + "key": 0, + "doc_count": 2 + }, + { + "key": 50, + "doc_count": 4 + }, + { + "key" : 100, + "doc_count" : 0 <1> + }, + { + "key": 150, + "doc_count": 3 + } + ] + } + } +} +-------------------------------------------------- + +<1> No documents were found that belong in this bucket, yet it is still returned with zero `doc_count`. + +==== Order + +By default the returned buckets are sorted by their `key` ascending, though the order behaviour can be controled +using the `order` setting. + +Ordering the buckets by their key - descending: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "prices" : { + "histogram" : { + "field" : "price", + "interval" : 50, + "order" : { "_key" : "desc" } + } + } + } +} +-------------------------------------------------- + +Ordering the buckets by their `doc_count` - ascending: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "prices" : { + "histogram" : { + "field" : "price", + "interval" : 50, + "order" : { "_count" : "asc" } + } + } + } +} +-------------------------------------------------- + +If the histogram aggregation has a direct metrics sub-aggregation, the latter can determine the order of the buckets: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "prices" : { + "histogram" : { + "field" : "price", + "interval" : 50, + "order" : { "price_stats.min" : "asc" } <1> + }, + "aggs" : { + "price_stats" : { "stats" : {} } <2> + } + } + } +} +-------------------------------------------------- + +<1> The `{ "price_stats.min" : asc" }` will sort the buckets based on `min` value of their their `price_stats` sub-aggregation. + +<2> There is no need to configure the `price` field for the `price_stats` aggregation as it will inherit it by default from its parent histogram aggregation. + +==== Minimum document count + +It is possible to only return buckets that have a document count that is greater than or equal to a configured +limit through the `min_doc_count` option. + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "prices" : { + "histogram" : { + "field" : "price", + "interval" : 50, + "min_doc_count": 10 + } + } + } +} +-------------------------------------------------- + +The above aggregation would only return buckets that contain 10 documents or more. Default value is `1`. + +NOTE: The special value `0` can be used to add empty buckets to the response between the minimum and the maximum buckets. +Here is an example of what the response could look like: + +[source,js] +-------------------------------------------------- +{ + "aggregations": { + "prices": { + "buckets": { + "0": { + "key": 0, + "doc_count": 2 + }, + "50": { + "key": 50, + "doc_count": 0 + }, + "150": { + "key": 150, + "doc_count": 3 + }, + "200": { + "key": 150, + "doc_count": 0 + }, + "250": { + "key": 150, + "doc_count": 0 + }, + "300": { + "key": 150, + "doc_count": 1 + } + } + } + } +} +-------------------------------------------------- + +==== Response Format + +By default, the buckets are returned as an ordered array. It is also possible to request the response as a hash +instead keyed by the buckets keys: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "prices" : { + "histogram" : { + "field" : "price", + "interval" : 50, + "keyed" : true + } + } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + "aggregations": { + "prices": { + "buckets": { + "0": { + "key": 0, + "doc_count": 2 + }, + "50": { + "key": 50, + "doc_count": 4 + }, + "150": { + "key": 150, + "doc_count": 3 + } + } + } + } +} +-------------------------------------------------- diff --git a/docs/reference/search/aggregations/bucket/iprange-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/iprange-aggregation.asciidoc new file mode 100644 index 0000000..def09e6 --- /dev/null +++ b/docs/reference/search/aggregations/bucket/iprange-aggregation.asciidoc @@ -0,0 +1,98 @@ +[[search-aggregations-bucket-iprange-aggregation]] +=== IPv4 Range + +Just like the dedicated <<search-aggregations-bucket-daterange-aggregation,date>> range aggregation, there is also a dedicated range aggregation for IPv4 typed fields: + +Example: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "ip_ranges" : { + "ip_range" : { + "field" : "ip", + "ranges" : [ + { "to" : "10.0.0.5" }, + { "from" : "10.0.0.5" } + ] + } + } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "ip_ranges": + "buckets" : [ + { + "to": 167772165, + "to_as_string": "10.0.0.5", + "doc_count": 4 + }, + { + "from": 167772165, + "from_as_string": "10.0.0.5", + "doc_count": 6 + } + ] + } + } +} +-------------------------------------------------- + +IP ranges can also be defined as CIDR masks: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "ip_ranges" : { + "ip_range" : { + "field" : "ip", + "ranges" : [ + { "mask" : "10.0.0.0/25" }, + { "mask" : "10.0.0.127/25" } + ] + } + } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + "aggregations": { + "ip_ranges": { + "buckets": [ + { + "key": "10.0.0.0/25", + "from": 1.6777216E+8, + "from_as_string": "10.0.0.0", + "to": 167772287, + "to_as_string": "10.0.0.127", + "doc_count": 127 + }, + { + "key": "10.0.0.127/25", + "from": 1.6777216E+8, + "from_as_string": "10.0.0.0", + "to": 167772287, + "to_as_string": "10.0.0.127", + "doc_count": 127 + } + ] + } + } +} +--------------------------------------------------
\ No newline at end of file diff --git a/docs/reference/search/aggregations/bucket/missing-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/missing-aggregation.asciidoc new file mode 100644 index 0000000..77526c2 --- /dev/null +++ b/docs/reference/search/aggregations/bucket/missing-aggregation.asciidoc @@ -0,0 +1,34 @@ +[[search-aggregations-bucket-missing-aggregation]] +=== Missing + +A field data based single bucket aggregation, that creates a bucket of all documents in the current document set context that are missing a field value (effectively, missing a field or having the configured NULL value set). This aggregator will often be used in conjunction with other field data bucket aggregators (such as ranges) to return information for all the documents that could not be placed in any of the other buckets due to missing field data values. + +Example: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "products_without_a_price" : { + "missing" : { "field" : "price" } + } + } +} +-------------------------------------------------- + +In the above example, we calculate the average price of all the products that are currently in-stock. + +Response: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggs" : { + "products_without_a_price" : { + "doc_count" : 10 + } + } +} +--------------------------------------------------
\ No newline at end of file diff --git a/docs/reference/search/aggregations/bucket/nested-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/nested-aggregation.asciidoc new file mode 100644 index 0000000..cd49f88 --- /dev/null +++ b/docs/reference/search/aggregations/bucket/nested-aggregation.asciidoc @@ -0,0 +1,67 @@ +[[search-aggregations-bucket-nested-aggregation]] +=== Nested + +A special single bucket aggregation that enables aggregating nested documents. + +For example, lets say we have a index of products, and each product holds the list of resellers - each having its own +price for the product. The mapping could look like: + +[source,js] +-------------------------------------------------- +{ + ... + + "product" : { + "properties" : { + "resellers" : { <1> + "type" : "nested" + "properties" : { + "name" : { "type" : "string" }, + "price" : { "type" : "double" } + } + } + } + } +} +-------------------------------------------------- + +<1> The `resellers` is an array that holds nested documents under the `product` object. + +The following aggregations will return the minimum price products can be purchased in: + +[source,js] +-------------------------------------------------- +{ + "query" : { + "match" : { "name" : "led tv" } + } + "aggs" : { + "resellers" : { + "nested" : { + "path" : "resellers" + }, + "aggs" : { + "min_price" : { "min" : { "field" : "nested.value" } } + } + } + } +} +-------------------------------------------------- + +As you can see above, the nested aggregation requires the `path` of the nested documents within the top level documents. +Then one can define any type of aggregation over these nested documents. + +Response: + +[source,js] +-------------------------------------------------- +{ + "aggregations": { + "resellers": { + "min_price": { + "value" : 350 + } + } + } +} +-------------------------------------------------- diff --git a/docs/reference/search/aggregations/bucket/range-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/range-aggregation.asciidoc new file mode 100644 index 0000000..41a4dd1 --- /dev/null +++ b/docs/reference/search/aggregations/bucket/range-aggregation.asciidoc @@ -0,0 +1,274 @@ +[[search-aggregations-bucket-range-aggregation]] +=== Range + +A multi-bucket value source based aggregation that enables the user to define a set of ranges - each representing a bucket. During the aggregation process, the values extracted from each document will be checked against each bucket range and "bucket" the relevant/matching document. + +Example: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "price_ranges" : { + "range" : { + "field" : "price", + "ranges" : [ + { "to" : 50 }, + { "from" : 50, "to" : 100 }, + { "from" : 100 } + ] + } + } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "price_ranges" : { + "buckets": [ + { + "to": 50, + "doc_count": 2 + }, + { + "from": 50, + "to": 100, + "doc_count": 4 + }, + { + "from": 100, + "doc_count": 4 + } + ] + } + } +} +-------------------------------------------------- + +==== Keyed Response + +Setting the `key` flag to `true` will associate a unique string key with each bucket and return the ranges as a hash rather than an array: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "price_ranges" : { + "range" : { + "field" : "price", + "keyed" : true, + "ranges" : [ + { "to" : 50 }, + { "from" : 50, "to" : 100 }, + { "from" : 100 } + ] + } + } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "price_ranges" : { + "buckets": { + "*-50.0": { + "to": 50, + "doc_count": 2 + }, + "50.0-100.0": { + "from": 50, + "to": 100, + "doc_count": 4 + }, + "100.0-*": { + "from": 100, + "doc_count": 4 + } + } + } + } +} +-------------------------------------------------- + +It is also possible to customize the key for each range: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "price_ranges" : { + "range" : { + "field" : "price", + "keyed" : true, + "ranges" : [ + { "key" : "cheap", "to" : 50 }, + { "key" : "average", "from" : 50, "to" : 100 }, + { "key" : "expensive", "from" : 100 } + ] + } + } + } +} +-------------------------------------------------- + +==== Script + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "price_ranges" : { + "range" : { + "script" : "doc['price'].value", + "ranges" : [ + { "to" : 50 }, + { "from" : 50, "to" : 100 }, + { "from" : 100 } + ] + } + } + } +} +-------------------------------------------------- + +==== Value Script + +Lets say the product prices are in USD but we would like to get the price ranges in EURO. We can use value script to convert the prices prior the aggregation (assuming conversion rate of 0.8) + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "price_ranges" : { + "range" : { + "field" : "price", + "script" : "_value * conversion_rate", + "params" : { + "conversion_rate" : 0.8 + }, + "ranges" : [ + { "to" : 35 }, + { "from" : 35, "to" : 70 }, + { "from" : 70 } + ] + } + } + } +} +-------------------------------------------------- + +==== Sub Aggregations + +The following example, not only "bucket" the documents to the different buckets but also computes statistics over the prices in each price range + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "price_ranges" : { + "range" : { + "field" : "price", + "ranges" : [ + { "to" : 50 }, + { "from" : 50, "to" : 100 }, + { "from" : 100 } + ] + }, + "aggs" : { + "price_stats" : { + "stats" : { "field" : "price" } + } + } + } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + "aggregations": { + "price_ranges" : { + "buckets": [ + { + "to": 50, + "doc_count": 2, + "price_stats": { + "count": 2, + "min": 20, + "max": 47, + "avg": 33.5, + "sum": 67 + } + }, + { + "from": 50, + "to": 100, + "doc_count": 4, + "price_stats": { + "count": 4, + "min": 60, + "max": 98, + "avg": 82.5, + "sum": 330 + } + }, + { + "from": 100, + "doc_count": 4, + "price_stats": { + "count": 4, + "min": 134, + "max": 367, + "avg": 216, + "sum": 864 + } + } + ] + } + } +} +-------------------------------------------------- + +If a sub aggregation is also based on the same value source as the range aggregation (like the `stats` aggregation in the example above) it is possible to leave out the value source definition for it. The following will return the same response as above: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "price_ranges" : { + "range" : { + "field" : "price", + "ranges" : [ + { "to" : 50 }, + { "from" : 50, "to" : 100 }, + { "from" : 100 } + ] + }, + "aggs" : { + "price_stats" : { + "stats" : {} <1> + } + } + } + } +} +-------------------------------------------------- + +<1> We don't need to specify the `price` as we "inherit" it by default from the parent `range` aggregation
\ No newline at end of file diff --git a/docs/reference/search/aggregations/bucket/terms-aggregation.asciidoc b/docs/reference/search/aggregations/bucket/terms-aggregation.asciidoc new file mode 100644 index 0000000..3cc7107 --- /dev/null +++ b/docs/reference/search/aggregations/bucket/terms-aggregation.asciidoc @@ -0,0 +1,294 @@ +[[search-aggregations-bucket-terms-aggregation]] +=== Terms + +A multi-bucket value source based aggregation where buckets are dynamically built - one per unique value. + +Example: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "genders" : { + "terms" : { "field" : "gender" } + } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations" : { + "genders" : { + "buckets" : [ + { + "key" : "male", + "doc_count" : 10 + }, + { + "key" : "female", + "doc_count" : 10 + }, + ] + } + } +} +-------------------------------------------------- + +By default, the `terms` aggregation will return the buckets for the top ten terms ordered by the `doc_count`. One can +change this default behaviour by setting the `size` parameter. + +==== Size & Shard Size + +The `size` parameter can be set to define how many term buckets should be returned out of the overall terms list. By +default, the node coordinating the search process will request each shard to provide its own top `size` term buckets +and once all shards respond, it will reduce the results to the final list that will then be returned to the client. +This means that if the number of unique terms is greater than `size`, the returned list is slightly off and not accurate +(it could be that the term counts are slightly off and it could even be that a term that should have been in the top +size buckets was not returned). + +The higher the requested `size` is, the more accurate the results will be, but also, the more expensive it will be to +compute the final results (both due to bigger priority queues that are managed on a shard level and due to bigger data +transfers between the nodes and the client). + +The `shard_size` parameter can be used to minimize the extra work that comes with bigger requested `size`. When defined, +it will determine how many terms the coordinating node will request from each shard. Once all the shards responded, the +coordinating node will then reduce them to a final result which will be based on the `size` parameter - this way, +one can increase the accuracy of the returned terms and avoid the overhead of streaming a big list of buckets back to +the client. + +NOTE: `shard_size` cannot be smaller than `size` (as it doesn't make much sense). When it is, elasticsearch will + override it and reset it to be equal to `size`. + + +==== Order + +The order of the buckets can be customized by setting the `order` parameter. By default, the buckets are ordered by +their `doc_count` descending. It is also possible to change this behaviour as follows: + +Ordering the buckets by their `doc_count` in an ascending manner: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "genders" : { + "terms" : { + "field" : "gender", + "order" : { "_count" : "asc" } + } + } + } +} +-------------------------------------------------- + +Ordering the buckets alphabetically by their terms in an ascending manner: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "genders" : { + "terms" : { + "field" : "gender", + "order" : { "_term" : "asc" } + } + } + } +} +-------------------------------------------------- + + +Ordering the buckets by single value metrics sub-aggregation (identified by the aggregation name): + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "genders" : { + "terms" : { + "field" : "gender", + "order" : { "avg_height" : "desc" } + }, + "aggs" : { + "avg_height" : { "avg" : { "field" : "height" } } + } + } + } +} +-------------------------------------------------- + +Ordering the buckets by multi value metrics sub-aggregation (identified by the aggregation name): + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "genders" : { + "terms" : { + "field" : "gender", + "order" : { "stats.avg" : "desc" } + }, + "aggs" : { + "height_stats" : { "stats" : { "field" : "height" } } + } + } + } +} +-------------------------------------------------- + +==== Minimum document count + +It is possible to only return terms that match more than a configured number of hits using the `min_doc_count` option: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "tags" : { + "terms" : { + "field" : "tag", + "min_doc_count": 10 + } + } + } +} +-------------------------------------------------- + +The above aggregation would only return tags which have been found in 10 hits or more. Default value is `1`. + +NOTE: Setting `min_doc_count`=`0` will also return buckets for terms that didn't match any hit. However, some of + the returned terms which have a document count of zero might only belong to deleted documents, so there is + no warranty that a `match_all` query would find a positive document count for those terms. + +WARNING: When NOT sorting on `doc_count` descending, high values of `min_doc_count` may return a number of buckets + which is less than `size` because not enough data was gathered from the shards. Missing buckets can be + back by increasing `shard_size`. + +==== Script + +Generating the terms using a script: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "genders" : { + "terms" : { + "script" : "doc['gender'].value" + } + } + } +} +-------------------------------------------------- + +==== Value Script + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "genders" : { + "terms" : { + "field" : "gender", + "script" : "'Gender: ' +_value" + } + } + } +} +-------------------------------------------------- + + +==== Filtering Values + +It is possible to filter the values for which buckets will be created. This can be done using the `include` and +`exclude` parameters which are based on regular expressions. + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "tags" : { + "terms" : { + "field" : "tags", + "include" : ".*sport.*", + "exclude" : "water_.*" + } + } + } +} +-------------------------------------------------- + +In the above example, buckets will be created for all the tags that has the word `sport` in them, except those starting +with `water_` (so the tag `water_sports` will no be aggregated). The `include` regular expression will determine what +values are "allowed" to be aggregated, while the `exclude` determines the values that should not be aggregated. When +both are defined, the `exclude` has precedence, meaning, the `include` is evaluated first and only then the `exclude`. + +The regular expression are based on the Java(TM) http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html[Pattern], +and as such, they it is also possible to pass in flags that will determine how the compiled regular expression will work: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "tags" : { + "terms" : { + "field" : "tags", + "include" : { + "pattern" : ".*sport.*", + "flags" : "CANON_EQ|CASE_INSENSITIVE" <1> + }, + "exclude" : { + "pattern" : "water_.*", + "flags" : "CANON_EQ|CASE_INSENSITIVE" + } + } + } + } +} +-------------------------------------------------- + +<1> the flags are concatenated using the `|` character as a separator + +The possible flags that can be used are: +http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#CANON_EQ[`CANON_EQ`], +http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#CASE_INSENSITIVE[`CASE_INSENSITIVE`], +http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#COMMENTS[`COMMENTS`], +http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#DOTALL[`DOTALL`], +http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#LITERAL[`LITERAL`], +http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#MULTILINE[`MULTILINE`], +http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#UNICODE_CASE[`UNICODE_CASE`], +http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#UNICODE_CHARACTER_CLASS[`UNICODE_CHARACTER_CLASS`] and +http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#UNIX_LINES[`UNIX_LINES`] + +==== Execution hint + +There are two mechanisms by which terms aggregations can be executed: either by using field values directly in order to aggregate +data per-bucket (`map`), or by using ordinals of the field values instead of the values themselves (`ordinals`). Although the +latter execution mode can be expected to be slightly faster, it is only available for use when the underlying data source exposes +those terms ordinals. Moreover, it may actually be slower if most field values are unique. Elasticsearch tries to have sensible +defaults when it comes to the execution mode that should be used, but in case you know that one execution mode may perform better +than the other one, you have the ability to "hint" it to Elasticsearch: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "tags" : { + "terms" : { + "field" : "tags", + "execution_hint": "map" <1> + } + } + } +} +-------------------------------------------------- + +<1> the possible values are `map` and `ordinals` + +Please note that Elasticsearch will ignore this execution hint if it is not applicable. diff --git a/docs/reference/search/aggregations/metrics.asciidoc b/docs/reference/search/aggregations/metrics.asciidoc new file mode 100644 index 0000000..068fda2 --- /dev/null +++ b/docs/reference/search/aggregations/metrics.asciidoc @@ -0,0 +1,15 @@ +[[search-aggregations-metrics]] + +include::metrics/min-aggregation.asciidoc[] + +include::metrics/max-aggregation.asciidoc[] + +include::metrics/sum-aggregation.asciidoc[] + +include::metrics/avg-aggregation.asciidoc[] + +include::metrics/stats-aggregation.asciidoc[] + +include::metrics/extendedstats-aggregation.asciidoc[] + +include::metrics/valuecount-aggregation.asciidoc[] diff --git a/docs/reference/search/aggregations/metrics/avg-aggregation.asciidoc b/docs/reference/search/aggregations/metrics/avg-aggregation.asciidoc new file mode 100644 index 0000000..e9d72e0 --- /dev/null +++ b/docs/reference/search/aggregations/metrics/avg-aggregation.asciidoc @@ -0,0 +1,73 @@ +[[search-aggregations-metrics-avg-aggregation]] +=== Avg + +A `single-value` metrics aggregation that computes the average of numeric values that are extracted from the aggregated documents. These values can be extracted either from specific numeric fields in the documents, or be generated by a provided script. + +Assuming the data consists of documents representing exams grades (between 0 and 100) of students + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "avg_grade" : { "avg" : { "field" : "grade" } } + } +} +-------------------------------------------------- + +The above aggregation computes the average grade over all documents. The aggregation type is `avg` and the `field` setting defines the numeric field of the documents the average will be computed on. The above will return the following: + + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "avg_grade": { + "value": 75 + } + } +} +-------------------------------------------------- + +The name of the aggregation (`avg_grade` above) also serves as the key by which the aggregation result can be retrieved from the returned response. + +==== Script + +Computing the average grade based on a script: + +[source,js] +-------------------------------------------------- +{ + ..., + + "aggs" : { + "avg_grade" : { "avg" : { "script" : "doc['grade'].value" } } + } +} +-------------------------------------------------- + +===== Value Script + +It turned out that the exam was way above the level of the students and a grade correction needs to be applied. We can use value script to get the new average: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + ... + + "aggs" : { + "avg_corrected_grade" : { + "avg" : { + "field" : "grade", + "script" : "_value * correction", + "params" : { + "correction" : 1.2 + } + } + } + } + } +} +--------------------------------------------------
\ No newline at end of file diff --git a/docs/reference/search/aggregations/metrics/extendedstats-aggregation.asciidoc b/docs/reference/search/aggregations/metrics/extendedstats-aggregation.asciidoc new file mode 100644 index 0000000..a632f4e --- /dev/null +++ b/docs/reference/search/aggregations/metrics/extendedstats-aggregation.asciidoc @@ -0,0 +1,82 @@ +[[search-aggregations-metrics-extendedstats-aggregation]] +=== Extended Stats + +A `multi-value` metrics aggregation that computes stats over numeric values extracted from the aggregated documents. These values can be extracted either from specific numeric fields in the documents, or be generated by a provided script. + +The `extended_stats` aggregations is an extended version of the <<search-aggregations-metrics-stats-aggregation,`stats`>> aggregation, where additional metrics are added such as `sum_of_squares`, `variance` and `std_deviation`. + +Assuming the data consists of documents representing exams grades (between 0 and 100) of students + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "grades_stats" : { "extended_stats" : { "field" : "grade" } } + } +} +-------------------------------------------------- + +The above aggregation computes the grades statistics over all documents. The aggregation type is `extended_stats` and the `field` setting defines the numeric field of the documents the stats will be computed on. The above will return the following: + + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "grades_stats": { + "count": 6, + "min": 72, + "max": 117.6, + "avg": 94.2, + "sum": 565.2, + "sum_of_squares": 54551.51999999999, + "variance": 218.2799999999976, + "std_deviation": 14.774302013969987 + } + } +} +-------------------------------------------------- + +The name of the aggregation (`grades_stats` above) also serves as the key by which the aggreagtion result can be retrieved from the returned response. + +==== Script + +Computing the grades stats based on a script: + +[source,js] +-------------------------------------------------- +{ + ..., + + "aggs" : { + "grades_stats" : { "extended_stats" : { "script" : "doc['grade'].value" } } + } +} +-------------------------------------------------- + +===== Value Script + +It turned out that the exam was way above the level of the students and a grade correction needs to be applied. We can use value script to get the new stats: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + ... + + "aggs" : { + "grades_stats" : { + "extended_stats" : { + "field" : "grade", + "script" : "_value * correction", + "params" : { + "correction" : 1.2 + } + } + } + } + } +} +--------------------------------------------------
\ No newline at end of file diff --git a/docs/reference/search/aggregations/metrics/max-aggregation.asciidoc b/docs/reference/search/aggregations/metrics/max-aggregation.asciidoc new file mode 100644 index 0000000..17b801e --- /dev/null +++ b/docs/reference/search/aggregations/metrics/max-aggregation.asciidoc @@ -0,0 +1,68 @@ +[[search-aggregations-metrics-max-aggregation]] +=== Max + +A `single-value` metrics aggregation that keeps track and returns the maximum value among the numeric values extracted from the aggregated documents. These values can be extracted either from specific numeric fields in the documents, or be generated by a provided script. + +Computing the max price value across all documents + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "max_price" : { "max" : { "field" : "price" } } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "max_price": { + "value": 35 + } + } +} +-------------------------------------------------- + +As can be seen, the name of the aggregation (`max_price` above) also serves as the key by which the aggregation result can be retrieved from the returned response. + +==== Script + +Computing the max price value across all document, this time using a script: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "max_price" : { "max" : { "script" : "doc['price'].value" } } + } +} +-------------------------------------------------- + + +==== Value Script + +Let's say that the prices of the documents in our index are in USD, but we would like to compute the max in EURO (and for the sake of this example, lets say the conversion rate is 1.2). We can use a value script to apply the conversion rate to every value before it is aggregated: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "max_price_in_euros" : { + "max" : { + "field" : "price", + "script" : "_value * conversion_rate", + "params" : { + "conversion_rate" : 1.2 + } + } + } + } +} +-------------------------------------------------- + diff --git a/docs/reference/search/aggregations/metrics/min-aggregation.asciidoc b/docs/reference/search/aggregations/metrics/min-aggregation.asciidoc new file mode 100644 index 0000000..d1cf0e3 --- /dev/null +++ b/docs/reference/search/aggregations/metrics/min-aggregation.asciidoc @@ -0,0 +1,67 @@ +[[search-aggregations-metrics-min-aggregation]] +=== Min + +A `single-value` metrics aggregation that keeps track and returns the minimum value among numeric values extracted from the aggregated documents. These values can be extracted either from specific numeric fields in the documents, or be generated by a provided script. + +Computing the min price value across all documents: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "min_price" : { "min" : { "field" : "price" } } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "min_price": { + "value": 10 + } + } +} +-------------------------------------------------- + +As can be seen, the name of the aggregation (`min_price` above) also serves as the key by which the aggreagtion result can be retrieved from the returned response. + +==== Script + +Computing the min price value across all document, this time using a script: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "min_price" : { "min" : { "script" : "doc['price'].value" } } + } +} +-------------------------------------------------- + + +==== Value Script + +Let's say that the prices of the documents in our index are in USD, but we would like to compute the min in EURO (and for the sake of this example, lets say the conversion rate is 1.2). We can use a value script to apply the conversion rate to every value before it is aggregated: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "min_price_in_euros" : { + "min" : { + "field" : "price", + "script" : "_value * conversion_rate", + "params" : { + "conversion_rate" : 1.2 + } + } + } + } +} +--------------------------------------------------
\ No newline at end of file diff --git a/docs/reference/search/aggregations/metrics/stats-aggregation.asciidoc b/docs/reference/search/aggregations/metrics/stats-aggregation.asciidoc new file mode 100644 index 0000000..940c113 --- /dev/null +++ b/docs/reference/search/aggregations/metrics/stats-aggregation.asciidoc @@ -0,0 +1,79 @@ +[[search-aggregations-metrics-stats-aggregation]] +=== Stats + +A `multi-value` metrics aggregation that computes stats over numeric values extracted from the aggregated documents. These values can be extracted either from specific numeric fields in the documents, or be generated by a provided script. + +The stats that are returned consist of: `min`, `max`, `sum`, `count` and `avg`. + +Assuming the data consists of documents representing exams grades (between 0 and 100) of students + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "grades_stats" : { "stats" : { "field" : "grade" } } + } +} +-------------------------------------------------- + +The above aggregation computes the grades statistics over all documents. The aggregation type is `stats` and the `field` setting defines the numeric field of the documents the stats will be computed on. The above will return the following: + + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "grades_stats": { + "count": 6, + "min": 60, + "max": 98, + "avg": 78.5, + "sum": 471 + } + } +} +-------------------------------------------------- + +The name of the aggregation (`grades_stats` above) also serves as the key by which the aggregation result can be retrieved from the returned response. + +==== Script + +Computing the grades stats based on a script: + +[source,js] +-------------------------------------------------- +{ + ..., + + "aggs" : { + "grades_stats" : { "stats" : { "script" : "doc['grade'].value" } } + } +} +-------------------------------------------------- + +===== Value Script + +It turned out that the exam was way above the level of the students and a grade correction needs to be applied. We can use a value script to get the new stats: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + ... + + "aggs" : { + "grades_stats" : { + "stats" : { + "field" : "grade", + "script" : "_value * correction", + "params" : { + "correction" : 1.2 + } + } + } + } + } +} +--------------------------------------------------
\ No newline at end of file diff --git a/docs/reference/search/aggregations/metrics/sum-aggregation.asciidoc b/docs/reference/search/aggregations/metrics/sum-aggregation.asciidoc new file mode 100644 index 0000000..69575f5 --- /dev/null +++ b/docs/reference/search/aggregations/metrics/sum-aggregation.asciidoc @@ -0,0 +1,77 @@ +[[search-aggregations-metrics-sum-aggregation]] +=== Sum + +A `single-value` metrics aggregation that sums up numeric values that are extracted from the aggregated documents. These values can be extracted either from specific numeric fields in the documents, or be generated by a provided script. + +Assuming the data consists of documents representing stock ticks, where each tick holds the change in the stock price from the previous tick. + +[source,js] +-------------------------------------------------- +{ + "query" : { + "filtered" : { + "query" : { "match_all" : {}}, + "filter" : { + "range" : { "timestamp" : { "from" : "now/1d+9.5h", "to" : "now/1d+16h" }} + } + } + }, + "aggs" : { + "intraday_return" : { "sum" : { "field" : "change" } } + } +} +-------------------------------------------------- + +The above aggregation sums up all changes in the today's trading stock ticks which accounts for the intraday return. The aggregation type is `sum` and the `field` setting defines the numeric field of the documents of which values will be summed up. The above will return the following: + + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "intraday_return": { + "value": 2.18 + } + } +} +-------------------------------------------------- + +The name of the aggregation (`intraday_return` above) also serves as the key by which the aggregation result can be retrieved from the returned response. + +==== Script + +Computing the intraday return based on a script: + +[source,js] +-------------------------------------------------- +{ + ..., + + "aggs" : { + "intraday_return" : { "sum" : { "script" : "doc['change'].value" } } + } +} +-------------------------------------------------- + +===== Value Script + +Computing the sum of squares over all stock tick changes: + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + ... + + "aggs" : { + "daytime_return" : { + "sum" : { + "field" : "change", + "script" : "_value * _value" } + } + } + } +} +-------------------------------------------------- diff --git a/docs/reference/search/aggregations/metrics/valuecount-aggregation.asciidoc b/docs/reference/search/aggregations/metrics/valuecount-aggregation.asciidoc new file mode 100644 index 0000000..b3f6bf4 --- /dev/null +++ b/docs/reference/search/aggregations/metrics/valuecount-aggregation.asciidoc @@ -0,0 +1,35 @@ +[[search-aggregations-metrics-valuecount-aggregation]] +=== Value Count + +A `single-value` metrics aggregation that counts the number of values that are extracted from the aggregated documents. +These values can be extracted either from specific fields in the documents. Typically, +this aggregator will be used in conjunction with other single-value aggregations. For example, when computing the `avg` +one might be interested in the number of values the average is computed over. + +[source,js] +-------------------------------------------------- +{ + "aggs" : { + "grades_count" : { "value_count" : { "field" : "grade" } } + } +} +-------------------------------------------------- + +Response: + +[source,js] +-------------------------------------------------- +{ + ... + + "aggregations": { + "grades_count": { + "value": 10 + } + } +} +-------------------------------------------------- + +The name of the aggregation (`grades_count` above) also serves as the key by which the aggregation result can be +retrieved from the returned response. + |