1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
|
[[setup-configuration]]
== Configuration
[float]
=== Environment Variables
Within the scripts, Elasticsearch comes with built in `JAVA_OPTS` passed
to the JVM started. The most important setting for that is the `-Xmx` to
control the maximum allowed memory for the process, and `-Xms` to
control the minimum allocated memory for the process (_in general, the
more memory allocated to the process, the better_).
Most times it is better to leave the default `JAVA_OPTS` as they are,
and use the `ES_JAVA_OPTS` environment variable in order to set / change
JVM settings or arguments.
The `ES_HEAP_SIZE` environment variable allows to set the heap memory
that will be allocated to elasticsearch java process. It will allocate
the same value to both min and max values, though those can be set
explicitly (not recommended) by setting `ES_MIN_MEM` (defaults to
`256m`), and `ES_MAX_MEM` (defaults to `1gb`).
It is recommended to set the min and max memory to the same value, and
enable <<setup-configuration-memory,`mlockall`>>.
[float]
[[system]]
=== System Configuration
[float]
[[file-descriptors]]
==== File Descriptors
Make sure to increase the number of open files descriptors on the
machine (or for the user running elasticsearch). Setting it to 32k or
even 64k is recommended.
In order to test how many open files the process can open, start it with
`-Des.max-open-files` set to `true`. This will print the number of open
files the process can open on startup.
Alternatively, you can retrieve the `max_file_descriptors` for each node
using the <<cluster-nodes-info>> API, with:
[source,js]
--------------------------------------------------
curl localhost:9200/_nodes/process?pretty
--------------------------------------------------
[float]
[[setup-configuration-memory]]
==== Memory Settings
There is an option to use
http://opengroup.org/onlinepubs/007908799/xsh/mlockall.html[mlockall] to
try to lock the process address space so it won't be swapped. For this
to work, the `bootstrap.mlockall` should be set to `true` and it is
recommended to set both the min and max memory allocation to be the
same. Note: This option is only available on Linux/Unix operating
systems.
In order to see if this works or not, set the `common.jna` logging to
DEBUG level. A solution to "Unknown mlockall error 0" can be to set
`ulimit -l unlimited`.
Note, `mlockall` might cause the JVM or shell
session to exit if it fails to allocate the memory (because not enough
memory is available on the machine).
[float]
[[settings]]
=== Elasticsearch Settings
*elasticsearch* configuration files can be found under `ES_HOME/config`
folder. The folder comes with two files, the `elasticsearch.yml` for
configuring Elasticsearch different
<<modules,modules>>, and `logging.yml` for
configuring the Elasticsearch logging.
The configuration format is http://www.yaml.org/[YAML]. Here is an
example of changing the address all network based modules will use to
bind and publish to:
[source,yaml]
--------------------------------------------------
network :
host : 10.0.0.4
--------------------------------------------------
[float]
[[paths]]
==== Paths
In production use, you will almost certainly want to change paths for
data and log files:
[source,yaml]
--------------------------------------------------
path:
logs: /var/log/elasticsearch
data: /var/data/elasticsearch
--------------------------------------------------
[float]
[[cluster-name]]
==== Cluster name
Also, don't forget to give your production cluster a name, which is used
to discover and auto-join other nodes:
[source,yaml]
--------------------------------------------------
cluster:
name: <NAME OF YOUR CLUSTER>
--------------------------------------------------
[float]
[[node-name]]
==== Node name
You may also want to change the default node name for each node to
something like the display hostname. By default Elasticsearch will
randomly pick a Marvel character name from a list of around 3000 names
when your node starts up.
[source,yaml]
--------------------------------------------------
node:
name: <NAME OF YOUR NODE>
--------------------------------------------------
Internally, all settings are collapsed into "namespaced" settings. For
example, the above gets collapsed into `node.name`. This means that
its easy to support other configuration formats, for example,
http://www.json.org[JSON]. If JSON is a preferred configuration format,
simply rename the `elasticsearch.yml` file to `elasticsearch.json` and
add:
[float]
[[styles]]
==== Configuration styles
[source,yaml]
--------------------------------------------------
{
"network" : {
"host" : "10.0.0.4"
}
}
--------------------------------------------------
It also means that its easy to provide the settings externally either
using the `ES_JAVA_OPTS` or as parameters to the `elasticsearch`
command, for example:
[source,sh]
--------------------------------------------------
$ elasticsearch -Des.network.host=10.0.0.4
--------------------------------------------------
Another option is to set `es.default.` prefix instead of `es.` prefix,
which means the default setting will be used only if not explicitly set
in the configuration file.
Another option is to use the `${...}` notation within the configuration
file which will resolve to an environment setting, for example:
[source,js]
--------------------------------------------------
{
"network" : {
"host" : "${ES_NET_HOST}"
}
}
--------------------------------------------------
The location of the configuration file can be set externally using a
system property:
[source,sh]
--------------------------------------------------
$ elasticsearch -Des.config=/path/to/config/file
--------------------------------------------------
[float]
[[configuration-index-settings]]
=== Index Settings
Indices created within the cluster can provide their own settings. For
example, the following creates an index with memory based storage
instead of the default file system based one (the format can be either
YAML or JSON):
[source,sh]
--------------------------------------------------
$ curl -XPUT http://localhost:9200/kimchy/ -d \
'
index :
store:
type: memory
'
--------------------------------------------------
Index level settings can be set on the node level as well, for example,
within the `elasticsearch.yml` file, the following can be set:
[source,yaml]
--------------------------------------------------
index :
store:
type: memory
--------------------------------------------------
This means that every index that gets created on the specific node
started with the mentioned configuration will store the index in memory
*unless the index explicitly sets it*. In other words, any index level
settings override what is set in the node configuration. Of course, the
above can also be set as a "collapsed" setting, for example:
[source,sh]
--------------------------------------------------
$ elasticsearch -Des.index.store.type=memory
--------------------------------------------------
All of the index level configuration can be found within each
<<index-modules,index module>>.
[float]
[[logging]]
=== Logging
Elasticsearch uses an internal logging abstraction and comes, out of the
box, with http://logging.apache.org/log4j/[log4j]. It tries to simplify
log4j configuration by using http://www.yaml.org/[YAML] to configure it,
and the logging configuration file is `config/logging.yml` file.
|
