1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
|
========================
Performance Improvements
========================
------------
Module: core
------------
:Author: Jan Kneschke
:Date: $Date: 2004/11/03 22:26:05 $
:Revision: $Revision: 1.3 $
:abstract:
handling performance issues in lighttpd
.. meta::
:keywords: lighttpd, performance
.. contents:: Table of Contents
Description
===========
Performance Issues
------------------
lighttpd is optimized into varying directions. The most important direction is
performance. The operation system has two major facilities to help lighttpd
a deliver its best performance.
HTTP Keep-Alive
---------------
Disabling keep-alive might help your server if you suffer from a large
number of open file descriptors.
The defaults for the server are: ::
server.max-keep-alive-requests = 128
server.max-keep-alive-idle = 30
server.max-read-idle = 60
server.max-write-idle = 360
handling 128 keep-alive requests in a row on a single connection, waiting 30 seconds
before an unused keep-alive connection gets dropped by lighttpd.
If you handle several connections at once under a high load (let's assume 500 connections
in parallel for 24h) you might run into the out-of-fd problem described below. ::
server.max-keep-alive-requests = 4
server.max-keep-alive-idle = 4
would release the connections earlier and would free file descriptors without a
detrimental performance loss.
Disabling keep-alive completely is the last resort if you are still short on file descriptors: ::
server.max-keep-alive-requests = 0
Event Handlers
--------------
The first one is the Event Handler which takes care of notifying the server
that one of the connections is ready to send or receive. As you can see,
every OS has at least the select() call which has some limitations.
============ ========== ===============
OS Method Config Value
============ ========== ===============
all select select
Unix poll poll
Linux 2.4+ rt-signals linux-rtsig
Linux 2.6+ epoll linux-sysepoll
Solaris /dev/poll solaris-devpoll
FreeBSD, ... kqueue freebsd-kqueue
============ ========== ===============
For more information on this topic take a look at http://www.kegel.com/c10k.html
Configuration
`````````````
The event handler can be set by specifying the 'Config Value' from above
in the ``server.event-handler`` variable
e.g.: ::
server.event-handler = "linux-sysepoll"
Network Handlers
----------------
The basic network interface for all platforms at the syscalls read() and
write(). Every modern OS provides its own syscall to help network servers
transfer files as fast as possible.
If you want to send out a file from the webserver, it doesn't make any sense
to copy the file into the webserver just to write() it back into a socket
in the next step.
sendfile() minimizes the work in the application and pushes a file directly
into the network card (ideally).
lighttpd supports all major platform-specific calls:
========== ==========
OS Method
========== ==========
all write
Unix writev
Linux 2.4+ sendfile
Linux 2.6+ sendfile64
Solaris sendfilev
FreeBSD sendfile
========== ==========
The best backend is selected at compile time. In case you want to use
another backend set: ::
server.network-backend = "writev"
You can find more information about network backend in:
http://blog.lighttpd.net/articles/2005/11/11/optimizing-lighty-for-high-concurrent-large-file-downloads
Max Connections
---------------
As lighttpd is a single-threaded server, its main resource limit is the
number of file descriptors, which is set to 1024 by default (on most systems).
If you are running a high-traffic site you might want to increase this limit
by setting ::
server.max-fds = 2048
This only works if lighttpd is started as root.
Out-of-fd condition
-------------------
Since file descriptors are used for TCP/IP sockets, files and directories,
a simple request for a PHP page might result in using 3 file descriptors:
1. the TCP/IP socket to the client
2. the TCP/IP and Unix domain socket to the FastCGI process
3. the filehandle to the file in the document root to check if it exists
If lighttpd runs out of file descriptors, it will stop accepting new
connections for awhile to use the existing file descriptors to handle the
currently-running requests.
If more than 90% of the file descriptors are used then the handling of new
connections is disabled. If it drops below 80% again new connections will
be accepted again.
Under some circumstances you will see ::
... accept() failed: Too many open files
in the error log. This tells you there were too many new requests at once
and lighttpd could not disable the incoming connections soon enough. The
connection was dropped and the client received an error message like 'connection
failed'. This is very rare and might only occur in test setups.
Increasing the ``server.max-fds`` limit will reduce the probability of this
problem.
stat() cache
============
A stat(2) can be expensive; caching it saves time and context switches.
Instead of using stat() every time to check for the existence of a file
you can stat() it once and monitor the directory the file is in for
modifications. As long as the directory doesn't change, the files in it
must all still be the same.
With the help of FAM or gamin you can use kernel events to assure that
your stat cache is up to date. ::
server.stat-cache-engine = "fam" # either fam, simple or disabled
Platform-Specific Notes
=======================
Linux
-----
For Linux 2.4.x you should think about compiling lighttpd with the option
``--disable-lfs`` to disable the support for files larger than 2GB. lighttpd will
fall back to the ``writev() + mmap()`` network calls which is ok, but not as
fast as possible but support files larger than 2GB.
Disabling the TCP options reduces the overhead of each TCP packet and might
help to get the last few percent of performance out of the server. Be aware that
disabling these options most likely decreases performance for high-latency and lossy
links.
- net.ipv4.tcp_sack = 0
- net.ipv4.tcp_timestamps = 0
Increasing the TCP send and receive buffers will increase the performance a
lot if (and only if) you have a lot of large files to send.
- net.ipv4.tcp_wmem = 4096 65536 524288
- net.core.wmem_max = 1048576
If you have a lot of large file uploads, increasing the receive buffers will help.
- net.ipv4.tcp_rmem = 4096 87380 524288
- net.core.rmem_max = 1048576
Keep in mind that every TCP connection uses the configured amount of memory for socket
buffers. If you've got many connections this can quickly drain the available memory.
See http://www.acc.umu.se/~maswan/linux-netperf.txt for more information on these parameters.
FreeBSD
-------
On FreeBSD you might gain some performance by enabling accept filters. Just
compile your kernel with: ::
options ACCEPT_FILTER_HTTP
For more ideas about tuning FreeBSD read: tuning(7)
Reducing the recvspace should always be ok if the server only handles HTTP
requests without large uploads. Increasing the sendspace would reduce the
system load if you have a lot of large files to be sent, but keep in mind that
you have to provide the memory in the kernel for each connection. 1024 * 64KB
would mean 64MB of kernel RAM. Keep this in mind.
- net.inet.tcp.recvspace = 4096
|
