path: root/doc/fastcgi.txt

=====================
the FastCGI Interface
=====================

-------------------
Module: mod_fastcgi
-------------------

:Author: Jan Kneschke
:Date: $Date: 2004/11/03 22:26:05 $
:Revision: $Revision: 1.3 $

:abstract:
  The FastCGI interface is the fastest and most secure way
  to interface external process-handlers like Perl, PHP and 
  your self-written applications. 
  
.. meta::
  :keywords: lighttpd, FastCGI
  
.. contents:: Table of Contents

Description
===========

lighttpd provides an interface to a external programs that 
support the FastCGI interface. The FastCGI Interface is 
defined by http://www.fastcgi.com/ and is a 
platform-independent and server independent interface between
a web-application and a webserver.

This means that FastCGI programs that run with the Apache 
Webserver will run seamlessly with lighttpd and vice versa.


FastCGI
-------

FastCGI is removes a lot of the limitations of CGI programs. 
CGI programs have the problem that they have to be restarted 
by the webserver for every request which leads to really bad 
performance values.

FastCGI removes this limitation by keeping the process running 
and handling the requests by this always running process. This 
removes the time used for the fork() and the overall startup 
and cleanup time which is necessary to create and destroy a 
process.

While CGI programs communicate to the server over pipes, 
FastCGI processes use Unix-Domain-Sockets or TCP/IP to talk 
with the webserver. This gives you the second advantage over 
simple CGI programs: FastCGI don't have to run on the Webserver
itself but everywhere in the network. 

lighttpd takes it a little bit further by providing a internal 
FastCGI load-balancer which can be used to balance the load 
over multiple FastCGI Servers. In contrast to other solutions 
only the FastCGI process has to be on the cluster and not the 
whole webserver. That gives the FastCGI process more resources
than a e.g. load-balancer+apache+mod_php solution.

If you compare FastCGI against a apache+mod_php solution you 
should note that FastCGI provides additional security as the 
FastCGI process can be run under different permissions that 
the webserver and can also live a chroot which might be 
different than the one the webserver is running in. 

Options
=======

lighttpd provides the FastCGI support via the fastcgi-module 
(mod_fastcgi) which provides 2 options in the config-file:

fastcgi.debug
  a value between 0 and 65535 to set the debug-level in the 
  FastCGI module. Currently only 0 and 1 are used. Use 1 to 
  enable some debug output, 0 to disable it.

fastcgi.server
  tell the module where to send FastCGI requests to. Every 
  file-extension can have it own handler. Load-Balancing is 
  done by specifying multiple handles for the same extension.
  
  structure of fastcgi.server section: ::
  
    ( <extension> => 
      (  
        ( "host" => <string> ,
          "port" => <integer> ,
	  "socket" => <string>,       # either socket
	                              # or host+port
	  "bin-path" => <string>,     # OPTIONAL 
	  "bin-environment" => <array>, # OPTIONAL 
	  "bin-copy-environment" => <array>, # OPTIONAL 
          "mode" => <string>,         # OPTIONAL
          "docroot" => <string> ,     # OPTIONAL if "mode" 
	                              # is not "authorizer"
          "check-local" => <string>,  # OPTIONAL
	  "min-procs" => <integer>,   # OPTIONAL
	  "max-procs" => <integer>,   # OPTIONAL
	  "max-load-per-proc" => <integer>, # OPTIONAL
	  "idle-timeout" => <integer>, # OPTIONAL
	  "broken-scriptfilename" => <boolean>, # OPTIONAL
          "disable-time" => <integer>, # optional
          "allow-x-send-file" => <boolean> # optional
	),
        ( "host" => ... 
        ) 
      )
    )
  
  :<extension>: is the file-extension or prefix 
                (if started with "/")
  :"host":      is hostname/ip of the FastCGI process
  :"port":      is tcp-port on the "host" used by the FastCGI
                process
  :"bin-path":  path to the local FastCGI binary which should be 
                started if no local FastCGI is running
  :"socket":    path to the unix-domain socket
  :"mode":      is the FastCGI protocol mode. 
                Default is "responder", also "authorizer" 
		mode is implemented.
  :"docroot":   is optional and is the docroot on the remote 
                host for default "responder" mode. For 
		"authorizer" mode it is MANDATORY and it points 
		to docroot for authorized requests. For security
		reasons it is recommended to keep this docroot
                outside of server.document-root tree.
  :"check-local": is optional and may be "enable" (default) or 
                "disable". If enabled the server first check 
		for a file in local server.document-root tree 
		and return 404 (Not Found) if no such file.
                If disabled, the server forward request to 
		FastCGI interface without this check.
  :"broken-scriptfilename": breaks SCRIPT_FILENAME in a wat that 
                PHP can extract PATH_INFO from it (default: disabled)
  :"disable-time": time to wait before a disabled backend is checked
                again
  :"allow-x-send-file": controls if X-LIGHTTPD-send-file headers
                are allowed

  If bin-path is set:

  :"min-procs": sets the minium processes to start
  :"max-procs": the upper limit of the processess to start
  :"max-load-per-proc": maximum number of waiting processes on
                average per process before a new process is 
                spawned
  :"idle-timeout": number of seconds before a unused process
                gets terminated
  :"bin-environment": put an entry into the environment of 
                the started process
  :"bin-copy-environement": clean up the environment and copy
                only the specified entries into the fresh 
                environment of the spawn process


Examples
--------

  Multiple extensions for the same host ::
  
    fastcgi.server = ( ".php" =>
      (( "host" => "127.0.0.1", 
         "port" => 1026,
	 "bin-path" => "/usr/local/bin/php"
      )),
      ".php4" =>
      (( "host" => "127.0.0.1",
	 "port" => 1026
      ))
    )

  Example with prefix: ::
  
    fastcgi.server = ( "/remote_scripts/" =>
      (( "host" => "192.168.0.3",
	 "port" => 9000,
         "check-local" => "disable",
         "docroot" => "/" # remote server may use 
	                  # it's own docroot
      ))
    )
		     
  The request `http://my.host.com/remote_scripts/test.cgi` will
  be forwarded to fastcgi server at 192.168.0.3 and the value
  "/remote_scripts/test.cgi" will be used for the SCRIPT_NAME
  variable. Remote server may prepend it with its own 
  document root. The handling of index files is also the 
  resposibility of remote server for this case.

  In the case that the prefix is not terminated with a slash 
  the prefix will be handled as file and /test.cgi would become
  a PATH_INFO instead of part of SCRIPT_NAME.


  Example for "authorizer" mode: ::
  
    fastcgi.server = ( "/remote_scripts/" =>
      (( "host" => "10.0.0.2",
	 "port" => 9000,
         "docroot" => "/path_to_private_docs",
         "mode" => "authorizer"
      ))
    )

  Note that if "docroot" is specified then its value will be 
  used in DOCUMENT_ROOT and SCRIPT_FILENAME variables passed
  to FastCGI server.

Load-Balancing
==============

The FastCGI plugin provides automaticly a load-balancing between 
multiple FastCGI servers. ::

  fastcgi.server = ( ".php" => 
     (( "host" => "10.0.0.2", "port" => 1030 ),
      ( "host" => "10.0.0.3", "port" => 1030 ))
    )


To understand how the load-balancing works you can enable the 
fastcgi.debug option and will get a similar output as here: ::

  proc: 127.0.0.1 1031  1 1 1 31454
  proc: 127.0.0.1 1028  1 1 1 31442
  proc: 127.0.0.1 1030  1 1 1 31449
  proc: 127.0.0.1 1029  1 1 2 31447
  proc: 127.0.0.1 1026  1 1 2 31438
  got proc: 34 31454
  release proc: 40 31438
  proc: 127.0.0.1 1026  1 1 1 31438
  proc: 127.0.0.1 1028  1 1 1 31442
  proc: 127.0.0.1 1030  1 1 1 31449
  proc: 127.0.0.1 1031  1 1 2 31454
  proc: 127.0.0.1 1029  1 1 2 31447

Even if this for multiple FastCGI children on the local machine 
the following explaination is valid for remote connections too.

The output shows: 

- IP, port, unix-socket (is empty here)
- is-local, state (0 - unset, 1 - running, ... )
- active connections (load)
- PID

As you can see the list is always sorted by the load field. 

Whenever a new connection is requested, the first entry (the one 
with the lowest load) is selected, the load is increased (got proc: ...) 
and the list is sorted again.

If a FastCGI request is done or the connection is dropped, the load on the 
FastCGI proc decreases and the list is sorted again (release proc: ...)

This behaviour is very light-weight in code and still very efficient 
as it keeps the fastcgi-servers equally loaded even if they have different 
CPUs. 

Adaptive Process Spawning
=========================

.. note:: This feature is disabled in 1.3.14 again. min-procs is 
          ignored in that release

Starting with 1.3.8 lighttpd can spawn processes on demand if 
a bin-path is specified and the FastCGI process runs locally.

If you want to have a least one FastCGI process running and 
more of the number of requests increases you can use min-procs 
and max-procs.

A new process is spawned as soon as the average number of 
requests waiting to be handle by a single process increases the
max-load-per-proc setting. 

The idle-timeout specifies how long a fastcgi-process should wait
for a new request before it kills itself. 

Example
-------
::

  fastcgi.server = ( ".php" => 
    (( "socket" => "/tmp/php.socket",
       "bin-path" => "/usr/local/bin/php",
       "min-procs" => 1,
       "max-procs" => 32,
       "max-load-per-proc" => 4,
       "idle-timeout" => 20 
    ))
   )

Disabling Adaptive Spawning
---------------------------

Adaptive Spawning is a quite new feature and it might misbehave 
for your setup. There are several ways to control how the spawing 
is done:

1. ``"max-load-per-proc" => 1``
   if that works for you, great.
   
2. If not set ``min-procs == max-procs``.
   
3. For PHP you can also use: ::
   
     $ PHP_FCGI_CHILDREN=384 ./lighttpd -f ./lighttpd.conf
     
     fastcgi.server = ( ".php" => 
  	(( "socket" => "/tmp/php.socket",
           "bin-path" => "/usr/local/bin/php",
           "min-procs" => 1,
	   "max-procs" => 1,
           "max-load-per-proc" => 4,
           "idle-timeout" => 20 
        ))
      )
   
   It will create one socket and let's PHP create the 384 processes itself.

4. If you don't want lighttpd to manage the fastcgi processes, remove the
   bin-path and use spawn-fcgi to spawn them itself.


FastCGI and Programming Languages
=================================		

Preparing PHP as a FastCGI program
----------------------------------

One of the most important application that has a FastCGI 
interface is php which can be downloaded from 
http://www.php.net/ . You have to recompile the php from 
source to enable the FastCGI interface as it is normally 
not enabled by default in the distributions.

If you already have a working installation of PHP on a 
webserver execute a small script which just contains ::

  <?php phpinfo(); ?>

and search for the line in that contains the configure call. 
You can use it as the base for the compilation. 

You have to remove all occurences of `--with-apxs`, `--with-apxs2` 
and the like which would build PHP with Apache support.  Add the
next three switches to compile PHP with FastCGI support::

  $ ./configure \
    --enable-fastcgi \
    --enable-force-cgi-redirect \
    ...
  
After compilation and installation check that your PHP 
binary contains FastCGI support by calling: ::

  $ php -v
  PHP 4.3.3RC2-dev (cgi-fcgi) (built: Oct 19 2003 23:19:17)

The important part is the (cgi-fcgi).


Starting a FastCGI-PHP
----------------------

Starting with version 1.3.6 lighttpd can spawn the FastCGI 
processes locally itself if necessary: ::

  fastcgi.server = ( ".php" =>
    (( "socket" => "/tmp/php-fastcgi.socket",
       "bin-path" => "/usr/local/bin/php"
    ))
  )

PHP provides 2 special environment variables which control the number of
spawned workes under the control of a single watching process
(PHP_FCGI_CHILDREN) and the number of requests what a single worker
handles before it kills itself. ::

  fastcgi.server = ( ".php" =>
     (( "socket" => "/tmp/php-fastcgi.socket",
        "bin-path" => "/usr/local/bin/php",
        "bin-environment" => ( 
          "PHP_FCGI_CHILDREN" => "16",
          "PHP_FCGI_MAX_REQUESTS" => "10000"
        )
     ))
   )

To increase the security of the started process you should only pass 
the necessary environment variables to the FastCGI process. ::

  fastcgi.server = ( ".php" =>
     (( "socket" => "/tmp/php-fastcgi.socket",
        "bin-path" => "/usr/local/bin/php",
        "bin-environment" => ( 
           "PHP_FCGI_CHILDREN" => "16",
           "PHP_FCGI_MAX_REQUESTS" => "10000" ),
        "bin-copy-environment" => (
	   "PATH", "SHELL", "USER" )
     ))
   )

Configuring PHP
---------------

If you want to use PATH_INFO and PHP_SELF in you PHP scripts you have to
configure php and lighttpd. The php.ini needs the option: ::

  cgi.fix_pathinfo = 1

and the option ``broken-scriptfilename`` in your fastcgi.server config: ::

  fastcgi.server = ( ".php" =>
     (( "socket" => "/tmp/php-fastcgi.socket",
        "bin-path" => "/usr/local/bin/php",
        "bin-environment" => ( 
          "PHP_FCGI_CHILDREN" => "16",
          "PHP_FCGI_MAX_REQUESTS" => "10000" ),
        "bin-copy-environment" => (
          "PATH", "SHELL", "USER" ),
        "broken-scriptfilename" => "enable"
     ))
   )

Why this ? the ``cgi.fix_pathinfo = 0`` would give you a working ``PATH_INFO`` 
but no ``PHP_SELF``. If you enable it, it turns around. To fix the
``PATH_INFO`` `--enable-discard-path` needs a SCRIPT_FILENAME which is against the CGI spec, a
broken-scriptfilename. With ``cgi.fix_pathinfo = 1`` in php.ini and
``broken-scriptfilename => "enable"`` you get both. 


External Spawning
-----------------

Spawning FastCGI processes directly in the webserver has some 
disadvantages like

- FastCGI process can only run locally
- has the same permissions as the webserver
- has the same base-dir as the webserver

As soon as you are using a seperate FastCGI Server to
take off some load from the webserver you have to control
the FastCGI process by a external program like spawn-fcgi.

spawn-fcgi is used to start a FastCGI process in its own
environment and set the user-id, group-id and change to 
another root-directory (chroot).

For convenience a wrapper script should be used which takes 
care of all the necessary option. Such a script in included 
in the lighttpd distribution and is call spawn-php.sh.

The script has a set of config variables you should take 
a look at: ::

  ## ABSOLUTE path to the spawn-fcgi binary
  SPAWNFCGI="/usr/local/sbin/spawn-fcgi"
  
  ## ABSOLUTE path to the PHP binary
  FCGIPROGRAM="/usr/local/bin/php"
  
  ## bind to tcp-port on localhost
  FCGIPORT="1026"
  
  ## bind to unix domain socket
  # FCGISOCKET="/tmp/php.sock"
  
  ## number of PHP childs to spawn
  PHP_FCGI_CHILDREN=10
  
  ## number of request server by a single php-process until
  ## is will be restarted
  PHP_FCGI_MAX_REQUESTS=1000
  
  ## IP adresses where PHP should access server connections
  ## from
  FCGI_WEB_SERVER_ADDRS="127.0.0.1,192.168.0.1"
  
  # allowed environment variables sperated by spaces
  ALLOWED_ENV="ORACLE_HOME PATH USER"
  
  ## if this script is run as root switch to the following user
  USERID=wwwrun
  GROUPID=wwwrun

If you have set the variables to values that fit to your 
setup you can start it by calling: ::

  $ spawn-php.sh
  spawn-fcgi.c.136: child spawned successfully: PID: 6925

If you get "child spawned successfully: PID:" the php 
processes could be started successfully. You should see them 
in your processlist: ::

  $ ps ax | grep php
  6925 ?        S      0:00 /usr/local/bin/php
  6928 ?        S      0:00 /usr/local/bin/php
  ...

The number of processes should be PHP_FCGI_CHILDREN + 1. 
Here the process 6925 is the master of the slaves which 
handle the work in parallel. Number of parallel workers can 
be set by PHP_FCGI_CHILDREN. A worker dies automaticly of 
handling PHP_FCGI_MAX_REQUESTS requests as PHP might have 
memory leaks.

If you start the script as user root php processes will be 
running as the user USERID and group GROUPID to drop the 
root permissions. Otherwise the php processes will run as 
the user you started script as.

As the script might be started from a unknown stage or even 
directly from the command-line it cleans the environment 
before starting the processes. ALLOWED_ENV contains all 
the external environement variables that should be available 
to the php-process.




Perl
----

For Perl you have to install the FCGI module from CPAN.

TCL
---

For TCL ...



Skeleton for remote authorizer
==============================

The basic functionality of authorizer is as follows (see
http://www.fastcgi.com/devkit/doc/fcgi-spec.html, 6.3 for 
details). ::

  #include <fcgi_stdio.h>
  #include <stdlib.h>
  #include <unistd.h>
  int main () {
    char* p;
  
    while (FCGI_Accept() >= 0) {   
      /* wait for fastcgi authorizer request */
  
      printf("Content-type: text/html\r\n");
  
      if ((p = getenv("QUERY_STRING")) == NULL) ||
           <QUERY_STRING is unauthorized>)
           printf("Status: 403 Forbidden\r\n\r\n");

      else printf("\r\n");  
        /* default Status is 200 - allow access */
    }
  
    return 0;
  }

It is possible to use any other variables provided by 
FastCGI interface for authorization check. Here is only an 
example.


Troubleshooting
===============

fastcgi.debug should be enabled for troubleshooting.

If you get: ::

  (fcgi.c.274) connect delayed:  8
  (fcgi.c.289) connect succeeded:  8
  (fcgi.c.745) unexpected end-of-file (perhaps the fastcgi 
     process died):  8

the fastcgi process accepted the connection but closed it 
right away. This happens if FCGI_WEB_SERVER_ADDRS doesn't 
include the host where you are connection from.

If you get ::

  (fcgi.c.274) connect delayed:  7
  (fcgi.c.1107) error: unexpected close of fastcgi connection 
     for /peterp/seite1.php (no fastcgi process on host/port ?)
  (fcgi.c.1015) emergency exit: fastcgi: connection-fd: 5 
     fcgi-fd: 7

the fastcgi process is not running on the host/port you are 
connection to. Check your configuration.

If you get ::

  (fcgi.c.274) connect delayed:  7
  (fcgi.c.289) connect succeeded:  7

everything is fine. The connect() call just was delayed a 
little bit and is completly normal.