Lighttpd

Table of Contents

Lighttpd

Credits

About the Author

About the Reviewer

Preface

What This Book Covers

What You Need For this Book

Who is This Book For

Conventions

Reader Feedback

Customer Support

Downloading the Example Code for the Book

Errata

Piracy

Questions

1. Introduction to Lighttpd

Installing Lighttpd

Building Lighttpd using Autotools

Building Lighttpd using CMake

Summary

2. Configuring and Running Lighttpd

Starting Lighttpd by Hand

Other Core Options

Mime Types

Selectors

Excursion: Regular Expressions

Play it Again, Sam

Are You There?

Decisions, Decisions

Group and Capture

Lucky Escape

Rewriting and Redirecting Requests

Including Variables, Files, and Shell-code

Summary

3. More Virtual Hosting and CGI

Extended Virtual Hosting

MySQL based Virtual Hosting

Installing MySQL

Bringing MySQL and mod_mysqlvhost Together

Going Dynamic

CGI with mod_cgi

FastCGI

SCGI

mod_proxy_core and backends

Summary

4. Downloads and Streams

Core Settings

Traffic Shaping

Showing Directory Contents

Securing Downloads

Streaming Content

Putting it All Together

Summary

5. Big Brother Lighttpd

Privacy

O Browser, Where Art Thou?

Access Logging

Tracking Users

Other Data Points

Summary

6. Encryption: SSL

Self-Signed Keys

Being our own Certificate Authority

Obtaining a Key Pair from a Third-Party Supplier

Configuring Lighttpd to use SSL

Summary

7. Securing Lighttpd

Barriers to Entry

Evading Denial of Service Attacks

Setting up Logrotate

Know Your Foe

RRDtool

Grepping the Logs

Summary

8. Containing Lighttpd

Giving up Privileges

Changing Root

Separating the Backend

Summary

9. Optimizing Lighttpd

Installing http_load

Running http_load Tests

Specific Optimizations

Example: Caching with mod_magnet

Measuring System Load

Profiling with gprof

Load Testing our Profiling Build

Summary

10. Migration from Apache

Adding Lighttpd to the Mix

Excursion: mod_proxy

Reducing Apache Load

mod_perl, mod_php, and mod_python

.htaccess

.htaccess and PHP

Rewriting Rules

WebDAV

Summary

11. CGI Revisited

Ruby on Rails

WordPress

phpMyAdmin

MediaWiki

Trac

AWStats

AjaxTerm

Summary

12. Using Lua with Lighttpd

Lua: A small Primer

Useful Lua Libraries

Lua/FastCGI

Installing Lua/FastCGI

GET and POST Requests

Looking at the Cache

Running mod_magnet

Example: A Shoutbox

Summary

13. Writing Lighttpd Modules

Handling Configuration

Rewriting the Request

Manipulating the Response

Summary

A. HTTP Status Codes

B. Module/Configuration Index

Internal

mod_access

mod_accesslog

mod_alias

mod_auth

mod_cgi

mod_cml

mod_chunked

mod_compress

mod_deflate

mod_dirlisting

mod_evasive

mod_evhost

mod_expire

mod_fastcgi

mod_flv_streaming

mod_indexfile

mod_magnet

mod_proxy

mod_proxy_core

mod_redirect

mod_rewrite

mod_rrdtool

mod_scgi

mod_secure_download

mod_setenv

mod_simple_vhost

mod_sql_vhost_core, mod_mysql_vhost

mod_ssi

mod_staticfile

mod_status

mod_trigger_b4_dl

mod_uploadprogress

mod_userdir

mod_usertrack

mod_webdav

Index

Lighttpd

Andre Bogus

Lighttpd

Copyright © 2008 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, Packt Publishing, nor its dealers or distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: October 2008

Production Reference: 1151008

Published by Packt Publishing Ltd.

32 Lincoln Road

Olton

Birmingham, B27 6PA, UK.

ISBN 978-1-847192-10-3

www.packtpub.com

Cover Image by Vinayak Chittar (<vinayak.chittar@gmail.com>)

Credits

Author

Andre Bogus

Reviewer

Peter Lavetsky

Development Editor

Swapna V. Verlekar

Technical Editors

Dhiraj Chandiramani

Rasika Sathe

Editorial Team Leader

Akshara Aware

Project Manager

Abhijeet Deobhakta

Project Coordinator

Abhijeet Deobhakta

Indexer

Monica Ajmera

Proofreader

Claire Lane

Production Coordinator

Shantanu Zagade

Cover Work

Shantanu Zagade

About the Author

Andre Bogus is a musician turned programmer. He has worked in different jobs from voice acting to programming to teaching to managing software projects. At the moment he works as a consultant and implementer for KOGIT GmbH, an Identity Management company based in Germany.

He found Lighttpd while searching for the ideal software for his personal web server and quickly learned the tricks to make it do what he wanted. He enjoys learning new things and telling others about them. When his full schedule allows it, he can be found on the #lighttpd IRC channel.

He wants to thank his wife, Ania, without whose support he would not have been able to finish this book. Also he appreciates his employer for allowing him to write besides his day job. The nice people at PACKT Publishing have also earned his gratitude by helping this book to become what it is.

About the Reviewer

Peter Lavetsky is a Senior Research and Development Analyst with Dealer.com, located in Burlington, VT. He has written multiple Lighttpd plugins as well as tuned many instances tailored to Dealer.com’s web serving needs. Peter currently works on integrating third-parties into the Dealer.com platform, including Google Base and Google AdWords. In his spare time he enjoys checkraising the turn and feeding the tiger shark.

Preface

This book explains downloading, installing, and configuring the Lighttpd HTTP server, illustrates how to extend it with modules and Lua code, shows a migration path from Apache httpd, gives case studies in setting up a number of popular web applications, and even demonstrates how to extend Lighttpd by writing our own modules.

The name Lighttpd (pronounced Lighty) is an abbreviation pulling together Light (as in weight) and HTTPD (which is an abbreviation for Hypertext Transport Protocol Daemon, in short web server). Early versions called themselves LightTPD to emphasize the lightweight part, but this led to confusion over pronunciation and meaning, so the capitalization was reduced.

What This Book Covers

Chapter 1 gives directions how to obtain Lighttpd. Regardless, if we want to use a binary package or build from source, everything is there. In addition, dependencies, optional packages, and compilation options are examined. After working through this chapter, we should have an installed Lighttpd to work with.

Chapter 2 introduces all elements of the configuration language by example. Usable examples include sending the correct MIME type, setting up multiple domains, rewriting, and redirecting. Also the command line options are explained. For those who are not fluent in regular expressions, the chapter has an excursion. At the end of this chapter, we have our Lighttpd up and running.

Chapter 3 builds on the concepts of the second chapter and discusses the configuration various CGI-like interfaces, three modules for virtual hosting, also introducing the MySQL database, which is used in one of the modules.

Chapter 4 shows how to set up Lighttpd as a download or streaming server, covering optimizations for large downloads as well as guarding our site against denial of service attacks, dealing with proxies, and restricting download speeds for anonymous clients.

Chapter 5 extends our Lighttpd to learn more about our users: Geo-tracking the location from the client IP address, dissecting the page traversal behavior (clickstream analysis) and other data points. Also responsible access logging practices are outlined.

Chapter 6 adds SSL support to our Lighttpd and walks through the steps to acquire or create the required certificates, whether we obtain a certificate from a public or corporate certificate authority, self-sign a certificate, or become our own certificate authority.

Chapter 7 helps us securing our Lighttpd by authorizing access, limiting traffic by IP to thwart denial-of-service attacks, and measuring our success by rigorously examination of our log files. Setting up log rotate and log parsers is also covered.

Chapter 8 concerns itself with limiting the potential damage a subverted Lighttpd could do to the system. The techniques to achieve this are reducing privileges and putting the whole Lighttpd in a secluded environment. Containing Lighttpd and a CGI backend in different environments is also demonstrated.

Chapter 9 shows a strategy to optimize our Lighttpd from system and configuration settings to the source code itself. The chapter also shows specific optimizations known to yield benefits across most systems.

Chapter 10 takes a pragmatic look on the migration path from Apache httpd. It shows how to port basic configuration, rewrite and redirect rules, how to deal with .htaccess files, and even discusses when not to migrate.

Chapter 11 revisits the CGI interfaces by getting various example applications from Ruby on Rails over WordPress, phpMyAdmin, trac, and AWstats to AjaxTerm up and running with our Lighttpd.

Chapter 12 adds the small and fast scripting language Lua to the mix, which can be used to extend the functionality of Lighttpd by mod_magnet or as a backend language by the Lua/FastCGI interface written by the same author as Lighttpd. Both options are discussed, along with an introduction to the language itself.

Chapter 13 gives a run down of extending Lighttpd by extending existing modules or even writing our own. With these modules, we can change the behavior of Lighttpd from request parsing to sending or altering content. This chapter is aimed at an average C programmer.

Appendix A lists the HTTP status codes that our Lighttpd can return on answering a request, giving directions which chapter or other source might have more information on each request.

Appendix B is the module and configuration index. Each configuration option for every Lighttpd module of the official distribution is explained here in one or two short sentences. Forgotten how a configuration option is written, what type it has or what it means? Look no further.

What You Need For this Book

To work through this book effectively, you will need at least a computer running on one of the supported operating systems (Refer to Chapter 1 on installation) connected to the Internet. Basic knowledge about computers, the Internet, (especially the HTTP protocol), and one or more programming language is also helpful.

Who is This Book For

This book pulls together all the information and gives helpful examples instead of complex theories. As Lighttpd is mostly used in an environment, common interfaces are also shown.

So, if you are a web developer or an administrator, and you want to learn how you can install, configure, secure, optimize (or even extend), and generally get the most out of Lighttpd, you should read this book.

Now, before reaping the benefits of Lighttpd, we first need to download and install it.

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text are shown as follows: We can include other contexts through the use of the include directive.

A block of code will be set as follows:

$HTTP[url] =~ .py { # use SCGI for python files

proxy_core.protocol = scgi

proxy-core.balancer = carp # tries to keep processes together

proxy-core.backends = { # we have 3 SCGI servers to balance:

127.0.0.1:3456, # a local port (by IP address)

otherhost.mydomain.net:3456, # a port on another host

unix:/tmp/python.socket # a unix socket

}

proxy-core.max-pool-size = 3 # for SCGI the number of backends

# for other options, see Appendix B

}

When we wish to draw your attention to a particular part of a code block, the relevant lines or items will be made bold:

while

(FCGI_Accept())

Any command-line input and output is written as follows:

$ gcc -Wall -O2 -g -o magnet magnet.c -lfcgi -llua -lm -ldl -Wl,-E

New terms and important words are introduced in a bold-type font. Words that you see on the screen, in menus or dialog boxes for example, appear in our text like this: clicking the Next button moves you to the next screen.

Note

Important notes appear in a box like this.

Note

Tips and tricks appear like this.

Reader Feedback

Feedback from our readers is always welcome. Let us know what you think about this book, what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply drop an email to <feedback@packtpub.com>, making sure to mention the book title in the subject of your message.

If there is a book that you need and would like to see us publish, please send us a note in the SUGGEST A TITLE form on www.packtpub.com or email .

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.

Customer Support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the Example Code for the Book

Visit http://www.packtpub.com/files/code/2103_Code.zip to directly download the example code.

The downloadable files contain instructions on how to use them.

Errata

Although we have taken every care to ensure the accuracy of our contents, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in text or code—we would be grateful if you would report this to us. By doing this you can save other readers from frustration, and help to improve subsequent versions of this book. If you find any errata, report them by visiting http://www.packtpub.com/support, selecting your book, clicking on the let us know link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata added to the list of existing errata. The existing errata can be viewed by selecting your title from http://www.packtpub.com/support.

Piracy

Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide the location address or website name immediately so we can pursue a remedy.

Please contact us at <copyright@packtpub.com> with a link to the suspected pirated material.

We appreciate your help in protecting our authors, and our ability to bring you valuable content.

Questions

You can contact us at <questions@packtpub.com> if you are having a problem with some aspect of the book, and we will do our best to address it.

Chapter 1. Introduction to Lighttpd

In this chapter, we will learn:

What Lighttpd is

How to install Lighttpd

How to build Lighttpd using:

Autotools

CMake

What is Lighttpd? Lighttpd, or Lighty, as it is affectionately called, is an extensible, modular, low-footprint, single-threaded, high performance, web server that will happily run on small servers, and outperform an Apache server or Microsoft IIS in most settings. Lighttpd powers many large sites, such as the YouTube video download servers and the image upload server of Wikipedia. At the time of this writing, Lighttpd has the fifth place in the netcraft web server top ten. The plugin architecture encourages developing custom modules and trying new ideas. The development community around Lighttpd is friendly, helpful and pragmatic, and the documentation, though a little scattered, is quite thorough, if you know where to look.

Installing Lighttpd

Lighttpd has very little dependencies considering the wealth of functionalities it provides. For most systems, getting Lighttpd is just a matter of downloading and installing a package. Before we go out and get one, we better know what we want. There are two branches of Lighttpd: a stable branch and a development branch.

The stable branch is very solid and changes at the most once every two months (if bug fixes are not counted, then about once a year), allowing the developers to concentrate on bug fixes. The development branch moves faster, with a new release every four to six weeks. The development snapshots contain new shiny features, but can also contain hidden bugs, break old features and can generally be less stable.

At the time of writing, version 1.4.19 is deemed to be the stable version, while pre-releases of the upcoming 1.5.0 version are distributed for more testing before the final release. Some systems might have packages of older versions, but anything older than the stable branch many contain known security holes.

For a live server, or if we want the latest versions, we usually compile Lighttpd from sources. For a development server, we might take the easy route and install a precompiled package to leave the worries about dependencies to whoever maintains the package database.

The last question is, on which system we should use Lighttpd? My pragmatic advice is to use what you have.