From normalperson at yhbt.net  Thu Apr  8 22:35:47 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 9 Apr 2010 02:35:47 +0000
Subject: [ANN] raindrops - real-time stats for preforking Rack servers
Message-ID: <20100409023547.GA5154@dcvr.yhbt.net>

(I announced this yesterday on ruby-talk, but I realized it would
be wise to actually announce it on the list of a server it was
designed for :)

Raindrops is a real time stats package to show statistics for Rack HTTP
servers.  It is designed for preforking servers such as Rainbows! and
Unicorn, but should support any Rack HTTP server under Ruby 1.9, 1.8 and
possibly Rubinius (untested) on platforms supporting POSIX shared memory
and compiled with GCC (for atomic builtins).

Raindrops includes a Struct-like Raindrops::Struct class that may be used
standalone to create atomic counters shared across any number of forked
processes under SMP.

* http://raindrops.bogomips.org/
* raindrops at librelist.com
* git://git.bogomips.org/raindrops.git

== Features

* counters are shared across all forked children and lock-free

* counters are kept on separate cache lines to reduce contention under SMP

* may expose server statistics as a Rack Middleware endpoint
  (default: "/_raindrops")

* middleware displays the number of actively processing and writing
  clients from a single request regardless of which worker process
  it hits.

== Linux-only Extra Features!

* Middleware response includes extra stats for bound TCP and
  Unix domain sockets (configurable, it can include stats from
  other TCP or UNIX domain socket servers).

* TCP socket stats use efficient inet_diag facilities via netlink
  instead of parsing /proc/net/tcp to minimize overhead.
  This was fun to discover and write.

== Install

raindrops requires GCC 4.x (or compatible) or later to support the
atomic builtins (__sync_{add,sub}_and_fetch()).  Atomic operations on
other compilers may be supported if there is demand.

If you're using a packaged Ruby distribution, make sure you have a C
compiler and the matching Ruby development libraries and headers.

If you use RubyGems:

    gem install raindrops

Otherwise grab the latest tarball from:

http://raindrops.bogomips.org/files/

Unpack it, and run "ruby setup.rb"

== Usage (Rainbows!/Unicorn preload_app=false)

If you're using preload_app=false (the default) in your Rainbows!/Unicorn
config file, you'll need to create the global Stats object before
forking.

        require 'raindrops'
        $stats ||= Raindrops::Middleware::Stats.new

In your Rack config.ru:

        use Raindrops::Middleware, :stats => $stats

== Usage (Rainbows!/Unicorn preload_app=true)

If you're using preload_app=true in your Rainbows!/Unicorn
config file, just add the middleware to your stack:

In your Rack config.ru:

        use Raindrops::Middleware

== Usage (Linux-extras)

To get bound listener statistics under Linux, you need to specify the
listener names for your server.  You can even include listen sockets for
*other* servers on the same machine.  This can be handy for monitoring
your nginx proxy as well.

In your Rack config.ru, just pass the :listeners argument as an array of
strings (along with any other arguments).  You can specify any
combination of TCP or Unix domain socket names:

        use Raindrops::Middleware, :listeners => %w(0.0.0.0:80 /tmp/.sock)

See the tests/ and examples/ directory for more examples

== Development

You can get the latest source via git from the following locations:

  git://git.bogomips.org/raindrops.git
  git://repo.or.cz/raindrops.git (mirror)

You may browse the code from the web and download the latest snapshot
tarballs here:

* http://git.bogomips.org/cgit/raindrops.git (cgit)
* http://repo.or.cz/w/raindrops.git (gitweb)

Inline patches (from "git format-patch") to the mailing list are
preferred because they allow code review and comments in the reply to
the patch.

We will adhere to mostly the same conventions for patch submissions as
git itself.  See the Documentation/SubmittingPatches document
distributed with git on on patch submission guidelines to follow.  Just
don't email the git mailing list or maintainer with raindrops patches.

== Contact

All feedback (bug reports, user/development discussion, patches, pull
requests) go to the mailing list: mailto:raindrops at librelist.com

-- 
Eric Wong

From normalperson at yhbt.net  Mon Apr 19 17:24:21 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 19 Apr 2010 21:24:21 +0000
Subject: [ANN] Rainbows! 0.91.1 - use a less-broken parser from Unicorn
Message-ID: <20100419212421.GA26722@dcvr.yhbt.net>

Rainbows! is an HTTP server for sleepy Rack applications.  It is based on
Unicorn, but designed to handle applications that expect long
request/response times and/or slow clients.  For Rack applications not
heavily bound by slow external network dependencies, consider Unicorn
instead as it simpler and easier to debug.

* http://rainbows.rubyforge.org/
* rainbows-talk at rubyforge.org
* git://git.bogomips.org/rainbows.git

Changes:

This release fixes a denial-of-service vector for deployments
exposed directly to untrusted clients.

The HTTP parser in Unicorn <= 0.97.0 would trip an assertion
(killing the associated worker process) on invalid
Content-Length headers instead of raising an exception.  Since
Rainbows! and Zbatery supports multiple clients per worker
process, all clients connected to the worker process that hit
the assertion would be aborted.

Deployments behind nginx are _not_ affected by this bug, as
nginx will reject clients that send invalid Content-Length
headers.

The status of deployments behind other HTTP-aware proxies is
unknown.  Deployments behind a non-HTTP-aware proxy (or no proxy
at all) are certainly affected by this DoS.

Users are strongly encouraged to upgrade as soon as possible,
there are no other changes besides this bug fix from Rainbows!
0.91.0 nor Unicorn 0.97.0

This bug affects all previously released versions of Rainbows!
and Zbatery.

-- 
Eric Wong

From normalperson at yhbt.net  Mon Apr 19 17:28:50 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 19 Apr 2010 21:28:50 +0000
Subject: [ANN] unicorn 0.97.1 - fix HTTP parser for Rainbows!/Zbatery
In-Reply-To: <20100419212513.GB26722@dcvr.yhbt.net>
References: <20100419212513.GB26722@dcvr.yhbt.net>
Message-ID: <20100419212850.GE26722@dcvr.yhbt.net>

Unicorn is an HTTP server for Rack applications designed to only serve
fast clients on low-latency, high-bandwidth connections and take
advantage of features in Unix/Unix-like kernels.  Slow clients should
only be served by placing a reverse proxy capable of fully buffering
both the the request and response in between Unicorn and slow clients.

* http://unicorn.bogomips.org/
* mongrel-unicorn at rubyforge.org
* git://git.bogomips.org/unicorn.git

Changes:

This release fixes a denial-of-service vector for derived
servers exposed directly to untrusted clients.

This bug does not affect most Unicorn deployments as Unicorn is
only supported with trusted clients (such as nginx) on a LAN.
nginx is known to reject clients that send invalid
Content-Length headers, so any deployments on a trusted LAN
and/or behind nginx are safe.

Servers affected by this bug include (but are not limited to)
Rainbows! and Zbatery.  This bug does not affect Thin nor
Mongrel, as neither got the request body filtering treatment
that the Unicorn HTTP parser got in August 2009.

The bug fixed in this release could result in a
denial-of-service as it would trigger a process-wide assertion
instead of raising an exception.  For servers such as
Rainbows!/Zbatery that serve multiple clients per worker
process, this could abort all clients connected to the
particular worker process that hit the assertion.

-- 
Eric Wong

From normalperson at yhbt.net  Mon Apr 19 17:30:16 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 19 Apr 2010 21:30:16 +0000
Subject: [ANN] Zbatery v0.2.1 - use a less-broken parser from Unicorn
In-Reply-To: <20100419212421.GA26722@dcvr.yhbt.net>
References: <20100419212421.GA26722@dcvr.yhbt.net>
Message-ID: <20100419213016.GG26722@dcvr.yhbt.net>

Zbatery is an HTTP server for Rack applications on systems that either
do not support fork(), or have no memory (nor need) to run the
master/worker model.  It is based on Rainbows! (which is based on
Unicorn (which is based on Mongrel)) and inherits parts of each.
Zbatery supports your choice of all the thread/fiber/event/actor-based
concurrency models and Rack middleware that Rainbows! supports (or will
ever support) in a single process.

* http://zbatery.bogomip.org/
* rainbows-talk at rubyforge.org
* git://git.bogomips.org/zbatery.git

Changes:

This release fixes a denial-of-service vector for deployments
exposed directly to untrusted clients.

The HTTP parser in Unicorn <= 0.97.0 would trip an assertion
(killing the associated worker process) on invalid
Content-Length headers instead of raising an exception.  Since
Rainbows! and Zbatery supports multiple clients per worker
process, all clients connected to the worker process that hit
the assertion would be aborted.

Deployments behind nginx are _not_ affected by this bug, as
nginx will reject clients that send invalid Content-Length
headers.

The status of deployments behind other HTTP-aware proxies is
unknown.  Deployments behind a non-HTTP-aware proxy (or no proxy
at all) are certainly affected by this DoS.

Users are strongly encouraged to upgrade as soon as possible,
there are no other changes besides this bug fix from Rainbows!
0.91.0 nor Unicorn 0.97.0

This bug affects all previously released versions of Rainbows!
and Zbatery.

-- 
Eric Wong

From normalperson at yhbt.net  Thu Apr 29 21:36:53 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 30 Apr 2010 01:36:53 +0000
Subject: client_max_body_size param coming
Message-ID: <20100430013653.GA27483@dcvr.yhbt.net>

Hi all,

There are currently no limits to the size of an HTTP request body.

I'm leaning towards making the default limit 1 megabyte (same as nginx).
This means if you're currently running Rainbows! and processing large
uploads, upgrading to a future version will require you to configure
this parameter.

I'm adding this because it's possible for somebody with a high bandwidth
connection to run your server out of disk space.  This is because Rack
(unfortunately) requires request bodies to be rewindable.

This primarily affects the EventMachine, NeverBlock and Rev backends.
They buffer the entire request body before dispatching to the Rack app,
so there's no chance for middleware to intercept calls to "rack.input"
to avoid the flood.

The rest of the concurrency models that use TeeInput never actually read
request bodies off the socket until the data is needed (though
Rack::Request used by most apps will also read it for POST params),
so I'm thinking about introducing middleware to wrap TeeInput and
have it automatically added to the stack.

I don't like automatically throwing middleware into things, but
I think having consistent behavior for:

   Rainbows! do
     use :Foo
     client_max_body_size 10*1024*1024
   end

is important, as it only involves reconfiguring Rainbows!

Rainbows! will would return a "413 Request Entity Too Large" error in
cases when the body is too large and close the connection.

This will only affect Rainbows! and Zbatery.  I won't add this to
Unicorn since Unicorn was only designed to run behind nginx (and trusted
local clients).  Since Unicorn always uses TeeInput, anybody who really
wants it can add the new middleware to their stack.

Thanks for reading.

-- 
Eric Wong