From sergio.lombrico at gmail.com  Thu Jun 17 12:50:57 2010
From: sergio.lombrico at gmail.com (Sergio Lombrico)
Date: Thu, 17 Jun 2010 18:50:57 +0200
Subject: [Mongrel] Chunked transfer
Message-ID: <AANLkTik4IX-NR5y7nK2NPbDWBlTvfJlVWvZf__RVPqJe@mail.gmail.com>

Hi.

Does mongrel support 'chunked' transfer encoding ?

We are sending images from a J2ME app, but do not seem to be able to read
the entire xml message that gets sent from the mobile device. The message is
posted to our RoR app running on a mongrel server.

Regards,

Sergio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mongrel-users/attachments/20100617/cfceb3ef/attachment.html>

From todd.fisher at gmail.com  Thu Jun 17 16:34:45 2010
From: todd.fisher at gmail.com (Todd Fisher)
Date: Thu, 17 Jun 2010 16:34:45 -0400
Subject: [Mongrel] Chunked transfer
In-Reply-To: <AANLkTik4IX-NR5y7nK2NPbDWBlTvfJlVWvZf__RVPqJe@mail.gmail.com>
References: <AANLkTik4IX-NR5y7nK2NPbDWBlTvfJlVWvZf__RVPqJe@mail.gmail.com>
Message-ID: <AANLkTinJk3YiI6KBk527P5_e0bsR_FjgXZH7JYU0Ubv_@mail.gmail.com>

I believe mongrel can be used to send chunked responses.   I did that with
the mongrel-esi server, haven't been keeping it updated but might still be
able to refer to that source as a reference...

On Thu, Jun 17, 2010 at 12:50 PM, Sergio Lombrico <sergio.lombrico at gmail.com
> wrote:

> Hi.
>
> Does mongrel support 'chunked' transfer encoding ?
>
> We are sending images from a J2ME app, but do not seem to be able to read
> the entire xml message that gets sent from the mobile device. The message is
> posted to our RoR app running on a mongrel server.
>
> Regards,
>
> Sergio
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mongrel-users/attachments/20100617/fe20a77a/attachment.html>

From normalperson at yhbt.net  Thu Jun 17 16:40:36 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 17 Jun 2010 13:40:36 -0700
Subject: [Mongrel] Chunked transfer
In-Reply-To: <AANLkTik4IX-NR5y7nK2NPbDWBlTvfJlVWvZf__RVPqJe@mail.gmail.com>
References: <AANLkTik4IX-NR5y7nK2NPbDWBlTvfJlVWvZf__RVPqJe@mail.gmail.com>
Message-ID: <20100617204036.GA9064@dcvr.yhbt.net>

Sergio Lombrico <sergio.lombrico at gmail.com> wrote:
> Hi.
> 
> Does mongrel support 'chunked' transfer encoding ?

Hi Sergio,

Not for requests, but of course it can generate chunked responses.

> We are sending images from a J2ME app, but do not seem to be able to read
> the entire xml message that gets sent from the mobile device. The message is
> posted to our RoR app running on a mongrel server.

<shameless plug(s)>

Rainbows![1] and Zbatery[2] both support chunked requests using the HTTP
parser found in Unicorn[3] (which is descended from the Mongrel parser).

The ThreadSpawn[4] concurrency model with Rainbows!/Zbatery is similar
to Mongrel, and can handle slow clients reasonably well.  Both Rainbows!
and Zbatery work great on *nix, and Zbatery _may_ work on Win32 as well.

Unicorn itself is only designed to run behind nginx, so it is wasteful
if you're dealing with slow connections from a J2ME mobile app.  I don't
remember nginx being able to parse chunked requests, but that may have
changed.  The added benefit of Rainbows!/Zbatery is they can read and
process the chunked request in your app incrementally while it's being
uploaded.

[1] http://rainbows.rubyforge.org/
[2] http://zbatery.bogomip.org/ (no, not a typo :)
[3] http://unicorn.bogomips.org/
[4] http://rainbows.rubyforge.org/Rainbows/ThreadSpawn.html

</shameless plug(s)>

-- 
Eric Wong

From sergio.lombrico at gmail.com  Thu Jun 17 16:53:28 2010
From: sergio.lombrico at gmail.com (Sergio Lombrico)
Date: Thu, 17 Jun 2010 22:53:28 +0200
Subject: [Mongrel] Chunked transfer
In-Reply-To: <AANLkTinJk3YiI6KBk527P5_e0bsR_FjgXZH7JYU0Ubv_@mail.gmail.com>
References: <AANLkTik4IX-NR5y7nK2NPbDWBlTvfJlVWvZf__RVPqJe@mail.gmail.com>
	<AANLkTinJk3YiI6KBk527P5_e0bsR_FjgXZH7JYU0Ubv_@mail.gmail.com>
Message-ID: <AANLkTimV_MdOyBGs8Atw6CfIu6GKH3P_-690kXI-Xsxr@mail.gmail.com>

Hi. Thanks for replying.

Unfortunately we need to receive chunked requests sent to a mongrel server.
Are there any other alternatives other than WEBrick ?

G.

On 17 June 2010 22:34, Todd Fisher <todd.fisher at gmail.com> wrote:

> I believe mongrel can be used to send chunked responses.   I did that with
> the mongrel-esi server, haven't been keeping it updated but might still be
> able to refer to that source as a reference...
>
> On Thu, Jun 17, 2010 at 12:50 PM, Sergio Lombrico <
> sergio.lombrico at gmail.com> wrote:
>
>> Hi.
>>
>> Does mongrel support 'chunked' transfer encoding ?
>>
>> We are sending images from a J2ME app, but do not seem to be able to read
>> the entire xml message that gets sent from the mobile device. The message is
>> posted to our RoR app running on a mongrel server.
>>
>> Regards,
>>
>> Sergio
>>
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>
>
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mongrel-users/attachments/20100617/0e5e2fa3/attachment-0001.html>

From snacktime at gmail.com  Mon Jun 21 15:58:58 2010
From: snacktime at gmail.com (snacktime)
Date: Mon, 21 Jun 2010 12:58:58 -0700
Subject: [Mongrel] scaling unicorn
Message-ID: <AANLkTilv4e_DPDKy440xotrlE7ucFIFXs74uHyGrzCKL@mail.gmail.com>

Interested in some feeback on this (does it sound right?), or maybe
this might be of interest to others.

We are launching a new facebook app in a couple weeks and we did some
load testing over the weekend on our unicorn web cluster.  The servers
are 8 way xeon's with 24gb ram.  Our app ended up being primarily cpu
bound.  So far the sweet spot for the number of unicorns seems to be
around 40.  This seemed to yield the most requests per second without
overloading the server or hitting memory bandwidth issues.  The
backlog is at the somaxconn default of 128, I'm still not sure if we
will bump that up or not.  Increasing the number of unicorns beyond a
certain point resulted in a noticable drop in the requests per second
the server could handle.   I'm pretty sure the cause is the box
running out of memory bandwidth.  The load average and resource usage
in general (except for memory) would keep going down but so did the
requests per second.  At 80 unicorns the requests per second dropped
by more then half.  I'm going to disable hyperthreading and rerun some
of the tests to see what impact that has.

Chris

From normalperson at yhbt.net  Mon Jun 21 20:16:32 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 21 Jun 2010 17:16:32 -0700
Subject: [Mongrel] scaling unicorn
In-Reply-To: <AANLkTilv4e_DPDKy440xotrlE7ucFIFXs74uHyGrzCKL@mail.gmail.com>
References: <AANLkTilv4e_DPDKy440xotrlE7ucFIFXs74uHyGrzCKL@mail.gmail.com>
Message-ID: <20100622001632.GA10082@dcvr.yhbt.net>

snacktime <snacktime at gmail.com> wrote:
> Interested in some feeback on this (does it sound right?), or maybe
> this might be of interest to others.

Hi Chris,

I think you meant to post this to the mongrel-unicorn at rubyforge.org
list, not mongrel-users at rubyforge.org :>

> We are launching a new facebook app in a couple weeks and we did some
> load testing over the weekend on our unicorn web cluster.  The servers
> are 8 way xeon's with 24gb ram.  Our app ended up being primarily cpu
> bound.  So far the sweet spot for the number of unicorns seems to be
> around 40.  This seemed to yield the most requests per second without
> overloading the server or hitting memory bandwidth issues.  The
> backlog is at the somaxconn default of 128, I'm still not sure if we
> will bump that up or not.

The default backlog we try to specify is actually 1024 (same as
Mongrel).  But it's always a murky value anyways, as it's
kernel/sysctl-dependent.  With Unix domain sockets, some folks use
crazy values like 2048 to look better on synthetic benchmarks :)

> Increasing the number of unicorns beyond a
> certain point resulted in a noticable drop in the requests per second
> the server could handle.   I'm pretty sure the cause is the box
> running out of memory bandwidth.  The load average and resource usage
> in general (except for memory) would keep going down but so did the
> requests per second.  At 80 unicorns the requests per second dropped
> by more then half.  I'm going to disable hyperthreading and rerun some
> of the tests to see what impact that has.

That's "8 way xeon" _before_ hyperthreading, right?  Which family of
Xeons are you using, the Pentium4-based crap or the awesome new ones?

How much memory is each Unicorn worker using for your app?

40 workers for 8 physical cores sounds reasonable.  Depending on the
app, I think the reasonable range is anywhere from 2-8 workers per
physical core.  More if you're (unfortunately) limited by external
network calls, but since you claim to be CPU bound, less.

Do you have actual performance numbers you're able to share?
Mean/median request times/rates would be very useful.  If your requests
run very quickly, you may be limited by contention with the accept()
syscall on the listen socket, too.

I assume you're using nginx as the proxy, is this with Unix domain
sockets or TCP sockets?  Unix domain sockets should give a small
performance over TCP if it's all on the same box.

With TCP, you should also check to see you have enough local ports
available if you're hitting extremely high (and probably unrealistic :)
request rates.

-- 
Eric Wong

From jftucker at gmail.com  Mon Jun 21 20:42:07 2010
From: jftucker at gmail.com (James Tucker)
Date: Mon, 21 Jun 2010 21:42:07 -0300
Subject: [Mongrel] scaling unicorn
In-Reply-To: <AANLkTilv4e_DPDKy440xotrlE7ucFIFXs74uHyGrzCKL@mail.gmail.com>
References: <AANLkTilv4e_DPDKy440xotrlE7ucFIFXs74uHyGrzCKL@mail.gmail.com>
Message-ID: <529E0EB0-1569-4482-940B-811765123A5B@gmail.com>

What was the request rate and total bandwidth flowing at your peak?

How far is that from your theoretical potential on the box?

On 21 Jun 2010, at 16:58, snacktime wrote:

> Interested in some feeback on this (does it sound right?), or maybe
> this might be of interest to others.
> 
> We are launching a new facebook app in a couple weeks and we did some
> load testing over the weekend on our unicorn web cluster.  The servers
> are 8 way xeon's with 24gb ram.  Our app ended up being primarily cpu
> bound.  So far the sweet spot for the number of unicorns seems to be
> around 40.  This seemed to yield the most requests per second without
> overloading the server or hitting memory bandwidth issues.  The
> backlog is at the somaxconn default of 128, I'm still not sure if we
> will bump that up or not.  Increasing the number of unicorns beyond a
> certain point resulted in a noticable drop in the requests per second
> the server could handle.   I'm pretty sure the cause is the box
> running out of memory bandwidth.  The load average and resource usage
> in general (except for memory) would keep going down but so did the
> requests per second.  At 80 unicorns the requests per second dropped
> by more then half.  I'm going to disable hyperthreading and rerun some
> of the tests to see what impact that has.
> 
> Chris
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From lists at ruby-forum.com  Wed Jun 23 13:50:04 2010
From: lists at ruby-forum.com (Rick Cockerham)
Date: Wed, 23 Jun 2010 19:50:04 +0200
Subject: [Mongrel] can't get mongrel to work as daemon
In-Reply-To: <846417e9bfae8e14305fcd01f1d6af40@ruby-forum.com>
References: <724f78d7f0f0bfcb35ebd5c3d98974aa@ruby-forum.com>
	<846417e9bfae8e14305fcd01f1d6af40@ruby-forum.com>
Message-ID: <c08a8242f773f2851e60d2086a711daf@ruby-forum.com>


I'm on Red Hat rhel 5.  Apache 2.2, Mongrel 1.1.5, Ruby 1.8.6, Rails 
2.3.5

When I run mongrel with -d it crashes the first time it's accessed.  It 
doesn't return anything.  It doesn't log anything in development.log or 
production.log depending on how I run it.  There is absolutely nothing 
interesting in the mongrel or apache logs even with -B.

I recently upgraded to Rails 2.3.5.  That's when it stopped working.
I tried going to 2.3.8, but that was worse.  I tried going back to 
mongrel 1.1.4, but that did the same thing.

I can run mongrel in the background then disown it, but that's a huge 
pain to restart a cluster of mongrels that way.  I've been googling for 
days and this is the only post that even comes close.  Of course, the 
solution wasn't posted...

Thanks,
Rick
-- 
Posted via http://www.ruby-forum.com/.