From nate at pivotallabs.com  Wed Jun  1 02:51:38 2011
From: nate at pivotallabs.com (Nate Clark)
Date: Wed, 1 Jun 2011 14:51:38 +0800
Subject: workers not utilizing multiple CPUs
In-Reply-To: <BANLkTikC5D+0pUDRDgvQbzu=dpwmdKNY=A@mail.gmail.com>
References: <BANLkTinR31Zeufc11sSffXpRiciosTGpUA@mail.gmail.com>
	<4DE4DABB.3070503@gmail.com>
	<BANLkTinX0t35a7Mu1vk=DwxwO6EZj-gptA@mail.gmail.com>
	<7E16F5B9-2C96-426F-BC75-670BEDD122A9@gmail.com>
	<20110531154803.GC10313@dcvr.yhbt.net>
	<BANLkTi=XbFTWSHg2mnriM=ANwMnHWFjPWA@mail.gmail.com>
	<BANLkTikC5D+0pUDRDgvQbzu=dpwmdKNY=A@mail.gmail.com>
Message-ID: <BANLkTimrZTdMhrr02XUWgBeS2yQchB+TJg@mail.gmail.com>

Thanks for the responses, all.

Eric, you were right, our load was not enough. We had just started on
load testing our app, and I think we started with too many app servers
and not enough load. Once we cranked up the load and used fewer
instances, we're now?definitely?seeing all CPU cores being utilized. I
was not aware that the kernel would optimize like you described.

Once we did start seeing heavier load, our collectd data and htop were
reporting usage on the virtual cores correctly.
Thanks again, very happy with the results so far,

Nate

On Wed, Jun 1, 2011 at 2:35 PM, Nate Clark <nate at pivotallabs.com> wrote:
>
> Thanks for the responses, all.
> Eric, you were right, our load was not enough. We had just started on load testing our app, and I think we started with too many app servers and not enough load. Once we cranked up the load and used fewer instances, we're now?definitely?seeing all CPU cores being utilized. I was not aware that the kernel would optimize like you described.
> Once we did start seeing heavier load, our collectd data and htop were reporting usage on the virtual cores correctly.
> Thanks again, very happy with the results so far,
> Nate
> On Tue, May 31, 2011 at 11:55 PM, Clifton King <cliftonk at gmail.com> wrote:
>>
>> Thanks Eric, I had expected that to be the case (we are under light
>> load as of now).
>>
>> On Tue, May 31, 2011 at 10:48 AM, Eric Wong <normalperson at yhbt.net> wrote:
>> > Clifton King <cliftonk at gmail.com> wrote:
>> >> We experience the same problem. I believe the problem has more to do
>> >> with the kernel CPU scheduler than anything else. If you figure put a
>> >> reliable way to spread the load, I'd like to hear it.
>> >
>> > Load not being spread is /not/ a problem unless there are requests that
>> > get stuck in the listen queue.
>> >
>> > If no requests are actually stuck in the queue (light load), the kernel
>> > is right to put requests into the most recently used worker since it can
>> > get better CPU cache behavior this way.
>> >
>> >
>> > == The real problem
>> >
>> > Under high loads (many cores, fast responses), Unicorn currently uses
>> > more resources because of non-blocking accept() + select(). ?This isn't
>> > a noticeable problem for most machines (1-16 cores).
>> >
>> > Future versions of Unicorn may take advantage of /blocking/ accept()
>> > optimizations under Linux. ?Rainbows! already lets you take advantage
>> > of this behavior if you meet the following requirements:
>> >
>> > * Ruby 1.9.x under Linux
>> > * only one listen socket (if worker_connections == 1 under Rainbows!)
>> > * use ThreadPool|XEpollThreadPool|XEpollThreadSpawn|XEpoll
>> >
>> > I haven't had a chance to benchmark any of this on very big machines so
>> > I have no idea how well it actually works compared to Unicorn, only how
>> > well it works in theory :)
>> >
>> >
>> > Blocking accept() under Ruby 1.9.x + Linux should distribute load evenly
>> > across workers in all situations, even in the non-busy cases where load
>> > distribution doesn't matter (your case :).
>> >
>> > [1] - http://rainbows.rubyforge.org/Rainbows/XEpollThreadPool.html
>> >
>> > --
>> > Eric Wong
>> > _______________________________________________
>> > Unicorn mailing list - mongrel-unicorn at rubyforge.org
>> > http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>> > Do not quote signatures (like this one) or top post when replying
>> >
>> _______________________________________________
>> Unicorn mailing list - mongrel-unicorn at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>> Do not quote signatures (like this one) or top post when replying
>

From normalperson at yhbt.net  Wed Jun  1 12:48:01 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 1 Jun 2011 16:48:01 +0000
Subject: unicorn stuck in sched_yield after ERESTARTNOHAND
In-Reply-To: <BANLkTinsXbKt2g5HZG0a3Q4Gr7D05HRRjA@mail.gmail.com>
References: <BANLkTikFid3n0QpsrnXf2oNansFmuJDyuw@mail.gmail.com>
	<BANLkTimF78PW9YgEAURS604Q8mucNwSDrg@mail.gmail.com>
	<BANLkTinPK-QBWE2YireOEqcT1G+RnRiyBw@mail.gmail.com>
	<20110531234829.GA9815@dcvr.yhbt.net>
	<BANLkTinsXbKt2g5HZG0a3Q4Gr7D05HRRjA@mail.gmail.com>
Message-ID: <20110601164801.GA18856@dcvr.yhbt.net>

Bharanee Rathna <deepfryed at gmail.com> wrote:
> On Wed, Jun 1, 2011 at 9:48 AM, Eric Wong <normalperson at yhbt.net> wrote:
> > Bharanee Rathna <deepfryed at gmail.com> wrote:
> >> I'm encountering a weird error where the unicorn workers are stuck in
> >> a loop after hitting a 500 on the backend sinatra app.
> >
> > Also, what extensions are you using in your app?
> 
> heaps of em. yajl, swift, rmagick, fastcaptcha, flock, nokogiri &
> curb.  except swift and curb none of the others would be touching the
> network.

So are any of them hitting Unicorn from a Unicorn worker itself?
This only happens when your app hits a 500?  I would not write
an app that does that, but if you do, be sure to shutdown any
open connections on a 500 (or avoid the error in the first place) ...

> >> strace at the point where it starts to go into a loop of death

Actually, the sched_yield() only started when you hit sent SIGINT.

> > What triggered SIGINT?
> 
> not sure
> >
> > Actually, after many lines of sched_yield() in your gist, I can see it
> > does actually exit the process. ?Did you kill it with SIGINT? ?If so, I
> > see nothing wrong...
> 
> yes i killed it after the worker looked stuck and wasn't responding for 30s

So you hit Ctrl-C (which sends SIGINT)?

So basically somebody from 10.1.1.4 opened a connection on Unicorn
and just let it sit idle there.  That somebody at 10.1.1.4 could've
been the worker that triggered the 500 (and forgot it had open
ccnnections)  or a third party that made a request but sit idle
because it couldn't handle the 500 which your Unicorn worker sent.

>From what I understand so far, it's not a Unicorn problem, but
something in your app.

Also, you wouldn't have this problem if nginx is in front of Unicorn
since nginx won't open a connection and sit idle before making a
request.

-- 
Eric Wong

From hongli at phusion.nl  Mon Jun  6 07:44:33 2011
From: hongli at phusion.nl (Hongli Lai)
Date: Mon, 6 Jun 2011 13:44:33 +0200
Subject: [PATCH] Document the method for building the Unicorn gem
Message-ID: <BANLkTinZzirEqJ9uxrdm9Ben=uWwiV1rqA@mail.gmail.com>

>From dcd47a609f4489bb37ce33ea1ce975bb2b3ab160 Mon Sep 17 00:00:00 2001
From: Hongli Lai (Phusion) <hongli at phusion.nl>
Date: Mon, 6 Jun 2011 13:36:57 +0200
Subject: [PATCH] Document the method for building the Unicorn gem.

Signed-off-by: Hongli Lai (Phusion) <hongli at phusion.nl>
---
 HACKING |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/HACKING b/HACKING
index 781c4ca..38f2f2a 100644
--- a/HACKING
+++ b/HACKING
@@ -107,6 +107,17 @@ git itself.  See the
Documentation/SubmittingPatches document
 distributed with git on on patch submission guidelines to follow.  Just
 don't email the git mailing list or maintainer with Unicorn patches :)

+== Building a Gem
+
+In order to build the gem, you must install the following components:
+
+ * wrongdoc
+ * pandoc
+
+You can build the Unicorn gem with the following command:
+
+  gmake gem
+
 == Running Development Versions

 It is easy to install the contents of your git working directory:
-- 
1.7.5

-- 
Phusion | Ruby & Rails deployment, scaling and tuning solutions

Web: http://www.phusion.nl/
E-mail: info at phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

From normalperson at yhbt.net  Mon Jun  6 12:58:33 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 6 Jun 2011 09:58:33 -0700
Subject: [PATCH] Document the method for building the Unicorn gem
In-Reply-To: <BANLkTinZzirEqJ9uxrdm9Ben=uWwiV1rqA@mail.gmail.com>
References: <BANLkTinZzirEqJ9uxrdm9Ben=uWwiV1rqA@mail.gmail.com>
Message-ID: <20110606165833.GA24016@dcvr.yhbt.net>

Hongli Lai <hongli at phusion.nl> wrote:
> >From dcd47a609f4489bb37ce33ea1ce975bb2b3ab160 Mon Sep 17 00:00:00 2001
> From: Hongli Lai (Phusion) <hongli at phusion.nl>
> Date: Mon, 6 Jun 2011 13:36:57 +0200
> Subject: [PATCH] Document the method for building the Unicorn gem.
> 
> Signed-off-by: Hongli Lai (Phusion) <hongli at phusion.nl>

Thanks, applied and pushed.

> index 781c4ca..38f2f2a 100644
> --- a/HACKING
> +++ b/HACKING
> @@ -107,6 +107,17 @@ git itself.  See the
> Documentation/SubmittingPatches document

The patch got line-wrapped there.  If your MUA of choice can't be taught
to leave lines unwrapped, try "git send-email".

> + * pandoc

If anybody has spare cycles to make this work with ronn instead of
pandoc, please send patches.  ronn should be an easier-to-install
for Rubyists since it doesn't depend on Haskell.

Can somebody confirm ronn doesn't introduce any JavaScript? (or can made
to be JS-free easily).

-- 
Eric Wong

From normalperson at yhbt.net  Mon Jun  6 13:06:01 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 6 Jun 2011 10:06:01 -0700
Subject: [PATCH] Document the method for building the Unicorn gem
In-Reply-To: <20110606165833.GA24016@dcvr.yhbt.net>
References: <BANLkTinZzirEqJ9uxrdm9Ben=uWwiV1rqA@mail.gmail.com>
	<20110606165833.GA24016@dcvr.yhbt.net>
Message-ID: <20110606170601.GA27918@dcvr.yhbt.net>

Eric Wong <normalperson at yhbt.net> wrote:
> Can somebody confirm ronn doesn't introduce any JavaScript? (or can made
> to be JS-free easily).

Actually, the wrongdoc documentation already uses ronn and wrongdoc
documentation is definitely JavaScript-free :)

-- 
Eric Wong

From hongli at phusion.nl  Mon Jun  6 13:31:14 2011
From: hongli at phusion.nl (Hongli Lai)
Date: Mon, 06 Jun 2011 19:31:14 +0200
Subject: [PATCH] Ensure that 'make gem' builds the documentation too.
Message-ID: <4DED0EE2.7040400@phusion.nl>

>From bfefc2cf0efb0913a42862886363b3140dcdbb2a Mon Sep 17 00:00:00 2001
From: Hongli Lai (Phusion) <hongli at phusion.nl>
Date: Mon, 6 Jun 2011 13:39:00 +0200
Subject: [PATCH] Ensure that 'make gem' builds the documentation too.

If autogenerated documentation files, like man pages, don't exist then
'make gem' will fail, complaining that some files are not found. By
depending the 'gem' target on the 'doc' target we ensure that 'make gem'
always works.

Signed-off-by: Hongli Lai (Phusion) <hongli at phusion.nl>
---
 GNUmakefile |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/GNUmakefile b/GNUmakefile
index 4072826..90cc451 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -257,7 +257,7 @@ gem: $(pkggem)
 install-gem: $(pkggem)
 	gem install $(CURDIR)/$<

-$(pkggem): .manifest fix-perms
+$(pkggem): .manifest fix-perms doc
 	gem build $(rfpackage).gemspec
 	mkdir -p pkg
 	mv $(@F) $@
-- 
1.7.5


From normalperson at yhbt.net  Mon Jun  6 13:51:23 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 6 Jun 2011 10:51:23 -0700
Subject: [PATCH] Ensure that 'make gem' builds the documentation too.
In-Reply-To: <4DED0EE2.7040400@phusion.nl>
References: <4DED0EE2.7040400@phusion.nl>
Message-ID: <20110606175123.GA29578@dcvr.yhbt.net>

Hongli Lai <hongli at phusion.nl> wrote:
> >From bfefc2cf0efb0913a42862886363b3140dcdbb2a Mon Sep 17 00:00:00 2001
> From: Hongli Lai (Phusion) <hongli at phusion.nl>
> Date: Mon, 6 Jun 2011 13:39:00 +0200
> Subject: [PATCH] Ensure that 'make gem' builds the documentation too.
> 
> If autogenerated documentation files, like man pages, don't exist then
> 'make gem' will fail, complaining that some files are not found. By
> depending the 'gem' target on the 'doc' target we ensure that 'make gem'
> always works.
> 
> Signed-off-by: Hongli Lai (Phusion) <hongli at phusion.nl>

Oops, this was a regression introduced when I switched to wrongdoc
in f62ef19a4aa3d3e4ce1aa37a499907ff776a8964

Perhaps this is better?  It'll also affect the tgz target and not
just the gem target.

diff --git a/GNUmakefile b/GNUmakefile
index da55052..61fb739 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -164,7 +164,7 @@ pkg_extra := GIT-VERSION-FILE ChangeLog LATEST NEWS \
 ChangeLog: GIT-VERSION-FILE .wrongdoc.yml
 	wrongdoc prepare
 
-.manifest: ChangeLog $(ext)/unicorn_http.c
+.manifest: ChangeLog $(ext)/unicorn_http.c man
 	(git ls-files && for i in $@ $(pkg_extra); do echo $$i; done) | \
 	  LC_ALL=C sort > $@+
 	cmp $@+ $@ || mv $@+ $@
-- 
Eric Wong

From normalperson at yhbt.net  Mon Jun  6 22:28:28 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 6 Jun 2011 19:28:28 -0700
Subject: [PATCH] examples/nginx.conf: add ipv6only comment
Message-ID: <20110607022828.GA24280@dcvr.yhbt.net>

IPv4 addresses started looking very ugly the first time I got
IPv6 working on bogomips.org.  In case somebody else can't stand
how IPv4-mapped-IPv6 addresses look, the workaround is to use
two listeners and ensure the IPv6 one is ipv6only.

Unicorn itself supports IPv6, too, but nobody uses/needs it.
I'll add :ipv6only support shortly (probably tomorrow).

>From 32b340b88915ec945ebdbfa11b7da242860a6f44 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson at yhbt.net>
Date: Mon, 6 Jun 2011 19:15:36 -0700
Subject: [PATCH] examples/nginx.conf: add ipv6only comment

IPv4-mapped-IPv6 addresses are fugly.
---
 examples/nginx.conf |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/examples/nginx.conf b/examples/nginx.conf
index 9f245c8..368e19e 100644
--- a/examples/nginx.conf
+++ b/examples/nginx.conf
@@ -87,6 +87,14 @@ http {
     # listen 80 default deferred; # for Linux
     # listen 80 default accept_filter=httpready; # for FreeBSD
 
+    # If you have IPv6, you'll likely want to have two separate listeners.
+    # One on IPv4 only (the default), and another on IPv6 only instead
+    # of a single dual-stack listener.  A dual-stack listener will make
+    # $remote_addr will make IPv4 addresses ugly (e.g ":ffff:10.0.0.1"
+    # instead of just "10.0.0.1") and potentially trigger bugs in
+    # some software.
+    # listen [::]:80 ipv6only=on; # deferred or accept_filter recommended
+
     client_max_body_size 4G;
     server_name _;
 
-- 
Eric Wong

From hongli at phusion.nl  Tue Jun  7 11:11:05 2011
From: hongli at phusion.nl (Hongli Lai)
Date: Tue, 7 Jun 2011 17:11:05 +0200
Subject: [PATCH] Ensure that 'make gem' builds the documentation too.
In-Reply-To: <20110606175123.GA29578@dcvr.yhbt.net>
References: <4DED0EE2.7040400@phusion.nl>
	<20110606175123.GA29578@dcvr.yhbt.net>
Message-ID: <BANLkTin1hZE7NjBq4Z-EW448pT3SP7x1jQ@mail.gmail.com>

On Mon, Jun 6, 2011 at 7:51 PM, Eric Wong <normalperson at yhbt.net> wrote:
> Oops, this was a regression introduced when I switched to wrongdoc
> in f62ef19a4aa3d3e4ce1aa37a499907ff776a8964
>
> Perhaps this is better? ?It'll also affect the tgz target and not
> just the gem target.

I'm fine with it if it works. :)

-- 
Phusion | Ruby & Rails deployment, scaling and tuning solutions

Web: http://www.phusion.nl/
E-mail: info at phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

From normalperson at yhbt.net  Tue Jun  7 13:04:22 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 7 Jun 2011 10:04:22 -0700
Subject: [PATCH] examples/nginx.conf: add ipv6only comment
In-Reply-To: <20110607022828.GA24280@dcvr.yhbt.net>
References: <20110607022828.GA24280@dcvr.yhbt.net>
Message-ID: <20110607170422.GA25560@dcvr.yhbt.net>

My wording sucked for the original comment, just pushed this out to
unicorn.git (6eefc641c84eaa86cb2be4a2b1983b15efcbfae1)

diff --git a/examples/nginx.conf b/examples/nginx.conf
index 368e19e..cc1038a 100644
--- a/examples/nginx.conf
+++ b/examples/nginx.conf
@@ -90,7 +90,7 @@ http {
     # If you have IPv6, you'll likely want to have two separate listeners.
     # One on IPv4 only (the default), and another on IPv6 only instead
     # of a single dual-stack listener.  A dual-stack listener will make
-    # $remote_addr will make IPv4 addresses ugly (e.g ":ffff:10.0.0.1"
+    # for ugly IPv4 addresses in $remote_addr (e.g ":ffff:10.0.0.1"
     # instead of just "10.0.0.1") and potentially trigger bugs in
     # some software.
     # listen [::]:80 ipv6only=on; # deferred or accept_filter recommended
-- 
Eric Wong

From normalperson at yhbt.net  Tue Jun  7 13:06:26 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 7 Jun 2011 10:06:26 -0700
Subject: [PATCH] Ensure that 'make gem' builds the documentation too.
In-Reply-To: <BANLkTin1hZE7NjBq4Z-EW448pT3SP7x1jQ@mail.gmail.com>
References: <4DED0EE2.7040400@phusion.nl>
	<20110606175123.GA29578@dcvr.yhbt.net>
	<BANLkTin1hZE7NjBq4Z-EW448pT3SP7x1jQ@mail.gmail.com>
Message-ID: <20110607170626.GA26220@dcvr.yhbt.net>

Hongli Lai <hongli at phusion.nl> wrote:
> On Mon, Jun 6, 2011 at 7:51 PM, Eric Wong <normalperson at yhbt.net> wrote:
> > Oops, this was a regression introduced when I switched to wrongdoc
> > in f62ef19a4aa3d3e4ce1aa37a499907ff776a8964
> >
> > Perhaps this is better? ?It'll also affect the tgz target and not
> > just the gem target.
> 
> I'm fine with it if it works. :)

Pushed out with the following commit message:

>From 0dc56fd03ea478ae054e3d0398703f43e017723b Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson at yhbt.net>
Date: Tue, 7 Jun 2011 09:56:30 -0700
Subject: [PATCH] build: ensure gem and tgz targets build manpages

Original patch by Hongli Lai <hongli at phusion.nl>:

> >From bfefc2cf0efb0913a42862886363b3140dcdbb2a Mon Sep 17 00:00:00 2001
> From: Hongli Lai (Phusion) <hongli at phusion.nl>
> Date: Mon, 6 Jun 2011 13:39:00 +0200
> Subject: [PATCH] Ensure that 'make gem' builds the documentation too.
>
> If autogenerated documentation files, like man pages, don't exist then
> 'make gem' will fail, complaining that some files are not found. By
> depending the 'gem' target on the 'doc' target we ensure that 'make gem'
> always works.
>
> Signed-off-by: Hongli Lai (Phusion) <hongli at phusion.nl>

ref: http://mid.gmane.org/4DED0EE2.7040400 at phusion.nl
-- 
Eric Wong

From normalperson at yhbt.net  Tue Jun  7 17:05:11 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 7 Jun 2011 14:05:11 -0700
Subject: [PATCH] configurator: add :ipv6only directive
In-Reply-To: <20110607022828.GA24280@dcvr.yhbt.net>
References: <20110607022828.GA24280@dcvr.yhbt.net>
Message-ID: <20110607210511.GA16295@dcvr.yhbt.net>

Eric Wong <normalperson at yhbt.net> wrote:
> Unicorn itself supports IPv6, too, but nobody uses/needs it.
> I'll add :ipv6only support shortly (probably tomorrow).

Pushed to unicorn.git (c3880bb0cc00821d1715a7dd94b0b76a03a7ace0)

Documentation below:

diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb
index b6ad022..0b84d53 100644
--- a/lib/unicorn/configurator.rb
+++ b/lib/unicorn/configurator.rb
@@ -281,6 +281,22 @@ class Unicorn::Configurator
   #
   #   Default: +true+ in \Unicorn 3.4+, +false+ in Rainbows!
   #
+  # [:ipv6only => true or false]
+  #
+  #   This option makes IPv6-capable TCP listeners IPv6-only and unable
+  #   to receive IPv4 queries on dual-stack systems.  A separate IPv4-only
+  #   listener is required if this is true.
+  #
+  #   This option is only available for Ruby 1.9.2 and later.
+  #
+  #   Enabling this option for the IPv6-only listener and having a
+  #   separate IPv4 listener is recommended if you wish to support IPv6
+  #   on the same TCP port.  Otherwise, the value of \env[\"REMOTE_ADDR\"]
+  #   will appear as an ugly IPv4-mapped-IPv6 address for IPv4 clients
+  #   (e.g ":ffff:10.0.0.1" instead of just "10.0.0.1").
+  #
+  #   Default: Operating-system dependent
+  #
   # [:tries => Integer]
   #
   #   Times to retry binding a socket if it is already in use
-- 
Eric Wong

From normalperson at yhbt.net  Thu Jun  9 16:55:12 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 9 Jun 2011 20:55:12 +0000
Subject: [ANN] unicorn 3.7.0 - minor feature update
Message-ID: <20110609205512.GA29446@dcvr.yhbt.net>

Changes:

* miscellaneous documentation improvements
* return 414 (instead of 400) for Request-URI Too Long
* strip leading and trailing linear whitespace in header values

User-visible improvements meant for Rainbows! users:

* add :ipv6only "listen" option (same as nginx)
---
* http://unicorn.bogomips.org/
* mongrel-unicorn at rubyforge.org
* git://bogomips.org/unicorn.git

-- 
Eric Wong

From normalperson at yhbt.net  Mon Jun 13 17:20:49 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 13 Jun 2011 14:20:49 -0700
Subject: Unicorn.run method removed in unicorn.git
Message-ID: <20110613212049.GA3490@dcvr.yhbt.net>

I don't know if there are any folks wrapping Unicorn, but
I've removed the Unicorn.run method in unicorn.git as part
of an effort to reduce stack size (which will improve GC
performance and reduce the cost of generating (and examining)
backtraces.

You can replace:

  Unicorn.run(app, options)

With:

  Unicorn::HttpServer.new(app, options).start.join

This will be in the upcoming 4.x release.

ref: git://bogomips.org/unicorn.git
commit 5d2284afdc2d4f4ff122394ae5fd78a32cb8c09e

-- 
Eric Wong

From normalperson at yhbt.net  Thu Jun 16 19:25:08 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 16 Jun 2011 23:25:08 +0000
Subject: [PATCH] replace fchmod()-based heartbeat with raindrops
Message-ID: <20110616232508.GA16538@dcvr.yhbt.net>

This means we no longer waste an extra file descriptor per
worker process in the master.  Now there's no need to set a
higher file descriptor limit for systems running >= 1024
workers.
---

 I just pushed this out to git://bogomips.org/unicorn.git and it'll
 be in Unicorn 4.x.

 The subset of raindrops used by Unicorn should work on all machines
 with mmap(2) + MAP_ANON/MAP_ANONYMOUS support, so *BSDs shouldn't be
 left out.

 Obviously the Linux-only INET_DIAG/TCP_INFO statistics support in
 raindrops won't ever be a requirement in Unicorn.

 If you're not using gcc 4+, you might need to install libatomic_ops
 http://www.hpl.hp.com/research/linux/atomic_ops/ to build
 raindrops.

 For more information on raindrops, see the raindrops website
 http://raindrops.bogomips.org/ and/or email the mailing list
 if you have questions, comments, patches, etc: raindrops at librelist.org
 (I'll also respond to email about raindrops here)

 DESIGN                     |    8 -------
 lib/unicorn/http_server.rb |   48 +++++++++++++-----------------------------
 lib/unicorn/worker.rb      |   49 ++++++++++++++++++++++++++++++++++++++++---
 script/isolate_for_tests   |    1 +
 test/unit/test_droplet.rb  |   28 +++++++++++++++++++++++++
 unicorn.gemspec            |    1 +
 6 files changed, 90 insertions(+), 45 deletions(-)
 create mode 100644 test/unit/test_droplet.rb

diff --git a/DESIGN b/DESIGN
index eb9fbea..2c98c2f 100644
--- a/DESIGN
+++ b/DESIGN
@@ -76,14 +76,6 @@
   Applications that use threads continue to work if Unicorn
   is only serving LAN or localhost clients.
 
-* Timeout implementation is done via fchmod(2) in each worker
-  on a shared file descriptor to update st_ctime on the inode.
-  Master process wakeups for checking on timeouts is throttled
-  one a second to minimize the performance impact and simplify
-  the code path within the worker.  Neither futimes(2) nor
-  pwrite(2)/pread(2) are supported by base MRI, nor are they as
-  portable on UNIX systems as fchmod(2).
-
 * SIGKILL is used to terminate the timed-out workers from misbehaving apps
   as reliably as possible on a UNIX system.  The default timeout is a
   generous 60 seconds (same default as in Mongrel).
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 059f040..0a9af86 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -373,7 +373,7 @@ class Unicorn::HttpServer
         self.pid = pid.chomp('.oldbin') if pid
         proc_name 'master'
       else
-        worker = WORKERS.delete(wpid) and worker.tmp.close rescue nil
+        worker = WORKERS.delete(wpid) and worker.close rescue nil
         m = "reaped #{status.inspect} worker=#{worker.nr rescue 'unknown'}"
         status.success? ? logger.info(m) : logger.error(m)
       end
@@ -430,22 +430,17 @@ class Unicorn::HttpServer
     proc_name 'master (old)'
   end
 
-  # forcibly terminate all workers that haven't checked in in timeout
-  # seconds.  The timeout is implemented using an unlinked File
-  # shared between the parent process and each worker.  The worker
-  # runs File#chmod to modify the ctime of the File.  If the ctime
-  # is stale for >timeout seconds, then we'll kill the corresponding
-  # worker.
+  # forcibly terminate all workers that haven't checked in in timeout seconds.  The timeout is implemented using an unlinked File
   def murder_lazy_workers
     t = @timeout
     next_sleep = 1
+    now = Time.now.to_i
     WORKERS.dup.each_pair do |wpid, worker|
-      stat = worker.tmp.stat
-      # skip workers that disable fchmod or have never fchmod-ed
-      stat.mode == 0100600 and next
-      diff = Time.now - stat.ctime
-      if diff <= t
-        tmp = t - diff
+      tick = worker.tick
+      0 == tick and next # skip workers that are sleeping
+      diff = now - tick
+      tmp = t - diff
+      if tmp >= 0
         next_sleep < tmp and next_sleep = tmp
         next
       end
@@ -472,7 +467,7 @@ class Unicorn::HttpServer
     worker_nr = -1
     until (worker_nr += 1) == @worker_processes
       WORKERS.values.include?(worker_nr) and next
-      worker = Worker.new(worker_nr, Unicorn::TmpIO.new)
+      worker = Worker.new(worker_nr)
       before_fork.call(self, worker)
       if pid = fork
         WORKERS[pid] = worker
@@ -549,10 +544,8 @@ class Unicorn::HttpServer
     proc_name "worker[#{worker.nr}]"
     START_CTX.clear
     init_self_pipe!
-    WORKERS.values.each { |other| other.tmp.close rescue nil }
     WORKERS.clear
     LISTENERS.each { |sock| sock.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) }
-    worker.tmp.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC)
     after_fork.call(self, worker) # can drop perms
     worker.user(*user) if user.kind_of?(Array) && ! worker.switched
     self.timeout /= 2.0 # halve it for select()
@@ -576,12 +569,11 @@ class Unicorn::HttpServer
     ppid = master_pid
     init_worker_process(worker)
     nr = 0 # this becomes negative if we need to reopen logs
-    alive = worker.tmp # tmp is our lifeline to the master process
     ready = LISTENERS.dup
 
     # closing anything we IO.select on will raise EBADF
     trap(:USR1) { nr = -65536; SELF_PIPE[0].close rescue nil }
-    trap(:QUIT) { alive = nil; LISTENERS.each { |s| s.close rescue nil }.clear }
+    trap(:QUIT) { worker = nil; LISTENERS.each { |s| s.close rescue nil }.clear }
     [:TERM, :INT].each { |sig| trap(sig) { exit!(0) } } # instant shutdown
     logger.info "worker=#{worker.nr} ready"
     m = 0
@@ -590,21 +582,12 @@ class Unicorn::HttpServer
       nr < 0 and reopen_worker_logs(worker.nr)
       nr = 0
 
-      # we're a goner in timeout seconds anyways if alive.chmod
-      # breaks, so don't trap the exception.  Using fchmod() since
-      # futimes() is not available in base Ruby and I very strongly
-      # prefer temporary files to be unlinked for security,
-      # performance and reliability reasons, so utime is out.  No-op
-      # changes with chmod doesn't update ctime on all filesystems; so
-      # we change our counter each and every time (after process_client
-      # and before IO.select).
-      alive.chmod(m = 0 == m ? 1 : 0)
-
       while sock = ready.shift
         if client = sock.kgio_tryaccept
+          worker.tick = Time.now.to_i
           process_client(client)
+          worker.tick = 0
           nr += 1
-          alive.chmod(m = 0 == m ? 1 : 0)
         end
         break if nr < 0
       end
@@ -619,18 +602,17 @@ class Unicorn::HttpServer
       end
 
       ppid == Process.ppid or return
-      alive.chmod(m = 0 == m ? 1 : 0)
 
       # timeout used so we can detect parent death:
       ret = IO.select(LISTENERS, nil, SELF_PIPE, timeout) and ready = ret[0]
     rescue Errno::EBADF
       nr < 0 or return
     rescue => e
-      if alive
+      if worker
         logger.error "Unhandled listen loop exception #{e.inspect}."
         logger.error e.backtrace.join("\n")
       end
-    end while alive
+    end while worker
   end
 
   # delivers a signal to a worker and fails gracefully if the worker
@@ -638,7 +620,7 @@ class Unicorn::HttpServer
   def kill_worker(signal, wpid)
     Process.kill(signal, wpid)
     rescue Errno::ESRCH
-      worker = WORKERS.delete(wpid) and worker.tmp.close rescue nil
+      worker = WORKERS.delete(wpid) and worker.close rescue nil
   end
 
   # delivers a signal to each worker
diff --git a/lib/unicorn/worker.rb b/lib/unicorn/worker.rb
index 39e9e32..cd41e22 100644
--- a/lib/unicorn/worker.rb
+++ b/lib/unicorn/worker.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+require "raindrops"
 
 # This class and its members can be considered a stable interface
 # and will not change in a backwards-incompatible fashion between
@@ -7,13 +8,53 @@
 #
 # Some users may want to access it in the before_fork/after_fork hooks.
 # See the Unicorn::Configurator RDoc for examples.
-class Unicorn::Worker < Struct.new(:nr, :tmp, :switched)
+class Unicorn::Worker
+  # :stopdoc:
+  attr_accessor :nr, :switched
+  attr_writer :tmp
+
+  PER_DROP = Raindrops::PAGE_SIZE / Raindrops::SIZE
+  DROPS = []
+
+  def initialize(nr)
+    drop_index = nr / PER_DROP
+    @raindrop = DROPS[drop_index] ||= Raindrops.new(PER_DROP)
+    @offset = nr % PER_DROP
+    @raindrop[@offset] = 0
+    @nr = nr
+    @tmp = @switched = false
+  end
 
   # worker objects may be compared to just plain Integers
   def ==(other_nr) # :nodoc:
-    nr == other_nr
+    @nr == other_nr
+  end
+
+  # called in the worker process
+  def tick=(value) # :nodoc:
+    @raindrop[@offset] = value
+  end
+
+  # called in the master process
+  def tick # :nodoc:
+    @raindrop[@offset]
   end
 
+  # only exists for compatibility
+  def tmp # :nodoc:
+    @tmp ||= begin
+      tmp = Unicorn::TmpIO.new
+      tmp.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC)
+      tmp
+    end
+  end
+
+  def close # :nodoc:
+    @tmp.close if @tmp
+  end
+
+  # :startdoc:
+
   # In most cases, you should be using the Unicorn::Configurator#user
   # directive instead.  This method should only be used if you need
   # fine-grained control of exactly when you want to change permissions
@@ -36,12 +77,12 @@ class Unicorn::Worker < Struct.new(:nr, :tmp, :switched)
     uid = Etc.getpwnam(user).uid
     gid = Etc.getgrnam(group).gid if group
     Unicorn::Util.chown_logs(uid, gid)
-    tmp.chown(uid, gid)
+    @tmp.chown(uid, gid) if @tmp
     if gid && Process.egid != gid
       Process.initgroups(user, gid)
       Process::GID.change_privilege(gid)
     end
     Process.euid != uid and Process::UID.change_privilege(uid)
-    self.switched = true
+    @switched = true
   end
 end
diff --git a/script/isolate_for_tests b/script/isolate_for_tests
index 1c9d9b1..96848c1 100755
--- a/script/isolate_for_tests
+++ b/script/isolate_for_tests
@@ -17,6 +17,7 @@ opts = {
 pid = fork do
   Isolate.now!(opts) do
     gem 'sqlite3-ruby', '1.2.5'
+    gem 'raindrops', '0.6.1'
     gem 'kgio', '2.3.3'
     gem 'rack', '1.2.2'
   end
diff --git a/test/unit/test_droplet.rb b/test/unit/test_droplet.rb
new file mode 100644
index 0000000..73cf38c
--- /dev/null
+++ b/test/unit/test_droplet.rb
@@ -0,0 +1,28 @@
+require 'test/unit'
+require 'unicorn'
+
+class TestDroplet < Test::Unit::TestCase
+  def test_create_many_droplets
+    now = Time.now.to_i
+    tmp = (0..1024).map do |i|
+      droplet = Unicorn::Worker.new(i)
+      assert droplet.respond_to?(:tick)
+      assert_equal 0, droplet.tick
+      assert_equal(now, droplet.tick = now)
+      assert_equal now, droplet.tick
+      assert_equal(0, droplet.tick = 0)
+      assert_equal 0, droplet.tick
+    end
+  end
+
+  def test_shared_process
+    droplet = Unicorn::Worker.new(0)
+    _, status = Process.waitpid2(fork { droplet.tick += 1; exit!(0) })
+    assert status.success?, status.inspect
+    assert_equal 1, droplet.tick
+
+    _, status = Process.waitpid2(fork { droplet.tick += 1; exit!(0) })
+    assert status.success?, status.inspect
+    assert_equal 2, droplet.tick
+  end
+end
diff --git a/unicorn.gemspec b/unicorn.gemspec
index 2ce6c59..851424f 100644
--- a/unicorn.gemspec
+++ b/unicorn.gemspec
@@ -35,6 +35,7 @@ Gem::Specification.new do |s|
   # *strongly* recommended for security reasons.
   s.add_dependency(%q<rack>)
   s.add_dependency(%q<kgio>, '~> 2.4')
+  s.add_dependency(%w<raindrops>, '~> 0.6')
 
   s.add_development_dependency('isolate', '~> 3.1')
   s.add_development_dependency('wrongdoc', '~> 1.5')
-- 
Eric Wong

From jason at lookbook.nu  Mon Jun 20 19:40:16 2011
From: jason at lookbook.nu (Jason Su)
Date: Mon, 20 Jun 2011 16:40:16 -0700
Subject: problem setting multiple cookies
Message-ID: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>

Hey guys,

When I try to set multiple cookies in one request, the 'Set-Cookies'
header in the http response doesn't look right -- all the cookies are
set without any line breaks.

Looks like this:

Set-Cookie:first=; domain=.domain.com; path=/; expires=Thu,
01-Jan-1970 00:00:00 GMTsecond=1; domain=.domain.com; path=/;
expires=Sun, 20-Jun-2021 23:34:30 GMT

Should look more like this, which is what I got from a fresh Rails project test:

Set-Cookie:first=33; path=/
second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
third=69; path=/

In the first example, neither cookies (first/second) are set. The
second example, all 3 cookies are set.

I think it's happening somewhere here, but I can't figure out what's
going on... http://bogomips.org/unicorn.git/tree/lib/unicorn/cgi_wrapper.rb

Any help appreciated, thanks in advance!

jason

From normalperson at yhbt.net  Mon Jun 20 20:06:25 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 21 Jun 2011 00:06:25 +0000
Subject: problem setting multiple cookies
In-Reply-To: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
Message-ID: <20110621000625.GA20225@dcvr.yhbt.net>

Jason Su <jason at lookbook.nu> wrote:
> Hey guys,
> 
> When I try to set multiple cookies in one request, the 'Set-Cookies'
> header in the http response doesn't look right -- all the cookies are
> set without any line breaks.
> 
> Looks like this:
> 
> Set-Cookie:first=; domain=.domain.com; path=/; expires=Thu,
> 01-Jan-1970 00:00:00 GMTsecond=1; domain=.domain.com; path=/;
> expires=Sun, 20-Jun-2021 23:34:30 GMT

Which versions of Rails and Rack are you using?  How is your app
setting cookies?

Older versions of Rack may have specified a different delimiter (or an
Array).  Current versions only use "\n" inside a string, so it should be
something like:

  headers["Set-Cookie"] = "first=33; path=/\n" \
                          "second=; path=/\n" \
                          "third=; path=/"

Using Rack::Utils.set_cookie_header! (or a Rails-level wrapper) is
recommended, though.

> Should look more like this, which is what I got from a fresh Rails project test:
> 
> Set-Cookie:first=33; path=/
> second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
> third=69; path=/

The response should have 3 individual Set-Cookie lines:

  Set-Cookie: first=33; path=/
  Set-Cookie: second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
  Set-Cookie: third=69; path=/

> In the first example, neither cookies (first/second) are set. The
> second example, all 3 cookies are set.
> 
> I think it's happening somewhere here, but I can't figure out what's
> going on... http://bogomips.org/unicorn.git/tree/lib/unicorn/cgi_wrapper.rb

The code that does this transformation of the Rack header to the
client socket is here:

http://bogomips.org/unicorn.git/tree/lib/unicorn/http_response.rb

cgi_wrapper.rb is only for Rails <= 2.2, where it is called before
http_response.rb

-- 
Eric Wong

From jason at lookbook.nu  Mon Jun 20 20:25:02 2011
From: jason at lookbook.nu (Jason Su)
Date: Mon, 20 Jun 2011 17:25:02 -0700
Subject: problem setting multiple cookies
In-Reply-To: <20110621000625.GA20225@dcvr.yhbt.net>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
	<20110621000625.GA20225@dcvr.yhbt.net>
Message-ID: <BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>

Hey Eric,

I'm using Rails 2.3.8, and Rack 1.1.2

I'm setting cookies like this:

cookies[:user_id] = { :domain => ".#{domain}", :value => "#{user.id}",
:expires => 10.years.from_now }

How can I get the response to show up correctly, with individual
Set-Cookie lines for each?

Thanks again,

Jason



On Mon, Jun 20, 2011 at 5:06 PM, Eric Wong <normalperson at yhbt.net> wrote:
> Jason Su <jason at lookbook.nu> wrote:
>> Hey guys,
>>
>> When I try to set multiple cookies in one request, the 'Set-Cookies'
>> header in the http response doesn't look right -- all the cookies are
>> set without any line breaks.
>>
>> Looks like this:
>>
>> Set-Cookie:first=; domain=.domain.com; path=/; expires=Thu,
>> 01-Jan-1970 00:00:00 GMTsecond=1; domain=.domain.com; path=/;
>> expires=Sun, 20-Jun-2021 23:34:30 GMT
>
> Which versions of Rails and Rack are you using? ?How is your app
> setting cookies?
>
> Older versions of Rack may have specified a different delimiter (or an
> Array). ?Current versions only use "\n" inside a string, so it should be
> something like:
>
> ?headers["Set-Cookie"] = "first=33; path=/\n" \
> ? ? ? ? ? ? ? ? ? ? ? ? ?"second=; path=/\n" \
> ? ? ? ? ? ? ? ? ? ? ? ? ?"third=; path=/"
>
> Using Rack::Utils.set_cookie_header! (or a Rails-level wrapper) is
> recommended, though.
>
>> Should look more like this, which is what I got from a fresh Rails project test:
>>
>> Set-Cookie:first=33; path=/
>> second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
>> third=69; path=/
>
> The response should have 3 individual Set-Cookie lines:
>
> ?Set-Cookie: first=33; path=/
> ?Set-Cookie: second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
> ?Set-Cookie: third=69; path=/
>
>> In the first example, neither cookies (first/second) are set. The
>> second example, all 3 cookies are set.
>>
>> I think it's happening somewhere here, but I can't figure out what's
>> going on... http://bogomips.org/unicorn.git/tree/lib/unicorn/cgi_wrapper.rb
>
> The code that does this transformation of the Rack header to the
> client socket is here:
>
> http://bogomips.org/unicorn.git/tree/lib/unicorn/http_response.rb
>
> cgi_wrapper.rb is only for Rails <= 2.2, where it is called before
> http_response.rb
>
> --
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>

From normalperson at yhbt.net  Mon Jun 20 20:46:07 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 20 Jun 2011 17:46:07 -0700
Subject: problem setting multiple cookies
In-Reply-To: <BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
	<20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
Message-ID: <20110621004607.GA29647@dcvr.yhbt.net>

Jason Su <jason at lookbook.nu> wrote:
> Hey Eric,
> 
> I'm using Rails 2.3.8, and Rack 1.1.2

Please don't top post, if you're just replying to the message in general
I'd rather you not quote at all.

> I'm setting cookies like this:
> 
> cookies[:user_id] = { :domain => ".#{domain}", :value => "#{user.id}",
> :expires => 10.years.from_now }

> How can I get the response to show up correctly, with individual
> Set-Cookie lines for each?

I wonder if it's a bug in older versions of Rails or Rack, there was
some confusion back in the day about how to handle multiple values
for a single header...

Can you try loading the following middleware in your app?

If it doesn't work, uncomment the 'p' statement and show us what data
structure is used for your headers.

---------------------- join_cookie.rb ---------------------------------
# Totally untested:
# usage (in config.ru):
#   use JoinCookie
#   run YourApp.new
class JoinCookie < Struct.new(:app)
  def call(env)
    status, headers, body = app.call(env)

    ## uncomment and show me the output of the next line if this doesn't work
    # p headers

    headers = Rack::Utils::HeaderHash.new(headers)
    case set_cookie = headers["Set-Cookie"]
    when Array
      headers["Set-Cookie"] = set_cookie.join("\n")
    end

    [ status, headers, body ]
  end
end
-----------------------------------------------------------------

-- 
Eric Wong

From jason at lookbook.nu  Mon Jun 20 21:34:31 2011
From: jason at lookbook.nu (Jason Su)
Date: Mon, 20 Jun 2011 18:34:31 -0700
Subject: problem setting multiple cookies
In-Reply-To: <20110621004607.GA29647@dcvr.yhbt.net>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
	<20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
Message-ID: <BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>

I don't think it worked, but here's the output:

{"X-Runtime"=>"174", "Content-Type"=>"text/html",
"Content-Length"=>"1", "Set-Cookie"=>"first=; domain=.test.com;
path=/; expires=Thu, 01-Jan-1970 00:00:00 GMTsecond=1;
domain=.test.com; path=/; expires=Mon, 21-Jun-2021 01:22:23
GMT\n_test_session=ef56a1c675e576bd0b922acb0027ebde; domain=.test.com;
path=/; HttpOnly", "Cache-Control"=>"no-cache"}

Here is how the response header looks:

Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
21-Jun-2021 01:32:26 GMT

Thanks,
Jason

From jason at lookbook.nu  Mon Jun 20 21:35:43 2011
From: jason at lookbook.nu (Jason Su)
Date: Mon, 20 Jun 2011 18:35:43 -0700
Subject: problem setting multiple cookies
In-Reply-To: <BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
	<20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
Message-ID: <BANLkTimDfQiu=rzuaj-6P4yPdEN96VGF2g@mail.gmail.com>

Sorry to post again, but the response headers were captured after I
cleared my cookies, so it's not consistent with the first output.

From normalperson at yhbt.net  Mon Jun 20 22:15:24 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 21 Jun 2011 02:15:24 +0000
Subject: problem setting multiple cookies
In-Reply-To: <BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
	<20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
Message-ID: <20110621021524.GA31142@dcvr.yhbt.net>

Jason Su <jason at lookbook.nu> wrote:
> I don't think it worked, but here's the output:
> 
> {"X-Runtime"=>"174", "Content-Type"=>"text/html",
> "Content-Length"=>"1", "Set-Cookie"=>"first=; domain=.test.com;
> path=/; expires=Thu, 01-Jan-1970 00:00:00 GMTsecond=1;
> domain=.test.com; path=/; expires=Mon, 21-Jun-2021 01:22:23
> GMT\n_test_session=ef56a1c675e576bd0b922acb0027ebde; domain=.test.com;
      ^-- this is suspicous, the _test_session= cookie *is* set
correctly...

> path=/; HttpOnly", "Cache-Control"=>"no-cache"}
> 
> Here is how the response header looks:
> 
> Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
> 00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
> 21-Jun-2021 01:32:26 GMT

And you got a second Set-Cookie: line with "_test_session" in there,
right?

Any chance you could upgrade Rails to 2.3.11?  Some rails commits
since 2.3.8 look interesting:
e0eb8e9c65ededce64169948d4dd51b0079cdd10
85b6d79d8a17fdef667770e31b44ac6647f8b584

-- 
Eric Wong

From jason at lookbook.nu  Mon Jun 20 22:48:05 2011
From: jason at lookbook.nu (Jason Su)
Date: Mon, 20 Jun 2011 19:48:05 -0700
Subject: problem setting multiple cookies
In-Reply-To: <20110621021524.GA31142@dcvr.yhbt.net>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
	<20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
	<20110621021524.GA31142@dcvr.yhbt.net>
Message-ID: <BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>

Hey Eric,

I called p headers in JoinCookie before manipulating the headers at
all, so what you saw was probably already set correctly by rails. I'm
not setting the _test_session cookie myself.

Here's what I get in the response, with the _test_session cookie line:

Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
21-Jun-2021 02:40:27 GMT
_test_session=40658e37c9e8b39c5f8389e09682c47b; domain=.test.com;
path=/; HttpOnly

It doesn't add a second Set-Cookie: line, but the line break seems to work...

2.3.11 is giving me a ton of trouble.. I've tried it before.. and I
just tried again with little luck... Is there anything that I can just
grab from the commits to make it work on 2.3.8?

Thanks,
Jason

From normalperson at yhbt.net  Mon Jun 20 23:15:34 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 21 Jun 2011 03:15:34 +0000
Subject: problem setting multiple cookies
In-Reply-To: <BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
	<20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
	<20110621021524.GA31142@dcvr.yhbt.net>
	<BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>
Message-ID: <20110621031534.GA8753@dcvr.yhbt.net>

Jason Su <jason at lookbook.nu> wrote:
> Here's what I get in the response, with the _test_session cookie line:
> 
> Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
> 00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
> 21-Jun-2021 02:40:27 GMT
> _test_session=40658e37c9e8b39c5f8389e09682c47b; domain=.test.com;
> path=/; HttpOnly
> 
> It doesn't add a second Set-Cookie: line, but the line break seems to work...

Huh?  Which version of unicorn are you using?  The /\n/ splitting in
headers has been in the earliest releases of unicorn...

> 2.3.11 is giving me a ton of trouble.. I've tried it before.. and I
> just tried again with little luck... Is there anything that I can just
> grab from the commits to make it work on 2.3.8?

Look at the individual commits in rails.git.  I just used

  git log -p v2.3.8..v2.3.11 -- actionpack

in rails.git to find those for you.   I just looked for things that
would manipulate cookies/headers.

-- 
Eric Wong

From jason at lookbook.nu  Mon Jun 20 23:38:52 2011
From: jason at lookbook.nu (Jason Su)
Date: Mon, 20 Jun 2011 20:38:52 -0700
Subject: problem setting multiple cookies
In-Reply-To: <20110621031534.GA8753@dcvr.yhbt.net>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
	<20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
	<20110621021524.GA31142@dcvr.yhbt.net>
	<BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>
	<20110621031534.GA8753@dcvr.yhbt.net>
Message-ID: <BANLkTi=byv5YZmmsydYoWS9UdGyRUt3agw@mail.gmail.com>

I'm using unicorn 3.6.2

What I was trying to say is that the JoinCookie middleware didn't do
anything. headers["Set-Cookie"] is a string, and the _test_session
cookie was always being set correctly. The problem happens only when I
try to set multiple cookies myself.

Here's the output of p headers in JoinCookie again

{"X-Runtime"=>"92", "Content-Type"=>"text/html",
"Content-Length"=>"1", "Set-Cookie"=>"first=; domain=.test.com;
path=/; expires=Thu, 01-Jan-1970 00:00:00 GMTsecond=1;
domain=.test.com; path=/; expires=Mon, 21-Jun-2021 03:35:30
GMT\n_test_session=1804126e6bad8cf4b25602b74218f003; domain=.test.com;
path=/; HttpOnly", "Cache-Control"=>"no-cache"}

Where is JoinCookie is getting it's headers from?


On Mon, Jun 20, 2011 at 8:15 PM, Eric Wong <normalperson at yhbt.net> wrote:
> Jason Su <jason at lookbook.nu> wrote:
>> Here's what I get in the response, with the _test_session cookie line:
>>
>> Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
>> 00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
>> 21-Jun-2021 02:40:27 GMT
>> _test_session=40658e37c9e8b39c5f8389e09682c47b; domain=.test.com;
>> path=/; HttpOnly
>>
>> It doesn't add a second Set-Cookie: line, but the line break seems to work...
>
> Huh? ?Which version of unicorn are you using? ?The /\n/ splitting in
> headers has been in the earliest releases of unicorn...
>
>> 2.3.11 is giving me a ton of trouble.. I've tried it before.. and I
>> just tried again with little luck... Is there anything that I can just
>> grab from the commits to make it work on 2.3.8?
>
> Look at the individual commits in rails.git. ?I just used
>
> ?git log -p v2.3.8..v2.3.11 -- actionpack
>
> in rails.git to find those for you. ? I just looked for things that
> would manipulate cookies/headers.
>
> --
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>

From normalperson at yhbt.net  Tue Jun 21 14:29:49 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 21 Jun 2011 11:29:49 -0700
Subject: problem setting multiple cookies
In-Reply-To: <BANLkTi=byv5YZmmsydYoWS9UdGyRUt3agw@mail.gmail.com>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
	<20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
	<20110621021524.GA31142@dcvr.yhbt.net>
	<BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>
	<20110621031534.GA8753@dcvr.yhbt.net>
	<BANLkTi=byv5YZmmsydYoWS9UdGyRUt3agw@mail.gmail.com>
Message-ID: <20110621182949.GB6054@dcvr.yhbt.net>

Jason Su <jason at lookbook.nu> wrote:
> I'm using unicorn 3.6.2
> 
> What I was trying to say is that the JoinCookie middleware didn't do
> anything. headers["Set-Cookie"] is a string, and the _test_session
> cookie was always being set correctly. The problem happens only when I
> try to set multiple cookies myself.

Yeah, you're not getting an Array (at least not at that stage) as I
semi-hoped so JoinCookie wouldn't solve it.

> Here's the output of p headers in JoinCookie again
> 
> {"X-Runtime"=>"92", "Content-Type"=>"text/html",
> "Content-Length"=>"1", "Set-Cookie"=>"first=; domain=.test.com;
> path=/; expires=Thu, 01-Jan-1970 00:00:00 GMTsecond=1;
> domain=.test.com; path=/; expires=Mon, 21-Jun-2021 03:35:30
> GMT\n_test_session=1804126e6bad8cf4b25602b74218f003; domain=.test.com;
> path=/; HttpOnly", "Cache-Control"=>"no-cache"}
> 
> Where is JoinCookie is getting it's headers from?

app.call(env) is what dispatches your Rails app, so the headers are what
Rails (and middlewares further down the stack) sets.

Any luck with looking at Rails changes that might impact this?

At this point it's clear it's a Rails/Rack issue (and for an old
version, at that) and not something that can be done with Unicorn.

Maybe you can fix whatever the Rails v2.3.8..v2.3.11 upgrade broke
in your app, but whatever's broken is broken at the application layer
and not in Unicorn.

-- 
Eric Wong

From jason at lookbook.nu  Tue Jun 21 21:01:28 2011
From: jason at lookbook.nu (Jason Su)
Date: Tue, 21 Jun 2011 18:01:28 -0700
Subject: problem setting multiple cookies
In-Reply-To: <20110621182949.GB6054@dcvr.yhbt.net>
References: <BANLkTikS7uukeLamG8ORdo97FpP7qK+jmg@mail.gmail.com>
	<20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
	<20110621021524.GA31142@dcvr.yhbt.net>
	<BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>
	<20110621031534.GA8753@dcvr.yhbt.net>
	<BANLkTi=byv5YZmmsydYoWS9UdGyRUt3agw@mail.gmail.com>
	<20110621182949.GB6054@dcvr.yhbt.net>
Message-ID: <BANLkTimyFe4MDmJLJTbYwUNVj=5s1WSHYQ@mail.gmail.com>

That's strange, doesn't really explain why cookies work fine on a
fresh Rails 2.3.8 app, nor why cookies stopped working for me after I
switched to Unicorn.

I'll keep looking..

From normalperson at yhbt.net  Tue Jun 21 21:59:34 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 22 Jun 2011 01:59:34 +0000
Subject: problem setting multiple cookies
In-Reply-To: <BANLkTimyFe4MDmJLJTbYwUNVj=5s1WSHYQ@mail.gmail.com>
References: <20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
	<20110621021524.GA31142@dcvr.yhbt.net>
	<BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>
	<20110621031534.GA8753@dcvr.yhbt.net>
	<BANLkTi=byv5YZmmsydYoWS9UdGyRUt3agw@mail.gmail.com>
	<20110621182949.GB6054@dcvr.yhbt.net>
	<BANLkTimyFe4MDmJLJTbYwUNVj=5s1WSHYQ@mail.gmail.com>
Message-ID: <20110622015934.GA18959@dcvr.yhbt.net>

Jason Su <jason at lookbook.nu> wrote:
> That's strange, doesn't really explain why cookies work fine on a
> fresh Rails 2.3.8 app, nor why cookies stopped working for me after I
> switched to Unicorn.

Actually, another option you could try is to use OldRails (designed for
Rails <= 2.2) and try to get that working with Rails 2.3.  This should
cause Rails 2.3 to use the cgi_wrapper.rb and not the native Rack
interface:

--------------- config.ru ------------------
require "unicorn/app/old_rails"
use Unicorn::App::OldRails::Static
run Unicorn::App::OldRails.new
--------------------------------------------

-- 
Eric Wong

From jason at lookbook.nu  Wed Jun 22 00:18:23 2011
From: jason at lookbook.nu (Jason Su)
Date: Tue, 21 Jun 2011 21:18:23 -0700
Subject: problem setting multiple cookies
In-Reply-To: <20110622015934.GA18959@dcvr.yhbt.net>
References: <20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
	<20110621021524.GA31142@dcvr.yhbt.net>
	<BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>
	<20110621031534.GA8753@dcvr.yhbt.net>
	<BANLkTi=byv5YZmmsydYoWS9UdGyRUt3agw@mail.gmail.com>
	<20110621182949.GB6054@dcvr.yhbt.net>
	<BANLkTimyFe4MDmJLJTbYwUNVj=5s1WSHYQ@mail.gmail.com>
	<20110622015934.GA18959@dcvr.yhbt.net>
Message-ID: <BANLkTinK9tPc6dHu==T=sDFMfjQdYhwp3w@mail.gmail.com>

Hey Eric,

Actually I think I am using OldRails -- don't know what it is or why
I'm doing it, but this is in my config.ru:

map '/' do
  if RAILS_ENV != 'production'
    begin
      require 'rack/bug'
      use Rack::Bug, :password => nil
    rescue LoadError
    end

    use Rails::Rack::Static
  end

  run Unicorn::App::OldRails.new
end

Should I not be using OldRails?



I'm also trying to upgrade to Rails 2.3.11 again. Here's the error
that's stopping me:

You have a nil object when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.split You have a nil object
when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.split
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/actionpack-2.3.11/lib/action_controller/cgi_process.rb:52:in
`dispatch_cgi'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:101:in
`dispatch_cgi'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:27:in
`dispatch'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/app/old_rails.rb:22:in
`call'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/app/old_rails/static.rb:56:in
`call'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:47:in
`call'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:41:in
`each'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:41:in
`call'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:545:in
`process_client'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/oob_gc.rb:60:in
`process_client'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:623:in
`worker_loop'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:621:in
`each'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:621:in
`worker_loop'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:509:in
`spawn_missing_workers'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:507:in
`fork'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:507:in
`spawn_missing_workers'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:503:in
`each'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:503:in
`spawn_missing_workers'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:516:in
`maintain_worker_count'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:166:in
`start'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn.rb:30:in
`run'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/bin/unicorn_rails:208
/Users/kittyflat/.rvm/gems/ruby-1.8.7-p334/bin/unicorn_rails:19:in `load'
/Users/kittyflat/.rvm/gems/ruby-1.8.7-p334/bin/unicorn_rails:19

I found this, which I think is somehow related to the fix:
https://gist.github.com/826692

Do you see anything that looks familiar?

Thanks,
jason

From jason at lookbook.nu  Wed Jun 22 00:20:40 2011
From: jason at lookbook.nu (Jason Su)
Date: Tue, 21 Jun 2011 21:20:40 -0700
Subject: problem setting multiple cookies
In-Reply-To: <BANLkTinK9tPc6dHu==T=sDFMfjQdYhwp3w@mail.gmail.com>
References: <20110621000625.GA20225@dcvr.yhbt.net>
	<BANLkTi=EzX=qiv=V_ZXVrWY3PETjVBbFYw@mail.gmail.com>
	<20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
	<20110621021524.GA31142@dcvr.yhbt.net>
	<BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>
	<20110621031534.GA8753@dcvr.yhbt.net>
	<BANLkTi=byv5YZmmsydYoWS9UdGyRUt3agw@mail.gmail.com>
	<20110621182949.GB6054@dcvr.yhbt.net>
	<BANLkTimyFe4MDmJLJTbYwUNVj=5s1WSHYQ@mail.gmail.com>
	<20110622015934.GA18959@dcvr.yhbt.net>
	<BANLkTinK9tPc6dHu==T=sDFMfjQdYhwp3w@mail.gmail.com>
Message-ID: <BANLkTimVnQHtJewysrGjn_FC=rdNyZDWEw@mail.gmail.com>

FYI I found that gist on this blog:
http://rcaguilar.wordpress.com/2011/02/14/broken-mongrels-after-rails-2-3-11-upgrade

From swissnld at ymail.com  Mon Jun 20 23:56:17 2011
From: swissnld at ymail.com (Swiss Lotto)
Date: Tue, 21 Jun 2011 05:56:17 +0200
Subject: No subject
Message-ID: <13086295180734aab4031ffdf1881c5508afb83f70@ymail.com>

SWISS LOTTO
Bloemenstraat 321, 2685 AG,
Poeldijk, Nederlands. 
www.swisslotto.ch

Dear Winner,

This is to inform you that you have won 850,000.00 Euro in the SWISS LOTTO E-Mail Lottery in which E-Mail address are selected randomly from the internet through the world wide web.

Your E-Mail address, attached to Reference Number NLD/17-I070182/09 with Serial Number FAI/EU01/SL drew the Lucky Number 10, and consequently won the E-Mail Lottery in the "A" Category. You have therefore been approved for the pay out of 850,000.00 Euro

To file for your claim, please contact our/your accredited agent:
Tomas Hendriks
Tel: +31-64-425-8175
Email: snfidagnt at aim.com

Remember to quote your lottery winning details in all your correspondences with us or our/your designated agent. 

Congratulations once more from all members and staff of this program and thank you for being part of our 2011 promotional program. 

Yours truly, 
Visser Van Leeuwen 
Online Games Director


From normalperson at yhbt.net  Wed Jun 22 02:02:55 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 22 Jun 2011 06:02:55 +0000
Subject: problem setting multiple cookies
In-Reply-To: <BANLkTinK9tPc6dHu==T=sDFMfjQdYhwp3w@mail.gmail.com>
References: <20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
	<20110621021524.GA31142@dcvr.yhbt.net>
	<BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>
	<20110621031534.GA8753@dcvr.yhbt.net>
	<BANLkTi=byv5YZmmsydYoWS9UdGyRUt3agw@mail.gmail.com>
	<20110621182949.GB6054@dcvr.yhbt.net>
	<BANLkTimyFe4MDmJLJTbYwUNVj=5s1WSHYQ@mail.gmail.com>
	<20110622015934.GA18959@dcvr.yhbt.net>
	<BANLkTinK9tPc6dHu==T=sDFMfjQdYhwp3w@mail.gmail.com>
Message-ID: <20110622060255.GA1838@dcvr.yhbt.net>

Jason Su <jason at lookbook.nu> wrote:
> Hey Eric,
> 
> Actually I think I am using OldRails -- don't know what it is or why
> I'm doing it, but this is in my config.ru:

It might be because you were on Rails 2.2 or earlier and upgraded?

> map '/' do
>   if RAILS_ENV != 'production'
>     begin
>       require 'rack/bug'
>       use Rack::Bug, :password => nil
>     rescue LoadError
>     end
> 
>     use Rails::Rack::Static
>   end
> 
>   run Unicorn::App::OldRails.new
> end
> 
> Should I not be using OldRails?

OldRails was for Rails <= 2.2, so you should probably try this instead:

  run ActionController::Dispatcher.new

This should use the pure Rack backend that avoids the old CGI
and CGIWrapper code which Rails <=2.2 relied on.

> I'm also trying to upgrade to Rails 2.3.11 again. Here's the error
> that's stopping me:
> 
> You have a nil object when you didn't expect it!
> You might have expected an instance of Array.
> The error occurred while evaluating nil.split You have a nil object
> when you didn't expect it!

<snip>

> I found this, which I think is somehow related to the fix:
> https://gist.github.com/826692
> 
> Do you see anything that looks familiar?

I haven't seen it before, but it looks to be on the right track.

OldRails uses the CGIWrapper code which is based on the original
code in Mongrel, too.

If you avoid CGIWrapper, you should be able to avoid the
workarounds/issues with it like the above gist.

-- 
Eric Wong

From jason at lookbook.nu  Wed Jun 22 08:51:37 2011
From: jason at lookbook.nu (Jason Su)
Date: Wed, 22 Jun 2011 05:51:37 -0700
Subject: problem setting multiple cookies
In-Reply-To: <20110622060255.GA1838@dcvr.yhbt.net>
References: <20110621004607.GA29647@dcvr.yhbt.net>
	<BANLkTimUQsSLagt22SwzNFawBuQQUpoeKA@mail.gmail.com>
	<20110621021524.GA31142@dcvr.yhbt.net>
	<BANLkTingtBuYeg6pPXrqNJF+=1vAG0Q67Q@mail.gmail.com>
	<20110621031534.GA8753@dcvr.yhbt.net>
	<BANLkTi=byv5YZmmsydYoWS9UdGyRUt3agw@mail.gmail.com>
	<20110621182949.GB6054@dcvr.yhbt.net>
	<BANLkTimyFe4MDmJLJTbYwUNVj=5s1WSHYQ@mail.gmail.com>
	<20110622015934.GA18959@dcvr.yhbt.net>
	<BANLkTinK9tPc6dHu==T=sDFMfjQdYhwp3w@mail.gmail.com>
	<20110622060255.GA1838@dcvr.yhbt.net>
Message-ID: <BANLkTi=wZRScGuv2fFBO5BHNwFTZOjSQOw@mail.gmail.com>

Upgrading to 2.3.11 and not using OldRails did it!

Thanks so much Eric

Jason

From mohit.chawla.binary at gmail.com  Thu Jun 23 12:36:20 2011
From: mohit.chawla.binary at gmail.com (Mohit Chawla)
Date: Thu, 23 Jun 2011 22:06:20 +0530
Subject: query string max length tweak ?
Message-ID: <BANLkTi=yME+NP9_XnfVFxwGJRNnKW0=uBw@mail.gmail.com>

Hi,

Is there a way that would allow us to increase the max length allowed
for query_string ? We have been getting 414s on large GET requests. We
are running the puppet master under unicorn behind nginx. I am not
sure of the general implications of this, whether its suggested/safe
even, but it is somewhat of a necessity. Any suggestions ?

From normalperson at yhbt.net  Thu Jun 23 14:37:32 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 23 Jun 2011 11:37:32 -0700
Subject: query string max length tweak ?
In-Reply-To: <BANLkTi=yME+NP9_XnfVFxwGJRNnKW0=uBw@mail.gmail.com>
References: <BANLkTi=yME+NP9_XnfVFxwGJRNnKW0=uBw@mail.gmail.com>
Message-ID: <20110623183732.GA12651@dcvr.yhbt.net>

Mohit Chawla <mohit.chawla.binary at gmail.com> wrote:
> Hi,
> 
> Is there a way that would allow us to increase the max length allowed
> for query_string ? We have been getting 414s on large GET requests. We
> are running the puppet master under unicorn behind nginx. I am not
> sure of the general implications of this, whether its suggested/safe
> even, but it is somewhat of a necessity. Any suggestions ?

You can change the following lines in ext/unicorn/global_variables.h
and rebuild the parser:

  DEF_MAX_LENGTH(REQUEST_URI, 1024 * 12);
  ...
  DEF_MAX_LENGTH(QUERY_STRING, (1024 * 10));

There's currently no way to configure them otherwise, but I can consider
dropping these per-field limits entirely.  We now have a config option
to set the maximum header size (only exposed via Rainbows!, though).

This means we would only care about the size of the entire header, and
not the size of any individual element.  I consider the size of the
entire header more important anyways.  We currently don't have
limits on the number of headers that can be sent.

-- 
Eric Wong

From mohit.chawla.binary at gmail.com  Thu Jun 23 15:31:10 2011
From: mohit.chawla.binary at gmail.com (Mohit Chawla)
Date: Fri, 24 Jun 2011 01:01:10 +0530
Subject: query string max length tweak ?
In-Reply-To: <20110623183732.GA12651@dcvr.yhbt.net>
References: <BANLkTi=yME+NP9_XnfVFxwGJRNnKW0=uBw@mail.gmail.com>
	<20110623183732.GA12651@dcvr.yhbt.net>
Message-ID: <BANLkTinCsg0ZQBboFRXojafrFhdSQoD0zA@mail.gmail.com>

Hi,

On Fri, Jun 24, 2011 at 12:07 AM, Eric Wong <normalperson at yhbt.net> wrote:
> You can change the following lines in ext/unicorn/global_variables.h
> and rebuild the parser:
>
> ?DEF_MAX_LENGTH(REQUEST_URI, 1024 * 12);
> ?...
> ?DEF_MAX_LENGTH(QUERY_STRING, (1024 * 10));
>
> There's currently no way to configure them otherwise, but I can consider
> dropping these per-field limits entirely. ?We now have a config option
> to set the maximum header size (only exposed via Rainbows!, though).
>
> This means we would only care about the size of the entire header, and
> not the size of any individual element. ?I consider the size of the
> entire header more important anyways. ?We currently don't have
> limits on the number of headers that can be sent.

Thanks a lot, yes I was trying out changing only the QUERY_STRING max
length, but that wasn't working. Will change the REQUEST_URI max
length as well. Also, I am new to using unicorn, hence not aware if
using Rainbows! in this scenario can be achieved/is suggested ? Nice
to know about limiting the entire size header in future releases ( in
prod we are using 1.1.4 ! ).

Also, sorry for the multiple html mails before this !

From normalperson at yhbt.net  Thu Jun 23 20:38:35 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 23 Jun 2011 17:38:35 -0700
Subject: query string max length tweak ?
In-Reply-To: <BANLkTinCsg0ZQBboFRXojafrFhdSQoD0zA@mail.gmail.com>
References: <BANLkTi=yME+NP9_XnfVFxwGJRNnKW0=uBw@mail.gmail.com>
	<20110623183732.GA12651@dcvr.yhbt.net>
	<BANLkTinCsg0ZQBboFRXojafrFhdSQoD0zA@mail.gmail.com>
Message-ID: <20110624003835.GA17186@dcvr.yhbt.net>

Mohit Chawla <mohit.chawla.binary at gmail.com> wrote:
> Also, I am new to using unicorn, hence not aware if
> using Rainbows! in this scenario can be achieved/is suggested ? Nice
> to know about limiting the entire size header in future releases ( in
> prod we are using 1.1.4 ! ).

No, Rainbows! is focused on network concurrency.  Unicorn can just use
nginx for stopping large requests (large_client_header_buffers in nginx)
and not be concerned about large headers at all.

> Also, sorry for the multiple html mails before this !

I just made the mailman config on rubyforge kill all HTML :)

-- 
Eric Wong

From normalperson at yhbt.net  Fri Jun 24 13:30:38 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 24 Jun 2011 10:30:38 -0700
Subject: [PATCH] replace fchmod()-based heartbeat with raindrops
In-Reply-To: <20110616232508.GA16538@dcvr.yhbt.net>
References: <20110616232508.GA16538@dcvr.yhbt.net>
Message-ID: <20110624173038.GA22131@dcvr.yhbt.net>

Eric Wong <normalperson at yhbt.net> wrote:
>  The subset of raindrops used by Unicorn should work on all machines
>  with mmap(2) + MAP_ANON/MAP_ANONYMOUS support, so *BSDs shouldn't be
>  left out.

I've had one report from a FreeBSD 8 user who couldn't get raindrops
to build.  I remember getting it to work under FreeBSD (7.2) in the
past (when I had access to a FreeBSD machine)

http://thread.gmane.org/gmane.comp.lang.ruby.raindrops.general/32

If anybody can help, that'd be great!

-- 
Eric Wong

From jeremyevans0 at gmail.com  Fri Jun 24 14:17:47 2011
From: jeremyevans0 at gmail.com (Jeremy Evans)
Date: Fri, 24 Jun 2011 11:17:47 -0700
Subject: [PATCH] replace fchmod()-based heartbeat with raindrops
In-Reply-To: <20110624173038.GA22131@dcvr.yhbt.net>
References: <20110616232508.GA16538@dcvr.yhbt.net>
	<20110624173038.GA22131@dcvr.yhbt.net>
Message-ID: <BANLkTi=a=jHsSVC0Qfx57xG-gXQk9iiB1w@mail.gmail.com>

On Fri, Jun 24, 2011 at 10:30 AM, Eric Wong <normalperson at yhbt.net> wrote:
> Eric Wong <normalperson at yhbt.net> wrote:
>> ?The subset of raindrops used by Unicorn should work on all machines
>> ?with mmap(2) + MAP_ANON/MAP_ANONYMOUS support, so *BSDs shouldn't be
>> ?left out.

FWIW, raindrops builds fine on OpenBSD 4.9 amd64.

Jeremy

From normalperson at yhbt.net  Fri Jun 24 16:51:58 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 24 Jun 2011 13:51:58 -0700
Subject: [PATCH] replace fchmod()-based heartbeat with raindrops
In-Reply-To: <BANLkTi=a=jHsSVC0Qfx57xG-gXQk9iiB1w@mail.gmail.com>
References: <20110616232508.GA16538@dcvr.yhbt.net>
	<20110624173038.GA22131@dcvr.yhbt.net>
	<BANLkTi=a=jHsSVC0Qfx57xG-gXQk9iiB1w@mail.gmail.com>
Message-ID: <20110624205158.GA5583@dcvr.yhbt.net>

Jeremy Evans <jeremyevans0 at gmail.com> wrote:
> On Fri, Jun 24, 2011 at 10:30 AM, Eric Wong <normalperson at yhbt.net> wrote:
> > Eric Wong <normalperson at yhbt.net> wrote:
> >> ?The subset of raindrops used by Unicorn should work on all machines
> >> ?with mmap(2) + MAP_ANON/MAP_ANONYMOUS support, so *BSDs shouldn't be
> >> ?left out.
> 
> FWIW, raindrops builds fine on OpenBSD 4.9 amd64.

Thanks for the confirmation!  I'm at a loss as to what's causing
issues under FreeBSD 8 for him...

-- 
Eric Wong

From asimic at gmail.com  Fri Jun 24 18:50:01 2011
From: asimic at gmail.com (Aleksandar Simic)
Date: Fri, 24 Jun 2011 23:50:01 +0100
Subject: [PATCH] replace fchmod()-based heartbeat with raindrops
In-Reply-To: <20110624205158.GA5583@dcvr.yhbt.net>
References: <20110616232508.GA16538@dcvr.yhbt.net>
	<20110624173038.GA22131@dcvr.yhbt.net>
	<BANLkTi=a=jHsSVC0Qfx57xG-gXQk9iiB1w@mail.gmail.com>
	<20110624205158.GA5583@dcvr.yhbt.net>
Message-ID: <BANLkTinV+8uE80CpubvzgJp_6-+RETSvRg@mail.gmail.com>

On Fri, Jun 24, 2011 at 9:51 PM, Eric Wong <normalperson at yhbt.net> wrote:
> Jeremy Evans <jeremyevans0 at gmail.com> wrote:
>> On Fri, Jun 24, 2011 at 10:30 AM, Eric Wong <normalperson at yhbt.net> wrote:
>> > Eric Wong <normalperson at yhbt.net> wrote:
>> >> ?The subset of raindrops used by Unicorn should work on all machines
>> >> ?with mmap(2) + MAP_ANON/MAP_ANONYMOUS support, so *BSDs shouldn't be
>> >> ?left out.
>>
>> FWIW, raindrops builds fine on OpenBSD 4.9 amd64.
>
> Thanks for the confirmation! ?I'm at a loss as to what's causing
> issues under FreeBSD 8 for him...


Hello,

the same OS & ruby version as the original poster who reported the
issue. I see the same symptoms.

Cowboy fix: adding "-D__BSD_VISIBLE" to CPPFLAGS gets raindrops gem to build.

Easily accessible and viewable sys/mmap.h:

http://www.gitorious.net/freebsd/freebsd/blobs/HEAD/sys/sys/mman.h

If required, I can give you access to my FreeBSD machine, just let me know.

Thanks,
Aleksandar

From normalperson at yhbt.net  Fri Jun 24 20:15:34 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 24 Jun 2011 17:15:34 -0700
Subject: [PATCH] replace fchmod()-based heartbeat with raindrops
In-Reply-To: <BANLkTinV+8uE80CpubvzgJp_6-+RETSvRg@mail.gmail.com>
References: <20110616232508.GA16538@dcvr.yhbt.net>
	<20110624173038.GA22131@dcvr.yhbt.net>
	<BANLkTi=a=jHsSVC0Qfx57xG-gXQk9iiB1w@mail.gmail.com>
	<20110624205158.GA5583@dcvr.yhbt.net>
	<BANLkTinV+8uE80CpubvzgJp_6-+RETSvRg@mail.gmail.com>
Message-ID: <20110625001534.GA18193@dcvr.yhbt.net>

Aleksandar Simic <asimic at gmail.com> wrote:
> Hello,
> 
> the same OS & ruby version as the original poster who reported the
> issue. I see the same symptoms.
> 
> Cowboy fix: adding "-D__BSD_VISIBLE" to CPPFLAGS gets raindrops gem to build.
> 
> Easily accessible and viewable sys/mmap.h:
> 
> http://www.gitorious.net/freebsd/freebsd/blobs/HEAD/sys/sys/mman.h

Aha!  So defining _XOPEN_SOURCE appears to cause __BSD_VISIBLE to
not be defined in sys/cdefs.h.  I've pushed out the below patch to
git://bogomips.org/raindrops.git

> If required, I can give you access to my FreeBSD machine, just let me know.

If the below patch doesn't fix it, yes, it'd be greatly appreciated,
thanks!

>From 1e7dc89cc38c5dec0b63ac452b23141297701f88 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson at yhbt.net>
Date: Fri, 24 Jun 2011 17:06:56 -0700
Subject: [PATCH] remove _XOPEN_SOURCE #define for FreeBSD

This appears to cause __BSD_VISIBLE to not be defined,
which is required for MAP_ANON to be visible in
sys/mman.h

Thanks for Aleksandar Simic for the hint and Troex Nevelin
for the bug report!
---
 ext/raindrops/extconf.rb |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/ext/raindrops/extconf.rb b/ext/raindrops/extconf.rb
index 825625d..9f5de95 100644
--- a/ext/raindrops/extconf.rb
+++ b/ext/raindrops/extconf.rb
@@ -7,7 +7,7 @@ have_func('munmap', 'sys/mman.h') or abort 'munmap() not found'
 $CPPFLAGS += " -D_GNU_SOURCE "
 have_func('mremap', 'sys/mman.h')
 
-$CPPFLAGS += " -D_BSD_SOURCE -D_XOPEN_SOURCE=600 "
+$CPPFLAGS += " -D_BSD_SOURCE "
 have_func("getpagesize", "unistd.h")
 have_func('rb_thread_blocking_region')
 have_func('rb_thread_io_blocking_region')
-- 
Eric Wong

From fxn at hashref.com  Sat Jun 25 12:08:59 2011
From: fxn at hashref.com (Xavier Noria)
Date: Sat, 25 Jun 2011 18:08:59 +0200
Subject: Unicorn and streaming in Rails 3.1
Message-ID: <BANLkTika2jE-ZFk1Q0VgWxAk44LHg16LJg@mail.gmail.com>

Streaming works with Unicorn + Apache. Both with and without deflating.

My understanding is that Unicorn + Apache is not a good combination
though because Apache does not buffer, and thus Unicorn has no fast
client in front. (I don't know which is the ultimate technical reason
Unicorn puts such an emphasis on fast clients, but will do some
research about it.)

I have seen in

    http://unicorn.bogomips.org/examples/nginx.conf

the comment

    "You normally want nginx to buffer responses to slow
    clients, even with Rails 3.1 streaming because otherwise a slow
    client can become a bottleneck of Unicorn."

If I understand how this works correctly, nginx buffers the entire
response from Unicorn. First filling what's configured in
proxy_buffer_size and proxy_buffers, and then going to disk if needed
as a last resort. Thus, even if the application streams, I believe the
client will receive the chunked response, but only after it has been
generated by the application and fully buffered by nginx. Which
defeats the purpose of streaming in the use case we have in mind in
Rails 3.1, which is to serve HEAD as soon as possible.

Is that comment in the example configuration file actually saying that
Unicorn with nginx buffering is not broken? I mean, that if your
application has some actions with stream enabled and you put it behind
this setup, the content will be delivered albeit not streamed?

If that is correct. Is it reasonable to send to nginx the header
X-Accel-Buffering to disable buffering only for streamed responses? Or
is it a very bad idea? If it is a real bad idea, is the recommendation
to Unicorn users that they should just ignore this new feature?

From normalperson at yhbt.net  Sat Jun 25 16:16:36 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Sat, 25 Jun 2011 20:16:36 +0000
Subject: Unicorn and streaming in Rails 3.1
In-Reply-To: <BANLkTika2jE-ZFk1Q0VgWxAk44LHg16LJg@mail.gmail.com>
References: <BANLkTika2jE-ZFk1Q0VgWxAk44LHg16LJg@mail.gmail.com>
Message-ID: <20110625201636.GA22343@dcvr.yhbt.net>

Xavier Noria <fxn at hashref.com> wrote:
> Streaming works with Unicorn + Apache. Both with and without deflating.
> 
> My understanding is that Unicorn + Apache is not a good combination
> though because Apache does not buffer, and thus Unicorn has no fast
> client in front. (I don't know which is the ultimate technical reason
> Unicorn puts such an emphasis on fast clients, but will do some
> research about it.)

Basically the per-connection overhead of Unicorn is huge, an entire Ruby
process (tens to several hundreds of megabytes).  The per-connection
overhead of nginx is tiny: maybe a few KB in userspace (including
buffers), and a few KB in in the kernel.  You don't want to maintain
connections to Unicorn for a long time because of that cost.

OK, if you have any specific questions that aren't answered in the
website, please ask.

> I have seen in
> 
>     http://unicorn.bogomips.org/examples/nginx.conf
> 
> the comment
> 
>     "You normally want nginx to buffer responses to slow
>     clients, even with Rails 3.1 streaming because otherwise a slow
>     client can become a bottleneck of Unicorn."
> 
> If I understand how this works correctly, nginx buffers the entire
> response from Unicorn. First filling what's configured in
> proxy_buffer_size and proxy_buffers, and then going to disk if needed
> as a last resort. Thus, even if the application streams, I believe the
> client will receive the chunked response, but only after it has been
> generated by the application and fully buffered by nginx. Which
> defeats the purpose of streaming

Yes.

> in the use case we have in mind in
> Rails 3.1, which is to serve HEAD as soon as possible.

Small nit: s/HEAD/the response header/   "HEAD" is a /request/ that only
expects to receive the response header.

nginx only sends HTTP/1.0 requests to unicorn, so Rack::Chunked won't
actually send a chunked/streamed response.  Rails 3.1 /could/ enable
streaming without chunking for HTTP/1.0, but only if the client
didn't set a non-standard HTTP/1.0 header to enable keepalive.  This
is because HTTP/1.0 (w/o keepalive) relies on the server to close
the connection to signal the end of a response.

> Is that comment in the example configuration file actually saying that
> Unicorn with nginx buffering is not broken? I mean, that if your
> application has some actions with stream enabled and you put it behind
> this setup, the content will be delivered albeit not streamed?

Correct.

> If that is correct. Is it reasonable to send to nginx the header
> X-Accel-Buffering to disable buffering only for streamed responses? Or
> is it a very bad idea? If it is a real bad idea, is the recommendation
> to Unicorn users that they should just ignore this new feature?

You can use "X-Accel-Buffering: no" if you know your responses are small
enough to fit into the kernel socket buffers.  There's two kernel
buffers (Unicorn + nginx), you can get a little more space there.  nginx
shouldn't make another request to Unicorn if it's blocked writing a
response to the client already, so an evil pipelining client should not
hurt unicorn in this case:

   require "socket"
   host = "example.com"
   s = TCPSocket.new(host, 80)
   req = "GET /something/big HTTP/1.1\r\nHost: #{host}\r\n\r\n"

   # pipeline a large number of requests, nginx won't send another
   # request to an upstream if it's still writing one
   30.times { s.write(req) }

   # don't read the response, or read it slowly, just keep the socket
   # open here...
   sleep

-- 
Eric Wong

From normalperson at yhbt.net  Sat Jun 25 16:33:29 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Sat, 25 Jun 2011 20:33:29 +0000
Subject: Unicorn and streaming in Rails 3.1
In-Reply-To: <BANLkTika2jE-ZFk1Q0VgWxAk44LHg16LJg@mail.gmail.com>
References: <BANLkTika2jE-ZFk1Q0VgWxAk44LHg16LJg@mail.gmail.com>
Message-ID: <20110625203329.GB22343@dcvr.yhbt.net>

Xavier Noria <fxn at hashref.com> wrote:
> If it is a real bad idea, is the recommendation
> to Unicorn users that they should just ignore this new feature?

Another thing, Rainbows! + ThreadSpawn/ThreadPool concurrency may do the
trick (without needing nginx at all).  The per-client overhead of
Rainbows! + threads (several hundred KB) is higher than nginx, but still
much lower than Unicorn.

All your Rails code must be thread-safe, though.

If you use Linux, XEpollThreadPool/XEpollThreadSpawn can be worth a
try, too.  The cost of completely idle keepalive clients should be
roughly inline with nginx.

If you want to forgo thread-safety, Rainbows! + StreamResponseEpoll[1]
+ ForceStreaming middleware[2] may also be an option, too (needs nginx).


Keep in mind that I don't know of anybody using Rainbows! for any
serious sites, so there could still be major bugs :)

Rainbows! http://rainbows.rubyforge.org/


[1] currently in rainbows.git, will probably be released this weekend...

[2] I'll probably move this to Rainbows! instead of a Unicorn branch:
    http://bogomips.org/unicorn.git/tree/lib/unicorn/force_streaming.rb?h=force_streaming
    This is 100% untested, I've never run it.

-- 
Eric Wong

From fxn at hashref.com  Sat Jun 25 18:23:34 2011
From: fxn at hashref.com (Xavier Noria)
Date: Sun, 26 Jun 2011 00:23:34 +0200
Subject: Unicorn and streaming in Rails 3.1
In-Reply-To: <20110625201636.GA22343@dcvr.yhbt.net>
References: <BANLkTika2jE-ZFk1Q0VgWxAk44LHg16LJg@mail.gmail.com>
	<20110625201636.GA22343@dcvr.yhbt.net>
Message-ID: <BANLkTi=DO+v5MogxyhezgjyqVmx7r1A8HQ@mail.gmail.com>

On Sat, Jun 25, 2011 at 10:16 PM, Eric Wong <normalperson at yhbt.net> wrote:

> Basically the per-connection overhead of Unicorn is huge, an entire Ruby
> process (tens to several hundreds of megabytes). ?The per-connection
> overhead of nginx is tiny: maybe a few KB in userspace (including
> buffers), and a few KB in in the kernel. ?You don't want to maintain
> connections to Unicorn for a long time because of that cost.

I see. I've read also the docs about design and philosophy in the website.

So if I understand it correctly, as far as memory consumption is
concerned the situation seems to be similar to the old days when
mongrel cluster was the standard for production, except perhaps for
setups with copy-on-write friendly interpreters, which weren't
available then.

So you configure only a few processes because of memory consumption,
and since there aren't many you want them to be ready to serve a new
request as soon as possible to handle some normal level of
concurrency. Hence the convenience of buffering in Nginx.

>> in the use case we have in mind in
>> Rails 3.1, which is to serve HEAD as soon as possible.
>
> Small nit: s/HEAD/the response header/ ? "HEAD" is a /request/ that only
> expects to receive the response header.

Oh yes, that was ambiguous. I actually meant the HEAD element of HTML
documents. The main use case in mind for adding streaming to Rails is
to be able to send the top of your layout (typically everything before
yielding to the view) so that the browser may issue requests for CSS
and JavaScript assets while the application builds an hypothetically
costly dynamic response.

> nginx only sends HTTP/1.0 requests to unicorn, so Rack::Chunked won't
> actually send a chunked/streamed response. ?Rails 3.1 /could/ enable
> streaming without chunking for HTTP/1.0, but only if the client
> didn't set a non-standard HTTP/1.0 header to enable keepalive. ?This
> is because HTTP/1.0 (w/o keepalive) relies on the server to close
> the connection to signal the end of a response.

It's clear then. Also, Rails has code that prevents streaming from
being triggered if the request is HTTP/1.0:

https://github.com/rails/rails/blob/master/actionpack/lib/action_controller/metal/streaming.rb#L243-244

> You can use "X-Accel-Buffering: no" if you know your responses are small
> enough to fit into the kernel socket buffers. ?There's two kernel
> buffers (Unicorn + nginx), you can get a little more space there. ?nginx
> shouldn't make another request to Unicorn if it's blocked writing a
> response to the client already, so an evil pipelining client should not
> hurt unicorn in this case:

Excellent.

Thanks Eric!

From asimic at gmail.com  Sun Jun 26 01:36:48 2011
From: asimic at gmail.com (Aleksandar Simic)
Date: Sun, 26 Jun 2011 06:36:48 +0100
Subject: [PATCH] replace fchmod()-based heartbeat with raindrops
In-Reply-To: <20110625001534.GA18193@dcvr.yhbt.net>
References: <20110616232508.GA16538@dcvr.yhbt.net>
	<20110624173038.GA22131@dcvr.yhbt.net>
	<BANLkTi=a=jHsSVC0Qfx57xG-gXQk9iiB1w@mail.gmail.com>
	<20110624205158.GA5583@dcvr.yhbt.net>
	<BANLkTinV+8uE80CpubvzgJp_6-+RETSvRg@mail.gmail.com>
	<20110625001534.GA18193@dcvr.yhbt.net>
Message-ID: <BANLkTimLKZUNH9uknon-jgjXp+s02QGQxQ@mail.gmail.com>

On Sat, Jun 25, 2011 at 1:15 AM, Eric Wong <normalperson at yhbt.net> wrote:
> Aleksandar Simic <asimic at gmail.com> wrote:
>> Hello,
>>
>> the same OS & ruby version as the original poster who reported the
>> issue. I see the same symptoms.
>>
>> Cowboy fix: adding "-D__BSD_VISIBLE" to CPPFLAGS gets raindrops gem to build.
>>
>> Easily accessible and viewable sys/mmap.h:
>>
>> http://www.gitorious.net/freebsd/freebsd/blobs/HEAD/sys/sys/mman.h
>
> Aha! ?So defining _XOPEN_SOURCE appears to cause __BSD_VISIBLE to
> not be defined in sys/cdefs.h. ?I've pushed out the below patch to
> git://bogomips.org/raindrops.git
>
>> If required, I can give you access to my FreeBSD machine, just let me know.
>
> If the below patch doesn't fix it, yes, it'd be greatly appreciated,
> thanks!
>
> >From 1e7dc89cc38c5dec0b63ac452b23141297701f88 Mon Sep 17 00:00:00 2001
> From: Eric Wong <normalperson at yhbt.net>
> Date: Fri, 24 Jun 2011 17:06:56 -0700
> Subject: [PATCH] remove _XOPEN_SOURCE #define for FreeBSD
>
> This appears to cause __BSD_VISIBLE to not be defined,
> which is required for MAP_ANON to be visible in
> sys/mman.h
>
> Thanks for Aleksandar Simic for the hint and Troex Nevelin
> for the bug report!
> ---
> ?ext/raindrops/extconf.rb | ? ?2 +-
> ?1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/ext/raindrops/extconf.rb b/ext/raindrops/extconf.rb
> index 825625d..9f5de95 100644
> --- a/ext/raindrops/extconf.rb
> +++ b/ext/raindrops/extconf.rb
> @@ -7,7 +7,7 @@ have_func('munmap', 'sys/mman.h') or abort 'munmap() not found'
> ?$CPPFLAGS += " -D_GNU_SOURCE "
> ?have_func('mremap', 'sys/mman.h')
>
> -$CPPFLAGS += " -D_BSD_SOURCE -D_XOPEN_SOURCE=600 "
> +$CPPFLAGS += " -D_BSD_SOURCE "
> ?have_func("getpagesize", "unistd.h")
> ?have_func('rb_thread_blocking_region')
> ?have_func('rb_thread_io_blocking_region')

Hello,

just to report back that this does fix the issue.

Thanks,
Aleksandar

From normalperson at yhbt.net  Mon Jun 27 03:32:13 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 27 Jun 2011 07:32:13 +0000
Subject: [PATCH] replace fchmod()-based heartbeat with raindrops
In-Reply-To: <BANLkTimLKZUNH9uknon-jgjXp+s02QGQxQ@mail.gmail.com>
References: <20110616232508.GA16538@dcvr.yhbt.net>
	<20110624173038.GA22131@dcvr.yhbt.net>
	<BANLkTi=a=jHsSVC0Qfx57xG-gXQk9iiB1w@mail.gmail.com>
	<20110624205158.GA5583@dcvr.yhbt.net>
	<BANLkTinV+8uE80CpubvzgJp_6-+RETSvRg@mail.gmail.com>
	<20110625001534.GA18193@dcvr.yhbt.net>
	<BANLkTimLKZUNH9uknon-jgjXp+s02QGQxQ@mail.gmail.com>
Message-ID: <20110627073212.GC15497@dcvr.yhbt.net>

Aleksandar Simic <asimic at gmail.com> wrote:
> just to report back that this does fix the issue.

Thanks for the confirmation!  Just released raindrops 0.7.0:

http://mid.gmane.org/20110627073050.GB15497 at dcvr.yhbt.net

-- 
Eric Wong

From normalperson at yhbt.net  Mon Jun 27 05:12:36 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 27 Jun 2011 09:12:36 +0000
Subject: [ANN] unicorn 4.0.0 - for mythical hardware!
Message-ID: <20110627091236.GA25451@dcvr.yhbt.net>

Changes:

A single Unicorn instance may manage more than 1024 workers
without needing privileges to modify resource limits.  As a
result of this, the "raindrops"[1] gem/library is now a required
dependency.

TCP socket defaults now favor low latency to mimic UNIX domain
socket behavior (tcp_nodelay: true, tcp_nopush: false).  This
hurts throughput, users who want to favor throughput should
specify "tcp_nodelay: false, tcp_nopush: true" in the listen
directive.

Error logging is more consistent and all lines should be
formatted correctly in backtraces.  This may break the
behavior of some log parsers.

The call stack is smaller and thus easier to examine backtraces
when debugging Rack applications.

There are some internal API changes and cleanups, but none that
affect applications designed for Rack.  See "git log v3.7.0.."
for details.

For users who cannot install kgio[2] or raindrops, Unicorn 1.1.x
remains supported indefinitely.  Unicorn 3.x will remain
supported if there is demand.  We expect raindrops to introduce
fewer portability problems than kgio did, however.

[1] http://raindrops.bogomips.org/
[2] http://bogomips.org/kgio/

* http://unicorn.bogomips.org/
* mongrel-unicorn at rubyforge.org
* git://bogomips.org/unicorn.git

-- 
Eric Wong

From normalperson at yhbt.net  Wed Jun 29 03:29:36 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 29 Jun 2011 07:29:36 +0000
Subject: [ANN] unicorn 4.0.0.2.g19f7 prerelease
Message-ID: <20110629072936.GA10100@dcvr.yhbt.net>

I just pushed a prerelease gem to RubyGems.org with the following
changes.  Please let us know if there were any other regressions in
4.0.0.

Expect a final 4.0.1 release soonish...

Full git changelog (pushed to git://bogomips.org/unicorn.git):

commit 19f798301ac1884f423640efafb277b071bb5439
Author: Eric Wong <normalperson at yhbt.net>
Date:   Wed Jun 29 07:19:32 2011 +0000

    fix per-worker listen directive in after_fork hook
    
    The testcase for this was broken, too, so we didn't notice
    this :<
    
    Reported-by: ghazel at gmail.com on the Rainbows! mailing list,
    http://mid.gmane.org/BANLkTi=oQXK5Casq9SuGD3edeUrDPvRm3A at mail.gmail.com

commit 38672501206c9e64d241e3d8571f70b198f0c1e5
Author: Eric Wong <normalperson at yhbt.net>
Date:   Mon Jun 27 20:51:16 2011 +0000

    configurator: truncate timeouts to 32-bit LONG_MAX
    
    IO.select in Ruby can't wait longer than this.  This
    means Unicorn can't support applications that take
    longer than 68 years to respond :(

-- 
Eric Wong

From normalperson at yhbt.net  Wed Jun 29 15:10:32 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 29 Jun 2011 19:10:32 +0000
Subject: [ANN] unicorn 4.0.1 - regression bugfixes
Message-ID: <20110629191032.GA3828@dcvr.yhbt.net>

Changes:

This release fixes things for users of per-worker "listen"
directives in the after_fork hook.  Thanks to ghazel at gmail.com
for reporting the bug.

The "timeout" configurator directive is now truncated to
0x7ffffffe seconds to prevent overflow when calling
IO.select.

* http://unicorn.bogomips.org/
* mongrel-unicorn at rubyforge.org
* git://bogomips.org/unicorn.git

-- 
Eric Wong

From normalperson at yhbt.net  Wed Jun 29 23:27:06 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 30 Jun 2011 03:27:06 +0000
Subject: Unicorn 1.x maintenance schedule?
Message-ID: <20110630032706.GA24937@dcvr.yhbt.net>

Just wondering, how much demand is there for continued 1.x maintenance?

I think 1.x (and 3.x) are both fairly bug-free by now and unlikely to
need much, but I'm willing to commit to long term maintenance if people
want it.  Maybe we can drop it when the Ruby core team stops supporting
Ruby 1.8?[1]

AFAIK, Unicorn 3.x with its kgio dependency currently works on all
*nix-like platforms with fork() support, even ugly non-Free OSes
that aren't _officially_ supported.

Unicorn 4.x adds the dependency on raindrops, but I think that would
cause fewer portability problems than kgio introduced with Unicorn 3.x.

Time will tell...



[1] For the record Unicorn 3.x/4.x works great with Ruby 1.8 and
    Unicorn 1.x works well with Ruby 1.9, too.   However, Ruby 1.9.2
    introduced a performance bug[2] that got me experimenting with
    kgio.  While this bug is fixed Ruby 1.9.3, but kgio is still
    faster.

[2] http://redmine.ruby-lang.org/issues/3905

Thanks for reading!
-- 
Eric Wong

From tscolari at gmail.com  Wed Jun 29 23:33:00 2011
From: tscolari at gmail.com (Tiago Scolari)
Date: Thu, 30 Jun 2011 00:33:00 -0300
Subject: "NoMethodError: undefined method" after switching to unicorn
In-Reply-To: <9E260BC166D04FF1B0B7CBC563B221BA@gmail.com>
References: <9E260BC166D04FF1B0B7CBC563B221BA@gmail.com>
Message-ID: <4AABABCB7BED4EAC9D72BFFCEF981CEC@gmail.com>

Hello,
I'm having a lot of "NoMethodError: undefined method" errors after I moved from thin to unicorn.

This seems to happen only with active records, and dynamic methods, like:
find_by_*, scoped_by_*, ...

It seems to be random, at least I haven't found relation yet.
I may reload the same page 10 times, sometimes it will work ok, some times it will give me this error.

I've 2 sites having this issue, one uses rails 3.0.9 (r 1.9.2) and other with 2.3.8 (r 1.8.7).
As i said, the issue started after I switched to unicorn.

Any idea what could cause this?



Here are some backtraces from hoptoad:
-- Rails 2.3.8 app:


[GEM_ROOT]/gems/activerecord-2.3.8/lib/active_record/base.rb:1872:in `method_missing_without_paginate'


vendor/plugins/will_paginate/lib/will_paginate/finder.rb:183:in `method_missing'


app/controllers/games_controller.rb:92:in `load_platform'


[GEM_ROOT]/gems/activesupport-2.3.8/lib/active_support/callbacks.rb:178:in `send'


[GEM_ROOT]/gems/activesupport-2.3.8/lib/active_support/callbacks.rb:178:in `evaluate_method'


[GEM_ROOT]/gems/activesupport-2.3.8/lib/active_support/callbacks.rb:166:in `call'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/filters.rb:225:in `call'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/filters.rb:629:in `run_before_filters'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/filters.rb:615:in `call_filters'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/filters.rb:610:in `perform_action_without_benchmark'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'


[GEM_ROOT]/gems/activesupport-2.3.8/lib/active_support/core_ext/benchmark.rb:17:in `ms'


/usr/local/lib/ruby/1.8/benchmark.rb:308:in `realtime'


[GEM_ROOT]/gems/activesupport-2.3.8/lib/active_support/core_ext/benchmark.rb:17:in `ms'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/rescue.rb:160:in `perform_action_without_flash'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/flash.rb:151:in `perform_action_without_newrelic_trace'


vendor/plugins/rpm/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:254:in `perform_action'


vendor/plugins/rpm/lib/new_relic/agent/method_tracer.rb:141:in `trace_execution_scoped'


vendor/plugins/rpm/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:247:in `perform_action'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/base.rb:532:in `send'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/base.rb:532:in `process_without_filters'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/filters.rb:606:in `process'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/base.rb:391:in `process'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/base.rb:386:in `call'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/routing/route_set.rb:438:in `call'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/dispatcher.rb:87:in `dispatch'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/dispatcher.rb:121:in `_call'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/dispatcher.rb:130


[GEM_ROOT]/gems/activerecord-2.3.8/lib/active_record/query_cache.rb:29:in `call'


[GEM_ROOT]/gems/activerecord-2.3.8/lib/active_record/query_cache.rb:29:in `call'


[GEM_ROOT]/gems/activerecord-2.3.8/lib/active_record/connection_adapters/abstract/query_cache.rb:34:in `cache'


[GEM_ROOT]/gems/activerecord-2.3.8/lib/active_record/query_cache.rb:9:in `cache'


[GEM_ROOT]/gems/activerecord-2.3.8/lib/active_record/query_cache.rb:28:in `call'


[GEM_ROOT]/gems/activerecord-2.3.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:361:in `call'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:50:in `call_app!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:32:in `call!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:19:in `call'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:50:in `call_app!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:32:in `call!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:19:in `call'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:50:in `call_app!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:32:in `call!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:19:in `call'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:50:in `call_app!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:32:in `call!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:19:in `call'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:50:in `call_app!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:32:in `call!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:19:in `call'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:50:in `call_app!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:32:in `call!'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/strategy.rb:19:in `call'


[GEM_ROOT]/gems/oa-core-0.1.6/lib/omniauth/builder.rb:30:in `call'


lib/flash_session_cookie_middleware.rb:13:in `call'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/string_coercion.rb:25:in `call'


[GEM_ROOT]/gems/rack-1.1.0/lib/rack/head.rb:9:in `call'


[GEM_ROOT]/gems/rack-1.1.0/lib/rack/methodoverride.rb:24:in `call'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/params_parser.rb:15:in `call'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/session/abstract_store.rb:128:in `call'


[GEM_ROOT]/gems/activesupport-2.3.8/lib/active_support/cache/strategy/local_cache.rb:25:in `call'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/failsafe.rb:26:in `call'


[GEM_ROOT]/gems/actionpack-2.3.8/lib/action_controller/dispatcher.rb:106:in `call'


[GEM_ROOT]/gems/rails-2.3.8/lib/rails/rack/static.rb:31:in `call'


[GEM_ROOT]/gems/rack-1.1.0/lib/rack/urlmap.rb:47:in `call'


[GEM_ROOT]/gems/rack-1.1.0/lib/rack/urlmap.rb:41:in `each'


[GEM_ROOT]/gems/rack-1.1.0/lib/rack/urlmap.rb:41:in `call'


[GEM_ROOT]/gems/unicorn-4.0.0/lib/unicorn/http_server.rb:518:in `process_client'


[GEM_ROOT]/gems/unicorn-4.0.0/lib/unicorn/http_server.rb:584:in `worker_loop'


vendor/plugins/rpm/lib/new_relic/agent/instrumentation/unicorn_instrumentation.rb:7:in `call'


vendor/plugins/rpm/lib/new_relic/agent/instrumentation/unicorn_instrumentation.rb:7:in `worker_loop'


[GEM_ROOT]/gems/unicorn-4.0.0/lib/unicorn/http_server.rb:475:in `spawn_missing_workers'


[GEM_ROOT]/gems/unicorn-4.0.0/lib/unicorn/http_server.rb:135:in `start'


[GEM_ROOT]/gems/unicorn-4.0.0/bin/unicorn_rails:209


/usr/local/bin/unicorn_rails:19:in `load'


/usr/local/bin/unicorn_rails:19













-- Rails 3.0.9 app:






[GEM_ROOT]/gems/activerecord-3.0.9/lib/active_record/base.rb:989:in `method_missing'


app/helpers/application_helper.rb:154:in `render_tv'


app/views/home/index.html.haml:8:in `_app_views_home_index_html_haml___1954184739576368685_58305940__2978367022487903250'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_view/template.rb:135:in `block in render'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/notifications.rb:54:in `instrument'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_view/template.rb:127:in `render'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_view/render/rendering.rb:59:in `block in _render_template'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/notifications.rb:52:in `block in instrument'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/notifications/instrumenter.rb:21:in `instrument'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/notifications.rb:52:in `instrument'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_view/render/rendering.rb:56:in `_render_template'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_view/render/rendering.rb:26:in `render'


[GEM_ROOT]/gems/haml-3.1.2/lib/haml/helpers/action_view_mods.rb:13:in `render_with_haml'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/abstract_controller/rendering.rb:115:in `_render_template'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/abstract_controller/rendering.rb:109:in `render_to_body'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/renderers.rb:47:in `render_to_body'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/compatibility.rb:55:in `render_to_body'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/abstract_controller/rendering.rb:102:in `render_to_string'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/abstract_controller/rendering.rb:93:in `render'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/rendering.rb:17:in `render'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/instrumentation.rb:40:in `block (2 levels) in render'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/core_ext/benchmark.rb:5:in `block in ms'


/home/todaela/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/benchmark.rb:309:in `realtime'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/core_ext/benchmark.rb:5:in `ms'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/instrumentation.rb:40:in `block in render'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/instrumentation.rb:78:in `cleanup_view_runtime'


[GEM_ROOT]/gems/activerecord-3.0.9/lib/active_record/railties/controller_runtime.rb:15:in `cleanup_view_runtime'


[GEM_ROOT]/gems/thinking-sphinx-2.0.4/lib/thinking_sphinx/action_controller.rb:12:in `cleanup_view_runtime'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/instrumentation.rb:39:in `render'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/implicit_render.rb:10:in `default_render'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/implicit_render.rb:5:in `send_action'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/abstract_controller/base.rb:150:in `process_action'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/rendering.rb:11:in `process_action'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/abstract_controller/callbacks.rb:18:in `block in process_action'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/callbacks.rb:451:in `_run__1893485971492116310__process_action__3270121157045284473__callbacks'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/callbacks.rb:410:in `_run_process_action_callbacks'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/callbacks.rb:94:in `run_callbacks'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/abstract_controller/callbacks.rb:17:in `process_action'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/notifications.rb:52:in `block in instrument'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/notifications/instrumenter.rb:21:in `instrument'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/notifications.rb:52:in `instrument'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/instrumentation.rb:29:in `process_action'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/rescue.rb:17:in `process_action'


[GEM_ROOT]/gems/newrelic_rpm-3.1.0/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:34:in `block in process_action'


[GEM_ROOT]/gems/newrelic_rpm-3.1.0/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:253:in `block in perform_action_with_newrelic_trace'


[GEM_ROOT]/gems/newrelic_rpm-3.1.0/lib/new_relic/agent/method_tracer.rb:191:in `trace_execution_scoped'


[GEM_ROOT]/gems/newrelic_rpm-3.1.0/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:248:in `perform_action_with_newrelic_trace'


[GEM_ROOT]/gems/newrelic_rpm-3.1.0/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:33:in `process_action'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/abstract_controller/base.rb:119:in `process'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/abstract_controller/rendering.rb:41:in `process'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal.rb:138:in `dispatch'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_controller/metal.rb:178:in `block in action'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/routing/route_set.rb:62:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/routing/route_set.rb:62:in `dispatch'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/routing/route_set.rb:27:in `call'


[GEM_ROOT]/gems/rack-mount-0.6.14/lib/rack/mount/route_set.rb:148:in `block in call'


[GEM_ROOT]/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:93:in `block in recognize'


[GEM_ROOT]/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:68:in `optimized_each'


[GEM_ROOT]/gems/rack-mount-0.6.14/lib/rack/mount/code_generation.rb:92:in `recognize'


[GEM_ROOT]/gems/rack-mount-0.6.14/lib/rack/mount/route_set.rb:139:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/routing/route_set.rb:493:in `call'


[GEM_ROOT]/gems/oa-core-0.2.6/lib/omniauth/strategy.rb:44:in `call!'


[GEM_ROOT]/gems/oa-core-0.2.6/lib/omniauth/strategy.rb:30:in `call'


[GEM_ROOT]/gems/oa-core-0.2.6/lib/omniauth/strategy.rb:44:in `call!'


[GEM_ROOT]/gems/oa-core-0.2.6/lib/omniauth/strategy.rb:30:in `call'


[GEM_ROOT]/gems/oa-core-0.2.6/lib/omniauth/strategy.rb:44:in `call!'


[GEM_ROOT]/gems/oa-core-0.2.6/lib/omniauth/strategy.rb:30:in `call'


[GEM_ROOT]/gems/barista-1.2.1/lib/barista/server.rb:33:in `call'


[GEM_ROOT]/gems/newrelic_rpm-3.1.0/lib/new_relic/rack/browser_monitoring.rb:18:in `call'


[GEM_ROOT]/gems/sass-3.1.2/lib/sass/plugin/rack.rb:54:in `call'


[GEM_ROOT]/gems/warden-1.0.4/lib/warden/manager.rb:35:in `block in call'


[GEM_ROOT]/gems/warden-1.0.4/lib/warden/manager.rb:34:in `catch'


[GEM_ROOT]/gems/warden-1.0.4/lib/warden/manager.rb:34:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/middleware/head.rb:14:in `call'


[GEM_ROOT]/gems/rack-1.2.3/lib/rack/methodoverride.rb:24:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/middleware/params_parser.rb:21:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/middleware/flash.rb:182:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/middleware/session/abstract_store.rb:149:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/middleware/cookies.rb:302:in `call'


[GEM_ROOT]/gems/ckeditor-3.5.4/lib/ckeditor/middleware.rb:15:in `call'


[GEM_ROOT]/gems/activerecord-3.0.9/lib/active_record/query_cache.rb:32:in `block in call'


[GEM_ROOT]/gems/activerecord-3.0.9/lib/active_record/connection_adapters/abstract/query_cache.rb:28:in `cache'


[GEM_ROOT]/gems/activerecord-3.0.9/lib/active_record/query_cache.rb:12:in `cache'


[GEM_ROOT]/gems/activerecord-3.0.9/lib/active_record/query_cache.rb:31:in `call'


[GEM_ROOT]/gems/activerecord-3.0.9/lib/active_record/connection_adapters/abstract/connection_pool.rb:354:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/middleware/callbacks.rb:46:in `block in call'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/callbacks.rb:416:in `_run_call_callbacks'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/middleware/callbacks.rb:44:in `call'


[GEM_ROOT]/gems/rack-1.2.3/lib/rack/sendfile.rb:107:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/middleware/remote_ip.rb:48:in `call'


[GEM_ROOT]/gems/actionpack-3.0.9/lib/action_dispatch/middleware/show_exceptions.rb:47:in `call'


[GEM_ROOT]/gems/railties-3.0.9/lib/rails/rack/logger.rb:13:in `call'


[GEM_ROOT]/gems/rack-1.2.3/lib/rack/runtime.rb:17:in `call'


[GEM_ROOT]/gems/activesupport-3.0.9/lib/active_support/cache/strategy/local_cache.rb:72:in `call'


[GEM_ROOT]/gems/rack-1.2.3/lib/rack/lock.rb:11:in `block in call'


:


[GEM_ROOT]/gems/rack-1.2.3/lib/rack/lock.rb:11:in `call'







Thank you

-- 
Tiago Scolari


From normalperson at yhbt.net  Thu Jun 30 01:04:20 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 29 Jun 2011 22:04:20 -0700
Subject: "NoMethodError: undefined method" after switching to unicorn
In-Reply-To: <4AABABCB7BED4EAC9D72BFFCEF981CEC@gmail.com>
References: <9E260BC166D04FF1B0B7CBC563B221BA@gmail.com>
	<4AABABCB7BED4EAC9D72BFFCEF981CEC@gmail.com>
Message-ID: <20110630050420.GA19876@dcvr.yhbt.net>

Tiago Scolari <tscolari at gmail.com> wrote:
> Hello, I'm having a lot of "NoMethodError: undefined method" errors
> after I moved from thin to unicorn.
> 
> This seems to happen only with active records, and dynamic methods,
> like: find_by_*, scoped_by_*, ...
> 
> It seems to be random, at least I haven't found relation yet.  I may
> reload the same page 10 times, sometimes it will work ok, some times
> it will give me this error.
> 
> I've 2 sites having this issue, one uses rails 3.0.9 (r 1.9.2) and
> other with 2.3.8 (r 1.8.7).  As i said, the issue started after I
> switched to unicorn.
> 
> Any idea what could cause this?

Are you using preload_app?

> vendor/plugins/rpm/lib/new_relic/agent/instrumentation/unicorn_instrumentation.rb:7:in
> `call'

Hrm, I wonder if the above file depends on (undocumented) Unicorn internals
that changed in unicorn 4.0.0?

> [GEM_ROOT]/gems/unicorn-4.0.0/lib/unicorn/http_server.rb:475:in
> `spawn_missing_workers'

It might be worth it to try unicorn 3.7.0 since 4.0.0 was just released
and some gems that depend on internal implementation details could
be broken.


Maybe somebody else here has a better explanation...

-- 
Eric Wong

From normalperson at yhbt.net  Thu Jun 30 15:45:15 2011
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 30 Jun 2011 12:45:15 -0700
Subject: "NoMethodError: undefined method" after switching to unicorn
In-Reply-To: <20110630050420.GA19876@dcvr.yhbt.net>
References: <9E260BC166D04FF1B0B7CBC563B221BA@gmail.com>
	<4AABABCB7BED4EAC9D72BFFCEF981CEC@gmail.com>
	<20110630050420.GA19876@dcvr.yhbt.net>
Message-ID: <20110630194515.GA29105@dcvr.yhbt.net>

Eric Wong <normalperson at yhbt.net> wrote:
> Tiago Scolari <tscolari at gmail.com> wrote:
> > Any idea what could cause this?
> 
> Are you using preload_app?

(private response said "yes")

> > vendor/plugins/rpm/lib/new_relic/agent/instrumentation/unicorn_instrumentation.rb:7:in
> > `call'
> 
> Hrm, I wonder if the above file depends on (undocumented) Unicorn internals
> that changed in unicorn 4.0.0?
> 
> > [GEM_ROOT]/gems/unicorn-4.0.0/lib/unicorn/http_server.rb:475:in
> > `spawn_missing_workers'
> 
> It might be worth it to try unicorn 3.7.0 since 4.0.0 was just released
> and some gems that depend on internal implementation details could
> be broken.

Reverting to 3.7.0 appears to have fixed the issue for the user.

-- 
Eric Wong