From ibc at aliax.net  Fri Jan  1 13:42:27 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Fri, 1 Jan 2010 19:42:27 +0100
Subject: PID is not regenerated when using "-P" and the master receives
	USR2
In-Reply-To: <20091231214104.GA19467@dcvr.yhbt.net>
References: <200912311901.14620.ibc@aliax.net>
	<200912312226.39253.ibc@aliax.net>
	<20091231214104.GA19467@dcvr.yhbt.net>
Message-ID: <201001011942.27694.ibc@aliax.net>

El Jueves, 31 de Diciembre de 2009, Eric Wong escribi?:
> You need to require 'unicorn/launcher' or 'unicorn' before you do any
> option parsing to stash ARGV in a safe place.

Thanks, that was the point.
In my script I load unicorn/launcher after args parsing so ARGV gets empty.

Thanks. 


-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Sat Jan  2 11:42:29 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Sat, 2 Jan 2010 17:42:29 +0100
Subject: Raised exceptions are not logged when USR2
Message-ID: <201001021742.29874.ibc@aliax.net>

Hi, I use the ready_pipe branch. I set "logger" to log into Syslog, but the 
issue I'll explain here is the same even if I log to a file.

I runned Unicorn daemonized. Now imagine I do a typo in my Rack config.ru (I 
use preload=true). When sending USR2 to the master it fails (ok) but the 
raised exception is not shown in the logger, instead it's lost as $stderr is 
/dev/null.

Even worse, imagine Unicorn is not running and I start it daemonized (with a 
init script). Due to the typo in config.ru I see:

  "master failed to start, check stderr log for details"

But the message is useless since stderr is /dev/null so I se nothing in 
Syslog.


So I suggest the following:

If the process fails after setting the logger, then rescue any exception, log 
it to the logger and raise it as normal:

  rescue => err
    logger.fatal "#{err.class}: #{err.message}\n#{err.backtrace.join("\n")}"
    raise err
  end

I'm trying to figure where exactly to include such code but it seems a bit 
complex. Any tip please?

Thanks a lot.


-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Sat Jan  2 16:27:31 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Sat, 2 Jan 2010 22:27:31 +0100
Subject: Raised exceptions are not logged when USR2
In-Reply-To: <201001021742.29874.ibc@aliax.net>
References: <201001021742.29874.ibc@aliax.net>
Message-ID: <201001022227.31999.ibc@aliax.net>

El S?bado, 2 de Enero de 2010, I?aki Baz Castillo escribi?:
> Hi, I use the ready_pipe branch. I set "logger" to log into Syslog, but the
> issue I'll explain here is the same even if I log to a file.
> 
> I runned Unicorn daemonized. Now imagine I do a typo in my Rack config.ru
>  (I use preload=true). When sending USR2 to the master it fails (ok) but
>  the raised exception is not shown in the logger, instead it's lost as
>  $stderr is /dev/null.
> 
> Even worse, imagine Unicorn is not running and I start it daemonized (with
>  a init script). Due to the typo in config.ru I see:
> 
>   "master failed to start, check stderr log for details"
> 
> But the message is useless since stderr is /dev/null so I se nothing in
> Syslog.
> 
> 
> So I suggest the following:
> 
> If the process fails after setting the logger, then rescue any exception,
>  log it to the logger and raise it as normal:
> 
>   rescue => err
>     logger.fatal "#{err.class}:
>  #{err.message}\n#{err.backtrace.join("\n")}" raise err
>   end
> 
> I'm trying to figure where exactly to include such code but it seems a bit
> complex. Any tip please?
> 
> Thanks a lot.

A simpler approach would be removing this line at the end of launcher.rb:

  Unicorn::Configurator::DEFAULTS[:stderr_path] = "/dev/null" 

With this, errors would be displayed in stderr. Of course for workers stderr 
should be reopened to point to /dev/null, but IMHO not for master process.

Perhaps errors preventing the master process to start should be displayed to 
stderr instead of to the logger (as it could be not set yet).

Opinnions?


-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Sat Jan  2 16:33:04 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Sat, 2 Jan 2010 22:33:04 +0100
Subject: Raised exceptions are not logged when USR2
In-Reply-To: <201001022227.31999.ibc@aliax.net>
References: <201001021742.29874.ibc@aliax.net>
	<201001022227.31999.ibc@aliax.net>
Message-ID: <201001022233.04664.ibc@aliax.net>

El S?bado, 2 de Enero de 2010, I?aki Baz Castillo escribi?:
> A simpler approach would be removing this line at the end of launcher.rb:
> 
>   Unicorn::Configurator::DEFAULTS[:stderr_path] = "/dev/null"
> 
> With this, errors would be displayed in stderr. Of course for workers
>  stderr should be reopened to point to /dev/null, but IMHO not for master
>  process.
> 
> Perhaps errors preventing the master process to start should be displayed
>  to stderr instead of to the logger (as it could be not set yet).

And the other already available workaround is setting the "stderr_path".
Unfortunatelly $stderr.reopen requires a file as parameter so is not possible 
to use Syslog or any other logger, is it?

BTW, how to reset the $stderr so it points again to the default one (the 
terminal)?

-- 
I?aki Baz Castillo <ibc at aliax.net>

From skaar at waste.org  Sat Jan  2 17:59:03 2010
From: skaar at waste.org (skaar)
Date: Sat, 2 Jan 2010 16:59:03 -0600
Subject: unicorn/reexec/bundler/cap deploy issue
Message-ID: <20100102225903.GC18964@waste.org>

Hi Eric,

we've run into an issue with Unicorn's reexec, when the rails app is
using Bundler and we deploy into dated release directories. Still not
sure what the correct fix is for this, but what happens is that bundler
modifies (acutally prefixes) PATH/RUBYOPT and when you reexec unicorn
these are passed along to the new master process - and Bundler will
again append to the PATH/RUBYOBT.

This becomes more of a problem when you deploy with capistrano and
include SIGUSR2 restart of your unicorn. And effectively it will attempt
to load all Bundler vendor/bundler_gems/environment.rb for all deploys
since unicorn was first started - and if you have, say, a keep only 5
releases strategy, you will soon get failures when a bundler
environment.rb file in RUBYOPT has been shifted out.

I'm still not sure what the right solution is (to modify unicorn, the
unicorn.rb config file or to address it in bundler), but the following
patch does it brute force in unicorn and introduce the assumption that
a reexec should take the same PATH/RUBYOPT variables as the initial
invokation of the unicorn process (using a similar strategy as with
PWD):


>From 3d6ca163723148668d69115ebaf00cb814db8f49 Mon Sep 17 00:00:00 2001
From: skaar <skaar at waste.org>
Date: Sat, 2 Jan 2010 12:29:38 -0500
Subject: [PATCH] filter PATH/RUBYOPT when rexec, and give new master same values as the
 original master process

---
 lib/unicorn.rb |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 225e00a..6b3e334 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -111,6 +111,8 @@ module Unicorn
             Dir.pwd
           end
         }.call,
+      :path => ENV['PATH'],
+      :rybyopt => ENV['RUBYOPT'],
       0 => $0.dup,
     }
 
@@ -488,6 +490,8 @@ module Unicorn
       self.reexec_pid = fork do
         listener_fds = LISTENERS.map { |sock| sock.fileno }
         ENV['UNICORN_FD'] = listener_fds.join(',')
+        ENV['PATH'] = START_CTX[:path]
+        ENV['RUBYOPT'] = START_CTX[:rubyopt]
         Dir.chdir(START_CTX[:cwd])
         cmd = [ START_CTX[0] ].concat(START_CTX[:argv])
 
-- 
1.6.6.rc3


-- 
/skaar

skaar at waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n

From skaar at waste.org  Sat Jan  2 13:29:02 2010
From: skaar at waste.org (skaar)
Date: Sat, 2 Jan 2010 12:29:02 -0600
Subject: unicorn/reexec/bundler/cap deploy issue
Message-ID: <20100102182902.GA18964@waste.org>

Hi Eric,

we've run into an issue with Unicorn's reexec, when the rails app is
using Bundler and we deploy into dated release directories. Still not
sure what the correct fix is for this, but what happens is that bundler
modifies (acutally prefixes) PATH/RUBYOPT and when you reexec unicorn
these are passed along to the new master process - and Bundler will
again append to the PATH/RUBYOBT.

This becomes more of a problem when you deploy with capistrano and
include SIGUSR2 restart of your unicorn. And effectively it will attempt
to load all Bundler vendor/bundler_gems/environment.rb for all deploys
since unicorn was first started - and if you have, say, a keep only 5
releases strategy, you will soon get failures when a bundler
environment.rb file in RUBYOPT has been shifted out.

I'm still not sure what the right solution is (to modify unicorn, the
unicorn.rb config file or to address it in bundler), but the following
patch does it brute force in unicorn and introduce the assumption that
a reexec should take the same PATH/RUBYOPT variables as the initial
invokation of the unicorn process (using a similar strategy as with
PWD):


>From 3d6ca163723148668d69115ebaf00cb814db8f49 Mon Sep 17 00:00:00 2001
From: skaar <skaar at waste.org>
Date: Sat, 2 Jan 2010 12:29:38 -0500
Subject: [PATCH] filter PATH/RUBYOPT when rexec, and give new master same values as the
 original master process

---
 lib/unicorn.rb |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 225e00a..6b3e334 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -111,6 +111,8 @@ module Unicorn
             Dir.pwd
           end
         }.call,
+      :path => ENV['PATH'],
+      :rybyopt => ENV['RUBYOPT'],
       0 => $0.dup,
     }
 
@@ -488,6 +490,8 @@ module Unicorn
       self.reexec_pid = fork do
         listener_fds = LISTENERS.map { |sock| sock.fileno }
         ENV['UNICORN_FD'] = listener_fds.join(',')
+        ENV['PATH'] = START_CTX[:path]
+        ENV['RUBYOPT'] = START_CTX[:rubyopt]
         Dir.chdir(START_CTX[:cwd])
         cmd = [ START_CTX[0] ].concat(START_CTX[:argv])
 
-- 
1.6.6.rc3



-- 
/skaar

skaar at waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n

From normalperson at yhbt.net  Sat Jan  2 19:04:01 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Sat, 2 Jan 2010 16:04:01 -0800
Subject: Raised exceptions are not logged when USR2
In-Reply-To: <201001022233.04664.ibc@aliax.net>
References: <201001021742.29874.ibc@aliax.net>
	<201001022227.31999.ibc@aliax.net>
	<201001022233.04664.ibc@aliax.net>
Message-ID: <20100103000401.GA13826@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El S?bado, 2 de Enero de 2010, I?aki Baz Castillo escribi?:
> > A simpler approach would be removing this line at the end of launcher.rb:
> > 
> >   Unicorn::Configurator::DEFAULTS[:stderr_path] = "/dev/null"
> > 
> > With this, errors would be displayed in stderr. Of course for workers
> >  stderr should be reopened to point to /dev/null, but IMHO not for master
> >  process.
> > 
> > Perhaps errors preventing the master process to start should be displayed
> >  to stderr instead of to the logger (as it could be not set yet).
> 
> And the other already available workaround is setting the "stderr_path".
> Unfortunatelly $stderr.reopen requires a file as parameter so is not possible 
> to use Syslog or any other logger, is it?

Hi I?aki,

Unfortunately, the stock Syslog module isn't compatible with the Ruby
Logger implementation.  Try grabbing the "SyslogLogger" gem and using
that instead.  I haven't had any experience with SyslogLogger, but a
cursory look says it's OK.

I would always keep stderr_path set somewhere in your config file since
by default "rack.errors" points to it, and it's the default place for
Kernel#warn to output to.

If you really want to, you may be able to subclass SyslogLogger and give
it IO-like methods (write/puts/flush) are require for Rack::Lint
compatibility and then set $stderr:

  # Totally untested, stick this in your Unicorn config file and let
  # us know if it works or blows up badly:

  require 'syslog_logger'

  class MySyslogLogger < SyslogLogger
    alias puts error
    alias write error
    def flush; self; end
  end

  $stderr = MySyslogLogger.new('foo')

> BTW, how to reset the $stderr so it points again to the default one (the 
> terminal)?

You would've had to dup stderr in the config file and keep it referenced
before it got redirected.  But a daemonized process shouldn't be holding
onto terminals...

-- 
Eric Wong

From normalperson at yhbt.net  Sat Jan  2 19:25:44 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Sat, 2 Jan 2010 16:25:44 -0800
Subject: unicorn/reexec/bundler/cap deploy issue
In-Reply-To: <20100102225903.GC18964@waste.org>
References: <20100102225903.GC18964@waste.org>
Message-ID: <20100103002544.GB13826@dcvr.yhbt.net>

skaar <skaar at waste.org> wrote:
> Hi Eric,
> 
> we've run into an issue with Unicorn's reexec, when the rails app is
> using Bundler and we deploy into dated release directories. Still not
> sure what the correct fix is for this, but what happens is that bundler
> modifies (acutally prefixes) PATH/RUBYOPT and when you reexec unicorn
> these are passed along to the new master process - and Bundler will
> again append to the PATH/RUBYOBT.
> 
> This becomes more of a problem when you deploy with capistrano and
> include SIGUSR2 restart of your unicorn. And effectively it will attempt
> to load all Bundler vendor/bundler_gems/environment.rb for all deploys
> since unicorn was first started - and if you have, say, a keep only 5
> releases strategy, you will soon get failures when a bundler
> environment.rb file in RUBYOPT has been shifted out.
> 
> I'm still not sure what the right solution is (to modify unicorn, the
> unicorn.rb config file or to address it in bundler),

Hi skaar,

Does having this in the Unicorn config file work for you?

  stash_env = %w(PATH RUBYOPT).map { |x| [ x, ENV[x] ] }
  before_exec do |_|
    stash_env.each { |(k,v)| ENV[k] = v }
  end

It's more generic and less surprising for people that (may) want
to change environment variables on upgrades:

> but the following
> patch does it brute force in unicorn and introduce the assumption that
> a reexec should take the same PATH/RUBYOPT variables as the initial
> invokation of the unicorn process (using a similar strategy as with
> PWD):

Early versions of Unicorn actually captured the entire ENV at startup
and restored it before exec.  I felt it was too heavy-handed since I
figured any process that changed its environment variables has good
reason to do so, so I've been trying to leave as much alone as
possible...

-- 
Eric Wong

From ibc at aliax.net  Sun Jan  3 11:22:20 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Sun, 3 Jan 2010 17:22:20 +0100
Subject: Raised exceptions are not logged when USR2
In-Reply-To: <20100103000401.GA13826@dcvr.yhbt.net>
References: <201001021742.29874.ibc@aliax.net>
	<201001022233.04664.ibc@aliax.net>
	<20100103000401.GA13826@dcvr.yhbt.net>
Message-ID: <201001031722.20468.ibc@aliax.net>

El Domingo, 3 de Enero de 2010, Eric Wong escribi?:

> Unfortunately, the stock Syslog module isn't compatible with the Ruby
> Logger implementation.  Try grabbing the "SyslogLogger" gem and using
> that instead.  I haven't had any experience with SyslogLogger, but a
> cursory look says it's OK.

Yes, in fact I already use a "bit" customized SyslogLogger :)

 
> If you really want to, you may be able to subclass SyslogLogger and give
> it IO-like methods (write/puts/flush) are require for Rack::Lint
> compatibility and then set $stderr:
> 
>   # Totally untested, stick this in your Unicorn config file and let
>   # us know if it works or blows up badly:
> 
>   require 'syslog_logger'
> 
>   class MySyslogLogger < SyslogLogger
>     alias puts error
>     alias write error
>     def flush; self; end
>   end
> 
>   $stderr = MySyslogLogger.new('foo')

Really great! It works fine :)

 
> > BTW, how to reset the $stderr so it points again to the default one (the
> > terminal)?
> 
> You would've had to dup stderr in the config file and keep it referenced
> before it got redirected.  But a daemonized process shouldn't be holding
> onto terminals...


Really thanks a lot. 


-- 
I?aki Baz Castillo <ibc at aliax.net>

From skaar at waste.org  Sun Jan  3 12:08:02 2010
From: skaar at waste.org (skaar)
Date: Sun, 3 Jan 2010 11:08:02 -0600
Subject: unicorn/reexec/bundler/cap deploy issue
In-Reply-To: <20100103002544.GB13826@dcvr.yhbt.net>
References: <20100102225903.GC18964@waste.org>
	<20100103002544.GB13826@dcvr.yhbt.net>
Message-ID: <20100103170802.GE18964@waste.org>

> Hi skaar,
> 
> Does having this in the Unicorn config file work for you?
> 
>   stash_env = %w(PATH RUBYOPT).map { |x| [ x, ENV[x] ] }
>   before_exec do |_|
>     stash_env.each { |(k,v)| ENV[k] = v }
>   end
> 
> It's more generic and less surprising for people that (may) want
> to change environment variables on upgrades:
 
yep, that does the job just fine - thanks.

> Early versions of Unicorn actually captured the entire ENV at startup
> and restored it before exec.  I felt it was too heavy-handed since I
> figured any process that changed its environment variables has good
> reason to do so, so I've been trying to leave as much alone as
> possible...

the approach of just stashing what you need from ENV as above is by far
a better approach than just forcing in the reexec code.

thanks again
-- 
/skaar

skaar at waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n

From ibc at aliax.net  Mon Jan  4 05:27:27 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Mon, 4 Jan 2010 11:27:27 +0100
Subject: Raised exceptions are not logged when USR2
In-Reply-To: <20100103000401.GA13826@dcvr.yhbt.net>
References: <201001021742.29874.ibc@aliax.net>
	<201001022233.04664.ibc@aliax.net>
	<20100103000401.GA13826@dcvr.yhbt.net>
Message-ID: <201001041127.27411.ibc@aliax.net>

El Domingo, 3 de Enero de 2010, Eric Wong escribi?:
> If you really want to, you may be able to subclass SyslogLogger and give
> it IO-like methods (write/puts/flush) are require for Rack::Lint
> compatibility and then set $stderr:
> 
>   # Totally untested, stick this in your Unicorn config file and let
>   # us know if it works or blows up badly:
> 
>   require 'syslog_logger'
> 
>   class MySyslogLogger < SyslogLogger
>     alias puts error
>     alias write error
>     def flush; self; end
>   end
> 
>   $stderr = MySyslogLogger.new('foo')
> 

They would be also needed these methods:

    def reopen(v); true; end
    def sync=(v); v; end

-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Mon Jan  4 17:14:52 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Mon, 4 Jan 2010 23:14:52 +0100
Subject: Raised exceptions are not logged when USR2
In-Reply-To: <20100103000401.GA13826@dcvr.yhbt.net>
References: <201001021742.29874.ibc@aliax.net>
	<201001022233.04664.ibc@aliax.net>
	<20100103000401.GA13826@dcvr.yhbt.net>
Message-ID: <201001042314.52709.ibc@aliax.net>

El Domingo, 3 de Enero de 2010, Eric Wong escribi?:

>   # Totally untested, stick this in your Unicorn config file and let
>   # us know if it works or blows up badly:
> 
>   require 'syslog_logger'
> 
>   class MySyslogLogger < SyslogLogger
>     alias puts error
>     alias write error
>     def flush; self; end
>   end
> 
>   $stderr = MySyslogLogger.new('foo')


Unfortunatelly it doesn'w work very well. First of all Sysloglogger relies on 
a constant called SYSLOG created in "initialize" method so it's required to 
add the above methods to SyslogLogger class rather than subclass.

Anyhow, the main problem is that by doing "$stderr = SyslogLogger.new('foo')" 
the std error is not redirected as /proc/PID/fd still shows:

  2 -> /dev/pts/2

Of course by calling $stderr.puts "error" the above works, but when a real 
error raised it's is printed in the screen (/dev/pts/2).

I strongly think that IO#reopen is required. IO#reopen accepts a string (file 
path) or other IO, so "converting" SyslogLogger into a IO is required.

I'll try to achieve it.

Regards.




-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Mon Jan  4 17:34:06 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 4 Jan 2010 14:34:06 -0800
Subject: Raised exceptions are not logged when USR2
In-Reply-To: <201001042314.52709.ibc@aliax.net>
References: <201001021742.29874.ibc@aliax.net>
	<201001022233.04664.ibc@aliax.net>
	<20100103000401.GA13826@dcvr.yhbt.net>
	<201001042314.52709.ibc@aliax.net>
Message-ID: <20100104223406.GB3566@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Domingo, 3 de Enero de 2010, Eric Wong escribi?:
> 
> >   # Totally untested, stick this in your Unicorn config file and let
> >   # us know if it works or blows up badly:
> > 
> >   require 'syslog_logger'
> > 
> >   class MySyslogLogger < SyslogLogger
> >     alias puts error
> >     alias write error
> >     def flush; self; end
> >   end
> > 
> >   $stderr = MySyslogLogger.new('foo')
> 
> Unfortunatelly it doesn'w work very well. First of all Sysloglogger relies on 
> a constant called SYSLOG created in "initialize" method so it's required to 
> add the above methods to SyslogLogger class rather than subclass.
> 
> Anyhow, the main problem is that by doing "$stderr = SyslogLogger.new('foo')" 
> the std error is not redirected as /proc/PID/fd still shows:
> 
>   2 -> /dev/pts/2

Yup, that's expected.  _Assigning_ $stderr does not do a redirect at the
OS-level, only at th Ruby level (sufficient for env["rack.errors"],
Kernel#warn).

> Of course by calling $stderr.puts "error" the above works, but when a real 
> error raised it's is printed in the screen (/dev/pts/2).

> I strongly think that IO#reopen is required. IO#reopen accepts a string (file 
> path) or other IO, so "converting" SyslogLogger into a IO is required.
> 
> I'll try to achieve it.

Yes, you need to do something that'll dup()/dup2() internally like
IO#reopen to replace fd=2.  But when you have the raw file descriptor,
you won't get syslog formatting...

The only way I can think of is to redirect fd=2 to the stdin of a
separate process to add formatting in...

  r, w = IO.pipe
  fork {
     $stdin.reopen(r)
     w.close
     exec('ruby', '-rsyslog', '-ane',
          'BEGIN{Syslog.open("foo")}; Syslog.warning $_')
  }
  r.close
  $stderr.reopen(w)

Not pretty...

-- 
Eric Wong

From ibc at aliax.net  Mon Jan  4 17:51:01 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Mon, 4 Jan 2010 23:51:01 +0100
Subject: Raised exceptions are not logged when USR2
In-Reply-To: <20100104223406.GB3566@dcvr.yhbt.net>
References: <201001021742.29874.ibc@aliax.net>
	<201001042314.52709.ibc@aliax.net>
	<20100104223406.GB3566@dcvr.yhbt.net>
Message-ID: <201001042351.02053.ibc@aliax.net>

El Lunes, 4 de Enero de 2010, Eric Wong escribi?:
> I?aki Baz Castillo <ibc at aliax.net> wrote:
> > El Domingo, 3 de Enero de 2010, Eric Wong escribi?:
> > >   # Totally untested, stick this in your Unicorn config file and let
> > >   # us know if it works or blows up badly:
> > >
> > >   require 'syslog_logger'
> > >
> > >   class MySyslogLogger < SyslogLogger
> > >     alias puts error
> > >     alias write error
> > >     def flush; self; end
> > >   end
> > >
> > >   $stderr = MySyslogLogger.new('foo')
> >
> > Unfortunatelly it doesn'w work very well. First of all Sysloglogger
> > relies on a constant called SYSLOG created in "initialize" method so it's
> > required to add the above methods to SyslogLogger class rather than
> > subclass.
> >
> > Anyhow, the main problem is that by doing "$stderr =
> > SyslogLogger.new('foo')" the std error is not redirected as /proc/PID/fd
> > still shows:
> >
> >   2 -> /dev/pts/2
> 
> Yup, that's expected.  _Assigning_ $stderr does not do a redirect at the
> OS-level, only at th Ruby level (sufficient for env["rack.errors"],
> Kernel#warn).
> 
> > Of course by calling $stderr.puts "error" the above works, but when a
> > real error raised it's is printed in the screen (/dev/pts/2).
> >
> > I strongly think that IO#reopen is required. IO#reopen accepts a string
> > (file path) or other IO, so "converting" SyslogLogger into a IO is
> > required.
> >
> > I'll try to achieve it.
> 
> Yes, you need to do something that'll dup()/dup2() internally like
> IO#reopen to replace fd=2.  But when you have the raw file descriptor,
> you won't get syslog formatting...
> 
> The only way I can think of is to redirect fd=2 to the stdin of a
> separate process to add formatting in...
> 
>   r, w = IO.pipe
>   fork {
>      $stdin.reopen(r)
>      w.close
>      exec('ruby', '-rsyslog', '-ane',
>           'BEGIN{Syslog.open("foo")}; Syslog.warning $_')
>   }
>   r.close
>   $stderr.reopen(w)
> 
> Not pretty...


Thanks. Definitively it seems too much complex so I'll change myidea and will 
use a normal text file for stderr when running daemonized.

Thanks a lot.


-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Tue Jan  5 05:26:32 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Tue, 5 Jan 2010 11:26:32 +0100
Subject: stderr_path doesn't work at the moment of being called
Message-ID: <201001051126.32449.ibc@aliax.net>

Hi, in unicorn.conf.rb I want to log a welcome message (current datetime 
basically) to the error log, so I do:

  if Process.ppid == 1 || Process.getpgrp != $$
    stderr_path "/tmp/error.log"
    $stderr.puts "#{Time.now} Starting..."

Unfortunatelly the message is displayed in the terminal rather than the file.

However with the following hacks the message is displayed in the file:

a)
  stderr_path "/tmp/error.log"
  $stderr.reopen "/tmp/error.log"
  $stderr.puts "#{Time.now} Starting..."

b)
  stderr_path "/tmp/error.log"
  File.open("/tmp/error.log", 'ab') { |fp| $stderr.reopen(fp) }
  $stderr.puts "#{Time.now} Starting..."
  

The second workaround is exactly what Unicorn does when calling to 
"stderr_path" ("redirect_io" method). So, why doesn't work for me without the 
hacks? do I miss something?

Thanks a lot.

-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Tue Jan  5 05:32:37 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Tue, 5 Jan 2010 11:32:37 +0100
Subject: stderr_path doesn't work at the moment of being called
In-Reply-To: <201001051126.32449.ibc@aliax.net>
References: <201001051126.32449.ibc@aliax.net>
Message-ID: <201001051132.37358.ibc@aliax.net>

El Martes, 5 de Enero de 2010, I?aki Baz Castillo escribi?:

> The second workaround is exactly what Unicorn does when calling to
> "stderr_path" ("redirect_io" method). So, why doesn't work for me without
>  the hacks? do I miss something?

By running 'exec "ls -l /proc/$$/fd"' after 'stderr_path' I've realized that 
the stderr is still pointing to /dev/pts/N. Perhaps the method is called later 
and now it's just "eval"-ulated?


-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Tue Jan  5 05:44:42 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Tue, 5 Jan 2010 11:44:42 +0100
Subject: 'before_fork' is executed in parallel by two processes 
	=?utf-8?q?=C2=BF=3F?=
Message-ID: <201001051144.42394.ibc@aliax.net>

Hi, by adding `echo $$ >> /tmp/kk` into 'before_fork' I've realized that two 
pids are printed to the file when starting Unicorn.
This means that 'before_fork' is being runned by two different processes in 
parallel. Is it the expected behaviour? perhaps something very wrong in my 
configuration?

Thanks.

-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Tue Jan  5 06:03:17 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Tue, 5 Jan 2010 12:03:17 +0100
Subject: 'before_fork' is executed in parallel by two processes 
	=?utf-8?q?=C2=BF=3F?=
In-Reply-To: <201001051144.42394.ibc@aliax.net>
References: <201001051144.42394.ibc@aliax.net>
Message-ID: <201001051203.17578.ibc@aliax.net>

El Martes, 5 de Enero de 2010, I?aki Baz Castillo escribi?:
> Hi, by adding `echo $$ >> /tmp/kk` into 'before_fork' I've realized that
>  two pids are printed to the file when starting Unicorn.
> This means that 'before_fork' is being runned by two different processes in
> parallel. Is it the expected behaviour? perhaps something very wrong in my
> configuration?

I've totally lost. before_fork is executed in parallel by the number of 
workers set in the "worker_processes". I've tested the following:

  before_fork do |server, worker|
    `echo "before_fork $$" >> /tmp/kk`
  end

  after_fork do |server, worker|
    `echo "after_fork  $$" >> /tmp/kk`
  end

And this is the file output:

  before_fork 3447
  after_fork  3456
  before_fork 3454
  before_fork 3463
  after_fork  3464
  after_fork  3472


Could I know what is happening please? This really breaks the concept I had.
In fact I use before_fork to check database connections and so, but I've 
realized that such check if done N (number of workers).

Thanks for any help.




-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Tue Jan  5 06:33:29 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Tue, 5 Jan 2010 12:33:29 +0100
Subject: 'before_fork' is executed in parallel by two processes 
	=?utf-8?q?=C2=BF=3F?=
In-Reply-To: <201001051203.17578.ibc@aliax.net>
References: <201001051144.42394.ibc@aliax.net>
	<201001051203.17578.ibc@aliax.net>
Message-ID: <201001051233.30107.ibc@aliax.net>

El Martes, 5 de Enero de 2010, I?aki Baz Castillo escribi?:
>   before_fork do |server, worker|
>     `echo "before_fork $$" >> /tmp/kk`
>   end


> And this is the file output:
> 
>   before_fork 3447
>   after_fork  3456
>   before_fork 3454
>   before_fork 3463
>   after_fork  3464
>   after_fork  3472


Hummm, this could be something strange related to how `` works...

If I do the following:

  before_fork do |server, worker|
    `echo "before_fork $$" >> /tmp/kk`
     File.open("/tmp/kk", "a") do |f|
       f.puts "$$=#{$$}"
     end
  end

then I get:

  before_fork 11989
  $$=11977
  before_fork 11994
  $$=11977
  before_fork 12000
  $$=11977

So I'm totally lost. However if I test my databases in "before_fork" then I 
see N number of DB connections (N = workers num).





-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Tue Jan  5 16:47:08 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 5 Jan 2010 13:47:08 -0800
Subject: stderr_path doesn't work at the moment of being called
In-Reply-To: <201001051132.37358.ibc@aliax.net>
References: <201001051126.32449.ibc@aliax.net>
	<201001051132.37358.ibc@aliax.net>
Message-ID: <20100105214708.GB15213@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Martes, 5 de Enero de 2010, I?aki Baz Castillo escribi?:
> 
> > The second workaround is exactly what Unicorn does when calling to
> > "stderr_path" ("redirect_io" method). So, why doesn't work for me without
> >  the hacks? do I miss something?
> 
> By running 'exec "ls -l /proc/$$/fd"' after 'stderr_path' I've realized that 
> the stderr is still pointing to /dev/pts/N. Perhaps the method is called later 
> and now it's just "eval"-ulated?

The value is stored and then set later.  I avoid changing internal
configuration variables during evaluation time for several reasons:

1. A Unicorn config file is typically the last thing configured for an
   application, when a user is testing their configuration it's easier
   to see the error message in the console if it's in the Unicorn
   config file

2. HUP-ing a with a bad config file should not leave the server in an
   unknown state.  There are still some things that bind immediately
   (working_directory, all of Rainbows!) for now, but for the most
   part it should be possible to recover on bad HUPs.

3. "working_directory" needs to be factored in for non-absolute paths.
   I prefer to work with absolute paths, but some people like
   non-absolute ones for various reasons.  So we always ensure
   stderr_path is set after "working_directory" is called.

-- 
Eric Wong

From normalperson at yhbt.net  Tue Jan  5 16:51:04 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 5 Jan 2010 13:51:04 -0800
Subject: 'before_fork' is =?iso-8859-1?Q?execut?=
	=?iso-8859-1?Q?ed_in_parallel_by_two_processes_=BF=3F?=
In-Reply-To: <201001051233.30107.ibc@aliax.net>
References: <201001051144.42394.ibc@aliax.net>
	<201001051203.17578.ibc@aliax.net>
	<201001051233.30107.ibc@aliax.net>
Message-ID: <20100105215104.GC15213@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Martes, 5 de Enero de 2010, I?aki Baz Castillo escribi?:
> >   before_fork do |server, worker|
> >     `echo "before_fork $$" >> /tmp/kk`
> >   end
> 
> 
> > And this is the file output:
> > 
> >   before_fork 3447
> >   after_fork  3456
> >   before_fork 3454
> >   before_fork 3463
> >   after_fork  3464
> >   after_fork  3472
> 
> 
> Hummm, this could be something strange related to how `` works...
> 
> If I do the following:
> 
>   before_fork do |server, worker|
>     `echo "before_fork $$" >> /tmp/kk`
>      File.open("/tmp/kk", "a") do |f|
>        f.puts "$$=#{$$}"
>      end
>   end

Odd, I'm confused by the way `` is working there too...  I wonder if
this is a Ruby bug...  Anyhow, in your use case system() or your
File.open block should be the way to go.  Don't use `` when system()
will do.

> then I get:
> 
>   before_fork 11989
>   $$=11977
>   before_fork 11994
>   $$=11977
>   before_fork 12000
>   $$=11977
> 
> So I'm totally lost. However if I test my databases in "before_fork" then I 
> see N number of DB connections (N = workers num).

How are you testing the database?  `` or via Ruby code?

-- 
Eric Wong

From stefanmaier at gmail.com  Tue Jan  5 17:12:05 2010
From: stefanmaier at gmail.com (Stefan Maier)
Date: Tue, 05 Jan 2010 23:12:05 +0100
Subject: Unicorn, Rails and Memcached
Message-ID: <4B43B935.7050601@gmail.com>

Hi,

I am running unicorn with rails and memcached. The combination usually
works flawlessly but sometimes when I retrieve an item from memcached i
get a different, unrelated item.

At the moment I'm more or less investigating in all directions and after
reading
http://www.modrails.com/documentation/Users%20guide.html#%5Fexample%5F1%5Fmemcached%5Fconnection%5Fsharing%5Fharmful
I thought that maybe the forking nature of unicorn has something to do
with it.

So, are there any steps you have to take to ensure unicorn works with
memcached?


Thanks in advance,
Stefan Maier

From normalperson at yhbt.net  Tue Jan  5 17:34:45 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 5 Jan 2010 14:34:45 -0800
Subject: Unicorn, Rails and Memcached
In-Reply-To: <4B43B935.7050601@gmail.com>
References: <4B43B935.7050601@gmail.com>
Message-ID: <20100105223445.GA1116@dcvr.yhbt.net>

Stefan Maier <stefanmaier at gmail.com> wrote:
> Hi,
> 
> I am running unicorn with rails and memcached. The combination usually
> works flawlessly but sometimes when I retrieve an item from memcached i
> get a different, unrelated item.
> 
> At the moment I'm more or less investigating in all directions and after
> reading
> http://www.modrails.com/documentation/Users%20guide.html#%5Fexample%5F1%5Fmemcached%5Fconnection%5Fsharing%5Fharmful
> I thought that maybe the forking nature of unicorn has something to do
> with it.

Hi Stefan,

If you're using "preload_app true" in your config file, then all the
gotchas from Passenger with sharing memcached sockets across processes
apply to Unicorn as well.

> So, are there any steps you have to take to ensure unicorn works with
> memcached?

Which memcached library are you using?

I believe some memcached libraries will delay opening a socket until
it's actually needed (inside the worker processes), but if you use
memcached in your app initialization code, then the socket will be
opened in the master process (which is bad).

So the safe thing would be to reconnect to your memcached servers in the
after_fork hook.

-- 
Eric Wong

From stefanmaier at gmail.com  Tue Jan  5 17:47:38 2010
From: stefanmaier at gmail.com (Stefan Maier)
Date: Tue, 05 Jan 2010 23:47:38 +0100
Subject: Unicorn, Rails and Memcached
In-Reply-To: <20100105223445.GA1116@dcvr.yhbt.net>
References: <4B43B935.7050601@gmail.com> <20100105223445.GA1116@dcvr.yhbt.net>
Message-ID: <4B43C18A.3020502@gmail.com>

Am 1/5/10 11:34 PM, schrieb Eric Wong:
> Stefan Maier <stefanmaier at gmail.com> wrote:
>> Hi,
>>
>> I am running unicorn with rails and memcached. The combination usually
>> works flawlessly but sometimes when I retrieve an item from memcached i
>> get a different, unrelated item.
>>
>> At the moment I'm more or less investigating in all directions and after
>> reading
>> http://www.modrails.com/documentation/Users%20guide.html#%5Fexample%5F1%5Fmemcached%5Fconnection%5Fsharing%5Fharmful
>> I thought that maybe the forking nature of unicorn has something to do
>> with it.
> 
> Hi Stefan,
> 
> If you're using "preload_app true" in your config file, then all the
> gotchas from Passenger with sharing memcached sockets across processes
> apply to Unicorn as well.
> 
>> So, are there any steps you have to take to ensure unicorn works with
>> memcached?
> 
> Which memcached library are you using?
> 
> I believe some memcached libraries will delay opening a socket until
> it's actually needed (inside the worker processes), but if you use
> memcached in your app initialization code, then the socket will be
> opened in the master process (which is bad).
> 
> So the safe thing would be to reconnect to your memcached servers in the
> after_fork hook.
> 


This is what i thought and lsof also confirms that the workers have
their own connections.
So it's not a unicorn problem,
At the moment it seems like memcached gets confused by a malformed key.

Thanks for the quick reply,

Stefan

From ibc at aliax.net  Tue Jan  5 17:49:43 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Tue, 5 Jan 2010 23:49:43 +0100
Subject: 'before_fork' is executed in parallel by two processes 
	=?iso-8859-1?q?=BF=3F?=
In-Reply-To: <20100105215104.GC15213@dcvr.yhbt.net>
References: <201001051144.42394.ibc@aliax.net>
	<201001051233.30107.ibc@aliax.net>
	<20100105215104.GC15213@dcvr.yhbt.net>
Message-ID: <201001052349.43567.ibc@aliax.net>

El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> > So I'm totally lost. However if I test my databases in "before_fork" then
> > I  see N number of DB connections (N = workers num).
> 
> How are you testing the database?  `` or via Ruby code?

I use Ruby Sequel ORM.
I've also added the following code to before_fork:

  File.open("/tmp/lalala", "ab") do |f|
    f.puts "My PID is #{$$}"
  end

I confirm that there are two lines printed in that file when Unicorn starts 
(being 2 the number of workers). Is it the expected behavior?

Regards.

-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Tue Jan  5 17:50:24 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Tue, 5 Jan 2010 23:50:24 +0100
Subject: stderr_path doesn't work at the moment of being called
In-Reply-To: <20100105214708.GB15213@dcvr.yhbt.net>
References: <201001051126.32449.ibc@aliax.net>
	<201001051132.37358.ibc@aliax.net>
	<20100105214708.GB15213@dcvr.yhbt.net>
Message-ID: <201001052350.24395.ibc@aliax.net>

El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> I?aki Baz Castillo <ibc at aliax.net> wrote:
> > El Martes, 5 de Enero de 2010, I?aki Baz Castillo escribi?:
> > > The second workaround is exactly what Unicorn does when calling to
> > > "stderr_path" ("redirect_io" method). So, why doesn't work for me
> > > without the hacks? do I miss something?
> >
> > By running 'exec "ls -l /proc/$$/fd"' after 'stderr_path' I've realized
> > that the stderr is still pointing to /dev/pts/N. Perhaps the method is
> > called later and now it's just "eval"-ulated?
> 
> The value is stored and then set later.  I avoid changing internal
> configuration variables during evaluation time for several reasons:
> 
> 1. A Unicorn config file is typically the last thing configured for an
>    application, when a user is testing their configuration it's easier
>    to see the error message in the console if it's in the Unicorn
>    config file
> 
> 2. HUP-ing a with a bad config file should not leave the server in an
>    unknown state.  There are still some things that bind immediately
>    (working_directory, all of Rainbows!) for now, but for the most
>    part it should be possible to recover on bad HUPs.
> 
> 3. "working_directory" needs to be factored in for non-absolute paths.
>    I prefer to work with absolute paths, but some people like
>    non-absolute ones for various reasons.  So we always ensure
>    stderr_path is set after "working_directory" is called.

Ok, it makes sense :)
Thanks. 


-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Tue Jan  5 17:57:52 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 5 Jan 2010 14:57:52 -0800
Subject: 'before_fork' is =?iso-8859-1?Q?execut?=
	=?iso-8859-1?Q?ed_in_parallel_by_two_processes_=BF=3F?=
In-Reply-To: <201001052349.43567.ibc@aliax.net>
References: <201001051144.42394.ibc@aliax.net>
	<201001051233.30107.ibc@aliax.net>
	<20100105215104.GC15213@dcvr.yhbt.net>
	<201001052349.43567.ibc@aliax.net>
Message-ID: <20100105225752.GA3377@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> > > So I'm totally lost. However if I test my databases in "before_fork" then
> > > I  see N number of DB connections (N = workers num).
> > 
> > How are you testing the database?  `` or via Ruby code?
> 
> I use Ruby Sequel ORM.
> I've also added the following code to before_fork:
> 
>   File.open("/tmp/lalala", "ab") do |f|
>     f.puts "My PID is #{$$}"
>   end
> 
> I confirm that there are two lines printed in that file when Unicorn starts 
> (being 2 the number of workers). Is it the expected behavior?

Yes, before_fork and after_fork are both called for every worker forked.

-- 
Eric Wong

From ibc at aliax.net  Tue Jan  5 18:04:40 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Wed, 6 Jan 2010 00:04:40 +0100
Subject: 'before_fork' is executed in parallel by two processes 
	=?iso-8859-1?q?=BF=3F?=
In-Reply-To: <201001052349.43567.ibc@aliax.net>
References: <201001051144.42394.ibc@aliax.net>
	<20100105215104.GC15213@dcvr.yhbt.net>
	<201001052349.43567.ibc@aliax.net>
Message-ID: <201001060004.40416.ibc@aliax.net>

El Martes, 5 de Enero de 2010, I?aki Baz Castillo escribi?:
> I confirm that there are two lines printed in that file when Unicorn starts 
> (being 2 the number of workers). Is it the expected behavior?

Simpler example:

-----------------
worker_processes 3

before_fork do |server, worker|
  puts "*** I'my PID #{$$}"
end
----------------


Then run Unicorn in foreground and I see:

  *** I'm PID 8791
  *** I'm PID 8791
  *** I'm PID 8791

8791 is the PID of the master process, so yes, it seems that workers don't 
exist yet in before_fork (as obviously expected). However the block into 
before_fork is executed "worker_processes" times by master process.

Do I miss something?

-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Tue Jan  5 18:30:49 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Wed, 6 Jan 2010 00:30:49 +0100
Subject: 'before_fork' is executed in parallel by two processes 
	=?iso-8859-1?q?=BF=3F?=
In-Reply-To: <20100105225752.GA3377@dcvr.yhbt.net>
References: <201001051144.42394.ibc@aliax.net>
	<201001052349.43567.ibc@aliax.net>
	<20100105225752.GA3377@dcvr.yhbt.net>
Message-ID: <201001060030.49486.ibc@aliax.net>

El Martes, 5 de Enero de 2010, Eric Wong escribi?:

> Yes, before_fork and after_fork are both called for every worker forked.

So if I just want to test a DB connection then I do better wrtitting such code 
out of before_fork in the config file, right? (at least it's the workaround 
that works for me).

Thanks. 


-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Tue Jan  5 18:55:52 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 5 Jan 2010 15:55:52 -0800
Subject: 'before_fork' is =?iso-8859-1?Q?execut?=
	=?iso-8859-1?Q?ed_in_parallel_by_two_processes_=BF=3F?=
In-Reply-To: <201001060030.49486.ibc@aliax.net>
References: <201001051144.42394.ibc@aliax.net>
	<201001052349.43567.ibc@aliax.net>
	<20100105225752.GA3377@dcvr.yhbt.net>
	<201001060030.49486.ibc@aliax.net>
Message-ID: <20100105235552.GA6027@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> 
> > Yes, before_fork and after_fork are both called for every worker forked.
> 
> So if I just want to test a DB connection then I do better wrtitting such code 
> out of before_fork in the config file, right? (at least it's the workaround 
> that works for me).

You can also do it on a certain worker only:

  before_fork do |server, worker|
    if worker.nr == 0
      ...
    end
  end

-- 
Eric Wong

From ibc at aliax.net  Tue Jan  5 19:33:05 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Wed, 6 Jan 2010 01:33:05 +0100
Subject: 'before_fork' is executed in parallel by two processes 
	=?iso-8859-1?q?=BF=3F?=
In-Reply-To: <20100105235552.GA6027@dcvr.yhbt.net>
References: <201001051144.42394.ibc@aliax.net>
	<201001060030.49486.ibc@aliax.net>
	<20100105235552.GA6027@dcvr.yhbt.net>
Message-ID: <201001060133.05193.ibc@aliax.net>

El Mi?rcoles, 6 de Enero de 2010, Eric Wong escribi?:
> I?aki Baz Castillo <ibc at aliax.net> wrote:
> > El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> > > Yes, before_fork and after_fork are both called for every worker
> > > forked.
> >
> > So if I just want to test a DB connection then I do better wrtitting such
> > code out of before_fork in the config file, right? (at least it's the
> > workaround that works for me).
> 
> You can also do it on a certain worker only:
> 
>   before_fork do |server, worker|
>     if worker.nr == 0
>       ...
>     end
>   end

So if I'm not wrong when before_fork block is runned the $stderr is already 
redirected to the IO set in "stderr_path", rigth?
Then any error in the config file into before_fork would not vi raised to the 
terminal screen but to the stderr (if it has been redirected).

Then I see no advantage on using what you suggest ("if worker.nr == 0 ...") 
over adding such code at the top of the config file. Do I miss something?

Thanks a lot.



-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Tue Jan  5 20:12:26 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 5 Jan 2010 17:12:26 -0800
Subject: 'before_fork' is =?iso-8859-1?Q?execut?=
	=?iso-8859-1?Q?ed_in_parallel_by_two_processes_=BF=3F?=
In-Reply-To: <201001060133.05193.ibc@aliax.net>
References: <201001051144.42394.ibc@aliax.net>
	<201001060030.49486.ibc@aliax.net>
	<20100105235552.GA6027@dcvr.yhbt.net>
	<201001060133.05193.ibc@aliax.net>
Message-ID: <20100106011226.GA6184@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Mi?rcoles, 6 de Enero de 2010, Eric Wong escribi?:
> > I?aki Baz Castillo <ibc at aliax.net> wrote:
> > > El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> > > > Yes, before_fork and after_fork are both called for every worker
> > > > forked.
> > >
> > > So if I just want to test a DB connection then I do better wrtitting such
> > > code out of before_fork in the config file, right? (at least it's the
> > > workaround that works for me).
> > 
> > You can also do it on a certain worker only:
> > 
> >   before_fork do |server, worker|
> >     if worker.nr == 0
> >       ...
> >     end
> >   end
> 
> So if I'm not wrong when before_fork block is runned the $stderr is already 
> redirected to the IO set in "stderr_path", rigth?

Yes, stderr is redirected before any before_fork hooks run.

> Then any error in the config file into before_fork would not vi raised to the 
> terminal screen but to the stderr (if it has been redirected).

If it's a Ruby syntax error, then it would've been sent to the
terminal at startup.  Otherwise it'll be redirected.

> Then I see no advantage on using what you suggest ("if worker.nr == 0 ...") 
> over adding such code at the top of the config file. Do I miss something?

I depends on whether you use preload_app or not, if you use preload_app,
then you'll get to have access to any objects your app creates at
initialization.

-- 
Eric Wong

From ibc at aliax.net  Thu Jan  7 10:26:49 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Thu, 7 Jan 2010 16:26:49 +0100
Subject: 'before_fork' is executed in parallel by two processes 
	=?iso-8859-1?q?=BF=3F?=
In-Reply-To: <20100105215104.GC15213@dcvr.yhbt.net>
References: <201001051144.42394.ibc@aliax.net>
	<201001051233.30107.ibc@aliax.net>
	<20100105215104.GC15213@dcvr.yhbt.net>
Message-ID: <201001071626.49880.ibc@aliax.net>

El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> >   before_fork do |server, worker|
> >     `echo "before_fork $$" >> /tmp/kk`
> >      File.open("/tmp/kk", "a") do |f|
> >        f.puts "$$=#{$$}"
> >      end
> >   end
> 
> Odd, I'm confused by the way `` is working there too...  I wonder if
> this is a Ruby bug...  Anyhow, in your use case system() or your
> File.open block should be the way to go.  Don't use `` when system()
> will do.

I think that the above makse sense since `` creates a new process (with its 
own pid). So each time it's called (once per worker) it will be a different 
process. 


-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Thu Jan  7 10:35:56 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Thu, 7 Jan 2010 16:35:56 +0100
Subject: Using a worker for a different purpose
Message-ID: <201001071635.56191.ibc@aliax.net>

Hi, I'm thinking in using a worker (i.e: "worker.nr == 0") to accomplish a 
diferent task than binding in the Unicorn socket.

It would behave as a different process which binds in a different socket as 
daemon, so the other workers would notify it after processing data.

Of course I could have a separate process but why not using an Unicorn worker 
for this? in this way it's automatically reaped by master process if it 
crashes and I don't need to manage two different services.

Is is suitable? The main question is: how to tell a worker not to bind in the 
Unicorn configured socket(s)? is it possible?

Thanks a lot.

-- 
I?aki Baz Castillo <ibc at aliax.net>

From mguterl at gmail.com  Thu Jan  7 11:05:15 2010
From: mguterl at gmail.com (Michael Guterl)
Date: Thu, 7 Jan 2010 11:05:15 -0500
Subject: Strange Thread related errors
Message-ID: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>

Our Rails app has started raising exceptions (caught by hoptoad
thankfully) and I can only imagine they're related to unicorn.  I only
*think* the errors are occurring on the request after we deploy, which
upgrades the Unicorn process.  I say this because the errors are
coming from many different actions, but I haven't been able to
reproduce.

After the most recent batch of errors, I upgraded from 0.95.1 to
0.95.3, but we have not deployed again.

ThreadError: stopping only thread note: use sleep to stop forever

/usr/local/lib/ruby/1.8/monitor.rb:285:in `stop'

/usr/local/lib/ruby/1.8/monitor.rb:285:in `mon_acquire'

/usr/local/lib/ruby/1.8/monitor.rb:214:in `mon_enter'

/usr/local/lib/ruby/1.8/monitor.rb:240:in `synchronize'

/usr/local/lib/ruby/1.8/logger.rb:496:in `write'

/usr/local/lib/ruby/1.8/logger.rb:326:in `add'

/usr/local/lib/ruby/1.8/logger.rb:374:in `info'

/home/deploy/public_html/rm/releases/20100107153636/vendor/plugins/newrelic_rpm/lib/new_relic/agent/agent.rb:46:in
`ensure_worker_thread_started'

/home/deploy/public_html/rm/releases/20100107153636/vendor/plugins/newrelic_rpm/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:248:in
`perform_action'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/base.rb:532:in
`send'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/base.rb:532:in
`process_without_filters'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/filters.rb:606:in
`process'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/base.rb:391:in
`process'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/base.rb:386:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/routing/route_set.rb:437:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:87:in
`dispatch'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:121:in
`_call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:130:in
`build_middleware_stack'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/query_cache.rb:29:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/query_cache.rb:29:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:34:in
`cache'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/query_cache.rb:9:in
`cache'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/query_cache.rb:28:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb:361:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/string_coercion.rb:25:in
`call'

/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/head.rb:9:in `call'

/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/methodoverride.rb:24:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/params_parser.rb:15:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/session/cookie_store.rb:93:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activesupport/lib/active_support/cache/strategy/local_cache.rb:24:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/failsafe.rb:26:in
`call'

/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `call'

/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in
`synchronize'

/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:106:in
`call'

/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/railties/lib/rails/rack/static.rb:31:in
`call'

/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:46:in `call'

/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:40:in `each'

/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:40:in `call'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:571:in
`process_client'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:643:in
`worker_loop'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:641:in `each'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:641:in
`worker_loop'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:534:in
`spawn_missing_workers'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:534:in `fork'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:534:in
`spawn_missing_workers'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:530:in `each'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:530:in
`spawn_missing_workers'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:540:in
`maintain_worker_count'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:215:in `start'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:28:in `run'

/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/bin/unicorn_rails:207

/usr/local/bin/unicorn_rails:19:in `load'

/usr/local/bin/unicorn_rails:19

From ibc at aliax.net  Thu Jan  7 11:31:49 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Thu, 7 Jan 2010 17:31:49 +0100
Subject: Strange Thread related errors
In-Reply-To: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>
References: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>
Message-ID: <201001071731.49847.ibc@aliax.net>

El Jueves, 7 de Enero de 2010, Michael Guterl escribi?:
> Our Rails app has started raising exceptions (caught by hoptoad
> thankfully) and I can only imagine they're related to unicorn.  I only
> think the errors are occurring on the request after we deploy, which
> upgrades the Unicorn process.  I say this because the errors are
> coming from many different actions, but I haven't been able to
> reproduce.

Let me explain a problem I had with Sequel and Unicorn (already solved), 
perhaps it's related:


I load Sequel database toolkit in the master process so it generates its own 
threads pool.
Then Unicorn forks into workers but each worker uses the same instance of 
Sequel database, this is, the same threads pool.

This causes errors since some thread could be used by different processes at 
the same time.

The solution is simple:

Don't set the Sequel instance in the master process, instead do it in each 
worker ("after_fork" block in unicorn config file).

In this way each process has its own Sequel instance with its own threads 
pool.

Also, using Sequel thread pool makes no sense when using Unicorn as each 
worker is single thread, so I gain some performance by setting 
"single_threaded = true" in Sequel database configuration (no thread pool 
stuff).

Probably you are getting a similar issue due to ActiveRecord or any other 
multithread library you load in the master process so the instance is shared 
between all the workers.
 


-- 
I?aki Baz Castillo <ibc at aliax.net>

From jeremyevans0 at gmail.com  Thu Jan  7 13:10:02 2010
From: jeremyevans0 at gmail.com (Jeremy Evans)
Date: Thu, 7 Jan 2010 10:10:02 -0800
Subject: Strange Thread related errors
In-Reply-To: <201001071731.49847.ibc@aliax.net>
References: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>
	<201001071731.49847.ibc@aliax.net>
Message-ID: <5c548e461001071010i330fc61fs68967fcb5af88313@mail.gmail.com>

On Thu, Jan 7, 2010 at 8:31 AM, I?aki Baz Castillo <ibc at aliax.net> wrote:
> Don't set the Sequel instance in the master process, instead do it in each
> worker ("after_fork" block in unicorn config file).

Alternatively, you can set up the Sequel Database in the master
process, and just call Database#disconnect before forking.

> In this way each process has its own Sequel instance with its own threads
> pool.
>
> Also, using Sequel thread pool makes no sense when using Unicorn as each
> worker is single thread, so I gain some performance by setting
> "single_threaded = true" in Sequel database configuration (no thread pool
> stuff).

This is good advice.

Jeremy

From ibc at aliax.net  Thu Jan  7 13:33:37 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Thu, 7 Jan 2010 19:33:37 +0100
Subject: Strange Thread related errors
In-Reply-To: <5c548e461001071010i330fc61fs68967fcb5af88313@mail.gmail.com>
References: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>
	<201001071731.49847.ibc@aliax.net>
	<5c548e461001071010i330fc61fs68967fcb5af88313@mail.gmail.com>
Message-ID: <201001071933.37928.ibc@aliax.net>

El Jueves, 7 de Enero de 2010, Jeremy Evans escribi?:
> > Don't set the Sequel instance in the master process, instead do it in
> > each worker ("after_fork" block in unicorn config file).
> 
> Alternatively, you can set up the Sequel Database in the master
> process, and just call Database#disconnect before forking.

Yes, in fact I do that to check the database connection when starting Unicorn 
and raise if some database fails.
Then I disconnect the databases and reconnect in each worker. 


-- 
I?aki Baz Castillo <ibc at aliax.net>

From mguterl at gmail.com  Thu Jan  7 13:49:52 2010
From: mguterl at gmail.com (Michael Guterl)
Date: Thu, 7 Jan 2010 13:49:52 -0500
Subject: Strange Thread related errors
In-Reply-To: <201001071731.49847.ibc@aliax.net>
References: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>
	<201001071731.49847.ibc@aliax.net>
Message-ID: <944a03771001071049j520f6580i26c3a3ae8b850a48@mail.gmail.com>

On Thu, Jan 7, 2010 at 11:31 AM, I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Jueves, 7 de Enero de 2010, Michael Guterl escribi?:
>> Our Rails app has started raising exceptions (caught by hoptoad
>> thankfully) and I can only imagine they're related to unicorn. ?I only
>> think the errors are occurring on the request after we deploy, which
>> upgrades the Unicorn process. ?I say this because the errors are
>> coming from many different actions, but I haven't been able to
>> reproduce.
>
> Let me explain a problem I had with Sequel and Unicorn (already solved),
> perhaps it's related:
>
>
> I load Sequel database toolkit in the master process so it generates its own
> threads pool.
> Then Unicorn forks into workers but each worker uses the same instance of
> Sequel database, this is, the same threads pool.
>
> This causes errors since some thread could be used by different processes at
> the same time.
>
> The solution is simple:
>
> Don't set the Sequel instance in the master process, instead do it in each
> worker ("after_fork" block in unicorn config file).
>
> In this way each process has its own Sequel instance with its own threads
> pool.
>
> Also, using Sequel thread pool makes no sense when using Unicorn as each
> worker is single thread, so I gain some performance by setting
> "single_threaded = true" in Sequel database configuration (no thread pool
> stuff).
>
> Probably you are getting a similar issue due to ActiveRecord or any other
> multithread library you load in the master process so the instance is shared
> between all the workers.
>

We currently disconnect from both databases in before_fork and
re-establish the connection in after_fork.

Here's our unicorn config: http://gist.github.com/271449

Maybe I should disable connection pooling?

Best regards,
Michael Guterl

From ibc at aliax.net  Thu Jan  7 14:08:38 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Thu, 7 Jan 2010 20:08:38 +0100
Subject: Strange Thread related errors
In-Reply-To: <944a03771001071049j520f6580i26c3a3ae8b850a48@mail.gmail.com>
References: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>
	<201001071731.49847.ibc@aliax.net>
	<944a03771001071049j520f6580i26c3a3ae8b850a48@mail.gmail.com>
Message-ID: <201001072008.39047.ibc@aliax.net>

El Jueves, 7 de Enero de 2010, Michael Guterl escribi?:
> We currently disconnect from both databases in before_fork and
> re-establish the connection in after_fork.
> 
> Here's our unicorn config: http://gist.github.com/271449

Unfortunately I don't know very well ho ActiveRecords works but you could get 
into the same probem even if you disconnect the db in the case you are sharing 
the same ActiveRecord instance in all the workers.
This issue also ocurred to me using Sequel: I generated the DB instances in 
master process and stored them in an Array that later used in each worker so 
the fact is that I was sharing the same DB object in all the workers, so when 
they reconcet they are in fact using a single connection (or a single 
connection pool) for all the workers.

But I think that ActiveRecords works in a different way as you don't create a 
DB instance but instead use directly a class ("class MyModel < 
ActiveRecords").

 
> Maybe I should disable connection pooling?
 
This should not be the real cause of your problem (if the above is not 
happening to you). Unicorn workers are single thread which means you can use 
threads into them safely (except if those threads are, accidentally, shared by 
other processes).

I suggest the following: Tun the server and ensure you do enough requests so 
all the worker perform at least a DB query. And them check the current DB 
connection in your DB. If you dissable coonectino pooling you should see just 
N connections /being N the number of Unicorn workers).

-- 
I?aki Baz Castillo <ibc at aliax.net>

From jeremyevans0 at gmail.com  Thu Jan  7 15:08:52 2010
From: jeremyevans0 at gmail.com (Jeremy Evans)
Date: Thu, 7 Jan 2010 12:08:52 -0800
Subject: Strange Thread related errors
In-Reply-To: <944a03771001071049j520f6580i26c3a3ae8b850a48@mail.gmail.com>
References: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>
	<201001071731.49847.ibc@aliax.net>
	<944a03771001071049j520f6580i26c3a3ae8b850a48@mail.gmail.com>
Message-ID: <5c548e461001071208u384a2a4bj7ea5e983381ad801@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:49 AM, Michael Guterl <mguterl at gmail.com> wrote:
> We currently disconnect from both databases in before_fork and
> re-establish the connection in after_fork.
>
> Here's our unicorn config: http://gist.github.com/271449
>
> Maybe I should disable connection pooling?

Try:

  ActiveRecord::Base.clear_active_connections!

instead of:

  ActiveRecord::Base.connection.disconnect!

Jeremy

From normalperson at yhbt.net  Thu Jan  7 15:13:40 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 7 Jan 2010 20:13:40 +0000
Subject: Strange Thread related errors
In-Reply-To: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>
References: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>
Message-ID: <20100107201340.GA27432@dcvr.yhbt.net>

Michael Guterl <mguterl at gmail.com> wrote:
> Our Rails app has started raising exceptions (caught by hoptoad
> thankfully) and I can only imagine they're related to unicorn.  I only
> *think* the errors are occurring on the request after we deploy, which
> upgrades the Unicorn process.  I say this because the errors are
> coming from many different actions, but I haven't been able to
> reproduce.
> 
> After the most recent batch of errors, I upgraded from 0.95.1 to
> 0.95.3, but we have not deployed again.
> 
> ThreadError: stopping only thread note: use sleep to stop forever
> 
> /usr/local/lib/ruby/1.8/monitor.rb:285:in `stop'
> 
> /usr/local/lib/ruby/1.8/monitor.rb:285:in `mon_acquire'
> 
> /usr/local/lib/ruby/1.8/monitor.rb:214:in `mon_enter'
> 
> /usr/local/lib/ruby/1.8/monitor.rb:240:in `synchronize'
> 
> /usr/local/lib/ruby/1.8/logger.rb:496:in `write'
> 
> /usr/local/lib/ruby/1.8/logger.rb:326:in `add'
> 
> /usr/local/lib/ruby/1.8/logger.rb:374:in `info'
> 
> /home/deploy/public_html/rm/releases/20100107153636/vendor/plugins/newrelic_rpm/lib/new_relic/agent/agent.rb:46:in
> `ensure_worker_thread_started'

Hi Michael,

It looks like there's a background thread with the NewRelic plugin...

With "preload_app true", then any threads spawned in the master will die
in workers.  This is true for Ruby 1.8 green threads by design.  With
1.9, it's not even to share native POSIX threads between processes.

So in your after_fork hook, you need to restart threads inside
each worker.

-- 
Eric Wong

From normalperson at yhbt.net  Thu Jan  7 15:26:32 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 7 Jan 2010 20:26:32 +0000
Subject: Using a worker for a different purpose
In-Reply-To: <201001071635.56191.ibc@aliax.net>
References: <201001071635.56191.ibc@aliax.net>
Message-ID: <20100107202632.GB27432@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> Hi, I'm thinking in using a worker (i.e: "worker.nr == 0") to accomplish a 
> diferent task than binding in the Unicorn socket.
> 
> It would behave as a different process which binds in a different socket as 
> daemon, so the other workers would notify it after processing data.
> 
> Of course I could have a separate process but why not using an Unicorn worker 
> for this? in this way it's automatically reaped by master process if it 
> crashes and I don't need to manage two different services.
> 
> Is is suitable? The main question is: how to tell a worker not to bind in the 
> Unicorn configured socket(s)? is it possible?

Hi I?aki,

You could _try_ something like:

  after_fork do |server, worker|
    if worker.nr == 0
      # new app
      server.app = Rack::Builder.new { ... }

      # clear the local listener set
      server.listeners = []

      # new listeners
      server.listen another_socket, socket_options
    end
  end

I make no guarantees that it'll work, though, and I'm hesitant
to support/encourage it even if it does.

-- 
Eric Wong

From ibc at aliax.net  Thu Jan  7 15:44:25 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Thu, 7 Jan 2010 21:44:25 +0100
Subject: Using a worker for a different purpose
In-Reply-To: <20100107202632.GB27432@dcvr.yhbt.net>
References: <201001071635.56191.ibc@aliax.net>
	<20100107202632.GB27432@dcvr.yhbt.net>
Message-ID: <201001072144.25944.ibc@aliax.net>

El Jueves, 7 de Enero de 2010, Eric Wong escribi?:
> I?aki Baz Castillo <ibc at aliax.net> wrote:
> > Hi, I'm thinking in using a worker (i.e: "worker.nr == 0") to accomplish
> > a diferent task than binding in the Unicorn socket.
> >
> > It would behave as a different process which binds in a different socket
> > as daemon, so the other workers would notify it after processing data.
> >
> > Of course I could have a separate process but why not using an Unicorn
> > worker for this? in this way it's automatically reaped by master process
> > if it crashes and I don't need to manage two different services.
> >
> > Is is suitable? The main question is: how to tell a worker not to bind in
> > the Unicorn configured socket(s)? is it possible?
> 
> Hi I?aki,
> 
> You could _try_ something like:
> 
>   after_fork do |server, worker|
>     if worker.nr == 0
>       # new app
>       server.app = Rack::Builder.new { ... }
> 
>       # clear the local listener set
>       server.listeners = []
> 
>       # new listeners
>       server.listen another_socket, socket_options
>     end
>   end
> 
> I make no guarantees that it'll work, though, and I'm hesitant
> to support/encourage it even if it does.

It seems interesting. Just a doubt: would it work with "preload_app true"?
I use preload_app since in case he config.ru is wrong then it raises in the 
master (instead of raising each worker and being reaped again and again).

Thanks, I'll try it. 


-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Thu Jan  7 15:53:15 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 7 Jan 2010 12:53:15 -0800
Subject: Using a worker for a different purpose
In-Reply-To: <201001072144.25944.ibc@aliax.net>
References: <201001071635.56191.ibc@aliax.net>
	<20100107202632.GB27432@dcvr.yhbt.net>
	<201001072144.25944.ibc@aliax.net>
Message-ID: <20100107205315.GA6074@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Jueves, 7 de Enero de 2010, Eric Wong escribi?:
> > I?aki Baz Castillo <ibc at aliax.net> wrote:
> > > Hi, I'm thinking in using a worker (i.e: "worker.nr == 0") to accomplish
> > > a diferent task than binding in the Unicorn socket.
> > >
> > > It would behave as a different process which binds in a different socket
> > > as daemon, so the other workers would notify it after processing data.
> > >
> > > Of course I could have a separate process but why not using an Unicorn
> > > worker for this? in this way it's automatically reaped by master process
> > > if it crashes and I don't need to manage two different services.
> > >
> > > Is is suitable? The main question is: how to tell a worker not to bind in
> > > the Unicorn configured socket(s)? is it possible?
> > 
> > Hi I?aki,
> > 
> > You could _try_ something like:
> > 
> >   after_fork do |server, worker|
> >     if worker.nr == 0
> >       # new app
> >       server.app = Rack::Builder.new { ... }
> > 
> >       # clear the local listener set
> >       server.listeners = []
> > 
> >       # new listeners
> >       server.listen another_socket, socket_options
> >     end
> >   end
> > 
> > I make no guarantees that it'll work, though, and I'm hesitant
> > to support/encourage it even if it does.
> 
> It seems interesting. Just a doubt: would it work with "preload_app true"?
> I use preload_app since in case he config.ru is wrong then it raises in the 
> master (instead of raising each worker and being reaped again and again).

It should work, but you won't be saving memory/spawning time with
"preload_app true" since you're replacing the app with a new one.

It's really dirty and smelly.  Processes are a great OS-level
abstraction to separate distinct services from each other.  IMHO it's
saner to just run another Unicorn instance.

Unicorn is not a VM, after all :)

-- 
Eric Wong

From ibc at aliax.net  Thu Jan  7 15:58:53 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Thu, 7 Jan 2010 21:58:53 +0100
Subject: Using a worker for a different purpose
In-Reply-To: <20100107205315.GA6074@dcvr.yhbt.net>
References: <201001071635.56191.ibc@aliax.net>
	<201001072144.25944.ibc@aliax.net>
	<20100107205315.GA6074@dcvr.yhbt.net>
Message-ID: <201001072158.53951.ibc@aliax.net>

El Jueves, 7 de Enero de 2010, Eric Wong escribi?:

> > It seems interesting. Just a doubt: would it work with "preload_app
> > true"? I use preload_app since in case he config.ru is wrong then it
> > raises in the master (instead of raising each worker and being reaped
> > again and again).
> 
> It should work, but you won't be saving memory/spawning time with
> "preload_app true" since you're replacing the app with a new one.
> 
> It's really dirty and smelly.  Processes are a great OS-level
> abstraction to separate distinct services from each other.  IMHO it's
> saner to just run another Unicorn instance.
> 
> Unicorn is not a VM, after all :)

Yes, you are right. In fact the purpose of such worker would not be a Rack 
handler (http server) but a different one.
Humm, I think I'll keep it simple and run an external process :) 


-- 
I?aki Baz Castillo <ibc at aliax.net>

From mguterl at gmail.com  Thu Jan  7 16:32:19 2010
From: mguterl at gmail.com (Michael Guterl)
Date: Thu, 7 Jan 2010 16:32:19 -0500
Subject: Strange Thread related errors
In-Reply-To: <20100107201340.GA27432@dcvr.yhbt.net>
References: <944a03771001070805k27356c9dwdc535bed8efa7bb6@mail.gmail.com>
	<20100107201340.GA27432@dcvr.yhbt.net>
Message-ID: <944a03771001071332g19610af6r76ea88cc193a245@mail.gmail.com>

On Thu, Jan 7, 2010 at 3:13 PM, Eric Wong <normalperson at yhbt.net> wrote:
> Michael Guterl <mguterl at gmail.com> wrote:
>> Our Rails app has started raising exceptions (caught by hoptoad
>> thankfully) and I can only imagine they're related to unicorn. ?I only
>> *think* the errors are occurring on the request after we deploy, which
>> upgrades the Unicorn process. ?I say this because the errors are
>> coming from many different actions, but I haven't been able to
>> reproduce.
>>
>> After the most recent batch of errors, I upgraded from 0.95.1 to
>> 0.95.3, but we have not deployed again.
>>
>> ThreadError: stopping only thread note: use sleep to stop forever
>>
>> /usr/local/lib/ruby/1.8/monitor.rb:285:in `stop'
>>
>> /usr/local/lib/ruby/1.8/monitor.rb:285:in `mon_acquire'
>>
>> /usr/local/lib/ruby/1.8/monitor.rb:214:in `mon_enter'
>>
>> /usr/local/lib/ruby/1.8/monitor.rb:240:in `synchronize'
>>
>> /usr/local/lib/ruby/1.8/logger.rb:496:in `write'
>>
>> /usr/local/lib/ruby/1.8/logger.rb:326:in `add'
>>
>> /usr/local/lib/ruby/1.8/logger.rb:374:in `info'
>>
>> /home/deploy/public_html/rm/releases/20100107153636/vendor/plugins/newrelic_rpm/lib/new_relic/agent/agent.rb:46:in
>> `ensure_worker_thread_started'
>
> Hi Michael,
>
> It looks like there's a background thread with the NewRelic plugin...
>
> With "preload_app true", then any threads spawned in the master will die
> in workers. ?This is true for Ruby 1.8 green threads by design. ?With
> 1.9, it's not even to share native POSIX threads between processes.
>
> So in your after_fork hook, you need to restart threads inside
> each worker.

Thanks for this Eric, much appreciated.  It turns out that New Relic
doesn't officially support Unicorn yet, but I upgraded their plugin
and I have deployed a few times and haven't seen anything since.

Michael Guterl

From normalperson at yhbt.net  Fri Jan  8 01:19:54 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 8 Jan 2010 06:19:54 +0000
Subject: [ANN] unicorn 0.96.0 - Rack 1.1 extras for fast clients
Message-ID: <20100108061953.GA20414@dcvr.yhbt.net>

Unicorn is an HTTP server for Rack applications designed to only serve
fast clients on low-latency, high-bandwidth connections and take
advantage of features in Unix/Unix-like kernels.  Slow clients should
only be served by placing a reverse proxy capable of fully buffering
both the the request and response in between Unicorn and slow clients.

* http://unicorn.bogomips.org/
* mongrel-unicorn at rubyforge.org
* git://git.bogomips.org/unicorn.git

Changes:

This release includes small changes for things allowed by Rack
1.1.  It is also now easier to detect if daemonized process
fails to start.  Manpages received some minor updates as well.

Rack 1.1 allowed us to make the following environment changes:

* "rack.logger" is now set to the "logger" specified in
  the Unicorn config file.  This defaults to a Logger
  instance pointing to $stderr.

* "rack.version" is now at [1,1].  Unicorn remains
  compatible with previous Rack versions if your app
  depends on it.

While only specified since Rack 1.1, Unicorn has always exposed
"rack.input" in binary mode (and has ridiculous integration
tests that go outside of Ruby to prove it!).

-- 
Eric Wong

From ibc at aliax.net  Sat Jan  9 16:54:13 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Sat, 9 Jan 2010 22:54:13 +0100
Subject: Why workers die when sending KILL to master?
Message-ID: <201001092254.13966.ibc@aliax.net>

Hi, in my customized "unicorn" executable I create N forks in master process 
to behave as posix_mq readers.

I added a "at_exit" block just in master process so they are killed when the 
master receives a signal to terminate.

The only issue I'm experimenting is the fact that they are not killed if the 
master process receives a KILL signal. However Unicorn workers are terminated 
after master is killed with KILL signal. Could I know how such cool feature is 
implemented?

AFAIK when a process receives a KILL signal it just dies and cannot capture 
such signal so this should be a feature implemented in workers (in some way 
they can detect if master has died, am I right?). If so, could I be pointed to 
the exact code in which the feature is implemented? I would like to replicate 
it in my extra-processes.

Thanks a lot.

-- 
I?aki Baz Castillo <ibc at aliax.net>

From skaar at waste.org  Sat Jan  9 17:16:57 2010
From: skaar at waste.org (skaar)
Date: Sat, 9 Jan 2010 16:16:57 -0600
Subject: X-Forwarded-Proto / X_FORWARDED_PROTO
Message-ID: <20100109221656.GA8846@waste.org>

Eric,

think I came across an issue with the parser in unicorn, with a request
(due to 2 layers of nginx proxying) coming across with both a
X_FORWARDED_PROTO and a X-Forwarded-Proto header. From the socket (in
HttpRequest) - we get:

  X_FORWARDED_PROTO: http
  X-Forwarded-Proto: https

which is parsed to 

  HTTP_X_FORWARDED_PROTO"=>"http,https

There was a passenger ticket that describes that from nginx's point of
view - X-Forwarded-Proto is the "correct" form. 

-- 
/skaar

skaar at waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n

From ibc at aliax.net  Sat Jan  9 17:33:41 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Sat, 9 Jan 2010 23:33:41 +0100
Subject: X-Forwarded-Proto / X_FORWARDED_PROTO
In-Reply-To: <20100109221656.GA8846@waste.org>
References: <20100109221656.GA8846@waste.org>
Message-ID: <201001092333.41898.ibc@aliax.net>

El S?bado, 9 de Enero de 2010, skaar escribi?:
> Eric,
> 
> think I came across an issue with the parser in unicorn, with a request
> (due to 2 layers of nginx proxying) coming across with both a
> X_FORWARDED_PROTO and a X-Forwarded-Proto header. From the socket (in
> HttpRequest) - we get:
> 
>   X_FORWARDED_PROTO: http
>   X-Forwarded-Proto: https
> 
> which is parsed to
> 
>   HTTP_X_FORWARDED_PROTO"=>"http,https
> 
> There was a passenger ticket that describes that from nginx's point of
> view - X-Forwarded-Proto is the "correct" form.

Rack specs require "-" being converted to "_" in header names, so both header 
X_FORWARDED_PROTO and X-Forwarded-Proto become the same header.

Then we have two headers with same name and different value. Some headers are 
defined as "multivalue" which means that they would have the same meaning if 
an unique header exists containing both values separated by comma (the top one 
first). So:

  X_FORWARDED_PROTO: http
  X-Forwarded-Proto: https

is equivalent to:

  X_FORWARDED_PROTO: http, https
 
However it's not true that *any* header behaves in this way, but just headers 
defined in such way (by the standards). For example two "Authorization" 
headers cannot be joined in a single header.

So it's not totally safe to assume that any header appearing various times 
could be a single header with their values separated by comma, however I think 
it's a good approach.

More correct would be that Unicorn parses uknown headers appearing more than 
one time by setting a Array as value:

  X_FORWARDED_PROTO: ["http", "https"]

Unfortunately this is not allowed in Rack specifications.


-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Sat Jan  9 17:39:03 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Sat, 9 Jan 2010 22:39:03 +0000
Subject: X-Forwarded-Proto / X_FORWARDED_PROTO
In-Reply-To: <20100109221656.GA8846@waste.org>
References: <20100109221656.GA8846@waste.org>
Message-ID: <20100109223903.GC7480@dcvr.yhbt.net>

skaar <skaar at waste.org> wrote:
> Eric,
> 
> think I came across an issue with the parser in unicorn, with a request
> (due to 2 layers of nginx proxying) coming across with both a
> X_FORWARDED_PROTO and a X-Forwarded-Proto header. From the socket (in
> HttpRequest) - we get:
> 
>   X_FORWARDED_PROTO: http
>   X-Forwarded-Proto: https
> 
> which is parsed to 
> 
>   HTTP_X_FORWARDED_PROTO"=>"http,https
> 
> There was a passenger ticket that describes that from nginx's point of
> view - X-Forwarded-Proto is the "correct" form. 

Hi skaar,

The Rack (and CGI) specs require that '-' be replaced with '_' for
HTTP header names, so Unicorn is doing the correct thing and treating
it as the same header.

Even though '_' appears to be allowed in header names by rfc 2616, it's
use is questionable as there are parsers that reject it.  Any chance of
fixing whatever sends the "X_FORWARDED_PROTO" header?  It's completely
pointless to set to "http"

-- 
Eric Wong

From normalperson at yhbt.net  Sat Jan  9 17:41:29 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Sat, 9 Jan 2010 22:41:29 +0000
Subject: Why workers die when sending KILL to master?
In-Reply-To: <201001092254.13966.ibc@aliax.net>
References: <201001092254.13966.ibc@aliax.net>
Message-ID: <20100109224129.GD7480@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> Hi, in my customized "unicorn" executable I create N forks in master process 
> to behave as posix_mq readers.
> 
> I added a "at_exit" block just in master process so they are killed when the 
> master receives a signal to terminate.
> 
> The only issue I'm experimenting is the fact that they are not killed if the 
> master process receives a KILL signal. However Unicorn workers are terminated 
> after master is killed with KILL signal. Could I know how such cool feature is 
> implemented?
> 
> AFAIK when a process receives a KILL signal it just dies and cannot capture 
> such signal so this should be a feature implemented in workers (in some way 
> they can detect if master has died, am I right?). If so, could I be pointed to 
> the exact code in which the feature is implemented? I would like to replicate 
> it in my extra-processes.
> 

Hi I?aki,

The worker_loop just compares Process.ppid with the original ppid it
was started with.  They wakeup from IO.select() every timeout/2 seconds
to check for original_ppid != ppid if the server is idle.

-- 
Eric Wong

From ibc at aliax.net  Sat Jan  9 17:58:00 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Sat, 9 Jan 2010 23:58:00 +0100
Subject: Why workers die when sending KILL to master?
In-Reply-To: <20100109224129.GD7480@dcvr.yhbt.net>
References: <201001092254.13966.ibc@aliax.net>
	<20100109224129.GD7480@dcvr.yhbt.net>
Message-ID: <201001092358.00950.ibc@aliax.net>

El S?bado, 9 de Enero de 2010, Eric Wong escribi?:
> The worker_loop just compares Process.ppid with the original ppid it
> was started with.  They wakeup from IO.select() every timeout/2 seconds
> to check for original_ppid != ppid if the server is idle.

Thanks, I was expecting exactly that as the ppid changes when the master 
process dies :) 


-- 
I?aki Baz Castillo <ibc at aliax.net>

From skaar at waste.org  Sat Jan  9 18:29:03 2010
From: skaar at waste.org (skaar)
Date: Sat, 9 Jan 2010 17:29:03 -0600
Subject: X-Forwarded-Proto / X_FORWARDED_PROTO
In-Reply-To: <20100109223903.GC7480@dcvr.yhbt.net>
References: <20100109221656.GA8846@waste.org>
	<20100109223903.GC7480@dcvr.yhbt.net>
Message-ID: <20100109232903.GB8846@waste.org>

> 
> The Rack (and CGI) specs require that '-' be replaced with '_' for
> HTTP header names, so Unicorn is doing the correct thing and treating
> it as the same header.

but should it aggregate the values?
 
> Even though '_' appears to be allowed in header names by rfc 2616, it's
> use is questionable as there are parsers that reject it.  Any chance of
> fixing whatever sends the "X_FORWARDED_PROTO" header?  It's completely
> pointless to set to "http"

in our case we can probably fix the proxy chain - we have a set of
dedicated SSL systems (which sets X-Forwarded-Proto) which forward to
the main nginx front end (which sets X_FORWARDED_PROTO). I'll also look
at removing the proxy-set-header cases that sets it to to 'http'.

Just for reference: we ran into this issue when replacing nginx to
mongrels over a TCP socket, with nginx to unicorn over a domain socket.
With mongrel we don't have this problem (although I see from
e26ebc985b882c38da50fb0104791a5f2c0f8522 - that you added support for it
in http11)

/skaar






> 
> -- 
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying

-- 
/skaar

skaar at waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n

From ibc at aliax.net  Sat Jan  9 18:45:37 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Sun, 10 Jan 2010 00:45:37 +0100
Subject: X-Forwarded-Proto / X_FORWARDED_PROTO
In-Reply-To: <20100109232903.GB8846@waste.org>
References: <20100109221656.GA8846@waste.org>
	<20100109223903.GC7480@dcvr.yhbt.net>
	<20100109232903.GB8846@waste.org>
Message-ID: <201001100045.37677.ibc@aliax.net>

El Domingo, 10 de Enero de 2010, skaar escribi?:
> > The Rack (and CGI) specs require that '-' be replaced with '_' for
> > HTTP header names, so Unicorn is doing the correct thing and treating
> > it as the same header.
> 
> but should it aggregate the values?

Hi, review my other response in this thread.

This is undefined. Some headers do allow multiple values separated by comma in 
the same header with the same meaning as varios headers with same name and 
single values. But this depends on each header specification.

In your case it seems valid for me (just an opinnion) as 
"HTTP_X_FORWARDED_PROTO: http,https" could mean that the request has been sent 
using HTTPS and an intermediary proxy has forwarded it using HTTP. Of course 
the final destination (Unicorn application) must be ready to support such 
syntax.


-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Sat Jan  9 18:47:04 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Sun, 10 Jan 2010 00:47:04 +0100
Subject: Why workers die when sending KILL to master?
In-Reply-To: <201001092358.00950.ibc@aliax.net>
References: <201001092254.13966.ibc@aliax.net>
	<20100109224129.GD7480@dcvr.yhbt.net>
	<201001092358.00950.ibc@aliax.net>
Message-ID: <201001100047.04837.ibc@aliax.net>

El S?bado, 9 de Enero de 2010, I?aki Baz Castillo escribi?:
> El S?bado, 9 de Enero de 2010, Eric Wong escribi?:
> > The worker_loop just compares Process.ppid with the original ppid it
> > was started with.  They wakeup from IO.select() every timeout/2 seconds
> > to check for original_ppid != ppid if the server is idle.
> 
> Thanks, I was expecting exactly that as the ppid changes when the master
> process dies :)

I've implememented the same concept for my extra-processes in a similar way:

        ppid = Process.ppid
        Thread.new do
          loop do
            sleep 4
            if Process.ppid != ppid
              log.fatal "master process did, exiting"
              exit!
            end
          end
        end


-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Sun Jan 10 01:12:09 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Sat, 9 Jan 2010 22:12:09 -0800
Subject: X-Forwarded-Proto / X_FORWARDED_PROTO
In-Reply-To: <201001100045.37677.ibc@aliax.net>
References: <20100109221656.GA8846@waste.org>
	<20100109223903.GC7480@dcvr.yhbt.net>
	<20100109232903.GB8846@waste.org>
	<201001100045.37677.ibc@aliax.net>
Message-ID: <20100110061209.GA1685@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Domingo, 10 de Enero de 2010, skaar escribi?:
> > > The Rack (and CGI) specs require that '-' be replaced with '_' for
> > > HTTP header names, so Unicorn is doing the correct thing and treating
> > > it as the same header.
> > 
> > but should it aggregate the values?
> 
> Hi, review my other response in this thread.
> 
> This is undefined. Some headers do allow multiple values separated by comma in 
> the same header with the same meaning as varios headers with same name and 
> single values. But this depends on each header specification.

>From reading rfc 2616, section 4.2 that all multi-value headers can be
combined with commas without changing semantics of the message[1].
There's also no other way (e.g. with an Array) to represent them for
Rack...

[1] In the Real-World(TM), this is not true for Set-Cookie headers in
    HTTP responses.

> In your case it seems valid for me (just an opinnion) as 
> "HTTP_X_FORWARDED_PROTO: http,https" could mean that the request has been sent 
> using HTTPS and an intermediary proxy has forwarded it using HTTP. Of course 
> the final destination (Unicorn application) must be ready to support such 
> syntax.

Is it safe to say that if there's an "https" *anywhere* in the
X-Forwarded-Proto chain, that "rack.url_scheme" should be set to
"https"?   Because I suppose most of the time there's only one
(client-facing) proxy using https.

-- 
Eric Wong

From normalperson at yhbt.net  Sun Jan 10 01:32:18 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Sat, 9 Jan 2010 22:32:18 -0800
Subject: X-Forwarded-Proto / X_FORWARDED_PROTO
In-Reply-To: <20100110061209.GA1685@dcvr.yhbt.net>
References: <20100109221656.GA8846@waste.org>
	<20100109223903.GC7480@dcvr.yhbt.net>
	<20100109232903.GB8846@waste.org>
	<201001100045.37677.ibc@aliax.net>
	<20100110061209.GA1685@dcvr.yhbt.net>
Message-ID: <20100110063218.GA3102@dcvr.yhbt.net>

Eric Wong <normalperson at yhbt.net> wrote:
> I?aki Baz Castillo <ibc at aliax.net> wrote:
> > In your case it seems valid for me (just an opinnion) as 
> > "HTTP_X_FORWARDED_PROTO: http,https" could mean that the request has been sent 
> > using HTTPS and an intermediary proxy has forwarded it using HTTP. Of course 
> > the final destination (Unicorn application) must be ready to support such 
> > syntax.
> 
> Is it safe to say that if there's an "https" *anywhere* in the
> X-Forwarded-Proto chain, that "rack.url_scheme" should be set to
> "https"?   Because I suppose most of the time there's only one
> (client-facing) proxy using https.

Maybe this will work...

diff --git a/ext/unicorn_http/global_variables.h b/ext/unicorn_http/global_variables.h
index e593cf6..6705851 100644
--- a/ext/unicorn_http/global_variables.h
+++ b/ext/unicorn_http/global_variables.h
@@ -74,7 +74,6 @@ void init_globals(void)
   DEF_GLOBAL(server_name, "SERVER_NAME");
   DEF_GLOBAL(server_port, "SERVER_PORT");
   DEF_GLOBAL(server_protocol, "SERVER_PROTOCOL");
-  DEF_GLOBAL(http_x_forwarded_proto, "HTTP_X_FORWARDED_PROTO");
   DEF_GLOBAL(port_80, "80");
   DEF_GLOBAL(port_443, "443");
   DEF_GLOBAL(localhost, "localhost");
diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
index 6232e2c..f3945b2 100644
--- a/ext/unicorn_http/unicorn_http.rl
+++ b/ext/unicorn_http/unicorn_http.rl
@@ -197,6 +197,14 @@ static void write_value(VALUE req, struct http_parser *hp,
     assert_frozen(f);
   }
 
+  /*
+   * any X-Forwarded-Proto: https means there's an https server in the
+   * proxy chain, and that server is most likely the one that actually
+   * sees the client, so help Rack apps generate URLs with "https"
+   */
+  if (f == g_http_x_forwarded_proto && STR_CSTR_EQ(v, "https"))
+    rb_hash_aset(req, g_rack_url_scheme, v);
+
   e = rb_hash_aref(req, f);
   if (NIL_P(e)) {
     hp->cont = rb_hash_aset(req, f, v);
@@ -393,12 +401,7 @@ static void finalize_header(struct http_parser *hp, VALUE req)
 
   /* set rack.url_scheme to "https" or "http", no others are allowed by Rack */
   if (NIL_P(temp)) {
-    temp = rb_hash_aref(req, g_http_x_forwarded_proto);
-    if (!NIL_P(temp) && STR_CSTR_EQ(temp, "https"))
-      server_port = g_port_443;
-    else
-      temp = g_http;
-    rb_hash_aset(req, g_rack_url_scheme, temp);
+    rb_hash_aset(req, g_rack_url_scheme, g_http);
   } else if (STR_CSTR_EQ(temp, "https")) {
     server_port = g_port_443;
   } else {
@@ -712,5 +715,6 @@ void Init_unicorn_http(void)
   SET_GLOBAL(g_http_transfer_encoding, "TRANSFER_ENCODING");
   SET_GLOBAL(g_content_length, "CONTENT_LENGTH");
   SET_GLOBAL(g_http_connection, "CONNECTION");
+  SET_GLOBAL(g_http_x_forwarded_proto, "X_FORWARDED_PROTO");
 }
 #undef SET_GLOBAL
-- 
Eric Wong

From hukl at berlin.ccc.de  Sun Jan 10 06:43:55 2010
From: hukl at berlin.ccc.de (John-Paul Bader)
Date: Sun, 10 Jan 2010 12:43:55 +0100
Subject: X-Forwarded-Proto / X_FORWARDED_PROTO
In-Reply-To: <20100110063218.GA3102@dcvr.yhbt.net>
References: <20100109221656.GA8846@waste.org>
	<20100109223903.GC7480@dcvr.yhbt.net>
	<20100109232903.GB8846@waste.org>
	<201001100045.37677.ibc@aliax.net>
	<20100110061209.GA1685@dcvr.yhbt.net>
	<20100110063218.GA3102@dcvr.yhbt.net>
Message-ID: <DEF84685-7CA8-4262-801C-3EFF6672FBC4@berlin.ccc.de>

Hey Guys,

I posted my setup before but for this thread I'll do it again.

For http I have: varnish(80) -> nginx(8080) -> unicorn

And for https: nginx(443) -> varnish(80) -> nginx(8080) -> unicorn

Nginx has these proxy settings for the 443 vhost: 

location / {
      ?
      access_log         off;
      proxy_redirect     off;
      proxy_pass         http://www.ccc.de;
      proxy_set_header   X-Real-IP  $remote_addr;
      proxy_set_header   X-FORWARDED-PROTO https;
    }
?

So I explicitly set this header. This way varnish(80), nginx(8080), unicorn and ultimately rails will get this Header and know that this was a https request. This works for me perfectly and maybe that is helpful for you ? If not never mind ;)

After all this header is a non standard extension introduced by squid. Theres a nice wikipedia page about it: http://en.wikipedia.org/wiki/X-Forwarded-For

Kind regards, John


On 10.01.2010, at 07:32, Eric Wong wrote:

> Eric Wong <normalperson at yhbt.net> wrote:
>> I?aki Baz Castillo <ibc at aliax.net> wrote:
>>> In your case it seems valid for me (just an opinnion) as 
>>> "HTTP_X_FORWARDED_PROTO: http,https" could mean that the request has been sent 
>>> using HTTPS and an intermediary proxy has forwarded it using HTTP. Of course 
>>> the final destination (Unicorn application) must be ready to support such 
>>> syntax.
>> 
>> Is it safe to say that if there's an "https" *anywhere* in the
>> X-Forwarded-Proto chain, that "rack.url_scheme" should be set to
>> "https"?   Because I suppose most of the time there's only one
>> (client-facing) proxy using https.
> 
> Maybe this will work...
> 
> diff --git a/ext/unicorn_http/global_variables.h b/ext/unicorn_http/global_variables.h
> index e593cf6..6705851 100644
> --- a/ext/unicorn_http/global_variables.h
> +++ b/ext/unicorn_http/global_variables.h
> @@ -74,7 +74,6 @@ void init_globals(void)
>   DEF_GLOBAL(server_name, "SERVER_NAME");
>   DEF_GLOBAL(server_port, "SERVER_PORT");
>   DEF_GLOBAL(server_protocol, "SERVER_PROTOCOL");
> -  DEF_GLOBAL(http_x_forwarded_proto, "HTTP_X_FORWARDED_PROTO");
>   DEF_GLOBAL(port_80, "80");
>   DEF_GLOBAL(port_443, "443");
>   DEF_GLOBAL(localhost, "localhost");
> diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
> index 6232e2c..f3945b2 100644
> --- a/ext/unicorn_http/unicorn_http.rl
> +++ b/ext/unicorn_http/unicorn_http.rl
> @@ -197,6 +197,14 @@ static void write_value(VALUE req, struct http_parser *hp,
>     assert_frozen(f);
>   }
> 
> +  /*
> +   * any X-Forwarded-Proto: https means there's an https server in the
> +   * proxy chain, and that server is most likely the one that actually
> +   * sees the client, so help Rack apps generate URLs with "https"
> +   */
> +  if (f == g_http_x_forwarded_proto && STR_CSTR_EQ(v, "https"))
> +    rb_hash_aset(req, g_rack_url_scheme, v);
> +
>   e = rb_hash_aref(req, f);
>   if (NIL_P(e)) {
>     hp->cont = rb_hash_aset(req, f, v);
> @@ -393,12 +401,7 @@ static void finalize_header(struct http_parser *hp, VALUE req)
> 
>   /* set rack.url_scheme to "https" or "http", no others are allowed by Rack */
>   if (NIL_P(temp)) {
> -    temp = rb_hash_aref(req, g_http_x_forwarded_proto);
> -    if (!NIL_P(temp) && STR_CSTR_EQ(temp, "https"))
> -      server_port = g_port_443;
> -    else
> -      temp = g_http;
> -    rb_hash_aset(req, g_rack_url_scheme, temp);
> +    rb_hash_aset(req, g_rack_url_scheme, g_http);
>   } else if (STR_CSTR_EQ(temp, "https")) {
>     server_port = g_port_443;
>   } else {
> @@ -712,5 +715,6 @@ void Init_unicorn_http(void)
>   SET_GLOBAL(g_http_transfer_encoding, "TRANSFER_ENCODING");
>   SET_GLOBAL(g_content_length, "CONTENT_LENGTH");
>   SET_GLOBAL(g_http_connection, "CONNECTION");
> +  SET_GLOBAL(g_http_x_forwarded_proto, "X_FORWARDED_PROTO");
> }
> #undef SET_GLOBAL
> -- 
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
> 


From peter.kieltyka at gmail.com  Mon Jan 11 17:27:51 2010
From: peter.kieltyka at gmail.com (Peter Kieltyka)
Date: Mon, 11 Jan 2010 17:27:51 -0500
Subject: Unicorn 0.96 doesn't play nice with Rails 2.3.5
Message-ID: <64540A96-A12F-468F-91F8-B4872A333ABC@gmail.com>

Hello,

I've experienced a bug while trying to setup Unicorn 0.96.0 with my Rails 2.3.5 app. Some details on my system: I'm running Freebsd 8.0-p2 (amd64) with Ruby 1.9.1 and Rubygems 1.3.5.

Pretty much, I tried running "unicorn_rails" in the root of my rails application, but it never loads. What happens is the master process load then it keeps trying to start the worker app but fails. I kept getting the message that I need to install Rails 2.3.5. At fist I thought I had configured something incorrectly and there was a bad reference to my gems, but no. The error message came from my Rails config/boot.rb file which is called after the gem list has been refreshed (via app.call in Unicorn). I traced through unicorn until I found that in the load_rails_gem method of the GemBoot class was raising the load error. When checking the exception message variable it told me: 

$ unicorn_rails                     
I, [2010-01-11T16:20:11.330566 #23391]  INFO -- : listening on addr=0.0.0.0:8080 fd=3
I, [2010-01-11T16:20:11.337793 #23391]  INFO -- : worker=0 spawning...
I, [2010-01-11T16:20:11.339090 #23391]  INFO -- : master process ready
I, [2010-01-11T16:20:11.339225 #23393]  INFO -- : worker=0 spawned pid=23393
I, [2010-01-11T16:20:11.379570 #23393]  INFO -- : Refreshing Gem list
Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
I, [2010-01-11T16:20:11.446208 #23391]  INFO -- : reaped #<Process::Status: pid 23393 exit 1> worker=0
I, [2010-01-11T16:20:11.446461 #23391]  INFO -- : worker=0 spawning...
I, [2010-01-11T16:20:11.447539 #23394]  INFO -- : worker=0 spawned pid=23394
I, [2010-01-11T16:20:11.488722 #23394]  INFO -- : Refreshing Gem list
Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
... etc. ... keeps going on and on and on.

The exception message in GemBoot tells me:
can't activate rack (~> 1.0.0, runtime) for ["actionpack-2.3.5", "rails-2.3.5"], already activated rack-1.1.0 for ["unicorn-0.96.0"]

So, I uninstalled rack 1.1.0, leaving behind just rack 1.0.1. Started up the unicorn_rails again and voila. The issue seemed to be a conflict with the rack 1.1 already being loaded by Unicorn, and when Rails begins to load and checks its dependencies, the rack version is too new for 2.3.5.

I'm not sure what the fix is here, but uninstalling rack 1.1 probably is not the answer.

Cheers,

Peter

From normalperson at yhbt.net  Mon Jan 11 22:42:57 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 11 Jan 2010 19:42:57 -0800
Subject: Unicorn 0.96 doesn't play nice with Rails 2.3.5
In-Reply-To: <64540A96-A12F-468F-91F8-B4872A333ABC@gmail.com>
References: <64540A96-A12F-468F-91F8-B4872A333ABC@gmail.com>
Message-ID: <20100112034257.GA17971@dcvr.yhbt.net>

Peter Kieltyka <peter.kieltyka at gmail.com> wrote:
> Hello,
> 
> I've experienced a bug while trying to setup Unicorn 0.96.0 with my
> Rails 2.3.5 app. Some details on my system: I'm running Freebsd 8.0-p2
> (amd64) with Ruby 1.9.1 and Rubygems 1.3.5.
> 
> Pretty much, I tried running "unicorn_rails" in the root of my rails
> application, but it never loads. What happens is the master process
> load then it keeps trying to start the worker app but fails. I kept
> getting the message that I need to install Rails 2.3.5. At fist I
> thought I had configured something incorrectly and there was a bad
> reference to my gems, but no. The error message came from my Rails
> config/boot.rb file which is called after the gem list has been
> refreshed (via app.call in Unicorn). I traced through unicorn until I
> found that in the load_rails_gem method of the GemBoot class was
> raising the load error. When checking the exception message variable
> it told me: 
> 
> $ unicorn_rails                     
> I, [2010-01-11T16:20:11.330566 #23391]  INFO -- : listening on addr=0.0.0.0:8080 fd=3
> I, [2010-01-11T16:20:11.337793 #23391]  INFO -- : worker=0 spawning...
> I, [2010-01-11T16:20:11.339090 #23391]  INFO -- : master process ready
> I, [2010-01-11T16:20:11.339225 #23393]  INFO -- : worker=0 spawned pid=23393
> I, [2010-01-11T16:20:11.379570 #23393]  INFO -- : Refreshing Gem list
> Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
> I, [2010-01-11T16:20:11.446208 #23391]  INFO -- : reaped #<Process::Status: pid 23393 exit 1> worker=0
> I, [2010-01-11T16:20:11.446461 #23391]  INFO -- : worker=0 spawning...
> I, [2010-01-11T16:20:11.447539 #23394]  INFO -- : worker=0 spawned pid=23394
> I, [2010-01-11T16:20:11.488722 #23394]  INFO -- : Refreshing Gem list
> Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
> ... etc. ... keeps going on and on and on.
> 
> The exception message in GemBoot tells me:
> 
> can't activate rack (~> 1.0.0, runtime) for ["actionpack-2.3.5",
> "rails-2.3.5"], already activated rack-1.1.0 for ["unicorn-0.96.0"]
> 
> So, I uninstalled rack 1.1.0, leaving behind just rack 1.0.1. Started
> up the unicorn_rails again and voila. The issue seemed to be a
> conflict with the rack 1.1 already being loaded by Unicorn, and when
> Rails begins to load and checks its dependencies, the rack version is
> too new for 2.3.5.
> 
> I'm not sure what the fix is here, but uninstalling rack 1.1 probably
> is not the answer.

Hi Peter,

Uninstalling Rack 1.1 may be the best answer, especially
if you don't need Rack for anything else.

There are several problems at hand:

1. Rails 2.3.5 declares it is only compatible with Rack 1.0.x
2. Unicorn does not require any particular Rack version
3. RubyGems defaults to the latest version of any Gem if
   no version is explicitly specified.

Since Unicorn is loaded before Rails, RubyGems will default to
loading the latest version of Rack based on the Unicorn gemspec

You have several other options here to work around this,
they all suck (in no particular order of suckiness):

1. edit the installed Rails/active* gemspecs[1] so they're not
   pinned on on 1.0.x anymore (they *should* be compatible enough)

1a. vendorize Rails/active* and edit the gemspecs there

2. edit your copy of /path/to/unicorn_rails and
   activate Rack 1.0.x in there before Unicorn is activated
   (this is the "unicorn_rails" wrapper RubyGems creates for you,
   not the actual "unicorn_rails" source).

3. create a "vendored" copy of unicorn_rails for your
   application (script/my_unicorn) and activate Rack 1.0.x
   in there first.

4. install Unicorn using setup.rb without RubyGems
   (uninstalls/upgrades won't be as nice to deal with, though).

5. edit the installed Unicorn gemspec[1] to depend on Rack ~> 1.0


[1] - RubyGems gemspecs are here:
      $prefix/lib/ruby/gems/$RUBY_VERSION/specifications/*.gemspec

-- 
Eric Wong

From peter.kieltyka at gmail.com  Tue Jan 12 09:36:14 2010
From: peter.kieltyka at gmail.com (Peter Kieltyka)
Date: Tue, 12 Jan 2010 09:36:14 -0500
Subject: Unicorn 0.96 doesn't play nice with Rails 2.3.5
In-Reply-To: <20100112034257.GA17971@dcvr.yhbt.net>
References: <64540A96-A12F-468F-91F8-B4872A333ABC@gmail.com>
	<20100112034257.GA17971@dcvr.yhbt.net>
Message-ID: <FC196464-B53F-47C0-9936-F3D34373926A@gmail.com>

On 2010-01-11, at 10:42 PM, Eric Wong wrote:

> Peter Kieltyka <peter.kieltyka at gmail.com> wrote:
>> Hello,
>> 
>> I've experienced a bug while trying to setup Unicorn 0.96.0 with my
>> Rails 2.3.5 app. Some details on my system: I'm running Freebsd 8.0-p2
>> (amd64) with Ruby 1.9.1 and Rubygems 1.3.5.
>> 
>> Pretty much, I tried running "unicorn_rails" in the root of my rails
>> application, but it never loads. What happens is the master process
>> load then it keeps trying to start the worker app but fails. I kept
>> getting the message that I need to install Rails 2.3.5. At fist I
>> thought I had configured something incorrectly and there was a bad
>> reference to my gems, but no. The error message came from my Rails
>> config/boot.rb file which is called after the gem list has been
>> refreshed (via app.call in Unicorn). I traced through unicorn until I
>> found that in the load_rails_gem method of the GemBoot class was
>> raising the load error. When checking the exception message variable
>> it told me: 
>> 
>> $ unicorn_rails                     
>> I, [2010-01-11T16:20:11.330566 #23391]  INFO -- : listening on addr=0.0.0.0:8080 fd=3
>> I, [2010-01-11T16:20:11.337793 #23391]  INFO -- : worker=0 spawning...
>> I, [2010-01-11T16:20:11.339090 #23391]  INFO -- : master process ready
>> I, [2010-01-11T16:20:11.339225 #23393]  INFO -- : worker=0 spawned pid=23393
>> I, [2010-01-11T16:20:11.379570 #23393]  INFO -- : Refreshing Gem list
>> Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
>> I, [2010-01-11T16:20:11.446208 #23391]  INFO -- : reaped #<Process::Status: pid 23393 exit 1> worker=0
>> I, [2010-01-11T16:20:11.446461 #23391]  INFO -- : worker=0 spawning...
>> I, [2010-01-11T16:20:11.447539 #23394]  INFO -- : worker=0 spawned pid=23394
>> I, [2010-01-11T16:20:11.488722 #23394]  INFO -- : Refreshing Gem list
>> Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
>> ... etc. ... keeps going on and on and on.
>> 
>> The exception message in GemBoot tells me:
>> 
>> can't activate rack (~> 1.0.0, runtime) for ["actionpack-2.3.5",
>> "rails-2.3.5"], already activated rack-1.1.0 for ["unicorn-0.96.0"]
>> 
>> So, I uninstalled rack 1.1.0, leaving behind just rack 1.0.1. Started
>> up the unicorn_rails again and voila. The issue seemed to be a
>> conflict with the rack 1.1 already being loaded by Unicorn, and when
>> Rails begins to load and checks its dependencies, the rack version is
>> too new for 2.3.5.
>> 
>> I'm not sure what the fix is here, but uninstalling rack 1.1 probably
>> is not the answer.
> 
> Hi Peter,
> 
> Uninstalling Rack 1.1 may be the best answer, especially
> if you don't need Rack for anything else.
> 
> There are several problems at hand:
> 
> 1. Rails 2.3.5 declares it is only compatible with Rack 1.0.x
> 2. Unicorn does not require any particular Rack version
> 3. RubyGems defaults to the latest version of any Gem if
>   no version is explicitly specified.
> 
> Since Unicorn is loaded before Rails, RubyGems will default to
> loading the latest version of Rack based on the Unicorn gemspec
> 
> You have several other options here to work around this,
> they all suck (in no particular order of suckiness):
> 
> 1. edit the installed Rails/active* gemspecs[1] so they're not
>   pinned on on 1.0.x anymore (they *should* be compatible enough)
> 
> 1a. vendorize Rails/active* and edit the gemspecs there
> 
> 2. edit your copy of /path/to/unicorn_rails and
>   activate Rack 1.0.x in there before Unicorn is activated
>   (this is the "unicorn_rails" wrapper RubyGems creates for you,
>   not the actual "unicorn_rails" source).
> 
> 3. create a "vendored" copy of unicorn_rails for your
>   application (script/my_unicorn) and activate Rack 1.0.x
>   in there first.
> 
> 4. install Unicorn using setup.rb without RubyGems
>   (uninstalls/upgrades won't be as nice to deal with, though).
> 
> 5. edit the installed Unicorn gemspec[1] to depend on Rack ~> 1.0
> 
> 
> [1] - RubyGems gemspecs are here:
>      $prefix/lib/ruby/gems/$RUBY_VERSION/specifications/*.gemspec
> 
> -- 
> Eric Wong

Hey Eric,

Thanks for the quick reply.

That is pretty much what I expected. I plan to begin experimenting with Rails 3 for this project in any regards but I thought I'd bring it to your attention if others have the same issue (which some already did acknowledge it on #rubyonrails at irc.freenode.net).

Strange though, on my OSX 10.6 system with the same setup (Ruby 1.9.1, Gems 1.3.5, Rack 1.1.0 & 1.0.1, and Unicorn 0.96.0) the issue doesn't come up. Ohh well..

Peter


From ibc at aliax.net  Sat Jan 16 20:44:55 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Sun, 17 Jan 2010 02:44:55 +0100
Subject: Issue when sending USR2 too soon
Message-ID: <201001170244.55418.ibc@aliax.net>

Hi, I've realized that if I send USR2 very quicky to master process (each time 
to the new master process, of course) then sometimes Unicorn just dies totally 
and returns 0.

Not sure, but I think that those times Unicorn receives the USR2 signal prior 
to load the code in which such signal is handled, could it be?

Regards.

-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Sat Jan 16 20:52:28 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Sun, 17 Jan 2010 02:52:28 +0100
Subject: Issue when sending USR2 too soon
In-Reply-To: <201001170244.55418.ibc@aliax.net>
References: <201001170244.55418.ibc@aliax.net>
Message-ID: <201001170252.28706.ibc@aliax.net>

El Domingo, 17 de Enero de 2010, I?aki Baz Castillo escribi?:
> Hi, I've realized that if I send USR2 very quicky to master process (each
>  time to the new master process, of course) then sometimes Unicorn just
>  dies totally and returns 0.

Sorry, I'm not sure if it returns 0 or not in such case since I'm doing the 
check with a init script which performs the "reload" action by sending USR2 to 
master process (which runs in background).


> Not sure, but I think that those times Unicorn receives the USR2 signal
>  prior to load the code in which such signal is handled, could it be?

It makes sense since by default USR2 terminates a Ruby program.

What about if Unicorn very quicky prepares the trap for USR2 so in case it 
receives it soon when starting it ignores it (and logs some warning)?
Does it make sense?

-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Sat Jan 16 20:59:59 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Sun, 17 Jan 2010 02:59:59 +0100
Subject: Issue when sending USR2 too soon
In-Reply-To: <201001170252.28706.ibc@aliax.net>
References: <201001170244.55418.ibc@aliax.net>
	<201001170252.28706.ibc@aliax.net>
Message-ID: <201001170259.59678.ibc@aliax.net>

El Domingo, 17 de Enero de 2010, I?aki Baz Castillo escribi?:
> What about if Unicorn very quicky prepares the trap for USR2 so in case it 
> receives it soon when starting it ignores it (and logs some warning)?
> Does it make sense?

Adding the following on the top of bin/unicorn solves the problem:

  # Ignore USR2 (instead of terminating script) in case it arrives too soon.
  trap("USR2") do
    $stderr.puts "WARN: USR2 signal (reload action) received too soon, ignored"
  end


-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Mon Jan 18 06:04:47 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 18 Jan 2010 11:04:47 +0000
Subject: Issue when sending USR2 too soon
In-Reply-To: <201001170259.59678.ibc@aliax.net>
References: <201001170244.55418.ibc@aliax.net>
	<201001170252.28706.ibc@aliax.net>
	<201001170259.59678.ibc@aliax.net>
Message-ID: <20100118110446.GA4625@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Domingo, 17 de Enero de 2010, I?aki Baz Castillo escribi?:
> > What about if Unicorn very quicky prepares the trap for USR2 so in case it 
> > receives it soon when starting it ignores it (and logs some warning)?
> > Does it make sense?
> 
> Adding the following on the top of bin/unicorn solves the problem:
> 
>   # Ignore USR2 (instead of terminating script) in case it arrives too soon.
>   trap("USR2") do
>     $stderr.puts "WARN: USR2 signal (reload action) received too soon, ignored"
>   end

No it doesn't, it just reduces the window where signals aren't setup.
On systems where Unicorn is installed as a RubyGem, that window of time
may be much bigger.

Is the issue with a script reading the pid file and sending the signal
because it exists?

In that case, it might be better to drop the pid file after
setting up signal handlers, but I seem to recall some tools wanting
the pid file earlier...  I'll get back to this on Tuesday.

-- 
Eric Wong

From ibc at aliax.net  Mon Jan 18 06:25:28 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Mon, 18 Jan 2010 12:25:28 +0100
Subject: Issue when sending USR2 too soon
In-Reply-To: <20100118110446.GA4625@dcvr.yhbt.net>
References: <201001170244.55418.ibc@aliax.net>
	<201001170259.59678.ibc@aliax.net>
	<20100118110446.GA4625@dcvr.yhbt.net>
Message-ID: <201001181225.28388.ibc@aliax.net>

El Lunes, 18 de Enero de 2010, Eric Wong escribi?:
> I?aki Baz Castillo <ibc at aliax.net> wrote:
> > El Domingo, 17 de Enero de 2010, I?aki Baz Castillo escribi?:
> > > What about if Unicorn very quicky prepares the trap for USR2 so in case
> > > it receives it soon when starting it ignores it (and logs some
> > > warning)? Does it make sense?
> >
> > Adding the following on the top of bin/unicorn solves the problem:
> >
> >   # Ignore USR2 (instead of terminating script) in case it arrives too
> > soon. trap("USR2") do
> >     $stderr.puts "WARN: USR2 signal (reload action) received too soon,
> > ignored" end
> 
> No it doesn't, it just reduces the window where signals aren't setup.
> On systems where Unicorn is installed as a RubyGem, that window of time
> may be much bigger.

But note that I've added:

  trap("USR2") do
    $stderr.puts "WARN: USR2 signal (reload action) received too soon,
                  ignored"
  end

at the beginning of the executable, so the time window is really reduced. Also 
note that in my modified version of the executable I load many other gems and 
them make the unicorn loading slower, increasing the time window.

 
> Is the issue with a script reading the pid file and sending the signal
> because it exists?

I've done a init script to start/stop/reload/log-rotate Unicorn. The action 
"log-rotate" takes the PID from the pid file and sends a USR1 (this call is 
made by logrotate). The action "reload" sends a USR2 and later a TERM 
(depending on if the PID has changed or not).

I just want to avoid that, in case a user invokes "reload" twice too fast, 
Unicorn receives USR2 before preparing it so it dies. With my "hack" is more 
difficult such possibility to occur.
 

Thanks.

-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Mon Jan 18 06:43:34 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 18 Jan 2010 11:43:34 +0000
Subject: Issue when sending USR2 too soon
In-Reply-To: <201001181225.28388.ibc@aliax.net>
References: <201001170244.55418.ibc@aliax.net>
	<201001170259.59678.ibc@aliax.net>
	<20100118110446.GA4625@dcvr.yhbt.net>
	<201001181225.28388.ibc@aliax.net>
Message-ID: <20100118114334.GC4625@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Lunes, 18 de Enero de 2010, Eric Wong escribi?:
> > I?aki Baz Castillo <ibc at aliax.net> wrote:
> > > El Domingo, 17 de Enero de 2010, I?aki Baz Castillo escribi?:
> > > > What about if Unicorn very quicky prepares the trap for USR2 so in case
> > > > it receives it soon when starting it ignores it (and logs some
> > > > warning)? Does it make sense?
> > >
> > > Adding the following on the top of bin/unicorn solves the problem:
> > >
> > >   # Ignore USR2 (instead of terminating script) in case it arrives too
> > > soon. trap("USR2") do
> > >     $stderr.puts "WARN: USR2 signal (reload action) received too soon,
> > > ignored" end
> > 
> > No it doesn't, it just reduces the window where signals aren't setup.
> > On systems where Unicorn is installed as a RubyGem, that window of time
> > may be much bigger.
> 
> But note that I've added:
> 
>   trap("USR2") do
>     $stderr.puts "WARN: USR2 signal (reload action) received too soon,
>                   ignored"
>   end
> 
> at the beginning of the executable, so the time window is really reduced. Also 
> note that in my modified version of the executable I load many other gems and 
> them make the unicorn loading slower, increasing the time window.

On systems with RubyGems, the executable is always an automatically
generated wrapper script that does "require 'rubygems'" first.

> > Is the issue with a script reading the pid file and sending the signal
> > because it exists?
> 
> I've done a init script to start/stop/reload/log-rotate Unicorn. The action 
> "log-rotate" takes the PID from the pid file and sends a USR1 (this call is 
> made by logrotate). The action "reload" sends a USR2 and later a TERM 
> (depending on if the PID has changed or not).
> 
> I just want to avoid that, in case a user invokes "reload" twice too fast, 
> Unicorn receives USR2 before preparing it so it dies. With my "hack" is more 
> difficult such possibility to occur.

I'll look into what the consequences of writing the pid file after
signal handlers are setup later this week.

Unfortunately pid files are inherently racy for the USR2 case.  Perhaps
your script can check the ctime of the pid file...

-- 
Eric Wong

From ibc at aliax.net  Mon Jan 18 06:48:49 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Mon, 18 Jan 2010 12:48:49 +0100
Subject: Issue when sending USR2 too soon
In-Reply-To: <20100118114334.GC4625@dcvr.yhbt.net>
References: <201001170244.55418.ibc@aliax.net>
	<201001181225.28388.ibc@aliax.net>
	<20100118114334.GC4625@dcvr.yhbt.net>
Message-ID: <201001181248.49502.ibc@aliax.net>

El Lunes, 18 de Enero de 2010, Eric Wong escribi?:

> On systems with RubyGems, the executable is always an automatically
> generated wrapper script that does "require 'rubygems'" first.

True, but I use Ruby 1.9 which doesn't require using "rubygems". However it's 
of course true that the binary does a "gem_load_bin('unicorn'..)" and so.

But in my case, for now, I use directly the *real* executable rather than the 
executable that Gem puts in /usr/bin or /usr/local/bin.

> > > Is the issue with a script reading the pid file and sending the signal
> > > because it exists?
> >
> > I've done a init script to start/stop/reload/log-rotate Unicorn. The
> > action "log-rotate" takes the PID from the pid file and sends a USR1
> > (this call is made by logrotate). The action "reload" sends a USR2 and
> > later a TERM (depending on if the PID has changed or not).
> >
> > I just want to avoid that, in case a user invokes "reload" twice too
> > fast, Unicorn receives USR2 before preparing it so it dies. With my
> > "hack" is more difficult such possibility to occur.
> 
> I'll look into what the consequences of writing the pid file after
> signal handlers are setup later this week.

Hum, yes, that sound good.


> Unfortunately pid files are inherently racy for the USR2 case.  Perhaps
> your script can check the ctime of the pid file...

Good suggestion :)


-- 
I?aki Baz Castillo <ibc at aliax.net>

From steve at steveklabnik.com  Tue Jan 19 16:42:33 2010
From: steve at steveklabnik.com (Steve Klabnik)
Date: Tue, 19 Jan 2010 21:42:33 +0000
Subject: Unicorn can't find gems?
Message-ID: <dd81403b1001191342w53e32dddn8499e8399540a876@mail.gmail.com>

Hello all. I've run into a strange problem with Unicorn, and Google
couldn't help me out. Basically, I'm going to switch from Passenger to
Unicorn for the next release of my Rails app. I installed it on my
local machine, and it ran great. But then, as I was building my new
server to deploy on, I ran into a snag.

When I run 'unicorn_rails', I got a strange error. Here's the output
of 'unicorn_rails -d' :

$ unicorn_rails -d
{:daemonize=>false,
 :unicorn_options=>{:listeners=>[]},
 :app=>
  #<Proc:0x000000000216dad0@/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/unicorn-0.96.0/bin/unicorn_rails:124>}
Exception `Errno::EEXIST' at
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/1.8/fileutils.rb:243 -
File exists - tmp/cache
Exception `Errno::EEXIST' at
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/1.8/fileutils.rb:243 -
File exists - tmp/pids
Exception `Errno::EEXIST' at
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/1.8/fileutils.rb:243 -
File exists - tmp/sessions
Exception `Errno::EEXIST' at
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/1.8/fileutils.rb:243 -
File exists - tmp/sockets
I, [2010-01-19T21:25:09.748670 #30184]  INFO -- : listening on
addr=0.0.0.0:8080 fd=3
Exception `LoadError' at
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31
- no such file to load -- Win32API
Exception `LoadError' at
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:38
- no such file to load -- Win32API
I, [2010-01-19T21:25:09.758670 #30184]  INFO -- : worker=0 spawning...
I, [2010-01-19T21:25:09.768670 #30184]  INFO -- : master process ready
I, [2010-01-19T21:25:09.768670 #30185]  INFO -- : worker=0 spawned pid=30185
I, [2010-01-19T21:25:09.828670 #30185]  INFO -- : Refreshing Gem list
Exception `Gem::LoadError' at
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/site_ruby/1.8/rubygems.rb:280
- can't activate rack (~> 1.0.0, runtime) for ["actionpack-2.3.5",
"rails-2.3.5"], already activated rack-1.1.0 for ["unicorn-0.96.0"]
Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`,
update your RAILS_GEM_VERSION setting in config/environment.rb for the
Rails version you do have installed, or comment out RAILS_GEM_VERSION
to use the latest version installed.
Exception `Errno::EAGAIN' at
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/unicorn-0.96.0/lib/unicorn.rb:436
- Resource temporarily unavailable
I, [2010-01-19T21:25:09.918670 #30184]  INFO -- : reaped
#<Process::Status: pid=30185,exited(1)> worker=0
Exception `Errno::ECHILD' at
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/unicorn-0.96.0/lib/unicorn.rb:454
- No child processes

And then it just repeats.

I'm using ree both locally and on the server (as you can see), but I'm
using it through rvm locally.

Any idea what this might be?

Thanks,
Steve

From normalperson at yhbt.net  Tue Jan 19 16:43:47 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 19 Jan 2010 13:43:47 -0800
Subject: X-Forwarded-Proto / X_FORWARDED_PROTO
In-Reply-To: <20100110063218.GA3102@dcvr.yhbt.net>
References: <20100109221656.GA8846@waste.org>
	<20100109223903.GC7480@dcvr.yhbt.net>
	<20100109232903.GB8846@waste.org>
	<201001100045.37677.ibc@aliax.net>
	<20100110061209.GA1685@dcvr.yhbt.net>
	<20100110063218.GA3102@dcvr.yhbt.net>
Message-ID: <20100119214347.GA8301@dcvr.yhbt.net>

Eric Wong <normalperson at yhbt.net> wrote:
> Eric Wong <normalperson at yhbt.net> wrote:
> > I?aki Baz Castillo <ibc at aliax.net> wrote:
> > > In your case it seems valid for me (just an opinnion) as 
> > > "HTTP_X_FORWARDED_PROTO: http,https" could mean that the request has been sent 
> > > using HTTPS and an intermediary proxy has forwarded it using HTTP. Of course 
> > > the final destination (Unicorn application) must be ready to support such 
> > > syntax.
> > 
> > Is it safe to say that if there's an "https" *anywhere* in the
> > X-Forwarded-Proto chain, that "rack.url_scheme" should be set to
> > "https"?   Because I suppose most of the time there's only one
> > (client-facing) proxy using https.
> 
> Maybe this will work...

Any feedback on this patch would be appreciated, thanks.

> diff --git a/ext/unicorn_http/global_variables.h b/ext/unicorn_http/global_variables.h
> index e593cf6..6705851 100644
> --- a/ext/unicorn_http/global_variables.h
> +++ b/ext/unicorn_http/global_variables.h
> @@ -74,7 +74,6 @@ void init_globals(void)
>    DEF_GLOBAL(server_name, "SERVER_NAME");
>    DEF_GLOBAL(server_port, "SERVER_PORT");
>    DEF_GLOBAL(server_protocol, "SERVER_PROTOCOL");
> -  DEF_GLOBAL(http_x_forwarded_proto, "HTTP_X_FORWARDED_PROTO");
>    DEF_GLOBAL(port_80, "80");
>    DEF_GLOBAL(port_443, "443");
>    DEF_GLOBAL(localhost, "localhost");
> diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
> index 6232e2c..f3945b2 100644
> --- a/ext/unicorn_http/unicorn_http.rl
> +++ b/ext/unicorn_http/unicorn_http.rl
> @@ -197,6 +197,14 @@ static void write_value(VALUE req, struct http_parser *hp,
>      assert_frozen(f);
>    }
>  
> +  /*
> +   * any X-Forwarded-Proto: https means there's an https server in the
> +   * proxy chain, and that server is most likely the one that actually
> +   * sees the client, so help Rack apps generate URLs with "https"
> +   */
> +  if (f == g_http_x_forwarded_proto && STR_CSTR_EQ(v, "https"))
> +    rb_hash_aset(req, g_rack_url_scheme, v);
> +
>    e = rb_hash_aref(req, f);
>    if (NIL_P(e)) {
>      hp->cont = rb_hash_aset(req, f, v);
> @@ -393,12 +401,7 @@ static void finalize_header(struct http_parser *hp, VALUE req)
>  
>    /* set rack.url_scheme to "https" or "http", no others are allowed by Rack */
>    if (NIL_P(temp)) {
> -    temp = rb_hash_aref(req, g_http_x_forwarded_proto);
> -    if (!NIL_P(temp) && STR_CSTR_EQ(temp, "https"))
> -      server_port = g_port_443;
> -    else
> -      temp = g_http;
> -    rb_hash_aset(req, g_rack_url_scheme, temp);
> +    rb_hash_aset(req, g_rack_url_scheme, g_http);
>    } else if (STR_CSTR_EQ(temp, "https")) {
>      server_port = g_port_443;
>    } else {
> @@ -712,5 +715,6 @@ void Init_unicorn_http(void)
>    SET_GLOBAL(g_http_transfer_encoding, "TRANSFER_ENCODING");
>    SET_GLOBAL(g_content_length, "CONTENT_LENGTH");
>    SET_GLOBAL(g_http_connection, "CONNECTION");
> +  SET_GLOBAL(g_http_x_forwarded_proto, "X_FORWARDED_PROTO");
>  }
>  #undef SET_GLOBAL
-- 
Eric Wong

From normalperson at yhbt.net  Tue Jan 19 16:48:13 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 19 Jan 2010 13:48:13 -0800
Subject: Unicorn can't find gems?
In-Reply-To: <dd81403b1001191342w53e32dddn8499e8399540a876@mail.gmail.com>
References: <dd81403b1001191342w53e32dddn8499e8399540a876@mail.gmail.com>
Message-ID: <20100119214813.GB8301@dcvr.yhbt.net>

Steve Klabnik <steve at steveklabnik.com> wrote:
> Hello all. I've run into a strange problem with Unicorn, and Google
> couldn't help me out. Basically, I'm going to switch from Passenger to
> Unicorn for the next release of my Rails app. I installed it on my
> local machine, and it ran great. But then, as I was building my new
> server to deploy on, I ran into a snag.
> 
> When I run 'unicorn_rails', I got a strange error. Here's the output
> of 'unicorn_rails -d' :
> 
> $ unicorn_rails -d

<snip>

> Exception `Gem::LoadError' at
> /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/site_ruby/1.8/rubygems.rb:280
> - can't activate rack (~> 1.0.0, runtime) for ["actionpack-2.3.5",
> "rails-2.3.5"], already activated rack-1.1.0 for ["unicorn-0.96.0"]
> Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`,
> update your RAILS_GEM_VERSION setting in config/environment.rb for the
> Rails version you do have installed, or comment out RAILS_GEM_VERSION
> to use the latest version installed.
> Exception `Errno::EAGAIN' at
> /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/unicorn-0.96.0/lib/unicorn.rb:436
> - Resource temporarily unavailable
> I, [2010-01-19T21:25:09.918670 #30184]  INFO -- : reaped
> #<Process::Status: pid=30185,exited(1)> worker=0
> Exception `Errno::ECHILD' at
> /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/unicorn-0.96.0/lib/unicorn.rb:454
> - No child processes
> 
> And then it just repeats.
> 
> I'm using ree both locally and on the server (as you can see), but I'm
> using it through rvm locally.
> 
> Any idea what this might be?

Hi Steve,

Rails 2.3.x isn't compatible with Rack 1.1.0.  Unicorn depends on Rack
and since Unicorn is always loaded before Rails is, the Unicorn
dependency will always load the newest version of Rack on the system.

Uninstalling Rack 1.1.0 for now should fix things, just keep Rack 1.0.1
installed).

I'll add this to the FAQ for now, not sure if there's a sane solution
for it...

-- 
Eric Wong

From ibc at aliax.net  Tue Jan 19 16:50:45 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Tue, 19 Jan 2010 22:50:45 +0100
Subject: Unicorn 0.96 doesn't play nice with Rails 2.3.5
In-Reply-To: <64540A96-A12F-468F-91F8-B4872A333ABC@gmail.com>
References: <64540A96-A12F-468F-91F8-B4872A333ABC@gmail.com>
Message-ID: <201001192250.45375.ibc@aliax.net>

El Lunes, 11 de Enero de 2010, Peter Kieltyka escribi?:

> The exception message in GemBoot tells me:
> can't activate rack (~> 1.0.0, runtime) for ["actionpack-2.3.5",
>  "rails-2.3.5"], already activated rack-1.1.0 for ["unicorn-0.96.0"]
> 
> So, I uninstalled rack 1.1.0, leaving behind just rack 1.0.1. Started up
>  the unicorn_rails again and voila. The issue seemed to be a conflict with
>  the rack 1.1 already being loaded by Unicorn, and when Rails begins to
>  load and checks its dependencies, the rack version is too new for 2.3.5.
> 
> I'm not sure what the fix is here, but uninstalling rack 1.1 probably is
>  not the answer.

I think this bug is related to a recent thread:

  http://www.mail-archive.com/mongrel-unicorn at rubyforge.org/msg00329.html


-- 
I?aki Baz Castillo <ibc at aliax.net>

From dvdplm at gmail.com  Tue Jan 19 16:56:16 2010
From: dvdplm at gmail.com (David Palm)
Date: Tue, 19 Jan 2010 22:56:16 +0100
Subject: Unicorn can't find gems?
In-Reply-To: <dd81403b1001191342w53e32dddn8499e8399540a876@mail.gmail.com>
References: <dd81403b1001191342w53e32dddn8499e8399540a876@mail.gmail.com>
Message-ID: <20100119225616946539.2b2c1329@gmail.com>

> Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`,
> update your RAILS_GEM_VERSION setting in config/environment.rb for the
> Rails version you do have installed, or comment out RAILS_GEM_VERSION
> to use the latest version installed.

I think I know what this is.

I went nuts on this today. It's Rack 1.1 that is messing with you and stupid stupid rails that, in boot.rb has this:

	rescue Gem::LoadError => load_error
	      $stderr.puts %(Missing the Rails #{version} gem. Please `gem install -v=#{version} rails`... ...
	end

Rails assumes the only reason for failure is a missing Rails install, but if you throw in a #{load_error.inspect} in there the Rack issue shows up loud and clear...

Always always include the actual error in a log statement when you rescue!

:)

> Exception `Errno::EAGAIN' at
> /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/unicorn-0.96.0/lib/unicorn.rb:436
> - Resource temporarily unavailable
> I, [2010-01-19T21:25:09.918670 #30184]  INFO -- : reaped
> #<Process::Status: pid=30185,exited(1)> worker=0
> Exception `Errno::ECHILD' at
> /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/unicorn-0.96.0/lib/unicorn.rb:454
> - No child processes
> 
> And then it just repeats.
> 
> I'm using ree both locally and on the server (as you can see), but I'm
> using it through rvm locally.
> 
> Any idea what this might be?
> 
> Thanks,
> Steve
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying

From steve at steveklabnik.com  Tue Jan 19 17:01:56 2010
From: steve at steveklabnik.com (Steve Klabnik)
Date: Tue, 19 Jan 2010 22:01:56 +0000
Subject: Unicorn can't find gems?
In-Reply-To: <20100119214813.GB8301@dcvr.yhbt.net>
References: <dd81403b1001191342w53e32dddn8499e8399540a876@mail.gmail.com>
	<20100119214813.GB8301@dcvr.yhbt.net>
Message-ID: <dd81403b1001191401j1ace18b7yb7a7fe8e97071501@mail.gmail.com>

Ah! Go figure. The new install gave me both versions of rack.

Thanks for the quick reply, all is well now.

-Steve

From normalperson at yhbt.net  Tue Jan 19 21:38:26 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 19 Jan 2010 18:38:26 -0800
Subject: [PATCH] initialize signal handlers before writing pid file
In-Reply-To: <20100118114334.GC4625@dcvr.yhbt.net>
References: <201001170244.55418.ibc@aliax.net>
	<201001170259.59678.ibc@aliax.net>
	<20100118110446.GA4625@dcvr.yhbt.net>
	<201001181225.28388.ibc@aliax.net>
	<20100118114334.GC4625@dcvr.yhbt.net>
Message-ID: <20100120023826.GA25165@dcvr.yhbt.net>

This prevents trigger-happy init scripts from reading the pid
file (and thus sending signals) to a not-fully initialized
master process to handle them.

This does NOT fix anything if other processes are sending
signals prematurely without relying on the presence of the pid
file.  It's not possible to prevent all cases of this in one
process, even in a purely C application, so we won't bother
trying.

We continue to always defer signal handling to the main loop
anyways, and signals sent to the master process will be
deferred/ignored until Unicorn::HttpServer#join is run.
---
  Eric Wong <normalperson at yhbt.net> wrote:
  > I'll look into what the consequences of writing the pid file after
  > signal handlers are setup later this week.

  I just pushed this out to git://git.bogomips.org/unicorn.git

  There were some test cases (that shouldn't be called "unit tests")
  that needed to be fixed but I don't think anything else is broken,
  but be sure to let us know either way.

 lib/unicorn.rb            |   16 +++++++++++-----
 test/test_helper.rb       |    5 +++++
 test/unit/test_server.rb  |    1 +
 test/unit/test_signals.rb |    6 +++++-
 test/unit/test_upload.rb  |    1 +
 5 files changed, 23 insertions(+), 6 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 7a1ef34..e3e4315 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -210,7 +210,18 @@ module Unicorn
       end
       config_listeners.each { |addr| listen(addr) }
       raise ArgumentError, "no listeners" if LISTENERS.empty?
+
+      # this pipe is used to wake us up from select(2) in #join when signals
+      # are trapped.  See trap_deferred.
+      init_self_pipe!
+
+      # setup signal handlers before writing pid file in case people get
+      # trigger happy and send signals as soon as the pid file exists.
+      # Note that signals don't actually get handled until the #join method
+      QUEUE_SIGS.each { |sig| trap_deferred(sig) }
+      trap(:CHLD) { |_| awaken_master }
       self.pid = config[:pid]
+
       self.master_pid = $$
       build_app! if preload_app
       maintain_worker_count
@@ -322,14 +333,9 @@ module Unicorn
     # one-at-a-time time and we'll happily drop signals in case somebody
     # is signalling us too often.
     def join
-      # this pipe is used to wake us up from select(2) in #join when signals
-      # are trapped.  See trap_deferred
-      init_self_pipe!
       respawn = true
       last_check = Time.now
 
-      QUEUE_SIGS.each { |sig| trap_deferred(sig) }
-      trap(:CHLD) { |sig_nr| awaken_master }
       proc_name 'master'
       logger.info "master process ready" # test_exec.rb relies on this message
       if ready_pipe
diff --git a/test/test_helper.rb b/test/test_helper.rb
index 3bdbeb1..5b750ee 100644
--- a/test/test_helper.rb
+++ b/test/test_helper.rb
@@ -294,3 +294,8 @@ def chunked_spawn(stdout, *cmd)
     end while true
   }
 end
+
+def reset_sig_handlers
+  sigs = %w(CHLD).concat(Unicorn::HttpServer::QUEUE_SIGS)
+  sigs.each { |sig| trap(sig, "DEFAULT") }
+end
diff --git a/test/unit/test_server.rb b/test/unit/test_server.rb
index 00705d0..36333a0 100644
--- a/test/unit/test_server.rb
+++ b/test/unit/test_server.rb
@@ -41,6 +41,7 @@ class WebServerTest < Test::Unit::TestCase
       File.truncate("test_stderr.#$$.log", 0)
       @server.stop(true)
     end
+    reset_sig_handlers
   end
 
   def test_preload_app_config
diff --git a/test/unit/test_signals.rb b/test/unit/test_signals.rb
index eb2af0b..71cf8f4 100644
--- a/test/unit/test_signals.rb
+++ b/test/unit/test_signals.rb
@@ -40,6 +40,10 @@ class SignalsTest < Test::Unit::TestCase
     @server = nil
   end
 
+  def teardown
+    reset_sig_handlers
+  end
+
   def test_worker_dies_on_dead_master
     pid = fork {
       app = lambda { |env| [ 200, {'X-Pid' => "#$$" }, [] ] }
@@ -190,7 +194,7 @@ class SignalsTest < Test::Unit::TestCase
     killer = fork { loop { Process.kill(:HUP, pid); sleep(0.0001) } }
     buf = ' ' * @bs
     @count.times { sock.syswrite(buf) }
-    Process.kill(:TERM, killer)
+    Process.kill(:KILL, killer)
     Process.waitpid2(killer)
     redirect_test_io { @server.stop(true) }
     # can't check for == since pending signals get merged
diff --git a/test/unit/test_upload.rb b/test/unit/test_upload.rb
index 7ac3c9e..dc0eb40 100644
--- a/test/unit/test_upload.rb
+++ b/test/unit/test_upload.rb
@@ -55,6 +55,7 @@ class UploadTest < Test::Unit::TestCase
   def teardown
     redirect_test_io { @server.stop(true) } if @server
     @random.close
+    reset_sig_handlers
   end
 
   def test_put
-- 
Eric Wong

From ibc at aliax.net  Wed Jan 20 03:45:33 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Wed, 20 Jan 2010 09:45:33 +0100
Subject: [PATCH] initialize signal handlers before writing pid file
In-Reply-To: <20100120023826.GA25165@dcvr.yhbt.net>
References: <201001170244.55418.ibc@aliax.net>
	<20100118114334.GC4625@dcvr.yhbt.net>
	<20100120023826.GA25165@dcvr.yhbt.net>
Message-ID: <201001200945.33438.ibc@aliax.net>

El Mi?rcoles, 20 de Enero de 2010, Eric Wong escribi?:
> This prevents trigger-happy init scripts from reading the pid
> file (and thus sending signals) to a not-fully initialized
> master process to handle them.
> 
> This does NOT fix anything if other processes are sending
> signals prematurely without relying on the presence of the pid
> file.  It's not possible to prevent all cases of this in one
> process, even in a purely C application, so we won't bother
> trying.
> 
> We continue to always defer signal handling to the main loop
> anyways, and signals sent to the master process will be
> deferred/ignored until Unicorn::HttpServer#join is run.


Hi Eric, I've already tested it. It works great :)


To check it I use the following init script "reload" action which hope could 
be useful for others (it requires loading "/lib/lsb/init-functions"):


reload|force-reload|safe-reload)
  # Return
  #   0 if daemon has been reloaded
  #   1 if daemon was not running
  #   1 if daemon could not be reloaded
  #
  log_daemon_msg "Safe reloading $DESC"

  oldpid=$(pidofproc -p $PIDFILE)
  if [ "$oldpid" ] ; then
    kill -s USR2 $oldpid
    log_action_msg "signal USR2 sent to $NAME master process (pid $oldpid) to
                    fork a new master and workers processes"

    log_action_msg "waiting for the new process to successfully start..."
    printf "."
    sleep 1
    while [ "$(pidofproc -p $PIDFILE.oldbin)" ]; do
      printf "."
      sleep 1
    done
    echo

    newpid=$(pidofproc -p $PIDFILE)
    # If the PID has definitively changed than the new master is running.
    if [ "$oldpid" != "$newpid" ] ; then
      log_success_msg "server correctly reloaded, new master process has pid
                       $newpid"
      log_end_msg 0
    # If not, the new master process couldn't start due to the new
    # configuration or code error.
    else
      log_failure_msg "error running a new master process, $NAME was not
                       reloaded (check error log)"
      log_end_msg 1
    fi
  else
    log_failure_msg "$NAME is not running"
    log_end_msg 1
  fi
  ;;



And I test the following loop to check it:

  while [ 0 ] ; do /etc/init.d/myserver reload ; done






-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Wed Jan 20 03:54:12 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Wed, 20 Jan 2010 09:54:12 +0100
Subject: Suggestion for reload action (USR2)
Message-ID: <201001200954.12169.ibc@aliax.net>

Hi, when Unicorn receives a USR2 it invokes the original script and arguments 
and creates a new master as a child.

However when running this new master there could be something to treat 
different as when we run Unicorn for first time.

For example:
- In the initial unicorn master I want it to create a Posix_mq, and in case it 
already exists then I want it to delete and create again.
- But in the new master (when USR2) I just want it to reuse the existing 
Posiq_mq.

Note this is just an example, perhaps not the best however.

So I suggest the following:

- When unicorn master receives a USR2 it executes the original executable 
withthe original arguments plus "--reload". This just would mean that the new 
instance is a reloaded process. The executable could store such option 
somewhere so the application could know it and react.

This would involve just three changes:

- Add an option "--reload" in OptionParser in bin/unicorn.
- Store such option somewhere so it can be readed later by the Rack 
application or other library.
- Add "--reload" to the list of original arguments when receiving USR2.

Opinnions?


-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Wed Jan 20 04:20:23 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Wed, 20 Jan 2010 10:20:23 +0100
Subject: Suggestion for reload action (USR2)
Message-ID: <201001201020.23413.ibc@aliax.net>

El Mi?rcoles, 20 de Enero de 2010, I?aki Baz Castillo escribi?:

> This would involve just three changes:
> 
> - Add an option "--reload" in OptionParser in bin/unicorn.
> - Store such option somewhere so it can be readed later by the Rack
> application or other library.
> - Add "--reload" to the list of original arguments when receiving USR2.

I attach a patch (for master branch) that would do the job (except the 
fact that I don't know how to store the "reload" option):


-----------------------------------------
diff --git a/bin/unicorn b/bin/unicorn
index 5af021d..43f5434 100755
--- a/bin/unicorn
+++ b/bin/unicorn
@@ -88,6 +88,10 @@ opts = OptionParser.new("", 24, '  ') do |opts|
     options[:config_file] = f
   end

+  opts.on("--reload", "HIDDEN") do |f|
+    # TODO: Store it somewhere
+  end
+
   # I'm avoiding Unicorn-specific config options on the command-line.
   # IMNSHO, config options on the command-line are redundant given
   # config files and make things unnecessarily complicated with multiple
@@ -96,7 +100,7 @@ opts = OptionParser.new("", 24, '  ') do |opts|
   opts.separator "Common options:"

   opts.on_tail("-h", "--help", "Show this message") do
-    puts opts.to_s.gsub(/^.*DEPRECATED.*$/s, '')
+    puts opts.to_s.gsub(/^.*DEPRECATED.*$/s, '').gsub(/^.*HIDDEN.*$/s, '')
     exit
   end

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index e3e4315..49ad79b 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -519,7 +519,7 @@ module Unicorn
         end
         logger.info "executing #{cmd.inspect} (in #{Dir.pwd})"
         before_exec.call(self)
-        exec(*cmd)
+        exec(*cmd << "--reload")
       end
       proc_name 'master (old)'
     end
-----------------------------------------


The patch is available here:
  http://oversip.net/public/unicorn_reload.patch

(unfortunatelly I cannot attach files in this mailist as the mail is 
rejected).


Regards.


-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Wed Jan 20 15:03:43 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 20 Jan 2010 20:03:43 +0000
Subject: Suggestion for reload action (USR2)
In-Reply-To: <201001201020.23413.ibc@aliax.net>
References: <201001201020.23413.ibc@aliax.net>
Message-ID: <20100120200343.GA9925@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Mi?rcoles, 20 de Enero de 2010, I?aki Baz Castillo escribi?:
> 
> > This would involve just three changes:
> > 
> > - Add an option "--reload" in OptionParser in bin/unicorn.
> > - Store such option somewhere so it can be readed later by the Rack
> > application or other library.
> > - Add "--reload" to the list of original arguments when receiving USR2.

Hi I?aki,

All you need to do is to check for the presence of ENV["UNICORN_FD"]
which is used to communicate file descriptor numbers for the upgrade
process

> I attach a patch (for master branch) that would do the job (except the 
> fact that I don't know how to store the "reload" option):

> (unfortunatelly I cannot attach files in this mailist as the mail is 
> rejected).

While the patch was not needed for this case, inline patches are strongly
preferred here.  Inline patches are far easier to read, reply-to and
apply than attachments.

-- 
Eric Wong

From ibc at aliax.net  Wed Jan 20 15:22:43 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Wed, 20 Jan 2010 21:22:43 +0100
Subject: Suggestion for reload action (USR2)
In-Reply-To: <20100120200343.GA9925@dcvr.yhbt.net>
References: <201001201020.23413.ibc@aliax.net>
	<20100120200343.GA9925@dcvr.yhbt.net>
Message-ID: <201001202122.43237.ibc@aliax.net>

El Mi?rcoles, 20 de Enero de 2010, Eric Wong escribi?:

> All you need to do is to check for the presence of ENV["UNICORN_FD"]
> which is used to communicate file descriptor numbers for the upgrade
> process

Great, I didn't know it!  :)

 
> > I attach a patch (for master branch) that would do the job (except the
> > fact that I don't know how to store the "reload" option):
> >
> > (unfortunatelly I cannot attach files in this mailist as the mail is
> > rejected).
> 
> While the patch was not needed for this case, inline patches are strongly
> preferred here.  Inline patches are far easier to read, reply-to and
> apply than attachments.

Sure but what about the fixed 80 columns of a mail?
Of course I can generate the mail without such constrain, but it doesn't look 
very cool :)

Thanks a lot.


-- 
I?aki Baz Castillo <ibc at aliax.net>

From normalperson at yhbt.net  Wed Jan 20 15:58:18 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 20 Jan 2010 20:58:18 +0000
Subject: Suggestion for reload action (USR2)
In-Reply-To: <201001202122.43237.ibc@aliax.net>
References: <201001201020.23413.ibc@aliax.net>
	<20100120200343.GA9925@dcvr.yhbt.net>
	<201001202122.43237.ibc@aliax.net>
Message-ID: <20100120205818.GA10293@dcvr.yhbt.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Mi?rcoles, 20 de Enero de 2010, Eric Wong escribi?:
> > While the patch was not needed for this case, inline patches are strongly
> > preferred here.  Inline patches are far easier to read, reply-to and
> > apply than attachments.
> 
> Sure but what about the fixed 80 columns of a mail?
> Of course I can generate the mail without such constrain, but it doesn't look 
> very cool :)

For mail, the soft limit is actually closer/around to 72 columns because
we take quoting into consideration.  But properly configured mailers
shouldn't care nor enforce this (git-send-email(1) does not).

http://git.kernel.org/?p=git/git.git;a=blob;f=Documentation/SubmittingPatches
has some good notes on various mailers (I usually use mutt to send
one-off patches and git send-email for a series).

-- 
Eric Wong

From ibc at aliax.net  Thu Jan 21 06:20:08 2010
From: ibc at aliax.net (=?iso-8859-1?q?I=F1aki_Baz_Castillo?=)
Date: Thu, 21 Jan 2010 12:20:08 +0100
Subject: Suggestion for reload action (USR2)
In-Reply-To: <20100120205818.GA10293@dcvr.yhbt.net>
References: <201001201020.23413.ibc@aliax.net>
	<201001202122.43237.ibc@aliax.net>
	<20100120205818.GA10293@dcvr.yhbt.net>
Message-ID: <201001211220.09051.ibc@aliax.net>

El Mi?rcoles, 20 de Enero de 2010, Eric Wong escribi?:
> I?aki Baz Castillo <ibc at aliax.net> wrote:
> > El Mi?rcoles, 20 de Enero de 2010, Eric Wong escribi?:
> > > While the patch was not needed for this case, inline patches are
> > > strongly preferred here.  Inline patches are far easier to read,
> > > reply-to and apply than attachments.
> >
> > Sure but what about the fixed 80 columns of a mail?
> > Of course I can generate the mail without such constrain, but it doesn't
> > look very cool :)
> 
> For mail, the soft limit is actually closer/around to 72 columns because
> we take quoting into consideration.  But properly configured mailers
> shouldn't care nor enforce this (git-send-email(1) does not).
> 
> http://git.kernel.org/?p=git/git.git;a=blob;f=Documentation/SubmittingPatch
> es has some good notes on various mailers (I usually use mutt to send
>  one-off patches and git send-email for a series).
> 

KMail
-----

This should help you to submit patches inline using KMail.

1) Prepare the patch as a text file.

2) Click on New Mail.

3) Go under "Options" in the Composer window and be sure that
"Word wrap" is not set.

4) Use Message -> Insert file... and insert the patch.

5) Back in the compose window: add whatever other text you wish to the
  514 message, complete the addressing and subject fields, and press send.


:)


-- 
I?aki Baz Castillo <ibc at aliax.net>

From travis at meticulo.com  Fri Jan 22 01:30:50 2010
From: travis at meticulo.com (Travis Bell)
Date: Thu, 21 Jan 2010 23:30:50 -0700
Subject: Getting a strange cookie error
Message-ID: <AAD43046-6AF4-4685-933D-41F7E1DC0795@meticulo.com>

Hey guys,

I've been developing with Thin since day 1 (many months) and today, as I started moving everything to test on production (with Unicorn) started getting this weird error.

I am using Sinatra, and have been setting my cookies like so:

         response.set_cookie('foo', {
            :value => "bar",
            :path => '/',
            :expires => Time.today+14400
         })

In Thin, Webrick, Passenger, this seems to work fine. However in Unicorn I get a big fat ugly error:

# Rack::Lint::LintError at /login
# a header value must be a String, but the value of 'Set-Cookie' is a Array
# /opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rack-1.1.0/lib/rack/lint.rb: in assert, line 18

I'm not really sure what this means since :value is a string. I am using the following versions:

- REE (1.8.7-2010.01)
- Rack 1.1.0
- Unicorn 0.96.0
- Latest Sinatra from source

Does anyone have any input as to why this is happening? I have been able to re-produce this on a 32-bit CentOS box and my local MacBook Pro (Snow Leopard).


Cheers,

-- 
Travis Bell
http://www.twitter.com/travisbell
http://www.twitter.com/themoviedb




From ibc at aliax.net  Fri Jan 22 10:16:41 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Fri, 22 Jan 2010 16:16:41 +0100
Subject: About new commit "xpath methods can return things other than nodesets"
Message-ID: <201001221616.41818.ibc@aliax.net>

Hi, I'm a bit afraid about this recet commit:

  http://github.com/tenderlove/nokogiri/commit/45c6d1d4902e100fecc3741985870aa4edda9a25
  "xpath methods can return things other than nodesets"

In my case I use Nokogiri in a server application which receives Xpath strings 
from remote clients and performs such Xpath operation within a XML document 
stored in the server.

However I do a strict pre-parsing of the Xpath string so it must have the form 
of a "typical" Xpath ("ns1:root/ns2:child/ns2:somenode[@name='lala']").

The server application expects to get a NodeSet after performing the Xpath 
operation and evaluates it (empty, number of entries....) by alwasying 
inspecting the NodeSet#size method.

Should I care about this new commit? Could occur that Xpath returns something 
but a NodeSet in my case?

Thanks.


-- 
I?aki Baz Castillo <ibc at aliax.net>

From ibc at aliax.net  Fri Jan 22 11:05:28 2010
From: ibc at aliax.net (=?utf-8?q?I=C3=B1aki_Baz_Castillo?=)
Date: Fri, 22 Jan 2010 17:05:28 +0100
Subject: About new commit "xpath methods can return things other than
	nodesets"
In-Reply-To: <201001221616.41818.ibc@aliax.net>
References: <201001221616.41818.ibc@aliax.net>
Message-ID: <201001221705.28170.ibc@aliax.net>

Sorry, wrong list.

-- 
I?aki Baz Castillo <ibc at aliax.net>

From travis at meticulo.com  Fri Jan 22 15:06:35 2010
From: travis at meticulo.com (Travis Bell)
Date: Fri, 22 Jan 2010 13:06:35 -0700
Subject: Getting a strange cookie error
In-Reply-To: <AAD43046-6AF4-4685-933D-41F7E1DC0795@meticulo.com>
References: <AAD43046-6AF4-4685-933D-41F7E1DC0795@meticulo.com>
Message-ID: <0B0E184C-1FD8-4208-BD48-0AC7DE2C33A0@meticulo.com>

Just a follow up, it looks like it was rack 1.1.0 causing the issues. I uninstalled it, and am now using rack 1.0.1 and the error has gone away.

Why it worked with Thin + Rack 1.1.0 and not Unicorn, still not sure.


Cheers guys.

-- 
Travis Bell
http://www.twitter.com/travisbell
http://www.twitter.com/themoviedb



On 2010-01-21, at 11:30 PM, Travis Bell wrote:

> Hey guys,
> 
> I've been developing with Thin since day 1 (many months) and today, as I started moving everything to test on production (with Unicorn) started getting this weird error.
> 
> I am using Sinatra, and have been setting my cookies like so:
> 
>         response.set_cookie('foo', {
>            :value => "bar",
>            :path => '/',
>            :expires => Time.today+14400
>         })
> 
> In Thin, Webrick, Passenger, this seems to work fine. However in Unicorn I get a big fat ugly error:
> 
> # Rack::Lint::LintError at /login
> # a header value must be a String, but the value of 'Set-Cookie' is a Array
> # /opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rack-1.1.0/lib/rack/lint.rb: in assert, line 18
> 
> I'm not really sure what this means since :value is a string. I am using the following versions:
> 
> - REE (1.8.7-2010.01)
> - Rack 1.1.0
> - Unicorn 0.96.0
> - Latest Sinatra from source
> 
> Does anyone have any input as to why this is happening? I have been able to re-produce this on a 32-bit CentOS box and my local MacBook Pro (Snow Leopard).
> 
> 
> Cheers,
> 
> -- 
> Travis Bell
> http://www.twitter.com/travisbell
> http://www.twitter.com/themoviedb
> 
> 
> 


From normalperson at yhbt.net  Fri Jan 22 16:48:51 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 22 Jan 2010 21:48:51 +0000
Subject: Getting a strange cookie error
In-Reply-To: <0B0E184C-1FD8-4208-BD48-0AC7DE2C33A0@meticulo.com>
References: <AAD43046-6AF4-4685-933D-41F7E1DC0795@meticulo.com>
	<0B0E184C-1FD8-4208-BD48-0AC7DE2C33A0@meticulo.com>
Message-ID: <20100122214851.GA26081@dcvr.yhbt.net>

Travis Bell <travis at meticulo.com> wrote:
> Just a follow up, it looks like it was rack 1.1.0 causing the issues.
> I uninstalled it, and am now using rack 1.0.1 and the error has gone
> away.
> 
> Why it worked with Thin + Rack 1.1.0 and not Unicorn, still not sure.

Sorry, I keep getting distracted/interrupted while looking into this,
it's probably related to the Rack::Utils::HeaderHash optimizations[1] in
Rack 1.1.0 somehow interacting badly with Sinatra + Rack::Lint

But Sinatra uses Rack::Response, so it shouldn't happen, but somehow
it does, I'll look at it more when I get a chance.


[1] - I made those optimizations to Rack, too :<

-- 
Eric Wong

From normalperson at yhbt.net  Fri Jan 22 18:03:37 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 22 Jan 2010 15:03:37 -0800
Subject: Getting a strange cookie error
In-Reply-To: <20100122214851.GA26081@dcvr.yhbt.net>
References: <AAD43046-6AF4-4685-933D-41F7E1DC0795@meticulo.com>
	<0B0E184C-1FD8-4208-BD48-0AC7DE2C33A0@meticulo.com>
	<20100122214851.GA26081@dcvr.yhbt.net>
Message-ID: <20100122230337.GA18815@dcvr.yhbt.net>

Eric Wong <normalperson at yhbt.net> wrote:
> Travis Bell <travis at meticulo.com> wrote:
> > Just a follow up, it looks like it was rack 1.1.0 causing the issues.
> > I uninstalled it, and am now using rack 1.0.1 and the error has gone
> > away.
> > 
> > Why it worked with Thin + Rack 1.1.0 and not Unicorn, still not sure.
> 
> Sorry, I keep getting distracted/interrupted while looking into this,
> it's probably related to the Rack::Utils::HeaderHash optimizations[1] in
> Rack 1.1.0 somehow interacting badly with Sinatra + Rack::Lint
> 
> But Sinatra uses Rack::Response, so it shouldn't happen, but somehow
> it does, I'll look at it more when I get a chance.

HI Travis,

I can't seem to reproduce this with Sinatra
466cc74b8f393f98de227122c78ed55cb745cbc6, and Rack 1.1.0 with a basic
"hello world" application.  I'll need to know more about what your
application does with response headers...

Are you running any other middlewares or patches that could affect
things inside Rack::Response or Rack::Utils?

Unicorn takes a very strict interpretation of the Rack spec and always
loads Rack::Lint in development mode.  This makes Unicorn much more
picky about applications/middleware that don't completely conform to the
Rack spec.

Thin itself doesn't appear to load Rack::Lint by default, but does
running your application with "rackup -s thin -E development..." let you
reproduce the failures you see with Unicorn?

Thanks for any more info you can give us!

-- 
Eric Wong

From travis at meticulo.com  Fri Jan 22 18:29:56 2010
From: travis at meticulo.com (Travis Bell)
Date: Fri, 22 Jan 2010 16:29:56 -0700
Subject: Getting a strange cookie error
In-Reply-To: <0B0E184C-1FD8-4208-BD48-0AC7DE2C33A0@meticulo.com>
References: <AAD43046-6AF4-4685-933D-41F7E1DC0795@meticulo.com>
	<0B0E184C-1FD8-4208-BD48-0AC7DE2C33A0@meticulo.com>
Message-ID: <71901C45-5A84-4A3B-A4A4-0895CE2D6974@meticulo.com>

Hi Eric,

Thanks for looking into this. I'll strip down my app and do a similar "Hello World" version and see how I make out. I will give that method of loading Thin a try too.

Report back when I know more.


Thanks!

-- 
Travis Bell




On 2010-01-22, at 1:06 PM, Travis Bell wrote:

> Just a follow up, it looks like it was rack 1.1.0 causing the issues. I uninstalled it, and am now using rack 1.0.1 and the error has gone away.
> 
> Why it worked with Thin + Rack 1.1.0 and not Unicorn, still not sure.
> 
> 
> Cheers guys.
> 
> -- 
> Travis Bell
> http://www.twitter.com/travisbell
> http://www.twitter.com/themoviedb
> 
> 
> 
> On 2010-01-21, at 11:30 PM, Travis Bell wrote:
> 
>> Hey guys,
>> 
>> I've been developing with Thin since day 1 (many months) and today, as I started moving everything to test on production (with Unicorn) started getting this weird error.
>> 
>> I am using Sinatra, and have been setting my cookies like so:
>> 
>>        response.set_cookie('foo', {
>>           :value => "bar",
>>           :path => '/',
>>           :expires => Time.today+14400
>>        })
>> 
>> In Thin, Webrick, Passenger, this seems to work fine. However in Unicorn I get a big fat ugly error:
>> 
>> # Rack::Lint::LintError at /login
>> # a header value must be a String, but the value of 'Set-Cookie' is a Array
>> # /opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rack-1.1.0/lib/rack/lint.rb: in assert, line 18
>> 
>> I'm not really sure what this means since :value is a string. I am using the following versions:
>> 
>> - REE (1.8.7-2010.01)
>> - Rack 1.1.0
>> - Unicorn 0.96.0
>> - Latest Sinatra from source
>> 
>> Does anyone have any input as to why this is happening? I have been able to re-produce this on a 32-bit CentOS box and my local MacBook Pro (Snow Leopard).
>> 
>> 
>> Cheers,
>> 
>> -- 
>> Travis Bell
>> http://www.twitter.com/travisbell
>> http://www.twitter.com/themoviedb
>> 
>> 
>> 
> 


From e.series2010 at milnews.es  Fri Jan 22 18:59:41 2010
From: e.series2010 at milnews.es (EURO MILLIONES GAMING AUTHORITY)
Date: Sat, 23 Jan 2010 00:59:41 +0100
Subject: You Email Emerged Winner [ REF. : TCC/0204/ESP/971 ]
Message-ID: <20100122235944.AE7B7625D89@smtp-03.servidoresdns.net>

                                 EURO MILLIONES LOTTERY BOARD.

REF. NUMBER: TCC/0204/ESP/971
BATCH NUMBER: 2010/BTH/30046
EMAIL: mongrel-unicorn at rubyforge.org

Dear Email Client,

This is to notify you that your email address shown above has won the EURO MILLIONES LOTTERY 
Online Computer ballot draw that was hosted in Madrid Spain. You have been awarded the prize 
of ?815,000 (EIGHT HUNDREDAND FIFTEEN THOUSAND EUROS) with the Winning information listed above.

If you are the accredited owner of this email address [ mongrel-unicorn at rubyforge.org ], please contact the appointed agent 
company (GLOBAL GESTORES S. A) with the below requirements including the winning datas above.

1. Fulnames
2. Telephone number
3. Mobile number
4. Country

GLOBAL GESTORES S. A.
MR. FERNAND CORTES (claim officer)
TEL: +34 672-892-907
FAX: +34 911-820-312
Email: ggestores at gmail.com or globalgestores at luckymail.com

All information will be strictly verified by your agent before payment will be carried out. 
Congratulations once again from our board of Directors.

Mrs. Angela De La Costa
EURO MILLIONES BAORD.
Madrid, 23rd Jan. 2010.

From aspirer at twocreative.nl  Sun Jan 24 14:54:27 2010
From: aspirer at twocreative.nl (Viagra on www.na47.com)
Date: Sun, 24 Jan 2010 20:54:27 +0100
Subject: downt rend  decri al
Message-ID: <4B5CA4ED.5040408@aaai.com.au>

mandr el  belie d  refle t  chiro graph ic


From swappable at rsedata.com  Mon Jan 25 11:00:58 2010
From: swappable at rsedata.com (Viagra on www.na47.com)
Date: Mon, 25 Jan 2010 17:00:58 +0100
Subject: spook y  tela  locat ed  genea logic al  logos
Message-ID: <4B5DBFC5.2030105@henkstoeten.nl>

deref erenc ed  highr oad  press urize s  proba tione rs  fugal ly


From normalperson at yhbt.net  Mon Jan 25 21:35:53 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 25 Jan 2010 18:35:53 -0800
Subject: [PATCH] KNOWN_ISSUES: background threads + Logger monkey patch
Message-ID: <20100126023553.GA7011@dcvr.yhbt.net>

Thanks to Michael Guterl for informing us of the issues
and testing the monkey patch.
---
 Pushed out to the website and unicorn.git

 KNOWN_ISSUES               |    8 ++++++++
 examples/logger_mp_safe.rb |   25 +++++++++++++++++++++++++
 2 files changed, 33 insertions(+), 0 deletions(-)
 create mode 100644 examples/logger_mp_safe.rb

diff --git a/KNOWN_ISSUES b/KNOWN_ISSUES
index 83b3584..e83e34e 100644
--- a/KNOWN_ISSUES
+++ b/KNOWN_ISSUES
@@ -3,6 +3,14 @@
 Occasionally odd {issues}[link:ISSUES.html] arise without a transparent or
 acceptable solution.  Those issues are documented here.
 
+* When using "preload_app true", with apps using background threads
+  need to restart them in the after_fork hook because threads are never
+  shared with child processes.  Additionally, any synchronization
+  primitives (Mutexes, Monitors, ConditionVariables) should be
+  reinitialized in case they are held during fork time to avoid
+  deadlocks.  The core Ruby Logger class needlessly uses a MonitorMutex
+  which can be disabled with a {monkey patch}[link:examples/logger_mp_safe.rb]
+
 * Rails 2.3.2 bundles its own version of Rack.  This may cause subtle
   bugs when simultaneously loaded with the system-wide Rack Rubygem
   which Unicorn depends on.  Upgrading to Rails 2.3.4 (or later) is
diff --git a/examples/logger_mp_safe.rb b/examples/logger_mp_safe.rb
new file mode 100644
index 0000000..05ad3fa
--- /dev/null
+++ b/examples/logger_mp_safe.rb
@@ -0,0 +1,25 @@
+# Multi-Processing-safe monkey patch for Logger
+#
+# This monkey patch fixes the case where "preload_app true" is used and
+# the application spawns a background thread upon being loaded.
+#
+# This removes all lock from the Logger code and solely relies on the
+# underlying filesystem to handle write(2) system calls atomically when
+# O_APPEND is used.  This is safe in the presence of both multiple
+# threads (native or green) and multiple processes when writing to
+# a filesystem with POSIX O_APPEND semantics.
+#
+# It should be noted that the original locking on Logger could _never_ be
+# considered reliable on non-POSIX filesystems with multiple processes,
+# either, so nothing is lost in that case.
+
+require 'logger'
+class Logger::LogDevice
+  def write(message)
+    @dev.syswrite(message)
+  end
+
+  def close
+    @dev.close
+  end
+end
-- 
Eric Wong

From normalperson at yhbt.net  Thu Jan 28 17:19:58 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 28 Jan 2010 14:19:58 -0800
Subject: Getting a strange cookie error
In-Reply-To: <71901C45-5A84-4A3B-A4A4-0895CE2D6974@meticulo.com>
References: <AAD43046-6AF4-4685-933D-41F7E1DC0795@meticulo.com>
	<0B0E184C-1FD8-4208-BD48-0AC7DE2C33A0@meticulo.com>
	<71901C45-5A84-4A3B-A4A4-0895CE2D6974@meticulo.com>
Message-ID: <20100128221958.GA13489@dcvr.yhbt.net>

Travis Bell <travis at meticulo.com> wrote:
> Hi Eric,
> 
> Thanks for looking into this. I'll strip down my app and do a similar
> "Hello World" version and see how I make out. I will give that method
> of loading Thin a try too.
> 
> Report back when I know more.

Hi Travis (or anybody else listening), any updates/info on this issue?
Thanks!

From evacuating at akjbabbelbox.nl  Sun Jan 31 05:41:32 2010
From: evacuating at akjbabbelbox.nl (Cheap Tamiflu on www.ve18.com)
Date: Sun, 31 Jan 2010 11:41:32 +0100
Subject: hepta valen t  snugg ling  initi ators  optio naliz e
Message-ID: <4B655D97.6020203@archineer.com>

handc raftm an  dost  schem atics  retyp ing  poser s  smoke  neces
sitie s  nonre alist ic  wooll ens  verti cally  bayan ihan  actab
ility  total ises  bayan ihan  gauzy  solon chak  metas tasiz es
inter surve y  phosp hatid ic  displ ayabl e  invoi cing  ridde r
benig nancy  eddys tone  urim  islan der  awake ning  piran dello
parre l  davit s  displ ant  losse s  rubeo lar  visio nal  legen d
abbre viati on  incid ent  spurn s  nurtu rant  penta grams  tamis
gauzy  panat ella  value d  parre l  fasti dious ly  decat hlons
surfi ng  disse cted  worsh ipped