From ajsharp at gmail.com Fri Jul 1 13:07:44 2011 From: ajsharp at gmail.com (Alex Sharp) Date: Fri, 1 Jul 2011 10:07:44 -0700 Subject: Timestamp in unicorn logs Message-ID: I would love to see a timestamp in each line of unicorn stderr and stdout logs. Has this been considered? Thanks. Best, alex sharp From ghazel at gmail.com Fri Jul 1 16:33:45 2011 From: ghazel at gmail.com (ghazel at gmail.com) Date: Fri, 1 Jul 2011 13:33:45 -0700 Subject: Timestamp in unicorn logs In-Reply-To: References: Message-ID: On Fri, Jul 1, 2011 at 10:07 AM, Alex Sharp wrote: > I would love to see a timestamp in each line of unicorn stderr and > stdout logs. Has this been considered? Thanks. +1 -Greg From normalperson at yhbt.net Fri Jul 1 16:54:34 2011 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 1 Jul 2011 20:54:34 +0000 Subject: Timestamp in unicorn logs In-Reply-To: References: Message-ID: <20110701205434.GA19317@dcvr.yhbt.net> Alex Sharp wrote: > I would love to see a timestamp in each line of unicorn stderr and > stdout logs. Has this been considered? Thanks. Unicorn itself never writes directly to stdout[1]. Once daemonized, it never writes to stderr directly, either. If it does either of these, it's a bug in Unicorn and needs to be reported and fixed. All writing through stderr happens through a Logger object which does timestamping/formatting, but Rails may screw up the formatting: >From the {FAQ}[http://unicorn.bogomips.org/FAQ.html]: | === Why are log messages from Unicorn are unformatted when using Rails? | | Current versions of Rails unfortunately overrides the default Logger | formatter. | | You can undo this behavior with the default logger in your Unicorn | config file: | | Configurator::DEFAULTS[:logger].formatter = Logger::Formatter.new | | Of course you can specify an entirely different logger as well | with the "logger" directive described by Unicorn::Configurator. It would be helpful for somebody who uses Rails to fix this behavior in Rails. [1] - stdout is reserved for the application; I configure {clogger}[http://clogger.rubyforge.org/] to log to stdout if I'm using clogger. -- Eric Wong From ghazel at gmail.com Fri Jul 1 17:23:50 2011 From: ghazel at gmail.com (ghazel at gmail.com) Date: Fri, 1 Jul 2011 14:23:50 -0700 Subject: Timestamp in unicorn logs In-Reply-To: <20110701205434.GA19317@dcvr.yhbt.net> References: <20110701205434.GA19317@dcvr.yhbt.net> Message-ID: On Fri, Jul 1, 2011 at 1:54 PM, Eric Wong wrote: > Alex Sharp wrote: >> I would love to see a timestamp in each line of unicorn stderr and >> stdout logs. Has this been considered? Thanks. > > Unicorn itself never writes directly to stdout[1]. ?Once daemonized, it > never writes to stderr directly, either. ?If it does either of these, > it's a bug in Unicorn and needs to be reported and fixed. > > All writing through stderr happens through a Logger object which does > timestamping/formatting, but Rails may screw up the formatting: I use these options in my config/unicorn.rb: stderr_path "/data/myapp/shared/log/unicorn.stderr.log" stdout_path "/data/myapp/shared/log/unicorn.stdout.log" So maybe that's what Alex is referring to as well. I do see un-timestamped output in both files (some from my OobGC-fork middleware). The logger configuration you mentioned is intended to be used with those? -Greg From ajsharp at gmail.com Fri Jul 1 17:40:58 2011 From: ajsharp at gmail.com (Alex Sharp) Date: Fri, 1 Jul 2011 14:40:58 -0700 Subject: Timestamp in unicorn logs In-Reply-To: References: <20110701205434.GA19317@dcvr.yhbt.net> Message-ID: On Fri, Jul 1, 2011 at 2:23 PM, wrote: > I use these options in my config/unicorn.rb: > stderr_path "/data/myapp/shared/log/unicorn.stderr.log" > stdout_path "/data/myapp/shared/log/unicorn.stdout.log" > > So maybe that's what Alex is referring to as well. I do see > un-timestamped output in both files (some from my OobGC-fork > middleware). The logger configuration you mentioned is intended to be > used with those? > Yes, these are the logs I was referring to, not application level logging. These logs output information when new workers are forked, when the server starts, etc. My apologies, I should have been more specific. Best, alex sharp github.com/ajsharp twitter.com/ajsharp alexjsharp.com 615.243.1472 From normalperson at yhbt.net Fri Jul 1 19:27:20 2011 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 1 Jul 2011 16:27:20 -0700 Subject: Timestamp in unicorn logs In-Reply-To: References: <20110701205434.GA19317@dcvr.yhbt.net> Message-ID: <20110701232720.GA25165@dcvr.yhbt.net> ghazel at gmail.com wrote: > I use these options in my config/unicorn.rb: > stderr_path "/data/myapp/shared/log/unicorn.stderr.log" > stdout_path "/data/myapp/shared/log/unicorn.stdout.log" > > So maybe that's what Alex is referring to as well. I do see > un-timestamped output in both files (some from my OobGC-fork > middleware). The logger configuration you mentioned is intended to be > used with those? Yes. You should be able to access the same Logger instance via env["rack.logger"] in your app/middleware (IIRC env["rack.logger"] is required for Rack 1.2). -- Eric Wong From ehudros at gmail.com Wed Jul 6 06:21:51 2011 From: ehudros at gmail.com (Ehud Rosenberg) Date: Wed, 6 Jul 2011 13:21:51 +0300 Subject: Unicorn completely ignores USR2 signal Message-ID: Hi all, I'm experiencing a rather strange problem with unicorn on my production server. Although the config file states preload_app true, sending USR2 to the master process does not generate any response, and it seems like unicorn is ignoring the signal altogether. On another server sending USR2 changes the master process to and (old) state and starts a new master process successfully. The problematic server is using RVM & bundler, so I'm assuming it's somehow related (the other one is vanilla ruby). Sending signals other than USR2 (QUIT, HUP) works just fine. Is there a way to trace what's going on behind the scenes here? Unicorn's log file is completely empty. Thanks :) From normalperson at yhbt.net Wed Jul 6 13:42:16 2011 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 6 Jul 2011 17:42:16 +0000 Subject: Unicorn completely ignores USR2 signal In-Reply-To: References: Message-ID: <20110706174216.GA1733@dcvr.yhbt.net> Ehud Rosenberg wrote: > Hi all, > I'm experiencing a rather strange problem with unicorn on my production server. > Although the config file states preload_app true, sending USR2 to the > master process does not generate any response, and it seems like > unicorn is ignoring the signal altogether. > On another server sending USR2 changes the master process to and (old) > state and starts a new master process successfully. > The problematic server is using RVM & bundler, so I'm assuming it's > somehow related (the other one is vanilla ruby). RVM could be a problem, especially if your ENV changed somehow and your path to. Can you dump out the START_CTX and ENV contents in the before_exec hook? before_exec do |server| File.open("/tmp/start_ctx.dump", "ab") do |fp| PP.pp Unicorn::HttpServer::START_CTX, fp PP.pp ENV, fp end end You may need to modify START_CTX if you're changing paths or if somehow RVM gave you the wrong path to unicorn START_CTX is documented here: http://unicorn.bogomips.org/Unicorn/HttpServer.html > Sending signals other than USR2 (QUIT, HUP) works just fine. > Is there a way to trace what's going on behind the scenes here? Run strace (Linux) or equivalent on the master process. > Unicorn's log file is completely empty. This is the log you setup to point to stderr_path? That shouldn't happen... -- Eric Wong From normalperson at yhbt.net Thu Jul 7 10:18:28 2011 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 7 Jul 2011 07:18:28 -0700 Subject: Unicorn completely ignores USR2 signal In-Reply-To: <20110706174216.GA1733@dcvr.yhbt.net> References: <20110706174216.GA1733@dcvr.yhbt.net> Message-ID: <20110707141828.GA8957@dcvr.yhbt.net> Any progress on this issue? I'll have limited network access the next few days... From ajsharp at gmail.com Thu Jul 7 13:52:52 2011 From: ajsharp at gmail.com (Alex Sharp) Date: Thu, 7 Jul 2011 19:52:52 +0200 Subject: Unicorn completely ignores USR2 signal In-Reply-To: <20110707141828.GA8957@dcvr.yhbt.net> References: <20110706174216.GA1733@dcvr.yhbt.net> <20110707141828.GA8957@dcvr.yhbt.net> Message-ID: Can you report the following? Ruby version (including the patch number), OS and version? The linux kernel shipped with Ubuntu natty 11.04 has a known bug with signal trapping that affects ruby 1.9.2. See here: * https://github.com/bvandenbos/resque-scheduler/issues/90 * https://bugzilla.kernel.org/show_bug.cgi?id=32922 On Thu, Jul 7, 2011 at 4:18 PM, Eric Wong wrote: > Any progress on this issue? ?I'll have limited network access > the next few days... > _______________________________________________ > Unicorn mailing list - mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > Do not quote signatures (like this one) or top post when replying > -- Best, alex sharp github.com/ajsharp twitter.com/ajsharp alexjsharp.com 615.243.1472 From ehudros at gmail.com Thu Jul 7 19:02:53 2011 From: ehudros at gmail.com (Ehud Rosenberg) Date: Thu, 7 Jul 2011 23:02:53 +0000 (UTC) Subject: Unicorn completely ignores USR2 signal References: <20110706174216.GA1733@dcvr.yhbt.net> Message-ID: Eric Wong yhbt.net> writes: > > Ehud Rosenberg gmail.com> wrote: > > Hi all, > > I'm experiencing a rather strange problem with unicorn on my production server. > > Although the config file states preload_app true, sending USR2 to the > > master process does not generate any response, and it seems like > > unicorn is ignoring the signal altogether. > > On another server sending USR2 changes the master process to and (old) > > state and starts a new master process successfully. > > The problematic server is using RVM & bundler, so I'm assuming it's > > somehow related (the other one is vanilla ruby). > > RVM could be a problem, especially if your ENV changed somehow and your > path to. Can you dump out the START_CTX and ENV contents in the > before_exec hook? > > before_exec do |server| > File.open("/tmp/start_ctx.dump", "ab") do |fp| > PP.pp Unicorn::HttpServer::START_CTX, fp > PP.pp ENV, fp > end > end > > You may need to modify START_CTX if you're changing paths or if > somehow RVM gave you the wrong path to unicorn > > START_CTX is documented here: > http://unicorn.bogomips.org/Unicorn/HttpServer.html > > > Sending signals other than USR2 (QUIT, HUP) works just fine. > > Is there a way to trace what's going on behind the scenes here? > > Run strace (Linux) or equivalent on the master process. > > > Unicorn's log file is completely empty. > > This is the log you setup to point to stderr_path? That shouldn't > happen... > Sorry for not getting back to you guys earlier. It seems like the entire setup is somehow broken, as the before_exec block does not seem to output anything to the logfiles/temp file (I know how weird that sounds). The stdout/stderr logs do not contain any info other than the regular "worker started" message. I've tried 'puts' outside of the before_exec block and got the following output straight into my stdout (right after running sudo /etc/init.d/unicorn start): {:argv=>["-D", "-E", "production", "-c", "/var/www/platform/current/config/unicorn.rb"], 0=>"/usr/local/rvm/gems/ruby- 1.9.2-head/bin/unicorn", :cwd=>"/var/www/platform/current"} {"OLDPWD"=>"/var/www/platform/current", "SUDO_UID"=>"1000", "LOGNAME"=>"root", "USERNAME"=>"root", "PATH"=>"/usr/local/rvm/gems/ruby-1.9.2- head/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6 /bin", "SHELL"=>"/bin/bash", "GEM_HOME"=>"/usr/local/rvm/gems/ruby-1.9.2-head", "PWD"=>"/var/www/platform/current", "RACK_ENV"=>"production", "BUNDLE_GEMFILE"=>"/var/www/platform/current/Gemfile", "BUNDLE_BIN_PATH"=>"/usr/local/rvm/gems/ruby-1.9.2-head/gems/bundler- 1.0.10/bin/bundle", "RUBYOPT"=>"-I/usr/local/rvm/gems/ruby-1.9.2- head/gems/bundler-1.0.10/lib -rbundler/setup"} The ENV output was truncated a little to remove irrelevant keys. strace can be found here: https://gist.github.com/54cd784880b99a2b46d5/77718daec3c08227639a0619e02d387c4d0 cd9fd I couldn't really figure out what's going on there - it complains the pid file can't be found although it's there, and it then writes its content to the oldbin file (which is never actually created in the pid directory). Permissions wise the directory is writable by pretty much anyone, and unicorn runs as root anyway. The server is Ubuntu 10.04, ruby 1.9.2p53. The server starts fine and the site is fully functional despite all the weirdness :) From zheelishas5nasaen at yahoo.com Thu Jul 7 18:41:50 2011 From: zheelishas5nasaen at yahoo.com (Rick) Date: Fri, 8 Jul 2011 06:41:50 +0800 Subject: =?GB2312?B?RGlnaXRhbCBQaG90byBFZGl0aW5nIFNlcnZpY2VzIC0gUGhvdG8gQ3V0b3V0?= Message-ID: <20110707225742.5770D82910F@rajexcise.net> You are receiving this email because we wish you to use our digital photo editing services. Digital Photo Editing Services : We are a China based Imaging Professionals. We offer basic and advanced digital photo Editing services and solutions like photo Cutout, morphological photo Editing, photoshop photo Editing, satellite photo Editing, color photo Editing and vector photo Editing using latest techniques. Our strength in providing Editing solutions for digital images like photo masking, photo re-touching, back ground cleaning & cloning, sharpening & restoration of color, all this derive from our vast experience. Our dedicated team of highly skilled and experienced professionals serves the photo Editing requirements of our clients. An in house quality checking mechanism assures value addition and delivers perfect job. Our team can deliver your job overnight or in time bounded fashion irrespective of job size. Our mainly services are: 1. Photo Cutout 2. Photo Enhancement 3. Photo Retouching 4. Vector Conversion 5. Pop Art 6. Images Masking 7. Clipping Path 8. Photo Restoration 9. Web Design Photo Editing Services: Our Photo Editing services includes advanced photo cutouts/clippings, balancing brightness / contrast in an image, repair of minor scratches, creases, minor dust, and spot removal to removal of major scratches, cracks, creases, and stains in all areas. We also repair seriously faded/damaged photographs and make it print ready. Best regards, Rick Ranmdanosn Imaging Professionals Contact: rickcontact at yeah.net Send address to koremoveideat at yeah.net for remove From marc at vailsys.com Fri Jul 8 14:30:23 2011 From: marc at vailsys.com (Marc Sernatinger) Date: Fri, 8 Jul 2011 13:30:23 -0500 Subject: SQL Server errors when using unicorn Message-ID: <09EC48D7-7E0E-43E3-9B42-641E045CF7D8@vailsys.com> Lately I have been seeing a large volume of database errors in a couple of applications at work. This has proven difficult to troubleshoot: after some restarts of unicorn, things seem fine. Then, a restart as part of a new deployment introduces these issues anew, with rollback to the previous version of the app still producing errors. One thing I have noticed is in my troubleshooting is that I seem to be able to reproduce these errors only after a period of idleness or low activity on the part of unicorn. Running unicorn on my workstation, running ab with a concurrency of 5 or so is enough to produce the issue reasonably often. I bring this question to the unicorn discussion because I noticed we don't have this problem using Passenger. I was wondering if there were any particulars of our setup that seem problematic. Here is an example of the error we see. It is not always exactly this error. Sometimes we will see a bad token error from SQL Server. Other times the error manifests itself in ActiveRecord dynamic methods not being properly created: [ WARN 11-07-08 11:42:29 25300:0x1001d6360 ] Internal Server Error ODBC::Error: 01000 (20017) [unixODBC][FreeTDS][SQL Server]Unexpected EOF from the server: SELECT columns.TABLE_NAME as table_name, columns.COLUMN_NAME as name, columns.DATA_TYPE as type, columns.COLUMN_DEFAULT as default_value, columns.NUMERIC_SCALE as numeric_scale, columns.NUMERIC_PRECISION as numeric_precision, CASE WHEN columns.DATA_TYPE IN ('nchar','nvarchar') THEN columns.CHARACTER_MAXIMUM_LENGTH ELSE COL_LENGTH(columns.TABLE_SCHEMA+'.'+columns.TABLE_NAME, columns.COLUMN_NAME) END as length, CASE WHEN columns.IS_NULLABLE = 'YES' THEN 1 ELSE NULL END as is_nullable, CASE WHEN COLUMNPROPERTY(OBJECT_ID(columns.TABLE_SCHEMA+'.'+columns.TABLE_NAME), columns.COLUMN_NAME, 'IsIdentity') = 0 THEN NULL ELSE 1 END as is_identity FROM INFORMATION_SCHEMA.COLUMNS columns WHERE columns.TABLE_NAME = 'users' AND columns.TABLE_SCHEMA = schema_name() ORDER BY columns.ordinal_position Our stack: Rails 3.0.7 activerecord-sqlserver-adapter 3.0.15 using ruby-odbc and FreeTDS 0.82 unicorn 3.5.0 with nginx in front SQL Server 2005 -- at a different site from our web servers, connected by WAN CentOS 5.9 I appreciate any suggestions as to where the problem may lie. Thanks From marc at vailsys.com Fri Jul 8 19:29:44 2011 From: marc at vailsys.com (Marc Sernatinger) Date: Fri, 8 Jul 2011 18:29:44 -0500 Subject: "NoMethodError: undefined method" after switching to unicorn Message-ID: <997EE861-8141-49D6-AE73-D834A7D00CCF@vailsys.com> Tiago Scolari gmail.com> writes: > > Hello, > I'm having a lot of "NoMethodError: undefined method" errors after I moved from thin to unicorn. > > This seems to happen only with active records, and dynamic methods, like: > find_by_*, scoped_by_*, ... > > It seems to be random, at least I haven't found relation yet. > I may reload the same page 10 times, sometimes it will work ok, some times it will give me this error. > > I've 2 sites having this issue, one uses rails 3.0.9 (r 1.9.2) and other with 2.3.8 (r 1.8.7). > As i said, the issue started after I switched to unicorn. > > Any idea what could cause this? > > Here are some backtraces from hoptoad: This is one of the errors we see as part of our issues with our database connection when using unicorn. Do you have any information about your stack to share? Thanks, Marc From normalperson at yhbt.net Mon Jul 11 04:11:30 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 11 Jul 2011 08:11:30 +0000 Subject: SQL Server errors when using unicorn In-Reply-To: <09EC48D7-7E0E-43E3-9B42-641E045CF7D8@vailsys.com> References: <09EC48D7-7E0E-43E3-9B42-641E045CF7D8@vailsys.com> Message-ID: <20110711081130.GA5176@dcvr.yhbt.net> Marc Sernatinger wrote: > I bring this question to the unicorn discussion because I noticed we > don't have this problem using Passenger. I was wondering if there > were any particulars of our setup that seem problematic. Do you have "preload_app" set to true? Try using the default (false) since it's likely to cause issues for some/most database drivers or anything else that opens a stream socket. You can't safely share stream sockets across multiple processes/threads so you have to disconnect/reconnect them in the before/after_fork hooks. > Our stack: Rails 3.0.7 activerecord-sqlserver-adapter 3.0.15 using > ruby-odbc and FreeTDS 0.82 unicorn 3.5.0 with nginx in front SQL > Server 2005 -- at a different site from our web servers, connected by > WAN CentOS 5.9 Maybe somebody else can help you if disabling preload_app doesn't help. I have no idea about any of the database components you use. -- Eric Wong From normalperson at yhbt.net Mon Jul 11 04:23:12 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 11 Jul 2011 08:23:12 +0000 Subject: "NoMethodError: undefined method" after switching to unicorn In-Reply-To: <997EE861-8141-49D6-AE73-D834A7D00CCF@vailsys.com> References: <997EE861-8141-49D6-AE73-D834A7D00CCF@vailsys.com> Message-ID: <20110711082312.GB5176@dcvr.yhbt.net> Marc Sernatinger wrote: > Tiago Scolari gmail.com> writes: > > Here are some backtraces from hoptoad: > > This is one of the errors we see as part of our issues with our > database connection when using unicorn. Do you have any information > about your stack to share? He was using preload_app, but seems to have fixed it by reverting to an older Unicorn (3.7.0, he was on 4.0.0, so you seem to be hitting a a different problem). ref: http://mid.gmane.org/20110630194515.GA29105 at dcvr.yhbt.net -- Eric Wong From matthewcalebsmith at gmail.com Mon Jul 11 12:07:59 2011 From: matthewcalebsmith at gmail.com (Matt Smith) Date: Mon, 11 Jul 2011 09:07:59 -0700 Subject: Unicorn vs Apache Message-ID: <1310400479.3665.44.camel@antium> Hello all, I have always deploys rails apps with unicorn and nginx as a reverse proxy in the past. However, I am working with a new firm and they would like to use Apache with mod_pagespeed in front of unicorn. We will be deploying a rails 3.1 app with streaming. To me, mod_pagespeed does not seem like a magic bullet, as it appears to be a collection of best practices which have to be rewritten as a page is being served. Why not code with the best practices in mind as to not have to scan and yet again reprocess the content? Searching on the web, I find few entries about mod_page speed and rails, and relatively few entries about apache with unicorn. To me it seems that using best practices, with appropriate use of caching, and nginx would be a better solution than apache with mod_pagespeed. Questions: 1) Does nginx provide any necessary services to unicorn, that apache cannot provide? 2) If apache can provide all necessary services, does it do them as well as nginx. 3) We would like the fastest user experience possible. Does apache with mod_pagespeed hold any weight here over nginx? 4) Part of a fast user experience is how fast you get the page, correct? Seems like nginx has the upper hand here, due to nginx's nio vs apache's threading. 5) Would it make since to put nginx in front of apache w/mod_pagespeed (or visa versa), to use the applicable mod_pagespeed filters? Or is this a bad idea to begin with? 6) Is there anything else I am missing? Are there any specific resources I need to look at? 7) Should we go with nginx or apache? (Opinions ok.) Much thanks, Matt Smith From steve at steveklabnik.com Mon Jul 11 12:50:41 2011 From: steve at steveklabnik.com (Steve Klabnik) Date: Mon, 11 Jul 2011 12:50:41 -0400 Subject: Unicorn vs Apache In-Reply-To: <1310400479.3665.44.camel@antium> References: <1310400479.3665.44.camel@antium> Message-ID: I only have a barely informed opinion here, but I will say two things: 1) You do know that 3.1 streaming is still very much experimental, right? 2) Introducing an additional layer in front of the application by putting nginx in front of apache doesn't seem to be a good idea to me, though I have done it in the past. Performance was the _last_ consideration with that application. 3) We can talk in theory all we want, but we don't have your application. Try it, and profile. Then you'll have a real answer, not some crap that we all make up. Especially with something like webserver setup, you should be able to just try all these combinations in a day or two, and see what works for your app. There is no silver bullet. -Steve From normalperson at yhbt.net Mon Jul 11 14:45:56 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 11 Jul 2011 11:45:56 -0700 Subject: Unicorn vs Apache In-Reply-To: <1310400479.3665.44.camel@antium> References: <1310400479.3665.44.camel@antium> Message-ID: <20110711184556.GA2024@dcvr.yhbt.net> Matt Smith wrote: > Hello all, > > I have always deploys rails apps with unicorn and nginx as a reverse > proxy in the past. However, I am working with a new firm and they would > like to use Apache with mod_pagespeed in front of unicorn. > > We will be deploying a rails 3.1 app with streaming. To me, > mod_pagespeed does not seem like a magic bullet, as it appears to be a > collection of best practices which have to be rewritten as a page is > being served. note: mod_pagespeed is totally independent if Rails 3.1 streaming. Keep in mind the most important feature of nginx Unicorn relies on (full request/response buffering) also defeats Rails 3.1 streaming because of how it's currently implemented. > Why not code with the best practices in mind as to not > have to scan and yet again reprocess the content? My thoughts exactly :) But implementing those best practices isn't cheap, either. If you have to run mod_pagespeed, the safest configuration would be: nginx <-> mod_pagespeed <-> unicorn Of course that would introduce latency. Maybe there's Rack middleware that works like mod_pagespeed... > Searching on the web, I find few entries about mod_page speed and rails, > and relatively few entries about apache with unicorn. Apache + Unicorn is still unsupported since (as far as anybody knows), it doesn't fully buffer responses and requests to completely isolate Unicorn from the harmful effects of slow clients. > To me it seems that using best practices, with appropriate use of > caching, and nginx would be a better solution than apache with > mod_pagespeed. I agree. > Questions: > > 1) Does nginx provide any necessary services to unicorn, that apache > cannot provide? nginx provides full request + response buffering (all uploads, and large response bodies that can't fit into socket buffers). nginx also provides inexpensive handling of idle keepalive connections; so you can have a lot of idle connections for little memory usage. Apache 2.x mpm_event /should/ be as good *if* (and only if) it an provide the full request/response buffering; but I don't have any experience with it. mpm_worker can be fairly efficient with Linux+NPTL if you can use smaller pthreads stack sizes (and are using 64-bit). > 2) If apache can provide all necessary services, does it do them as well > as nginx. Buffering + keepalive maintenance is the most important thing I care about, and as far as I know, Apache doesn't do that as well as nginx. > 3) We would like the fastest user experience possible. Does apache with > mod_pagespeed hold any weight here over nginx? The full request/response buffering of nginx also hurts latency a little, so the user experience is slightly worse under low load. However, nginx does the best job of keeping things going under high load. > 4) Part of a fast user experience is how fast you get the page, correct? > Seems like nginx has the upper hand here, due to nginx's nio vs apache's > threading. Not sure what "nio" means... non-blocking I/O? Apache can use mpm_event which is like the non-blocking + event loop nginx uses, but AFAIK the buffering of requests responses is not like what nginx implements. > 5) Would it make since to put nginx in front of apache w/mod_pagespeed > (or visa versa), to use the applicable mod_pagespeed filters? Or is this > a bad idea to begin with? nginx <-> mod_pagespeed <-> unicorn should be alright, but of course not ideal because of the extra copying + latency involved. For non-Ruby/Rack apps, I've always recommended nginx sit in front of any Apache 1.3 or 2.x+mpm_prefork apps that are exposed to slow clients, too. > 6) Is there anything else I am missing? Are there any specific resources > I need to look at? If you're willing to take a risk with Rainbows!, I posted some notes about making it work with Rails 3.1 streaming here: http://thread.gmane.org/gmane.comp.lang.ruby.rainbows.general/229 Keep in mind nobody I know of uses Rainbows! in production; but it /should/ be able to work without nginx (or any proxy). > 7) Should we go with nginx or apache? (Opinions ok.) nginx remains the only supported reverse proxy for Unicorn at this moment. -- Eric Wong From normalperson at yhbt.net Mon Jul 11 14:50:26 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 11 Jul 2011 11:50:26 -0700 Subject: Unicorn vs Apache In-Reply-To: References: <1310400479.3665.44.camel@antium> Message-ID: <20110711185026.GB2024@dcvr.yhbt.net> Steve Klabnik wrote: > 3) We can talk in theory all we want, but we don't have your > application. Try it, and profile. Then you'll have a real answer, not > some crap that we all make up. Especially with something like > webserver setup, you should be able to just try all these combinations > in a day or two, and see what works for your app. Be careful about relying on profiling/benchmarking tools, most of them don't replicate network latency and simulate slow/idle/keepalive clients at all. I wrote david many years ago to simulate slow client behavior and to convince Apache mpm_prefork + (mod_php|mod_perl) users to put nginx in front of their apps regardless of what benchmarks were saying: http://bogomips.org/david.git -- Eric Wong From steve at steveklabnik.com Mon Jul 11 14:57:27 2011 From: steve at steveklabnik.com (Steve Klabnik) Date: Mon, 11 Jul 2011 14:57:27 -0400 Subject: Unicorn vs Apache In-Reply-To: <20110711184556.GA2024@dcvr.yhbt.net> References: <1310400479.3665.44.camel@antium> <20110711184556.GA2024@dcvr.yhbt.net> Message-ID: > Of course that would introduce latency. ?Maybe there's Rack middleware > that works like mod_pagespeed... I knew I remembered something like this: http://rack-pagespeed.heroku.com/ I think that's what this is... From steve at steveklabnik.com Mon Jul 11 15:22:40 2011 From: steve at steveklabnik.com (Steve Klabnik) Date: Mon, 11 Jul 2011 15:22:40 -0400 Subject: Unicorn vs Apache In-Reply-To: <20110711185026.GB2024@dcvr.yhbt.net> References: <1310400479.3665.44.camel@antium> <20110711185026.GB2024@dcvr.yhbt.net> Message-ID: Quite fair. Like I said, barely informed opinion here. :) From matthewcalebsmith at gmail.com Mon Jul 11 17:43:12 2011 From: matthewcalebsmith at gmail.com (Matt Smith) Date: Mon, 11 Jul 2011 14:43:12 -0700 Subject: Unicorn vs Apache Message-ID: <1310420592.3665.81.camel@antium> Thanks Eric and Steve, I really appreciate the input and time. And thanks for reading through my typos... You understood exactly what I meant to say. I have read about Rainbows! before. I will have to give it a shot on some upcomming projects, as I really enjoy using Unicorn. Thanks for pointing out rack-pagespeed. Much thanks, Matt Smith From normalperson at yhbt.net Tue Jul 12 21:28:36 2011 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 13 Jul 2011 01:28:36 +0000 Subject: [PATCH] http: reject non-LWS CTL chars (0..31 + 127) in field values Message-ID: <20110713012836.GA29441@dcvr.yhbt.net> Would anybody be negatively affected by this change? I've been seeing \x00 bytes in HTTP headers from clients and would rather stop those clients earlier rather than later. >From 4a8ddcd017a75b9bc99190dc565880615709d810 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Tue, 12 Jul 2011 23:52:33 +0000 Subject: [PATCH] http: reject non-LWS CTL chars (0..31 + 127) in field values RFC 2616 doesn't appear to allow most CTL bytes even though Mongrel always did. Rack::Lint disallows 0..31, too, though we allow "\t" (HT, 09) since it's LWS and allowed by RFC 2616. --- ext/unicorn_http/unicorn_http_common.rl | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/ext/unicorn_http/unicorn_http_common.rl b/ext/unicorn_http/unicorn_http_common.rl index cf93fec..cc1d455 100644 --- a/ext/unicorn_http/unicorn_http_common.rl +++ b/ext/unicorn_http/unicorn_http_common.rl @@ -20,6 +20,7 @@ pchar = (uchar | ":" | "@" | "&" | "=" | "+"); tspecials = ("(" | ")" | "<" | ">" | "@" | "," | ";" | ":" | "\\" | "\"" | "/" | "[" | "]" | "?" | "=" | "{" | "}" | " " | "\t"); lws = (" " | "\t"); + content = ((any -- CTL) | lws); # elements token = (ascii -- (CTL | tspecials)); @@ -50,9 +51,9 @@ field_name = ( token -- ":" )+ >start_field $snake_upcase_field %write_field; - field_value = any* >start_value %write_value; + field_value = content* >start_value %write_value; - value_cont = lws+ any* >start_value %write_cont_value; + value_cont = lws+ content* >start_value %write_cont_value; message_header = ((field_name ":" lws* field_value)|value_cont) :> CRLF; chunk_ext_val = token*; -- Eric Wong From normalperson at yhbt.net Wed Jul 13 19:52:52 2011 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 13 Jul 2011 23:52:52 +0000 Subject: [PATCH] http: reject non-LWS CTL chars (0..31 + 127) in field values In-Reply-To: <20110713012836.GA29441@dcvr.yhbt.net> References: <20110713012836.GA29441@dcvr.yhbt.net> Message-ID: <20110713235252.GA4615@dcvr.yhbt.net> Eric Wong wrote: > Would anybody be negatively affected by this change? I've been seeing > \x00 bytes in HTTP headers from clients and would rather stop those > clients earlier rather than later. Pushed out with tests to git://bogomips.org/unicorn.git From pin1960 at att.net Sat Jul 16 05:53:25 2011 From: pin1960 at att.net (Joseph Osei) Date: Sat, 16 Jul 2011 02:53:25 -0700 Subject: HELLO URGENT Message-ID: Greetings, I have interest of investing in your country as well as seek your services in a private and confidential matter. I am a senior Auditor heading computing department here in our bank. On my routine inspection I discovered a dormant domiciliary account with a Balance of ($159. Million USD}.I write to seek your indulgence and assistance in transferring this fund to your country through legal means. I am proposing to make this transfer to a designated bank account of your choice in your country for investment. At the conclusion of this transaction, you will be given 30% of the total amount, 70% will be for me and my Bank colleagues who will help make this business a success. Let me know if you can handle it. When I hear from you, we then commence all arrangements and I will give you more details on how we should handle this project. Please treat this business with utmost confidentiality and send me the following as asap: (1) Full names: (2) Private phone number: (3) Current residential address: (4) Occupation with company name if available: (5) Age and Sex: Regards, Joseph Osei From pin1960 at att.net Sun Jul 17 09:52:32 2011 From: pin1960 at att.net (Joseph Osei) Date: Sun, 17 Jul 2011 06:52:32 -0700 Subject: HELLO URGENT Message-ID: Greetings, I have interest of investing in your country as well as seek your services in a private and confidential matter. I am a senior Auditor heading computing department here in our bank. On my routine inspection I discovered a dormant domiciliary account with a Balance of ($159. Million USD}.I write to seek your indulgence and assistance in transferring this fund to your country through legal means. I am proposing to make this transfer to a designated bank account of your choice in your country for investment. At the conclusion of this transaction, you will be given 30% of the total amount, 70% will be for me and my Bank colleagues who will help make this business a success. Let me know if you can handle it. When I hear from you, we then commence all arrangements and I will give you more details on how we should handle this project. Please treat this business with utmost confidentiality and send me the following as asap: (1) Full names: (2) Private phone number: (3) Current residential address: (4) Occupation with company name if available: (5) Age and Sex: Regards, Joseph Osei From pin1960 at att.net Sun Jul 17 22:26:22 2011 From: pin1960 at att.net (Joseph Osei) Date: Sun, 17 Jul 2011 19:26:22 -0700 Subject: HELLO URGENT Message-ID: Greetings, I have interest of investing in your country as well as seek your services in a private and confidential matter. I am a senior Auditor heading computing department here in our bank. On my routine inspection I discovered a dormant domiciliary account with a Balance of ($159. Million USD}.I write to seek your indulgence and assistance in transferring this fund to your country through legal means. I am proposing to make this transfer to a designated bank account of your choice in your country for investment. At the conclusion of this transaction, you will be given 30% of the total amount, 70% will be for me and my Bank colleagues who will help make this business a success. Let me know if you can handle it. When I hear from you, we then commence all arrangements and I will give you more details on how we should handle this project. Please treat this business with utmost confidentiality and send me the following as asap: (1) Full names: (2) Private phone number: (3) Current residential address: (4) Occupation with company name if available: (5) Age and Sex: Regards, Joseph Osei From pin1960 at att.net Mon Jul 18 04:34:30 2011 From: pin1960 at att.net (Joseph Osei) Date: Mon, 18 Jul 2011 01:34:30 -0700 Subject: HELLO URGENT Message-ID: Greetings, I have interest of investing in your country as well as seek your services in a private and confidential matter. I am a senior Auditor heading computing department here in our bank. On my routine inspection I discovered a dormant domiciliary account with a Balance of ($159. Million USD}.I write to seek your indulgence and assistance in transferring this fund to your country through legal means. I am proposing to make this transfer to a designated bank account of your choice in your country for investment. At the conclusion of this transaction, you will be given 30% of the total amount, 70% will be for me and my Bank colleagues who will help make this business a success. Let me know if you can handle it. When I hear from you, we then commence all arrangements and I will give you more details on how we should handle this project. Please treat this business with utmost confidentiality and send me the following as asap: (1) Full names: (2) Private phone number: (3) Current residential address: (4) Occupation with company name if available: (5) Age and Sex: Regards, Joseph Osei From douglasmoura27 at gmail.com Mon Jul 18 09:34:57 2011 From: douglasmoura27 at gmail.com (Douglas =?utf-8?b?QW5kcsOp?=) Date: Mon, 18 Jul 2011 13:34:57 +0000 (UTC) Subject: Undefined method after switching to unicorn References: <9E260BC166D04FF1B0B7CBC563B221BA@gmail.com> <4AABABCB7BED4EAC9D72BFFCEF981CEC@gmail.com> <20110630050420.GA19876@dcvr.yhbt.net> <20110630194515.GA29105@dcvr.yhbt.net> Message-ID: Hello, I'm having the same issue with Unicorn 3.7.0 and rails 3.0.9. I don't have any idea what could cause this problem. Please help me!! From normalperson at yhbt.net Mon Jul 18 15:42:07 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 18 Jul 2011 12:42:07 -0700 Subject: Undefined method after switching to unicorn In-Reply-To: References: <9E260BC166D04FF1B0B7CBC563B221BA@gmail.com> <4AABABCB7BED4EAC9D72BFFCEF981CEC@gmail.com> <20110630050420.GA19876@dcvr.yhbt.net> <20110630194515.GA29105@dcvr.yhbt.net> Message-ID: <20110718194207.GA27898@dcvr.yhbt.net> Douglas Andr? wrote: > Hello, I'm having the same issue with Unicorn 3.7.0 and rails 3.0.9. > I don't have any idea what could cause this problem. Please help me!! Did you try disabling preload_app? If that doesn't help, help us help you by telling us everything you can about your setup, including (and not limited to): * Anything that maintains network connections/open files * All gems and versions you use, including the versions of 3rd party libraries (e.g. OpenSSL version) * OS version * Ruby version * CPU architecture -- Eric Wong From desantis.maurizio at gmail.com Sat Jul 23 16:18:29 2011 From: desantis.maurizio at gmail.com (Maurizio De Santis) Date: Sat, 23 Jul 2011 22:18:29 +0200 Subject: Problems running unicorn_rails without bundle exec (in a RVM context) Message-ID: Hello, I cannot run unicorn_rails out without using bundle exec . This is the error: /home/user/.rvm/gems/ruby-1.9.2-p180/gems/bundler-1.0.15/lib/bundler/runtime.rb:31:in `block in setup': You have already activated rack 1.3.1, but your Gemfile requires rack 1.2.3. Consider using bundle exec. (Gem::LoadError) Maybe RVM is involved? Maybe I am involved? Please add me in the CC replies, I am not subscribed to the newsletter. Cheers -- Maurizio De Santis From normalperson at yhbt.net Sat Jul 23 16:57:57 2011 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 23 Jul 2011 13:57:57 -0700 Subject: Problems running unicorn_rails without bundle exec (in a RVM context) In-Reply-To: References: Message-ID: <20110723205757.GA22777@dcvr.yhbt.net> Maurizio De Santis wrote: > Hello, > > I cannot run unicorn_rails out without using bundle exec . This is the error: > > /home/user/.rvm/gems/ruby-1.9.2-p180/gems/bundler-1.0.15/lib/bundler/runtime.rb:31:in > `block in setup': You have already activated rack 1.3.1, but your > Gemfile requires rack 1.2.3. Consider using bundle exec. > (Gem::LoadError) > > Maybe RVM is involved? Maybe I am involved? You should use bundle exec if your app is using bundler. Otherwise you can try uninstall the conflicting version of rack from your system/RVM installation, but that defeats the purpose of bundler... > Please add me in the CC replies, I am not subscribed to the newsletter. Thanks for making a note of this :> -- Eric Wong From desantis.maurizio at gmail.com Sat Jul 23 19:03:42 2011 From: desantis.maurizio at gmail.com (Maurizio De Santis) Date: Sun, 24 Jul 2011 01:03:42 +0200 Subject: Problems running unicorn_rails without bundle exec (in a RVM context) In-Reply-To: <20110723205757.GA22777@dcvr.yhbt.net> References: <20110723205757.GA22777@dcvr.yhbt.net> Message-ID: > You should use bundle exec if your app is using bundler. ?Otherwise you > can try uninstall the conflicting version of rack from your system/RVM > installation, but that defeats the purpose of bundler... uhm, this will make my life more difficoult because I'm trying to write an upstart srcript for unicorn_rails, and so I have to consider this too... more work for me :) >> Please add me in the CC replies, I am not subscribed to the newsletter. > > Thanks for making a note of this :> > I read the ISSUES page carefully before to write :) Thank you! -- Maurizio De Santis From swindsor at gmail.com Sat Jul 23 20:48:12 2011 From: swindsor at gmail.com (Scott Windsor) Date: Sat, 23 Jul 2011 17:48:12 -0700 Subject: Problems running unicorn_rails without bundle exec (in a RVM context) In-Reply-To: References: <20110723205757.GA22777@dcvr.yhbt.net> Message-ID: On Sat, Jul 23, 2011 at 4:03 PM, Maurizio De Santis wrote: >> You should use bundle exec if your app is using bundler. ?Otherwise you >> can try uninstall the conflicting version of rack from your system/RVM >> installation, but that defeats the purpose of bundler... > > uhm, this will make my life more difficoult because I'm trying to > write an upstart srcript for unicorn_rails, and so I have to consider > this too... more work for me :) > Have you looked at foreman? http://blog.daviddollar.org/2011/05/06/introducing-foreman.html It's a pretty good approach to process management and can export the upstart scripts for you... - scott From jesse at jc00ke.com Mon Jul 25 16:21:31 2011 From: jesse at jc00ke.com (Jesse Cooke) Date: Mon, 25 Jul 2011 13:21:31 -0700 Subject: SIGTERM not actually killing processes Message-ID: Hi, Unicorn is saying it's terminating but it's not actually. Check out the gist:?https://gist.github.com/1104930 Using: - Ruby 1.9.2p180 - unicorn 4.0.1 - kgio 2.6.0 - bundler 1.0.15 - Linux maynard 2.6.38-10-generic #46-Ubuntu SMP Tue Jun 28 15:07:17 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux Please let me know if there's any other info I can provide. Thanks, Jesse -------------------------------------------- Jesse Cooke ::?N-tier Engineer jc00ke.com?/ @jc00ke From normalperson at yhbt.net Mon Jul 25 17:22:15 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 25 Jul 2011 14:22:15 -0700 Subject: SIGTERM not actually killing processes In-Reply-To: References: Message-ID: <20110725212215.GA17101@dcvr.yhbt.net> Jesse Cooke wrote: > Hi, > Unicorn is saying it's terminating but it's not actually. Check out > the gist:?https://gist.github.com/1104930 > > Using: > - Ruby 1.9.2p180 > - unicorn 4.0.1 > - kgio 2.6.0 > - bundler 1.0.15 > - Linux maynard 2.6.38-10-generic #46-Ubuntu SMP Tue Jun 28 15:07:17 > UTC 2011 x86_64 x86_64 x86_64 GNU/Linux > > Please let me know if there's any other info I can provide. Since it's a 2.6.38 kernel, I it could be this bug: https://bugzilla.kernel.org/show_bug.cgi?id=32922 Linux 2.6.38.4 should have the fix, but I'm not sure if Ubuntu backported that fix to its 2.6.38-10 package. Rubyists have seen this issue a few times already: http://redmine.ruby-lang.org/issues/4777 http://redmine.ruby-lang.org/issues/4939 Also, can you try sending signals directly to Unicorn without foreman? I'm not familiar with foreman at all, maybe somebody else on this list is... -- Eric Wong From jesse at jc00ke.com Mon Jul 25 19:20:53 2011 From: jesse at jc00ke.com (Jesse Cooke) Date: Mon, 25 Jul 2011 16:20:53 -0700 Subject: SIGTERM not actually killing processes In-Reply-To: References: <20110725212215.GA17101@dcvr.yhbt.net> Message-ID: Sending TERM directly to the master process killed it. -------------------------------------------- Jesse Cooke ::?N-tier Engineer jc00ke.com?/ @jc00ke On Mon, Jul 25, 2011 at 3:30 PM, Jesse Cooke wrote: > > Sending TERM directly to the master process killed it. > > -------------------------------------------- > Jesse Cooke ::?N-tier Engineer > jc00ke.com?/ @jc00ke > > > On Mon, Jul 25, 2011 at 2:32 PM, Alex Sharp wrote: >> >> On Monday, July 25, 2011 at 2:22 PM, Eric Wong wrote: >> >> Jesse Cooke wrote: >> >> Hi, >> Unicorn is saying it's terminating but it's not actually. Check out >> the gist:?https://gist.github.com/1104930 >> >> Using: >> - Ruby 1.9.2p180 >> - unicorn 4.0.1 >> - kgio 2.6.0 >> - bundler 1.0.15 >> - Linux maynard 2.6.38-10-generic #46-Ubuntu SMP Tue Jun 28 15:07:17 >> UTC 2011 x86_64 x86_64 x86_64 GNU/Linux >> >> Please let me know if there's any other info I can provide. >> >> Since it's a 2.6.38 kernel, I it could be this bug: >> https://bugzilla.kernel.org/show_bug.cgi?id=32922 >> >> Linux 2.6.38.4 should have the fix, but I'm not sure if Ubuntu >> backported that fix to its 2.6.38-10 package. >> >> Rubyists have seen this issue a few times already: >> http://redmine.ruby-lang.org/issues/4777 >> http://redmine.ruby-lang.org/issues/4939 >> >> >> Also, can you try sending signals directly to Unicorn without foreman? >> I'm not familiar with foreman at all, maybe somebody else on this list >> is... >> >> Not familiar with foreman, but I've verified the issues you've linked to with another non-unicorn problem (god, actually). As of a couple of weeks ago, the kernel bug still existed in ubuntu 11.04, and to my knowledge, this is the only affected major version. We're in the process of moving to ubuntu 10.10 (mostly specifically because of this bug), which is the most recent LTS release, and not affected by the signal trapping bug. >> >> -- >> Eric Wong >> _______________________________________________ >> Unicorn mailing list - mongrel-unicorn at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mongrel-unicorn >> Do not quote signatures (like this one) or top post when replying >> >