From chris at codeintensity.com  Sat Sep  1 00:54:53 2007
From: chris at codeintensity.com (Christopher Bailey)
Date: Fri, 31 Aug 2007 21:54:53 -0700
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
Message-ID: <8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>

We are finding that anytime our application sends back large files to the
requestor, we start chewing up memory fast.  We haven't determined the
precise cause or how to solve it yet, but it sounds like the same kind of
situation.  In our case, the particular large files are usually not
generated by Builder, but typically are actually files being
downloaded/transferred (e.g. images, documents, etc.).

On 8/19/07, Chris Carter <cdcarter at gmail.com> wrote:
>
> On 8/19/07, Wayne E. Seguin <wayneeseguin at gmail.com> wrote:
> >
> > Massimo,
> >
> > You should post this to ruby-talk mailing list. Make sure to include the
> > version of ruby, the version  rails, as well as any plugins and/or gems
> your
> > app uses. If at all possible try to include some code for analysis as
> many
> > people on that list will jump right on that to tell you what might be
> > causing it and a better way to go about it (if one exists).
> >
> > Best,
> >
> >   ~Wayne
>
> I think rails-talk would be even more appropriate.
>
>
> --
> Chris Carter
> concentrationstudios.com
> brynmawrcs.com
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070831/82795dc0/attachment-0001.html 

From zedshaw at zedshaw.com  Sat Sep  1 01:40:29 2007
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Sat, 1 Sep 2007 01:40:29 -0400
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
Message-ID: <20070901014029.837dba7d.zedshaw@zedshaw.com>

On Fri, 31 Aug 2007 21:54:53 -0700
"Christopher Bailey" <chris at codeintensity.com> wrote:

> We are finding that anytime our application sends back large files to the
> requestor, we start chewing up memory fast.  We haven't determined the
> precise cause or how to solve it yet, but it sounds like the same kind of
> situation.  In our case, the particular large files are usually not
> generated by Builder, but typically are actually files being
> downloaded/transferred (e.g. images, documents, etc.).

This is because Mongrel collects your large file response into a StringIO so that it can keep rails happy.  After the StringIO is done and rails leaves the locked section of code it then shoves that out the door on the socket.

You are expecting to write a file, in rails, using Ruby's crappy IO, and that it would go immediately on the socket.  That's probably why you're seeing this.

In reality, if it's a large file, and you know where it is, then you should let a real web server handle it, or at a minimum write a Mongrel Handler to do the real heavy lifting.

There's plenty of information on doing this, but seriously, do not ever use send_file in rails or similar.  It's just a waste of resources, and even if you need to authenticate you can use X-Sendfile in apache or nginx and that'll let you auth someone then send the file.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From jgeiger at gmail.com  Sat Sep  1 08:41:32 2007
From: jgeiger at gmail.com (Joey Geiger)
Date: Sat, 1 Sep 2007 08:41:32 -0400
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <20070901014029.837dba7d.zedshaw@zedshaw.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
	<20070901014029.837dba7d.zedshaw@zedshaw.com>
Message-ID: <466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>

Now, I'm opening a different can of worms, but how do you suggest
sending an image file from the database without using send_data, or
does send_data not suffer from the same issues as send_file?

I tried using X-Sendfile and ran into issues that files weren't being
pulled out of the DB correctly. (Probably should be able to fix this
now, after I figured it out)

On 9/1/07, Zed A. Shaw <zedshaw at zedshaw.com> wrote:
> On Fri, 31 Aug 2007 21:54:53 -0700
> "Christopher Bailey" <chris at codeintensity.com> wrote:
>
> > We are finding that anytime our application sends back large files to the
> > requestor, we start chewing up memory fast.  We haven't determined the
> > precise cause or how to solve it yet, but it sounds like the same kind of
> > situation.  In our case, the particular large files are usually not
> > generated by Builder, but typically are actually files being
> > downloaded/transferred (e.g. images, documents, etc.).
>
> This is because Mongrel collects your large file response into a StringIO so that it can keep rails happy.  After the StringIO is done and rails leaves the locked section of code it then shoves that out the door on the socket.
>
> You are expecting to write a file, in rails, using Ruby's crappy IO, and that it would go immediately on the socket.  That's probably why you're seeing this.
>
> In reality, if it's a large file, and you know where it is, then you should let a real web server handle it, or at a minimum write a Mongrel Handler to do the real heavy lifting.
>
> There's plenty of information on doing this, but seriously, do not ever use send_file in rails or similar.  It's just a waste of resources, and even if you need to authenticate you can use X-Sendfile in apache or nginx and that'll let you auth someone then send the file.
>
> --
> Zed A. Shaw
> - Hate: http://savingtheinternetwithhate.com/
> - Good: http://www.zedshaw.com/
> - Evil: http://yearofevil.com/
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>

From ezmobius at gmail.com  Sat Sep  1 11:55:31 2007
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Sat, 1 Sep 2007 08:55:31 -0700
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
	<20070901014029.837dba7d.zedshaw@zedshaw.com>
	<466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>
Message-ID: <0F2D93B1-BACD-440C-BEA7-D95F2D08B434@brainspl.at>


On Sep 1, 2007, at 5:41 AM, Joey Geiger wrote:

> Now, I'm opening a different can of worms, but how do you suggest
> sending an image file from the database without using send_data, or
> does send_data not suffer from the same issues as send_file?
>
> I tried using X-Sendfile and ran into issues that files weren't being
> pulled out of the DB correctly. (Probably should be able to fix this
> now, after I figured it out)


	You should never store files in the database in the first place.  
send_data has the same leaky problem as send_file. The filesystem is  
a database for files already and is much more efficient at storing  
files then using the database for it.

Cheers-
-- Ezra Zygmuntowicz 
-- Founder & Ruby Hacker
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From will at hotgazpacho.com  Sat Sep  1 13:46:18 2007
From: will at hotgazpacho.com (Will Green)
Date: Sat, 01 Sep 2007 13:46:18 -0400
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <0F2D93B1-BACD-440C-BEA7-D95F2D08B434@brainspl.at>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>	<20070901014029.837dba7d.zedshaw@zedshaw.com>	<466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>
	<0F2D93B1-BACD-440C-BEA7-D95F2D08B434@brainspl.at>
Message-ID: <46D9A56A.1010408@hotgazpacho.com>

Not necessarily so, Ezra. Storing images in the database is perfectly 
legitimate.  However, just like Rails HTML views, you could implement 
caching of the images on the filesystem (i.e. write them to both the FS 
and the DB). Whatever action "renders" the image could take care of 
caching on the FS, serving the FS version if the DB version has, for 
example, the same MD5 hash as the one in the DB.

Yes, performance will be a bit less than pure FS, but backups are a 
whole lot simpler (just backup and restore the DB). Besides, servers are 
cheap compared to developers (just ask the 37s guys), right?

==
Will Green

Find out why this email is 5 sentences or less at http://five.sentec.es/

From _ at whats-your.name  Sat Sep  1 17:46:30 2007
From: _ at whats-your.name (cr)
Date: Sat, 1 Sep 2007 17:46:30 -0400
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <46D9A56A.1010408@hotgazpacho.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
	<20070901014029.837dba7d.zedshaw@zedshaw.com>
	<466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>
	<0F2D93B1-BACD-440C-BEA7-D95F2D08B434@brainspl.at>
	<46D9A56A.1010408@hotgazpacho.com>
Message-ID: <20070901214630.GA2748@replic.net>

> Not necessarily so, Ezra. Storing images in the database is perfectly 
> legitimate.  However, just like Rails HTML views, you could implement 
> caching of the images on the filesystem (i.e. write them to both the FS 
> and the DB). Whatever action "renders" the image could take care of 
> caching on the FS, serving the FS version if the DB version has, for 
> example, the same MD5 hash as the one in the DB.

and why do all that then put them on the fs when you can just put them on the FS in the first place?

> 
> Yes, performance will be a bit less than pure FS, but backups are a 
> whole lot simpler (just backup and restore the DB).

rsync and tar are very simple

> Besides, servers are 
> cheap compared to developers

you get a significant productivity boost out of putting binary files into the database? really?

From zedshaw at zedshaw.com  Sat Sep  1 17:51:09 2007
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Sat, 1 Sep 2007 17:51:09 -0400
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <46D9A56A.1010408@hotgazpacho.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
	<20070901014029.837dba7d.zedshaw@zedshaw.com>
	<466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>
	<0F2D93B1-BACD-440C-BEA7-D95F2D08B434@brainspl.at>
	<46D9A56A.1010408@hotgazpacho.com>
Message-ID: <20070901175109.7983daa0.zedshaw@zedshaw.com>

On Sat, 01 Sep 2007 13:46:18 -0400
Will Green <will at hotgazpacho.com> wrote:

> Not necessarily so, Ezra. Storing images in the database is perfectly 
> legitimate.  However, just like Rails HTML views, you could implement 
> caching of the images on the filesystem (i.e. write them to both the FS 
> and the DB). Whatever action "renders" the image could take care of 
> caching on the FS, serving the FS version if the DB version has, for 
> example, the same MD5 hash as the one in the DB.

No, not right at all.  All RDBMS were originally designed to store relations, not files.  It's only recently that people started putting every damn thing they could into a RDBMS.  The smart folks just put the data on a file system behind a specialized image web server.  Then, when you need to serve the image, you, uh serve it.  Doesn't get much easier than that.

> Yes, performance will be a bit less than pure FS, but backups are a 
> whole lot simpler (just backup and restore the DB). Besides, servers are 
> cheap compared to developers (just ask the 37s guys), right?

That's it?  Backups?  Seriously man, that's a lame reason to do anything.  Especially since backing up a file system is *infinitely* easier than a database.  In real companies around the world there are little children crying because every night the DBA has to shut the production databases down to back them up, even if Oracle says they don't.

Backing up a file system does not require shutting it down, and can even be done with just simple rsync scripts.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From will at hotgazpacho.com  Sat Sep  1 19:17:04 2007
From: will at hotgazpacho.com (Will Green)
Date: Sat, 01 Sep 2007 19:17:04 -0400
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <20070901175109.7983daa0.zedshaw@zedshaw.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>	<20070901014029.837dba7d.zedshaw@zedshaw.com>	<466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>	<0F2D93B1-BACD-440C-BEA7-D95F2D08B434@brainspl.at>	<46D9A56A.1010408@hotgazpacho.com>
	<20070901175109.7983daa0.zedshaw@zedshaw.com>
Message-ID: <46D9F2F0.2070007@hotgazpacho.com>

Please, by all means, feel free to make this argument to my boss...


Zed A. Shaw wrote:
> On Sat, 01 Sep 2007 13:46:18 -0400
> Will Green <will at hotgazpacho.com> wrote:
> 
>> Not necessarily so, Ezra. Storing images in the database is perfectly 
>> legitimate.  However, just like Rails HTML views, you could implement 
>> caching of the images on the filesystem (i.e. write them to both the FS 
>> and the DB). Whatever action "renders" the image could take care of 
>> caching on the FS, serving the FS version if the DB version has, for 
>> example, the same MD5 hash as the one in the DB.
> 
> No, not right at all.  All RDBMS were originally designed to store relations, not files.  It's only recently that people started putting every damn thing they could into a RDBMS.  The smart folks just put the data on a file system behind a specialized image web server.  Then, when you need to serve the image, you, uh serve it.  Doesn't get much easier than that.
> 
>> Yes, performance will be a bit less than pure FS, but backups are a 
>> whole lot simpler (just backup and restore the DB). Besides, servers are 
>> cheap compared to developers (just ask the 37s guys), right?
> 
> That's it?  Backups?  Seriously man, that's a lame reason to do anything.  Especially since backing up a file system is *infinitely* easier than a database.  In real companies around the world there are little children crying because every night the DBA has to shut the production databases down to back them up, even if Oracle says they don't.
> 
> Backing up a file system does not require shutting it down, and can even be done with just simple rsync scripts.
> 

-- 
==
Will Green

Find out why this email is 5 sentences or less at http://five.sentec.es/

From evan at cloudbur.st  Sun Sep  2 01:59:56 2007
From: evan at cloudbur.st (Evan Weaver)
Date: Sun, 2 Sep 2007 01:59:56 -0400
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <46D9F2F0.2070007@hotgazpacho.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
	<20070901014029.837dba7d.zedshaw@zedshaw.com>
	<466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>
	<0F2D93B1-BACD-440C-BEA7-D95F2D08B434@brainspl.at>
	<46D9A56A.1010408@hotgazpacho.com>
	<20070901175109.7983daa0.zedshaw@zedshaw.com>
	<46D9F2F0.2070007@hotgazpacho.com>
Message-ID: <b6f68fc60709012259uf771103obc1662a562a91fdd@mail.gmail.com>

The experimental MyBS MySQL engine can serve columns directly from the
db to the client.

http://www.blobstreaming.org/download/index.php

Just saying.

Evan

On 9/1/07, Will Green <will at hotgazpacho.com> wrote:
> Please, by all means, feel free to make this argument to my boss...
>
>
> Zed A. Shaw wrote:
> > On Sat, 01 Sep 2007 13:46:18 -0400
> > Will Green <will at hotgazpacho.com> wrote:
> >
> >> Not necessarily so, Ezra. Storing images in the database is perfectly
> >> legitimate.  However, just like Rails HTML views, you could implement
> >> caching of the images on the filesystem (i.e. write them to both the FS
> >> and the DB). Whatever action "renders" the image could take care of
> >> caching on the FS, serving the FS version if the DB version has, for
> >> example, the same MD5 hash as the one in the DB.
> >
> > No, not right at all.  All RDBMS were originally designed to store relations, not files.  It's only recently that people started putting every damn thing they could into a RDBMS.  The smart folks just put the data on a file system behind a specialized image web server.  Then, when you need to serve the image, you, uh serve it.  Doesn't get much easier than that.
> >
> >> Yes, performance will be a bit less than pure FS, but backups are a
> >> whole lot simpler (just backup and restore the DB). Besides, servers are
> >> cheap compared to developers (just ask the 37s guys), right?
> >
> > That's it?  Backups?  Seriously man, that's a lame reason to do anything.  Especially since backing up a file system is *infinitely* easier than a database.  In real companies around the world there are little children crying because every night the DBA has to shut the production databases down to back them up, even if Oracle says they don't.
> >
> > Backing up a file system does not require shutting it down, and can even be done with just simple rsync scripts.
> >
>
> --
> ==
> Will Green
>
> Find out why this email is 5 sentences or less at http://five.sentec.es/
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>


-- 
Evan Weaver
Cloudburst, LLC

From j.list at blueboxdev.com  Sun Sep  2 03:17:46 2007
From: j.list at blueboxdev.com (Jesse Proudman)
Date: Sun, 2 Sep 2007 00:17:46 -0700
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <b6f68fc60709012259uf771103obc1662a562a91fdd@mail.gmail.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
	<20070901014029.837dba7d.zedshaw@zedshaw.com>
	<466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>
	<0F2D93B1-BACD-440C-BEA7-D95F2D08B434@brainspl.at>
	<46D9A56A.1010408@hotgazpacho.com>
	<20070901175109.7983daa0.zedshaw@zedshaw.com>
	<46D9F2F0.2070007@hotgazpacho.com>
	<b6f68fc60709012259uf771103obc1662a562a91fdd@mail.gmail.com>
Message-ID: <7013DB4D-AFCF-4AE2-929C-FDDF904D381A@blueboxdev.com>

Does any one else find pleasure in the name MyBS?

--

Jesse Proudman,  Blue Box Group, LLC




On Sep 1, 2007, at 10:59 PM, Evan Weaver wrote:

> The experimental MyBS MySQL engine can serve columns directly from the
> db to the client.


From jgeiger at gmail.com  Sun Sep  2 09:18:09 2007
From: jgeiger at gmail.com (Joey Geiger)
Date: Sun, 2 Sep 2007 09:18:09 -0400
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <7013DB4D-AFCF-4AE2-929C-FDDF904D381A@blueboxdev.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
	<20070901014029.837dba7d.zedshaw@zedshaw.com>
	<466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>
	<0F2D93B1-BACD-440C-BEA7-D95F2D08B434@brainspl.at>
	<46D9A56A.1010408@hotgazpacho.com>
	<20070901175109.7983daa0.zedshaw@zedshaw.com>
	<46D9F2F0.2070007@hotgazpacho.com>
	<b6f68fc60709012259uf771103obc1662a562a91fdd@mail.gmail.com>
	<7013DB4D-AFCF-4AE2-929C-FDDF904D381A@blueboxdev.com>
Message-ID: <466af3440709020618g79f22e17k243d268289a9ee7c@mail.gmail.com>

I'm storing the images in a DB because they are shared between 4 web
servers, any of which can submit an image. We are rsynching the MAIN
webserver to the 3 other subboxes, but we aren't going the other way.

In a perfect world, I'd have a separate image server or file storage
area, but we don't have the time or money to do that. I'm not serving
the images every time from the DB, rails is actually caching the file
on the first read anyway.


On 9/2/07, Jesse Proudman <j.list at blueboxdev.com> wrote:
> Does any one else find pleasure in the name MyBS?
>
> --
>
> Jesse Proudman,  Blue Box Group, LLC
>
>
>
>
> On Sep 1, 2007, at 10:59 PM, Evan Weaver wrote:
>
> > The experimental MyBS MySQL engine can serve columns directly from the
> > db to the client.
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>

From wayneeseguin at gmail.com  Sun Sep  2 21:29:58 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Sun, 2 Sep 2007 21:29:58 -0400
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <7013DB4D-AFCF-4AE2-929C-FDDF904D381A@blueboxdev.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
	<20070901014029.837dba7d.zedshaw@zedshaw.com>
	<466af3440709010541n1100051elb4ee8b6271d6227@mail.gmail.com>
	<0F2D93B1-BACD-440C-BEA7-D95F2D08B434@brainspl.at>
	<46D9A56A.1010408@hotgazpacho.com>
	<20070901175109.7983daa0.zedshaw@zedshaw.com>
	<46D9F2F0.2070007@hotgazpacho.com>
	<b6f68fc60709012259uf771103obc1662a562a91fdd@mail.gmail.com>
	<7013DB4D-AFCF-4AE2-929C-FDDF904D381A@blueboxdev.com>
Message-ID: <30BAD1B2-AEDD-4809-A11F-83FB4C1BF002@gmail.com>

On Sep 02, 2007, at 03:17 , Jesse Proudman wrote:
> Does any one else find pleasure in the name MyBS?

Very much so :)

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070902/c8a23bd3/attachment.html 

From flujan at gmail.com  Mon Sep  3 19:09:40 2007
From: flujan at gmail.com (Fernando Lujan)
Date: Mon, 3 Sep 2007 20:09:40 -0300
Subject: [Mongrel] Mongrel and Memory usage.
Message-ID: <631837390709031609x3cca5608s6a2756e04e9a6f6f@mail.gmail.com>

Hi,

I am using mongrel for a long time... But just now I see a problem. I am
running 3 instances using mongrel_cluster and nginx as a reverse proxy. When
I start the cluster I have a memory usage of 20MB per instance... In the end
of the day this usage is something about 230MB for each instance. It should
be a memory leak?

Thanks for the help.

-- 
Fernando Lujan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070903/3b3d3f1a/attachment.html 

From bk at benjaminkrause.com  Tue Sep  4 03:27:59 2007
From: bk at benjaminkrause.com (Benjamin Krause)
Date: Tue, 4 Sep 2007 09:27:59 +0200
Subject: [Mongrel] Mongrel and Memory usage.
In-Reply-To: <631837390709031609x3cca5608s6a2756e04e9a6f6f@mail.gmail.com>
References: <631837390709031609x3cca5608s6a2756e04e9a6f6f@mail.gmail.com>
Message-ID: <5A4D2AA6-B55D-4D26-BF4A-D4B0B582773C@benjaminkrause.com>

Fernando,

> I am using mongrel for a long time... But just now I see a problem.  
> I am running 3 instances using mongrel_cluster and nginx as a  
> reverse proxy. When I start the cluster I have a memory usage of  
> 20MB per instance... In the end of the day this usage is something  
> about 230MB for each instance. It should be a memory leak?

This is most probably a memory leak, but not necessarily a mongrel  
related issue.

You should check your app with bleak_house[1] or any other memory  
profiling tool to see, where the leak comes from. I had the same  
problem and the leak was caused by ferret[2]. Some native extensions  
(like rmagick) might be the problem.

Ben

[1] http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/ 
README.html
[2] http://blog.omdb-beta.org/2007/7/29/tracking-down-a-memory-leak- 
in-ferret-0-11-4


From matte at ruckuswireless.com  Tue Sep  4 16:37:52 2007
From: matte at ruckuswireless.com (Matte Edens)
Date: Tue, 04 Sep 2007 13:37:52 -0700
Subject: [Mongrel] Mongrel and Memory usage.
In-Reply-To: <5A4D2AA6-B55D-4D26-BF4A-D4B0B582773C@benjaminkrause.com>
References: <631837390709031609x3cca5608s6a2756e04e9a6f6f@mail.gmail.com>
	<5A4D2AA6-B55D-4D26-BF4A-D4B0B582773C@benjaminkrause.com>
Message-ID: <46DDC220.3020904@ruckuswireless.com>

Here's another possibility that I encountered before I was smarter (heh)

Are you providing download capabilities of large files.  I manage a 
support site for firmware downloads and other documents.  Before I 
discovered the X-Sendfile header, mongrel was loading and serving the 
files.  I had MAJOR memory spikes.  Now that Apache is grabbing that 
request (and sending the file) before proxying the request to Mongrel, 
no more problems.

matte - webmonkey
webmaster at ruckuswireless.com

Benjamin Krause wrote:
> Fernando,
>
>   
>> I am using mongrel for a long time... But just now I see a problem.  
>> I am running 3 instances using mongrel_cluster and nginx as a  
>> reverse proxy. When I start the cluster I have a memory usage of  
>> 20MB per instance... In the end of the day this usage is something  
>> about 230MB for each instance. It should be a memory leak?
>>     
>
> This is most probably a memory leak, but not necessarily a mongrel  
> related issue.
>
> You should check your app with bleak_house[1] or any other memory  
> profiling tool to see, where the leak comes from. I had the same  
> problem and the leak was caused by ferret[2]. Some native extensions  
> (like rmagick) might be the problem.
>
> Ben
>
> [1] http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/ 
> README.html
> [2] http://blog.omdb-beta.org/2007/7/29/tracking-down-a-memory-leak- 
> in-ferret-0-11-4
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070904/bd49ce45/attachment.html 

From chris at codeintensity.com  Tue Sep  4 19:55:36 2007
From: chris at codeintensity.com (Christopher Bailey)
Date: Tue, 4 Sep 2007 16:55:36 -0700
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <20070901014029.837dba7d.zedshaw@zedshaw.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
	<20070901014029.837dba7d.zedshaw@zedshaw.com>
Message-ID: <8c8513100709041655v381657baj36553e82cc338442@mail.gmail.com>

Thanks Zed.  This is the direction we are going, but it is non-trivial due
to how our storage and authentication system works.  A reality
none-the-less.

On 8/31/07, Zed A. Shaw <zedshaw at zedshaw.com> wrote:
>
> On Fri, 31 Aug 2007 21:54:53 -0700
> "Christopher Bailey" <chris at codeintensity.com> wrote:
>
> > We are finding that anytime our application sends back large files to
> the
> > requestor, we start chewing up memory fast.  We haven't determined the
> > precise cause or how to solve it yet, but it sounds like the same kind
> of
> > situation.  In our case, the particular large files are usually not
> > generated by Builder, but typically are actually files being
> > downloaded/transferred (e.g. images, documents, etc.).
>
> This is because Mongrel collects your large file response into a StringIO
> so that it can keep rails happy.  After the StringIO is done and rails
> leaves the locked section of code it then shoves that out the door on the
> socket.
>
> You are expecting to write a file, in rails, using Ruby's crappy IO, and
> that it would go immediately on the socket.  That's probably why you're
> seeing this.
>
> In reality, if it's a large file, and you know where it is, then you
> should let a real web server handle it, or at a minimum write a Mongrel
> Handler to do the real heavy lifting.
>
> There's plenty of information on doing this, but seriously, do not ever
> use send_file in rails or similar.  It's just a waste of resources, and even
> if you need to authenticate you can use X-Sendfile in apache or nginx and
> that'll let you auth someone then send the file.
>
> --
> Zed A. Shaw
> - Hate: http://savingtheinternetwithhate.com/
> - Good: http://www.zedshaw.com/
> - Evil: http://yearofevil.com/
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070904/42ce64f6/attachment.html 

From mrueckert at suse.de  Wed Sep  5 09:47:02 2007
From: mrueckert at suse.de (Marcus Rueckert)
Date: Wed, 5 Sep 2007 15:47:02 +0200
Subject: [Mongrel] Possible memory leak problem...
In-Reply-To: <8c8513100709041655v381657baj36553e82cc338442@mail.gmail.com>
References: <61B5B973-C9E5-48AC-AD8F-E748FD22F4CA@zero.it>
	<C2DB09C9-5852-46B2-91FD-17F75758B2D7@gmail.com>
	<86987bdf0708192050r282351bu7256e44cbba28b7@mail.gmail.com>
	<8c8513100708312154l16071187te3e8e75270ef063@mail.gmail.com>
	<20070901014029.837dba7d.zedshaw@zedshaw.com>
	<8c8513100709041655v381657baj36553e82cc338442@mail.gmail.com>
Message-ID: <20070905134702.GV3255@suse.de>

On 2007-09-04 16:55:36 -0700, Christopher Bailey wrote:
> Thanks Zed.  This is the direction we are going, but it is non-trivial due
> to how our storage and authentication system works.  A reality
> none-the-less.

at least the authentication/authorization issues can be solved with
x-(lighttpd-)sendfile

    darix

-- 
          openSUSE - SUSE Linux is my linux
              openSUSE is good for you
                  www.opensuse.org

From cjwoodward at gmail.com  Thu Sep  6 13:55:37 2007
From: cjwoodward at gmail.com (Carl Woodward)
Date: Thu,  6 Sep 2007 18:55:37 +0100 (BST)
Subject: [Mongrel] Invite from Carl Woodward (cjwoodward@gmail.com)
Message-ID: <20070906175537.766442B0CF3@mail2.quechup.com>

CarlWoodward (cjwoodward at gmail.com)
has invited you as a friend on Quechup... 
	...the social networking platform sweeping the globe

Go to: http://quechup.com/join.php/aT0wMDAwMDAwMDA5NTI5MTMxJmM9OTkyMDg%3D
to accept Carl's invite

You can use Quechup to meet new people, catch up with old friends,
maintain a blog, share videos &amp; photos, chat with other members, play
games, and more.
It's no wonder Quechup is fast becoming 'The Social Networking site to be
on'

Join Carl and his friends today:
http://quechup.com/join.php/aT0wMDAwMDAwMDA5NTI5MTMxJmM9OTkyMDg%3D
------------------------------------------------------------------
You received this because Carl Woodward (cjwoodward at gmail.com) knows and
agreed to invite you. You will only receive one invitation from
cjwoodward at gmail.com. Quechup will not spam or sell your email address, see
our privacy policy - http://quechup.com/privacy.php
Go to
http://quechup.com/emailunsubscribe.php/ZW09bW9uZ3JlbC11c2Vyc0BydWJ5Zm9yZ2Uub3Jn
if you do not wish to receive any more emails from Quechup.
------------------------------------------------------------------
Copyright Quechup.com 2007.
------------------------------------
Go to
http://quechup.com/emailunsubscribe.php/ZW09bW9uZ3JlbC11c2Vyc0BydWJ5Zm9yZ2Uub3Jn
if you do not wish to receive any more emails from Quechup
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070906/8331d0e4/attachment-0001.html 

From casper at lol.dk  Fri Sep  7 03:30:23 2007
From: casper at lol.dk (Casper Fabricius)
Date: Fri, 07 Sep 2007 09:30:23 +0200
Subject: [Mongrel] Fails to restart mongrel cluster after Capistrano
 rollback (PIDs not deleted)
Message-ID: <389A7842-CD02-435F-8B76-D3BE5EB2C4E0@lol.dk>

Hi,

I am deploying a Rails website using Capistrano to a VPS running the  
site through Apache and Mongrel.

'cap deploy' works fine, and mongrels are restarted. When I execute  
'cap deploy:rollback', however, my script fails to restart the  
mongrels giving this error:
!!! PID file tmp/pids/mongrel.5000.pid already exists.  Mongrel could  
be running already.  Check your log/mongrel.5000.log for errors.

I have a hunch that the error is related to the fact that I link /tmp/ 
pids into the shared directory upon each release in the  
"after_update" task, but I'm not sure how I end up with this problem,  
as the old dir I'm rolling back also has the link setup. The mongrels  
are stopped, but for some reason they don't clean up their PID files,  
and thus they fail to start again.

The full output of the rollback looks like this:
http://pastie.caboo.se/94602

My deploy.rb looks like this:
http://pastie.caboo.se/94600

Any help on this issue would be appreciated - thanks!

Best regards,
Casper Fabricius

From wayneeseguin at gmail.com  Fri Sep  7 12:15:34 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Fri, 7 Sep 2007 12:15:34 -0400
Subject: [Mongrel] Fails to restart mongrel cluster after Capistrano
	rollback (PIDs not deleted)
In-Reply-To: <389A7842-CD02-435F-8B76-D3BE5EB2C4E0@lol.dk>
References: <389A7842-CD02-435F-8B76-D3BE5EB2C4E0@lol.dk>
Message-ID: <29182826-820D-43A3-A673-9987549D8461@gmail.com>

On Sep 07, 2007, at 03:30 , Casper Fabricius wrote:
> Hi,
>
> I am deploying a Rails website using Capistrano to a VPS running the
> site through Apache and Mongrel.
>
> 'cap deploy' works fine, and mongrels are restarted. When I execute
> 'cap deploy:rollback', however, my script fails to restart the
> mongrels giving this error:
> !!! PID file tmp/pids/mongrel.5000.pid already exists.  Mongrel could
> be running already.  Check your log/mongrel.5000.log for errors.
>
> I have a hunch that the error is related to the fact that I link /tmp/
> pids into the shared directory upon each release in the
> "after_update" task, but I'm not sure how I end up with this problem,
> as the old dir I'm rolling back also has the link setup. The mongrels
> are stopped, but for some reason they don't clean up their PID files,
> and thus they fail to start again.
>
> The full output of the rollback looks like this:
> http://pastie.caboo.se/94602
>
> My deploy.rb looks like this:
> http://pastie.caboo.se/94600
>
> Any help on this issue would be appreciated - thanks!
>
> Best regards,
> Casper Fabricius
> ________________________________________

Casper,

I'd rewrite the default rollback task to:
1. Stop mongrel cluster
2. Rollback
3. Restart mongrel cluster

Between 2-3 you might also want to ensure that the new (old)  
"current" directory links the pid directory correctly.

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/f6484653/attachment.html 

From rogerpack2005 at gmail.com  Fri Sep  7 14:33:20 2007
From: rogerpack2005 at gmail.com (Roger Pack)
Date: Fri, 7 Sep 2007 12:33:20 -0600
Subject: [Mongrel] multi threaded theoretically useful?
Message-ID: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>

So here's a random question:  if a (Ruby) multi-threaded rails server could
exist (bug free), would it be faster than using a mongrel cluster?
(I.e. 10 mongrel processes versus 10 Ruby threads).  I'm not sure if
it would.  RAM it
might save, though.  Any thoughts?
-Roger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/d2fed9f0/attachment.html 

From ezmobius at gmail.com  Fri Sep  7 14:43:24 2007
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Fri, 7 Sep 2007 11:43:24 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
Message-ID: <B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>


On Sep 7, 2007, at 11:33 AM, Roger Pack wrote:

> So here's a random question: if a (Ruby) multi-threaded rails  
> server could exist (bug free), would it be faster than using a  
> mongrel cluster? (I.e. 10 mongrel processes versus 10 Ruby  
> threads). I'm not sure if it would. RAM it might save, though. Any  
> thoughts? -Roger
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


Hey Roger-

	No it would not be as fast at all. Current ruby threads are green  
threads, meaning that they do not use native OS threads so there is  
no real parallel execution. Ruby has an internal timer and just  
switches between threads really fast. So 10 mongrels will trounce one  
thread safe mongrel.

	Ruby 1.9, Jruby and Rubinius will eventually have native threads and  
may make this situation better but for now such is life.

Cheers-
-- Ezra Zygmuntowicz 
-- Founder & Ruby Hacker
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From erik.hetzner at ucop.edu  Fri Sep  7 15:02:55 2007
From: erik.hetzner at ucop.edu (Erik Hetzner)
Date: Fri, 07 Sep 2007 12:02:55 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
Message-ID: <87d4wuz4bk.wl%erik.hetzner@ucop.edu>

At Fri, 7 Sep 2007 11:43:24 -0700,
Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:
> Hey Roger-
>
> 	No it would not be as fast at all. Current ruby threads are green
> threads, meaning that they do not use native OS threads so there is
> no real parallel execution. Ruby has an internal timer and just
> switches between threads really fast. So 10 mongrels will trounce one
> thread safe mongrel.
>
> 	Ruby 1.9, Jruby and Rubinius will eventually have native
> threads and may make this situation better but for now such is life.

I don?t want to muddy the waters here, as my knowledge of Ruby
threading performance is minimal, but there isn?t any ?real? parallel
execution on any uniprocessor machine: it?s all emulation. Erlang uses
green threads (but calls them processes) but context switches quite a
bit faster (if I understand correctly) than native threads or
processes. True, on a multiprocessor machine Ruby isn?t going to take
advantage of the situation without native threads or running multiple
processes. But it may turn out that a mixture of multiple processes
AND green threads provides greater performance than plain old multiple
processes.

This paper, for instance, from 2002, claims significantly better
performance on the JVM (Linux) with green threads than native.
<http://cselab.snu.ac.kr/member/naehyuck/papers/%5BJ14%5DComparative%20performance%20evaluation%20of%20Java%20threads%20for%20embedded%20applications.pdf>

best,
Erik Hetzner
;; Erik Hetzner, California Digital Library
;; gnupg key id: 1024D/01DB07E3
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/129a6b26/attachment.bin 

From cleaner416 at gmail.com  Fri Sep  7 13:37:06 2007
From: cleaner416 at gmail.com (cleaner416)
Date: Fri, 7 Sep 2007 13:37:06 -0400
Subject: [Mongrel] memcached and fragment storage,
	session storage with a mongrel cluster
Message-ID: <758206FE-C02C-422F-BAFF-61C6E21F7A08@gmail.com>

Greetings,

I posted this not on the Rails mailing list and didn't get a definitive
response so I thought I'd post it here, since I figure lots of you  
are using
mongrel with Rails apps. Any suggestions would be greatly appreciated.

I've been using file-based fragment caching and DB-based session
caching quite extensively.  I decided to try out memcached for both to
see if I could achieve a meaningful performance gain for a particular  
app.

I searched around a bit and simply added two lines to my
development.rb:

config.action_controller.fragment_cache_store = :mem_cache_store
config.action_controller.session_store = :mem_cache_store

(I also changed config.action_controller.perform_caching = true for
testing )

This works like a charm on my development box.  (Mac) I've got  
memcached -vv
running in another terminal window I can see it doing it's thing.

However, as soon as I tried this on my production box, (by adding the
above lines to production.rb) I ran into some odd problems.   My
production setup is pretty vanilla: Apache 2.2 load balancing to a
couple of mongrel instances via mongrel cluster on a fedora core 5
box.   It seems as if a memcache pool is being created for each
mongrel instance, instead of the app.  Is there some other kind of
config I need to do in production.rb?  The stuff I've found via
googling seems to apply to Rails < 1.2

Thanks much

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/308447f3/attachment-0001.html 

From wyhaines at gmail.com  Fri Sep  7 15:32:22 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Fri, 7 Sep 2007 12:32:22 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <87d4wuz4bk.wl%erik.hetzner@ucop.edu>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
Message-ID: <f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>

On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote:
> processes. True, on a multiprocessor machine Ruby isn't going to take
> advantage of the situation without native threads or running multiple
> processes. But it may turn out that a mixture of multiple processes
> AND green threads provides greater performance than plain old multiple
> processes.

I can state with a great deal of confidence that in this problem
domain, Ruby's green threading will not get you greater throughput.

A given process can only do so much work. If, in the course of doing
that work, there is an external latency, such as waiting on some IO,
that can be done in a non-blocking way, then it's possible to get more
work out of that process per unit time by using threads.

The work involved in handling web browser requests and rendering
responses back to them, though, doesn't tend to fall into this
category. It tends to be CPU intensive, and when one is waiting for
something external, like a database query, one is usually waiting
inside the body of the low level extension based database driver, so
thread context switching doesn't happen there, anyway.

The most efficient way of dealing with these sorts of things, from a
throughput perspective, is to avoid the overhead of threads completely
and just pound them through, one at a time, with 100% of the process
resources working on that single request at a time.

This doesn't change if you have one process or ten processes for your app/site.


Kirk Haines

swiftiply.swiftcore.org
analogger.swiftcore.org

From pete at nextengine.com  Fri Sep  7 15:06:41 2007
From: pete at nextengine.com (Pete DeLaurentis)
Date: Fri, 7 Sep 2007 12:06:41 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <87d4wuz4bk.wl%erik.hetzner@ucop.edu>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
Message-ID: <9795143F-AE04-49F2-B8E5-907EEC910417@nextengine.com>

Here's one question that I think would have an impact on how useful  
green threads are for many server apps:

If you are blocking on something like a database SELECT in one green  
thread, will the Ruby interpreter switch to another green thread  
while it's waiting for the response?

Or does a blocking request block all green threads?

Thanks,
Pete


On Sep 7, 2007, at 12:02 PM, Erik Hetzner wrote:

> At Fri, 7 Sep 2007 11:43:24 -0700,
> Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:
>> Hey Roger-
>>
>> 	No it would not be as fast at all. Current ruby threads are green
>> threads, meaning that they do not use native OS threads so there is
>> no real parallel execution. Ruby has an internal timer and just
>> switches between threads really fast. So 10 mongrels will trounce one
>> thread safe mongrel.
>>
>> 	Ruby 1.9, Jruby and Rubinius will eventually have native
>> threads and may make this situation better but for now such is life.
>
> I don?t want to muddy the waters here, as my knowledge of Ruby
> threading performance is minimal, but there isn?t any ?real? parallel
> execution on any uniprocessor machine: it?s all emulation. Erlang uses
> green threads (but calls them processes) but context switches quite a
> bit faster (if I understand correctly) than native threads or
> processes. True, on a multiprocessor machine Ruby isn?t going to take
> advantage of the situation without native threads or running multiple
> processes. But it may turn out that a mixture of multiple processes
> AND green threads provides greater performance than plain old multiple
> processes.
>
> This paper, for instance, from 2002, claims significantly better
> performance on the JVM (Linux) with green threads than native.
> <http://cselab.snu.ac.kr/member/naehyuck/papers/%5BJ14%5DComparative 
> %20performance%20evaluation%20of%20Java%20threads%20for%20embedded% 
> 20applications.pdf>
>
> best,
> Erik Hetzner
> ;; Erik Hetzner, California Digital Library
> ;; gnupg key id: 1024D/01DB07E3
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From terry.reese at oregonstate.edu  Fri Sep  7 16:01:03 2007
From: terry.reese at oregonstate.edu (Reese, Terry)
Date: Fri, 7 Sep 2007 13:01:03 -0700
Subject: [Mongrel] memcached and fragment storage,
	session storage with a mongrel cluster
In-Reply-To: <758206FE-C02C-422F-BAFF-61C6E21F7A08@gmail.com>
References: <758206FE-C02C-422F-BAFF-61C6E21F7A08@gmail.com>
Message-ID: <1454CCD2A8D4B74F9EEE8CB68C891CC901B6255D@NWS-EXCH3.nws.oregonstate.edu>

So we had some similar problems related to running our rails application
using either memcache or an sql-based sessioning solution using a number
of mongrels with Apache's balancer.  In the end, I think the problem we
had was related to a race condition when setting session data between
ajax calls.  On the rails trac instance, there has been a plugin
developed for these types of conditions called smartsessionstore.  It
works with the sql_session plugin, meaning your sessioning will be sql
based so not quite as fast as memcache, but I'm not sure if the
difference is actually noticable to users.  Here's the link:
http://www.texperts.com/2007/05/01/race-conditions-in-rails-sessions-and
-how-to-fix-them/
 
--TR 


________________________________

	From: mongrel-users-bounces at rubyforge.org
[mailto:mongrel-users-bounces at rubyforge.org] On Behalf Of cleaner416
	Sent: Friday, September 07, 2007 10:37 AM
	To: mongrel-users at rubyforge.org
	Subject: [Mongrel] memcached and fragment storage,session
storage with a mongrel cluster
	
	
	Greetings,

	I posted this not on the Rails mailing list and didn't get a
definitive 
	response so I thought I'd post it here, since I figure lots of
you are using
	mongrel with Rails apps. Any suggestions would be greatly
appreciated.
	
	
	I've been using file-based fragment caching and DB-based session
	caching quite extensively.  I decided to try out memcached for
both to
	see if I could achieve a meaningful performance gain for a
particular app.

	I searched around a bit and simply added two lines to my
	development.rb:

	config.action_controller.fragment_cache_store = :mem_cache_store
	config.action_controller.session_store = :mem_cache_store

	(I also changed config.action_controller.perform_caching = true
for
	testing )

	This works like a charm on my development box.  (Mac) I've got
memcached -vv
	running in another terminal window I can see it doing it's
thing.

	However, as soon as I tried this on my production box, (by
adding the
	above lines to production.rb) I ran into some odd problems.   My
	production setup is pretty vanilla: Apache 2.2 load balancing to
a
	couple of mongrel instances via mongrel cluster on a fedora core
5
	box.   It seems as if a memcache pool is being created for each
	mongrel instance, instead of the app.  Is there some other kind
of
	config I need to do in production.rb?  The stuff I've found via
	googling seems to apply to Rails < 1.2

	Thanks much
	
	

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/725fbbd6/attachment.html 

From wyhaines at gmail.com  Fri Sep  7 16:14:09 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Fri, 7 Sep 2007 13:14:09 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <9795143F-AE04-49F2-B8E5-907EEC910417@nextengine.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<9795143F-AE04-49F2-B8E5-907EEC910417@nextengine.com>
Message-ID: <f4cd26df0709071314o70430be5q22d0f9d0158809c0@mail.gmail.com>

On 9/7/07, Pete DeLaurentis <pete at nextengine.com> wrote:
> Here's one question that I think would have an impact on how useful
> green threads are for many server apps:
>
> If you are blocking on something like a database SELECT in one green
> thread, will the Ruby interpreter switch to another green thread
> while it's waiting for the response?
>
> Or does a blocking request block all green threads?

When the flow of execution leaves Ruby and goes into some 3rd party
extension (like that of a DB library), then ruby can't context switch.

So yes, when one does a DB query that is going through a C extension,
all of the other green threads are blocked, so that latency can't be
captured.

If it could be, then yes, there would be some benefit from some number
of green threads per process for typical web apps.


Kirk Haines

From faisal at faisal.com  Fri Sep  7 16:14:29 2007
From: faisal at faisal.com (Faisal N Jawdat)
Date: Fri, 7 Sep 2007 16:14:29 -0400
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
Message-ID: <7AAA915C-358F-4C6D-9AA1-6983F10BDE6E@faisal.com>

Are you assuming that the memory hit per-thread is about the same as  
per mongrel process?

-faisal

On Sep 7, 2007, at 2:43 PM, Ezra Zygmuntowicz wrote:

>
> On Sep 7, 2007, at 11:33 AM, Roger Pack wrote:
>
>> So here's a random question: if a (Ruby) multi-threaded rails
>> server could exist (bug free), would it be faster than using a
>> mongrel cluster? (I.e. 10 mongrel processes versus 10 Ruby
>> threads). I'm not sure if it would. RAM it might save, though. Any
>> thoughts? -Roger
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>
>
> Hey Roger-
>
> 	No it would not be as fast at all. Current ruby threads are green
> threads, meaning that they do not use native OS threads so there is
> no real parallel execution. Ruby has an internal timer and just
> switches between threads really fast. So 10 mongrels will trounce one
> thread safe mongrel.
>
> 	Ruby 1.9, Jruby and Rubinius will eventually have native threads and
> may make this situation better but for now such is life.
>
> Cheers-
> -- Ezra Zygmuntowicz 
> -- Founder & Ruby Hacker
> -- ez at engineyard.com
> -- Engine Yard, Serious Rails Hosting
> -- (866) 518-YARD (9273)
>
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From wyhaines at gmail.com  Fri Sep  7 16:35:53 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Fri, 7 Sep 2007 13:35:53 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <7AAA915C-358F-4C6D-9AA1-6983F10BDE6E@faisal.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<7AAA915C-358F-4C6D-9AA1-6983F10BDE6E@faisal.com>
Message-ID: <f4cd26df0709071335l17f9cecevab876166097885bf@mail.gmail.com>

On 9/7/07, Faisal N Jawdat <faisal at faisal.com> wrote:
> Are you assuming that the memory hit per-thread is about the same as
> per mongrel process?

No.  It's irrelevant, because there's only so much work that a single
ruby process can do, regardless of the number of threads it contains.
Once the process is doing all of the work that it can, you only get
more work done by adding another process.  Repeat until all of your
CPUs are doing all of the work that they can.

I have a personal hunch that most of the time people are running more
Mongrels than they need to run to maximize the throughput on their
hardware, though.  Anyone who is concerned about the RAM footprint of
their application should step back and take a look at how many mongrel
processes they are running.  Do some analysis and see if they can
reduce that number without affecting throughput.

Maybe take a look at using Swiftiply+swiftiplied_mongrel
evented_mongrel instead of the threaded mongrel and look again.  I
think that in many cases a person can run a lot fewer processes than
they are, and still get the same, or maybe even better throughput.
Once your CPUs are at max utilization, more processes don't help.


Kirk Haines

From erik.hetzner at ucop.edu  Fri Sep  7 16:40:20 2007
From: erik.hetzner at ucop.edu (Erik Hetzner)
Date: Fri, 07 Sep 2007 13:40:20 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
Message-ID: <87bqceyzt7.wl%erik.hetzner@ucop.edu>

At Fri, 7 Sep 2007 12:32:22 -0700,
"Kirk Haines" <wyhaines at gmail.com> wrote:
>
> On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote:
> > processes. True, on a multiprocessor machine Ruby isn't going to take
> > advantage of the situation without native threads or running multiple
> > processes. But it may turn out that a mixture of multiple processes
> > AND green threads provides greater performance than plain old multiple
> > processes.
>
> I can state with a great deal of confidence that in this problem
> domain, Ruby's green threading will not get you greater throughput.

What problem domain? Web servers in general? Web applications? Web
applications which are heavily database dependent? I show below
that there are web apps whose throughput can be improved by green
threading.

> A given process can only do so much work. If, in the course of doing
> that work, there is an external latency, such as waiting on some IO,
> that can be done in a non-blocking way, then it's possible to get more
> work out of that process per unit time by using threads.

A process can use almost all the cycles available on a machine if the
scheduler allows it to, and if it has something to do. Threads and a
thread scheduler allow a process to have something to do.

> The work involved in handling web browser requests and rendering
> responses back to them, though, doesn't tend to fall into this
> category. It tends to be CPU intensive, and when one is waiting for
> something external, like a database query, one is usually waiting
> inside the body of the low level extension based database driver, so
> thread context switching doesn't happen there, anyway.

In my experience web applications tend to spend most of their time
waiting for other things, from databases, web services, etc. So
obviously your mileage may vary.

> The most efficient way of dealing with these sorts of things, from a
> throughput perspective, is to avoid the overhead of threads completely
> and just pound them through, one at a time, with 100% of the process
> resources working on that single request at a time.
>
> This doesn't change if you have one process or ten processes for
> your app/site.

Below is an example of what I?m getting at. You can tweak it to test
various things. It is a simple mongrel server that sleeps for 5 usecs
and then calculates the fibonacci number of 15. You can turn it into a
multi-threaded server by commented out the synchronize stuff. On my
uniprocessor machine you get half again the performance in terms of
requests/sec from multithreaded.

This probably does not model a typical web app. But my original
response was to a blanket statement that it was impossible to get
better performance from more threads. Here is an example of a web app
that gets better performance with more threads.

best,
Erik Hetzner

===============

require 'rubygems'
require 'mongrel'
require 'monitor'
port = ARGV[0]

class MyHttpHandler < Mongrel::HttpHandler
  @@lock = Monitor.new
  def fib(n)
    if n == 1 or n == 2 then
      return 1
    else
      return fib(n - 1) + fib(n - 2)
    end
  end
  def process(request,response)
    @@lock.synchronize do
      sleep 0.005
      response.start(200) do |head,body|
        body << fib(15).to_s
      end
    end
  end
end

config = Mongrel::Configurator.new :host => "localhost" do
  listener :port => port do
    uri "/", :handler => MyHttpHandler.new
  end
end
config.run
begin
  while true do
    sleep 1000000
  end
rescue Interrupt
end

===============;; Erik Hetzner, California Digital Library
;; gnupg key id: 1024D/01DB07E3
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/d7d2cc73/attachment-0001.bin 

From wyhaines at gmail.com  Fri Sep  7 16:50:50 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Fri, 7 Sep 2007 13:50:50 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <87bqceyzt7.wl%erik.hetzner@ucop.edu>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
Message-ID: <f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>

On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote

> What problem domain? Web servers in general? Web applications? Web
> applications which are heavily database dependent? I show below
> that there are web apps whose throughput can be improved by green
> threading.

The general area of Ruby web applications and the serving thereof.

> A process can use almost all the cycles available on a machine if the
> scheduler allows it to, and if it has something to do. Threads and a
> thread scheduler allow a process to have something to do.

Sure, but the _only_ way to get this with Ruby green threads is if you
have threads that are blocking on something within Ruby code, so that
Ruby can context switch to a different thread and get something done
with it while the first one sits around, waiting.  You artificially
construct exactly that scenario in your example below.  I'll comment
on it more in a minute.  In real apps, though, most of the time, that
just isn't happening.

> > The work involved in handling web browser requests and rendering
> > responses back to them, though, doesn't tend to fall into this
> > category. It tends to be CPU intensive, and when one is waiting for
> > something external, like a database query, one is usually waiting
> > inside the body of the low level extension based database driver, so
> > thread context switching doesn't happen there, anyway.
>
> In my experience web applications tend to spend most of their time
> waiting for other things, from databases, web services, etc. So
> obviously your mileage may vary.

And when you are waiting on something like a database, you are doing
it within the DB driver extension, so there are no context switches
happening -- your green threads are blocked.

> Below is an example of what I'm getting at. You can tweak it to test
> various things. It is a simple mongrel server that sleeps for 5 usecs
> and then calculates the fibonacci number of 15. You can turn it into a
> multi-threaded server by commented out the synchronize stuff. On my
> uniprocessor machine you get half again the performance in terms of
> requests/sec from multithreaded.

The reason you get more performance when multithreading is because of
that sleep call.  It is all happening inside of Ruby, so when running
in a multithreaded context, when one thread goes to sleep, the others
just get the execution cycles.  It's artificial, though.  Read Ruby
web apps don't tend to have latencies like that.  When they are
waiting, they are waiting inside of things that block the entire Ruby
process, and not just waiting for something that is blocking a single
thread.


Kirk Haines

From cleaner416 at gmail.com  Fri Sep  7 17:26:08 2007
From: cleaner416 at gmail.com (cleaner416)
Date: Fri, 7 Sep 2007 17:26:08 -0400
Subject: [Mongrel] memcached and fragment storage,
	session storage with a mongrel cluster
In-Reply-To: <1454CCD2A8D4B74F9EEE8CB68C891CC901B6255D@NWS-EXCH3.nws.oregonstate.edu>
References: <758206FE-C02C-422F-BAFF-61C6E21F7A08@gmail.com>
	<1454CCD2A8D4B74F9EEE8CB68C891CC901B6255D@NWS-EXCH3.nws.oregonstate.edu>
Message-ID: <77ED31AB-42C8-4129-BCA6-C89622FD1206@gmail.com>

Thanks, Terry.  That's very helpful.


On Sep 7, 2007, at 4:01 PM, Reese, Terry wrote:

> So we had some similar problems related to running our rails  
> application using either memcache or an sql-based sessioning  
> solution using a number of mongrels with Apache's balancer.  In the  
> end, I think the problem we had was related to a race condition  
> when setting session data between ajax calls.  On the rails trac  
> instance, there has been a plugin developed for these types of  
> conditions called smartsessionstore.  It works with the sql_session  
> plugin, meaning your sessioning will be sql based so not quite as  
> fast as memcache, but I'm not sure if the difference is actually  
> noticable to users.  Here's the link: http://www.texperts.com/ 
> 2007/05/01/race-conditions-in-rails-sessions-and-how-to-fix-them/
>
> --TR
>
> From: mongrel-users-bounces at rubyforge.org [mailto:mongrel-users- 
> bounces at rubyforge.org] On Behalf Of cleaner416
> Sent: Friday, September 07, 2007 10:37 AM
> To: mongrel-users at rubyforge.org
> Subject: [Mongrel] memcached and fragment storage,session storage  
> with a mongrel cluster
>
> Greetings,
>
> I posted this not on the Rails mailing list and didn't get a  
> definitive
> response so I thought I'd post it here, since I figure lots of you  
> are using
> mongrel with Rails apps. Any suggestions would be greatly appreciated.
>
> I've been using file-based fragment caching and DB-based session
> caching quite extensively.  I decided to try out memcached for both to
> see if I could achieve a meaningful performance gain for a  
> particular app.
>
> I searched around a bit and simply added two lines to my
> development.rb:
>
> config.action_controller.fragment_cache_store = :mem_cache_store
> config.action_controller.session_store = :mem_cache_store
>
> (I also changed config.action_controller.perform_caching = true for
> testing )
>
> This works like a charm on my development box.  (Mac) I've got  
> memcached -vv
> running in another terminal window I can see it doing it's thing.
>
> However, as soon as I tried this on my production box, (by adding the
> above lines to production.rb) I ran into some odd problems.   My
> production setup is pretty vanilla: Apache 2.2 load balancing to a
> couple of mongrel instances via mongrel cluster on a fedora core 5
> box.   It seems as if a memcache pool is being created for each
> mongrel instance, instead of the app.  Is there some other kind of
> config I need to do in production.rb?  The stuff I've found via
> googling seems to apply to Rails < 1.2
>
> Thanks much
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/8c47179d/attachment.html 

From erik.hetzner at ucop.edu  Fri Sep  7 17:36:56 2007
From: erik.hetzner at ucop.edu (Erik Hetzner)
Date: Fri, 07 Sep 2007 14:36:56 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
Message-ID: <87abryyx6v.wl%erik.hetzner@ucop.edu>

At Fri, 7 Sep 2007 13:50:50 -0700,
"Kirk Haines" <wyhaines at gmail.com> wrote:
>
> On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote
>
> > What problem domain? Web servers in general? Web applications? Web
> > applications which are heavily database dependent? I show below
> > that there are web apps whose throughput can be improved by green
> > threading.
>
> The general area of Ruby web applications and the serving thereof.

Which in my experience involves waiting for external web services,
among other things. If you want to say that this is something that
isn?t a typical rails app, fine. I?m telling you that my threads are
often i/o bound.

> Sure, but the _only_ way to get this with Ruby green threads is if
> you have threads that are blocking on something within Ruby code, so
> that Ruby can context switch to a different thread and get something
> done with it while the first one sits around, waiting. You
> artificially construct exactly that scenario in your example below.
> I'll comment on it more in a minute. In real apps, though, most of
> the time, that just isn't happening.

I understand how threads work, and I don?t think that my construction
is artificial.

> And when you are waiting on something like a database, you are doing
> it within the DB driver extension, so there are no context switches
> happening -- your green threads are blocked.

Surely it is possible to write a C extension for Ruby that allows
context switching from within it? This is outside my understanding of
Ruby threads.

> The reason you get more performance when multithreading is because
> of that sleep call. It is all happening inside of Ruby, so when
> running in a multithreaded context, when one thread goes to sleep,
> the others just get the execution cycles. It's artificial, though.
> Read Ruby web apps don't tend to have latencies like that. When they
> are waiting, they are waiting inside of things that block the entire
> Ruby process, and not just waiting for something that is blocking a
> single thread.

I know why we are getting greater performance, and again, I don?t
think it?s artificial. But, in any case, it was a demonstration,
mostly for the original responder, who claimed that you would never
get more performance from Ruby threads.

Again, your mileage may vary. And it?s all moot for rails users since
I don?t see rails going multithreaded.

best,
Erik Hetzner
;; Erik Hetzner, California Digital Library
;; gnupg key id: 1024D/01DB07E3
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/ec5d9f95/attachment.bin 

From brandorr at opensolaris.org  Fri Sep  7 17:45:56 2007
From: brandorr at opensolaris.org (Brandorr)
Date: Fri, 7 Sep 2007 17:45:56 -0400
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <9795143F-AE04-49F2-B8E5-907EEC910417@nextengine.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<9795143F-AE04-49F2-B8E5-907EEC910417@nextengine.com>
Message-ID: <5b5090780709071445w300a8444jeb0ed50ab50a51df@mail.gmail.com>

On 9/7/07, Pete DeLaurentis <pete at nextengine.com> wrote:
> Here's one question that I think would have an impact on how useful
> green threads are for many server apps:
>
> If you are blocking on something like a database SELECT in one green
> thread, will the Ruby interpreter switch to another green thread
> while it's waiting for the response?
>
> Or does a blocking request block all green threads?

Well considering that a single-threaded process will map to a single
kernel thread in Solaris 9+ and Linux 2.6+. A process with green
threads that makes a system call will be blocked, until the call
returns, that means that all green threads in that process are
blocked. (Green threads are now known as fibers) ;)

This is one reason why IO intensive workloads, which make intensive
use of system calls, should benefit from "real" kernel threads, while
CPU intensive workloads can have the edge when implemented using green
threads, vs kernel threads. (The context switching cost being  lower.)

> Thanks,
> Pete
>
>
> On Sep 7, 2007, at 12:02 PM, Erik Hetzner wrote:
>
> > At Fri, 7 Sep 2007 11:43:24 -0700,
> > Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:
> >> Hey Roger-
> >>
> >>      No it would not be as fast at all. Current ruby threads are green
> >> threads, meaning that they do not use native OS threads so there is
> >> no real parallel execution. Ruby has an internal timer and just
> >> switches between threads really fast. So 10 mongrels will trounce one
> >> thread safe mongrel.
> >>
> >>      Ruby 1.9, Jruby and Rubinius will eventually have native
> >> threads and may make this situation better but for now such is life.
> >
> > I don't want to muddy the waters here, as my knowledge of Ruby
> > threading performance is minimal, but there isn't any 'real' parallel
> > execution on any uniprocessor machine: it's all emulation. Erlang uses
> > green threads (but calls them processes) but context switches quite a
> > bit faster (if I understand correctly) than native threads or
> > processes. True, on a multiprocessor machine Ruby isn't going to take
> > advantage of the situation without native threads or running multiple
> > processes. But it may turn out that a mixture of multiple processes
> > AND green threads provides greater performance than plain old multiple
> > processes.
> >
> > This paper, for instance, from 2002, claims significantly better
> > performance on the JVM (Linux) with green threads than native.
> > <http://cselab.snu.ac.kr/member/naehyuck/papers/%5BJ14%5DComparative
> > %20performance%20evaluation%20of%20Java%20threads%20for%20embedded%
> > 20applications.pdf>
> >
> > best,
> > Erik Hetzner
> > ;; Erik Hetzner, California Digital Library
> > ;; gnupg key id: 1024D/01DB07E3
> > _______________________________________________
> > Mongrel-users mailing list
> > Mongrel-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mongrel-users
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>


-- 
- Brian Gupta

http://opensolaris.org/os/project/nycosug/

From wyhaines at gmail.com  Fri Sep  7 18:34:49 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Fri, 7 Sep 2007 15:34:49 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <87abryyx6v.wl%erik.hetzner@ucop.edu>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
Message-ID: <f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>

On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote:

> > And when you are waiting on something like a database, you are doing
> > it within the DB driver extension, so there are no context switches
> > happening -- your green threads are blocked.
>
> Surely it is possible to write a C extension for Ruby that allows
> context switching from within it? This is outside my understanding of
> Ruby threads.

Maybe, but it's irrelevant.  The MySQL/Pg/whatever drivers aren't
written to be aware of Ruby internals.

> > The reason you get more performance when multithreading is because
> > of that sleep call. It is all happening inside of Ruby, so when
> > running in a multithreaded context, when one thread goes to sleep,
> > the others just get the execution cycles. It's artificial, though.
> > Read Ruby web apps don't tend to have latencies like that. When they
> > are waiting, they are waiting inside of things that block the entire
> > Ruby process, and not just waiting for something that is blocking a
> > single thread.
>
> I know why we are getting greater performance, and again, I don't
> think it's artificial. But, in any case, it was a demonstration,
> mostly for the original responder, who claimed that you would never
> get more performance from Ruby threads.

Ezra?  His response simplified it.  Of course there are situations
where green threads improve performance.  They can be quantified very
simply, as I already have.  When there are external latencies that are
not blocking the entire process, green threads can capture those
latencies and turn them into additional work.  A sleep() call is such
a latency.  Any pure ruby code that stays in rubyland and doesn't go
into an external extension is such a latency.  A DB query that spends
most of its time inside of some C library that is external to Ruby is
not such a latency.

> Again, your mileage may vary. And it's all moot for rails users since
> I don't see rails going multithreaded.

It's not very difficult, actually, to do it.  Just make a copy of the
rails.rb mongrel handler for Rails.  Remove the bit that wraps the
call to Dispatcher.dispatch() in a @guard.synchronize block.

Whether all of Rails is actually threadsafe, and whether one's code
is....that's another thing, but it's easy to run Rails in multiple
threads under Mongrel if one wants to play with that.


Kirk Haines

From ezmobius at gmail.com  Fri Sep  7 18:39:33 2007
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Fri, 7 Sep 2007 15:39:33 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
Message-ID: <0BD07355-FCC0-4029-BA1E-A3542C2EF32A@gmail.com>

Hi~

On Sep 7, 2007, at 3:34 PM, Kirk Haines wrote:

> On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote:
>
>>> And when you are waiting on something like a database, you are doing
>>> it within the DB driver extension, so there are no context switches
>>> happening -- your green threads are blocked.
>>
>> Surely it is possible to write a C extension for Ruby that allows
>> context switching from within it? This is outside my understanding of
>> Ruby threads.
>
> Maybe, but it's irrelevant.  The MySQL/Pg/whatever drivers aren't
> written to be aware of Ruby internals.
>
>>> The reason you get more performance when multithreading is because
>>> of that sleep call. It is all happening inside of Ruby, so when
>>> running in a multithreaded context, when one thread goes to sleep,
>>> the others just get the execution cycles. It's artificial, though.
>>> Read Ruby web apps don't tend to have latencies like that. When they
>>> are waiting, they are waiting inside of things that block the entire
>>> Ruby process, and not just waiting for something that is blocking a
>>> single thread.
>>
>> I know why we are getting greater performance, and again, I don't
>> think it's artificial. But, in any case, it was a demonstration,
>> mostly for the original responder, who claimed that you would never
>> get more performance from Ruby threads.

Yeah I actually said that you would never get more performance out of  
one mongrel running 10 threads then you would with 10 mongrels  
running one thread. Just to clarify ;)


-- Ezra



From erik.hetzner at ucop.edu  Fri Sep  7 19:23:18 2007
From: erik.hetzner at ucop.edu (Erik Hetzner)
Date: Fri, 07 Sep 2007 16:23:18 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <0BD07355-FCC0-4029-BA1E-A3542C2EF32A@gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<0BD07355-FCC0-4029-BA1E-A3542C2EF32A@gmail.com>
Message-ID: <878x7iys9l.wl%erik.hetzner@ucop.edu>

At Fri, 7 Sep 2007 15:39:33 -0700,
Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:
> [?]
>
> > On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote:

> [?]

> >> I know why we are getting greater performance, and again, I don't
> >> think it's artificial. But, in any case, it was a demonstration,
> >> mostly for the original responder, who claimed that you would never
> >> get more performance from Ruby threads.
>
> Yeah I actually said that you would never get more performance out of
> one mongrel running 10 threads then you would with 10 mongrels
> running one thread. Just to clarify ;)

My apologies for misquoting you, and also apologies for referring to
you as ?the original responder? - a bit rude, really. :) Here is your
original message:

> Hey Roger-

> No it would not be as fast at all. Current ruby threads are green
> threads, meaning that they do not use native OS threads so there is
> no real parallel execution. Ruby has an internal timer and just
> switches between threads really fast. So 10 mongrels will trounce
> one thread safe mongrel.

> Ruby 1.9, Jruby and Rubinius will eventually have native threads and
> may make this situation better but for now such is life.

I?ve just tested this on my uniprocessor machine. 10 single-threaded
mongrels of my sample sleep + fibonacci web server, balanced behind an
apache load balancer, versus 1 multithreaded mongrels without a load
balancer. The performance, in terms of requests/second, is almost
identical.

best,
Erik Hetzner
;; Erik Hetzner, California Digital Library
;; gnupg key id: 1024D/01DB07E3
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/d1198563/attachment.bin 

From work at ashleymoran.me.uk  Sat Sep  8 12:39:37 2007
From: work at ashleymoran.me.uk (Ashley Moran)
Date: Sat, 8 Sep 2007 17:39:37 +0100
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
Message-ID: <1448881C-BF5D-48F3-BBA7-521B52428BD8@ashleymoran.me.uk>


On Sep 07, 2007, at 11:34 pm, Kirk Haines wrote:

> Whether all of Rails is actually threadsafe, and whether one's code
> is....that's another thing, but it's easy to run Rails in multiple
> threads under Mongrel if one wants to play with that.

Just out of curiosity, does anyone here use Nitro?  I'm going to  
switch from Rails to Nitro for one of my own projects.  I've heard it  
runs happily multi-threaded.  Anyone noticed any difference in  
performance?  (Or for that matter, with other frameworks that run  
multi-threaded like Camping and Merb?)  To my knowledge, the only  
apps big enough for scalability to be a serious issue are all in Rails.

Also, this thread has answered the question I've wondered lately,  
namely that MRI can't context switch if one thread is in a C  
extension like a database adapter?.  Will Rubinius go any way to help  
this, assuming the adapter is re-entrant?

Ashley

From wyhaines at gmail.com  Sat Sep  8 17:02:56 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Sat, 8 Sep 2007 14:02:56 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <1448881C-BF5D-48F3-BBA7-521B52428BD8@ashleymoran.me.uk>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<1448881C-BF5D-48F3-BBA7-521B52428BD8@ashleymoran.me.uk>
Message-ID: <f4cd26df0709081402g2d43e9ebs924027646ba769e1@mail.gmail.com>

On 9/8/07, Ashley Moran <work at ashleymoran.me.uk> wrote:

> Just out of curiosity, does anyone here use Nitro?  I'm going to
> switch from Rails to Nitro for one of my own projects.  I've heard it
> runs happily multi-threaded.  Anyone noticed any difference in
> performance?  (Or for that matter, with other frameworks that run
> multi-threaded like Camping and Merb?)  To my knowledge, the only
> apps big enough for scalability to be a serious issue are all in Rails.

I don't use Nitro, but I have followed its development for a couple
years, and have done some benchmarks with it.

It is somewhat faster than Rails (though there was a very extensive
rewrite that is nearing completion for the 0.50 release, and I have no
idea what the rewrite has done to overall performance), and yes, it's
Mongrel handler runs in a multithreaded mode.

Actually, every framework except for Rails will Run in that way.
Merb, as an example, runs multithreaded, and is one of the fastest
Ruby frameworks (in my benchmarks, only IOWA is faster), and it has
some projects that do use clusters (I am helping someone setup a small
clustered app for demo purposes this weekend, for example).  I have
written several applications that were designed for clustering and for
heavy, multiserver use with IOWA (but the customers never ended up
taking them there, alas).  Ramaze is also a nice, multithread-safe
framework that merits some attention.  It is kind of like Nitro's
Merb.  Inspired by Nitro, but lighter weight, reasonably fast, with
some good ideas in it.

> Also, this thread has answered the question I've wondered lately,
> namely that MRI can't context switch if one thread is in a C
> extension like a database adapter?.  Will Rubinius go any way to help
> this, assuming the adapter is re-entrant?

With Rubinius (or JRuby) it is not the same issue because they are
using OS level threads.


Kirk Haines

From work at ashleymoran.me.uk  Sat Sep  8 17:53:27 2007
From: work at ashleymoran.me.uk (Ashley Moran)
Date: Sat, 8 Sep 2007 22:53:27 +0100
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709081402g2d43e9ebs924027646ba769e1@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<1448881C-BF5D-48F3-BBA7-521B52428BD8@ashleymoran.me.uk>
	<f4cd26df0709081402g2d43e9ebs924027646ba769e1@mail.gmail.com>
Message-ID: <49065EA3-33E9-46A6-888A-780FE1D2FC46@ashleymoran.me.uk>


On Sep 08, 2007, at 10:02 pm, Kirk Haines wrote:

> Ramaze is also a nice, multithread-safe
> framework that merits some attention.  It is kind of like Nitro's
> Merb.  Inspired by Nitro, but lighter weight, reasonably fast, with
> some good ideas in it.

That one had passed me by.  Thanks for the heads up!


> With Rubinius (or JRuby) it is not the same issue because they are
> using OS level threads.

I appreciate that the issue is different with JRuby, since what  
matters there is whether the Java libraries are thread-safe (although  
I saw this week a project to give JRuby access to C libraries using  
JNI, name escapes me).

As for Rubinius, I figured it is in the same position as the Apache  
web server, where you can't use the worker MPM if you want to load  
thread-unsafe modules or modules that depend on thread-unsafe  
libraries (eg mod_php).  And I guess Rubinius still has the option of  
going the Erlang way of mapping N(requests) green threads onto N 
(cores) OS threads.

Please forgive and wildly incorrect statements I may have made...  
concurrent programming isn't something I know much about, mainly  
because I've avoided it due to the mind-boggling complexity of it  
(well, of threads).

Ashley

From erik.hetzner at ucop.edu  Sat Sep  8 18:19:28 2007
From: erik.hetzner at ucop.edu (Erik Hetzner)
Date: Sat, 08 Sep 2007 15:19:28 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
Message-ID: <873axoztov.wl%erik.hetzner@ucop.edu>

At Fri, 7 Sep 2007 15:34:49 -0700,
"Kirk Haines" <wyhaines at gmail.com> wrote:
> Maybe, but it's irrelevant.  The MySQL/Pg/whatever drivers aren't
> written to be aware of Ruby internals.

It is my estimation that any application is going to have a lot of
places where a ruby thread context switch is possible.

> Ezra?  His response simplified it.  Of course there are situations
> where green threads improve performance.  They can be quantified very
> simply, as I already have.  When there are external latencies that are
> not blocking the entire process, green threads can capture those
> latencies and turn them into additional work.  A sleep() call is such
> a latency.  Any pure ruby code that stays in rubyland and doesn't go
> into an external extension is such a latency.  A DB query that spends
> most of its time inside of some C library that is external to Ruby is
> not such a latency.

Here is what I am trying to get at.

(a) It is nonsense to say that because ruby threads are not ?real?
they will never be as fast as ?real? threads or multiple processes. It
is dependent, as I take you to mean, on the balance of your code
between code from which context switching can happen & code from which
it cannot.

(b) *If* you have an application that is threadsafe, you ought to
*test* it with multiple threads before you discount the possibility
that you will be able to get *comparable* performance out of ruby?s
threading than you will from multiple processes, esp. because running
multiple threads rather than multiple processes provides some
benefits.

It is my belief that in general with a threadsafe mongrel app you are
going to find that the best system is one that combines multiple
processes to take advantage of multiple cores & of preemptive
multitasking and multiple threads to take advantage of lower memory
consumption and the faster context switching which ruby should
provide.

Obviously everything depends on (a) your code, (b) your ruby
implementation, (c) your os?s scheduler, and (d) the phase of the
moon, which is why you ought to test to find out what works best.

best,
Erik Hetzner

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070908/310c053b/attachment.bin 

From work at ashleymoran.me.uk  Sat Sep  8 18:50:34 2007
From: work at ashleymoran.me.uk (Ashley Moran)
Date: Sat, 8 Sep 2007 23:50:34 +0100
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <873axoztov.wl%erik.hetzner@ucop.edu>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<873axoztov.wl%erik.hetzner@ucop.edu>
Message-ID: <ADD3770F-EDA5-4AE0-8695-5AA1F7F8D630@ashleymoran.me.uk>


On Sep 08, 2007, at 11:19 pm, Erik Hetzner wrote:

> It is my belief that in general with a threadsafe mongrel app you are
> going to find that the best system is one that combines multiple
> processes to take advantage of multiple cores & of preemptive
> multitasking and multiple threads to take advantage of lower memory
> consumption and the faster context switching which ruby should
> provide.

Like I touched on in my last post, this is the strategy that Erlang  
takes, right?  (They seem to have the concurrency problem solved.)   
My immediate interest here is mainly lower memory consumption.  The  
server (read - dirt cheap PC) I just bought "only" has 1GB RAM, and  
will only hold 2GB.  Simply because I can't afford to go and buy  
anything bigger right now, I'd like to know I don't have to waste RAM  
running a shedload of 50-200MB processes.


> Obviously everything depends on (a) your code, (b) your ruby
> implementation, (c) your os?s scheduler, and (d) the phase of the
> moon

True.  Lycanthropy is not conducive to good software development.

Ashley

From _ at whats-your.name  Sat Sep  8 15:34:30 2007
From: _ at whats-your.name (cdr)
Date: Sat, 8 Sep 2007 19:34:30 +0000
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709081402g2d43e9ebs924027646ba769e1@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<1448881C-BF5D-48F3-BBA7-521B52428BD8@ashleymoran.me.uk>
	<f4cd26df0709081402g2d43e9ebs924027646ba769e1@mail.gmail.com>
Message-ID: <20070908193430.GA2750@replic.net>

> taking them there, alas).  Ramaze is also a nice, multithread-safe
> framework that merits some attention.  It is kind of like Nitro's
> Merb.  Inspired by Nitro, but lighter weight, reasonably fast, with

now that MRI1.9 has Coroutines, it might be interseting to check out some of the Lua frameworks/webservers multiplexed on coroutines rather than threads, like Copas / Xavante.

er, i guess that would have to wait for anything to run on MRI1.9, nevermind..


From wyhaines at gmail.com  Sat Sep  8 19:49:58 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Sat, 8 Sep 2007 16:49:58 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <ADD3770F-EDA5-4AE0-8695-5AA1F7F8D630@ashleymoran.me.uk>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<873axoztov.wl%erik.hetzner@ucop.edu>
	<ADD3770F-EDA5-4AE0-8695-5AA1F7F8D630@ashleymoran.me.uk>
Message-ID: <f4cd26df0709081649r518df9bi38c1996de2df8b21@mail.gmail.com>

On 9/8/07, Ashley Moran <work at ashleymoran.me.uk> wrote:

> Like I touched on in my last post, this is the strategy that Erlang
> takes, right?  (They seem to have the concurrency problem solved.)
> My immediate interest here is mainly lower memory consumption.  The
> server (read - dirt cheap PC) I just bought "only" has 1GB RAM, and
> will only hold 2GB.  Simply because I can't afford to go and buy
> anything bigger right now, I'd like to know I don't have to waste RAM
> running a shedload of 50-200MB processes.

Just as a practical example, I run about 60 separate backend ruby
processes on one very modest server with 2GB of RAM, and a bunch more
across a few other servers.  These 60+ processes represent a cross
section of communications/request management technique, all backed by
the same basic framework, IOWA.

Some are multithreaded with a mod_ruby based handler for
communications to the backends (these are the oldest ones).

Some use FCGI as the communications conduit, to multithreaded backends.

Some run in Mongrel via a mongrel handler, multi-threaded.

Some use the Mongrel HTTP parser inside an event based application
loop, rendering them effectively single threaded, event based apps.

And some of them run behind Swiftiply, some with one node, and some
with multiple nodes.

On the one machine that has 60 backends on it, each of those
represents a separate site.  That is, all of the sites are ran off of
a single process.  They all use the same basic framework -- IOWA --
regardless of the communications model being used.

The sites with the greatest throughput capacity are invariably the
ones that run an event based communications model -- either the
mongrel HTTP parser inside an event loop, or behind Swiftiply.  I have
sites that, with every page rendered dynamically (from content stored
in a database), are capable of handling 285 requests/second with no
concurrency, up to about 360/second with a concurrency of 50 (measured
using ab).  If I switch it to a multithreaded model (mongrel handler),
the best that I can do, with no concurrency, is 220 page
requests/second, and if I have much concurrency at all, that number
quickly drops down towards 150/second.

I have two points, here.  First, you can run a considerable number of
ruby based sites on a single modest server like yours, and they can
handle a rather impressive traffic load on a single process, depending
on your framework.  And second, running the same code in an event
based pipeline versus a multithreaded pipeline invariably gives me
better peformance in my apps.


Kirk Haines

From work at ashleymoran.me.uk  Sat Sep  8 20:34:56 2007
From: work at ashleymoran.me.uk (Ashley Moran)
Date: Sun, 9 Sep 2007 01:34:56 +0100
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709081649r518df9bi38c1996de2df8b21@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<873axoztov.wl%erik.hetzner@ucop.edu>
	<ADD3770F-EDA5-4AE0-8695-5AA1F7F8D630@ashleymoran.me.uk>
	<f4cd26df0709081649r518df9bi38c1996de2df8b21@mail.gmail.com>
Message-ID: <BE6B1856-0CC1-4440-A58E-FF4A81439F90@ashleymoran.me.uk>


On Sep 09, 2007, at 12:49 am, Kirk Haines wrote:

> I have two points, here.  First, you can run a considerable number of
> ruby based sites on a single modest server like yours, and they can
> handle a rather impressive traffic load on a single process, depending
> on your framework.  And second, running the same code in an event
> based pipeline versus a multithreaded pipeline invariably gives me
> better peformance in my apps.

Interesting!  What about the case of long-running requests, eg file  
uploads, outgoing web service requests or complex calculations?  Do  
you have any of those, and if so, how does it affect the balance of  
event-based vs multithreaded?

Ashley

From wyhaines at gmail.com  Sat Sep  8 21:31:48 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Sat, 8 Sep 2007 18:31:48 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <BE6B1856-0CC1-4440-A58E-FF4A81439F90@ashleymoran.me.uk>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<873axoztov.wl%erik.hetzner@ucop.edu>
	<ADD3770F-EDA5-4AE0-8695-5AA1F7F8D630@ashleymoran.me.uk>
	<f4cd26df0709081649r518df9bi38c1996de2df8b21@mail.gmail.com>
	<BE6B1856-0CC1-4440-A58E-FF4A81439F90@ashleymoran.me.uk>
Message-ID: <f4cd26df0709081831y1dd9e493t81bacc1636908985@mail.gmail.com>

On 9/8/07, Ashley Moran <work at ashleymoran.me.uk> wrote:
>
> Interesting!  What about the case of long-running requests, eg file
> uploads, outgoing web service requests or complex calculations?  Do
> you have any of those, and if so, how does it affect the balance of
> event-based vs multithreaded?

Yeah.

File uploads -- the most efficient way to deal with these is really to
run a separate, dedicated mini-app just for them.  If I remember my
history correctly, this is actually what originally gave rize to Merb.
 Ezra needed something fast and lightweight to efficiently handle file
uploads.

In Ruby, a dedicated file upload handler running event based, though,
will still outperform the multithreaded approach. That might make an
interesting topic for a blog post sometime soon....

Regarding accessing web services, it all depends on whether those
accesses are done in a way that block other threads or not.  If they
do not, then heavy web service access would be a place where latencies
could also be captured.

Heavy calculations, on the other hand, are a hands down winner for the
single threaded approach.  There are no latencies to capture -- it's
CPU all the way, so you just need to bring all the resources available
to the process to bear on one problem at a time.


Kirk haines

From work at ashleymoran.me.uk  Sun Sep  9 06:52:24 2007
From: work at ashleymoran.me.uk (Ashley Moran)
Date: Sun, 9 Sep 2007 11:52:24 +0100
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709081831y1dd9e493t81bacc1636908985@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<873axoztov.wl%erik.hetzner@ucop.edu>
	<ADD3770F-EDA5-4AE0-8695-5AA1F7F8D630@ashleymoran.me.uk>
	<f4cd26df0709081649r518df9bi38c1996de2df8b21@mail.gmail.com>
	<BE6B1856-0CC1-4440-A58E-FF4A81439F90@ashleymoran.me.uk>
	<f4cd26df0709081831y1dd9e493t81bacc1636908985@mail.gmail.com>
Message-ID: <27ACD9D0-84D4-4616-A292-2287F49AAA6A@ashleymoran.me.uk>


On Sep 09, 2007, at 2:31 am, Kirk Haines wrote:

> In Ruby, a dedicated file upload handler running event based, though,
> will still outperform the multithreaded approach. That might make an
> interesting topic for a blog post sometime soon....

Definately!



From neongrau at gmail.com  Mon Sep 10 11:09:26 2007
From: neongrau at gmail.com (Ralf Vitasek)
Date: Mon, 10 Sep 2007 17:09:26 +0200
Subject: [Mongrel] mongrel_rails service on windows incompatible with recent
	ruby builds?
Message-ID: <7adba8e60709100809sa1f19c8g4466f1703db48e1@mail.gmail.com>

i was suffering from a massive memory leak in my application. after
searching and testing for quite a while i finally found that it wasn't
my app that had a leak but the win32ole support that was used by the
ADO driver in sqlserver adapter.

the solution to fix the leak was to install ruby 1.8.5p52 or 1.8.6p36

but sadly as soon i install one of the fixed releases mongrel won't
start as service anymore.

the latest version i was able to run as a service is 1.8.5p35 but that
doesn't have the leak fixed.

"mongrel_rails start" will run the app in dev or production mode on
the console without problems.

when trying to start the service in mmc it fails with the message that
the service was started and that it stopped right away.

when running mongrel_service.exe with the "console" option:
###
C:\informer_rails>"c:/ruby/bin/mongrel_service.exe" console -e development -p 30
00 -a 0.0.0.0 -l "log/mongrel.log" -P "log/mongrel.pid" -c "c:/informer_rails" -
t 0 -r "public" -n 1024
Mongrel Win32 Service, version 0.3.1
(c) 2006 The Mongrel development team.

Starting service 'single' in console mode, please wait...
Service is in running state.
Press Ctrl-C to stop it.
###

i see a ruby.exe for a second in the task manager that seems to die
right away. no log files are created.

i also wonder why it says "Mongrel Win32 Service, version 0.3.1"
because i have 0.3.2 installed. i tried uninstalling and then
reinstalling mongrel, mongrel_service and win32-service without any
luck.

and the error doesn't seem to come from win32-service as i'm able to
run an acts_as_ferret serving DRb without any problems with every ruby
version.

please check the attached servicefb.log and mongrel_service.log
as i can't see anything wrong in those logs.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: servicefb.log
Url: http://rubyforge.org/pipermail/mongrel-users/attachments/20070910/4689b21f/attachment.pl 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: mongrel_service.log
Url: http://rubyforge.org/pipermail/mongrel-users/attachments/20070910/4689b21f/attachment-0001.pl 

From jan.svitok at gmail.com  Mon Sep 10 12:56:49 2007
From: jan.svitok at gmail.com (Jano Svitok)
Date: Mon, 10 Sep 2007 18:56:49 +0200
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <7adba8e60709100809sa1f19c8g4466f1703db48e1@mail.gmail.com>
References: <7adba8e60709100809sa1f19c8g4466f1703db48e1@mail.gmail.com>
Message-ID: <8d9b3d920709100956v20967799t15f957b6e6a88d56@mail.gmail.com>

On 9/10/07, Ralf Vitasek <neongrau at gmail.com> wrote:
> i also wonder why it says "Mongrel Win32 Service, version 0.3.1"
> because i have 0.3.2 installed. i tried uninstalling and then
> reinstalling mongrel, mongrel_service and win32-service without any
> luck.

It's a bug in the service code:
http://mongrel.rubyforge.org/svn/trunk/projects/mongrel_service/native/mongrel_service.bi

No need to worry.

I've reported to
http://rubyforge.org/tracker/index.php?func=detail&aid=13823&group_id=1306&atid=5145

From erik.hetzner at ucop.edu  Mon Sep 10 14:25:47 2007
From: erik.hetzner at ucop.edu (Erik Hetzner)
Date: Mon, 10 Sep 2007 11:25:47 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709081649r518df9bi38c1996de2df8b21@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<873axoztov.wl%erik.hetzner@ucop.edu>
	<ADD3770F-EDA5-4AE0-8695-5AA1F7F8D630@ashleymoran.me.uk>
	<f4cd26df0709081649r518df9bi38c1996de2df8b21@mail.gmail.com>
Message-ID: <87wsuyxtqs.wl%erik.hetzner@ucop.edu>

At Sat, 8 Sep 2007 16:49:58 -0700,
"Kirk Haines" <wyhaines at gmail.com> wrote:
> I have two points, here.  First, you can run a considerable number of
> ruby based sites on a single modest server like yours, and they can
> handle a rather impressive traffic load on a single process, depending
> on your framework.  And second, running the same code in an event
> based pipeline versus a multithreaded pipeline invariably gives me
> better peformance in my apps.

Hi Kirk,

Thanks so much for your information here. I?m going to have a closer
look at your event driven mongrel and see what kind of results I can
get out of it for my systems.

best,
Erik Hetzner



-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070910/a677a1d0/attachment.bin 

From erik.hetzner at ucop.edu  Mon Sep 10 14:26:04 2007
From: erik.hetzner at ucop.edu (Erik Hetzner)
Date: Mon, 10 Sep 2007 11:26:04 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709081649r518df9bi38c1996de2df8b21@mail.gmail.com>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<B220AB03-C117-4537-9E01-C25003C93D6C@brainspl.at>
	<87d4wuz4bk.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<873axoztov.wl%erik.hetzner@ucop.edu>
	<ADD3770F-EDA5-4AE0-8695-5AA1F7F8D630@ashleymoran.me.uk>
	<f4cd26df0709081649r518df9bi38c1996de2df8b21@mail.gmail.com>
Message-ID: <87veaixtqb.wl%erik.hetzner@ucop.edu>

At Sat, 8 Sep 2007 16:49:58 -0700,
"Kirk Haines" <wyhaines at gmail.com> wrote:
> I have two points, here.  First, you can run a considerable number of
> ruby based sites on a single modest server like yours, and they can
> handle a rather impressive traffic load on a single process, depending
> on your framework.  And second, running the same code in an event
> based pipeline versus a multithreaded pipeline invariably gives me
> better peformance in my apps.

Hi Kirk,

Thanks so much for your information here. I?m going to have a closer
look at your event driven mongrel and see what kind of results I can
get out of it for my systems.

best,
Erik Hetzner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070910/7ac39b85/attachment-0001.bin 

From wyhaines at gmail.com  Mon Sep 10 14:31:45 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Mon, 10 Sep 2007 11:31:45 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <87veaixtqb.wl%erik.hetzner@ucop.edu>
References: <966599840709071133y20757821rcda9b36a3d73cc4e@mail.gmail.com>
	<f4cd26df0709071232g1c9d7d5eu7ab880838c0df20a@mail.gmail.com>
	<87bqceyzt7.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071350m347fd19fx43f57f670a7f6f83@mail.gmail.com>
	<87abryyx6v.wl%erik.hetzner@ucop.edu>
	<f4cd26df0709071534t50bd0981m683777c958b490ce@mail.gmail.com>
	<873axoztov.wl%erik.hetzner@ucop.edu>
	<ADD3770F-EDA5-4AE0-8695-5AA1F7F8D630@ashleymoran.me.uk>
	<f4cd26df0709081649r518df9bi38c1996de2df8b21@mail.gmail.com>
	<87veaixtqb.wl%erik.hetzner@ucop.edu>
Message-ID: <f4cd26df0709101131k48a43ddaqffb4aebb2424fbfe@mail.gmail.com>

On 9/10/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote:

> Thanks so much for your information here. I'm going to have a closer
> look at your event driven mongrel and see what kind of results I can
> get out of it for my systems.

Cool.  Feel free to email me if you have any questions.  There is a
new release Swiftiply and of evented_mongrel coming just as soon as I
manage to disentangle myself from client facing work I'm getting
buried with.  From a scalability standpoint, the new release of
evented_mongrel enables it to use epoll on platforms that support that
(Linux 2.6.x platforms), instead of select.  I've been saying for two
weeks, now, "This week...."  but...hopefully this week I get to the
light at the end of the tunnel.  ;)


Kirk Haines

From luislavena at gmail.com  Mon Sep 10 15:05:52 2007
From: luislavena at gmail.com (Luis Lavena)
Date: Mon, 10 Sep 2007 16:05:52 -0300
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <7adba8e60709100809sa1f19c8g4466f1703db48e1@mail.gmail.com>
References: <7adba8e60709100809sa1f19c8g4466f1703db48e1@mail.gmail.com>
Message-ID: <71166b3b0709101205m57645994rccc9ea6e29957ccc@mail.gmail.com>

On 9/10/07, Ralf Vitasek <neongrau at gmail.com> wrote:
> i was suffering from a massive memory leak in my application. after
> searching and testing for quite a while i finally found that it wasn't
> my app that had a leak but the win32ole support that was used by the
> ADO driver in sqlserver adapter.
>

Ralf, we need to clarify that win32-service, mongrel and even
mongrel_service are quite different stuff from quite different
developers.

> the solution to fix the leak was to install ruby 1.8.5p52 or 1.8.6p36
>
> but sadly as soon i install one of the fixed releases mongrel won't
> start as service anymore.
>
> the latest version i was able to run as a service is 1.8.5p35 but that
> doesn't have the leak fixed.
>
> "mongrel_rails start" will run the app in dev or production mode on
> the console without problems.
>
> when trying to start the service in mmc it fails with the message that
> the service was started and that it stopped right away.
>
> when running mongrel_service.exe with the "console" option:
> ###
> C:\informer_rails>"c:/ruby/bin/mongrel_service.exe" console -e development -p 30
> 00 -a 0.0.0.0 -l "log/mongrel.log" -P "log/mongrel.pid" -c "c:/informer_rails" -
> t 0 -r "public" -n 1024
> Mongrel Win32 Service, version 0.3.1
> (c) 2006 The Mongrel development team.
>
> Starting service 'single' in console mode, please wait...
> Service is in running state.
> Press Ctrl-C to stop it.
> ###
>
> i see a ruby.exe for a second in the task manager that seems to die
> right away. no log files are created.
>
> i also wonder why it says "Mongrel Win32 Service, version 0.3.1"
> because i have 0.3.2 installed. i tried uninstalling and then
> reinstalling mongrel, mongrel_service and win32-service without any
> luck.
>

Version 0.3.1 was a error in the definition headers, my mistake.

> and the error doesn't seem to come from win32-service as i'm able to
> run an acts_as_ferret serving DRb without any problems with every ruby
> version.
>

For the record, the problem with win32-service was not able to serve
Rails application properly, for other ruby tasks, it works fine.

Also, mongrel_service only uses win32-service to install/remove the
service from the SCM, nothing else.

> please check the attached servicefb.log and mongrel_service.log
> as i can't see anything wrong in those logs.
>

>From mongrel_service.log, could you try running this command line?
ruby.exe c:\ruby\bin\mongrel_rails start -e development -p 3000 -a
0.0.0.0 -l log/mongrel.log -P log/mongrel.pid -c c:/informer_rails -t
0 -r public -n 1024

It seems it's trying to run ruby.exe from the PATH, maybe you don't
have it in your environment variables?

Latest version (0.3.2) properly uses C:\ruby\bin\ruby.exe as ruby executable.

For get that fixes, you need to remove and reinstall the service.

Sorry the troubles, and please let us know if that help you :-)

-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

From casper at lol.dk  Tue Sep 11 03:31:11 2007
From: casper at lol.dk (Casper Fabricius)
Date: Tue, 11 Sep 2007 09:31:11 +0200
Subject: [Mongrel] Fails to restart mongrel cluster after Capistrano
 rollback (PIDs not deleted)
In-Reply-To: <29182826-820D-43A3-A673-9987549D8461@gmail.com>
References: <389A7842-CD02-435F-8B76-D3BE5EB2C4E0@lol.dk>
	<29182826-820D-43A3-A673-9987549D8461@gmail.com>
Message-ID: <C476B4CF-CCEA-45B0-B5DC-C97938D53879@lol.dk>

Thanks Wayne,

That does the trick for me! I simply added this:

   task :rollback do
     stop
     rollback_code
     start
   end

Cheers,
Casper


On 07/09/2007, at 18:15, Wayne E. Seguin wrote:

> On Sep 07, 2007, at 03:30 , Casper Fabricius wrote:
>> Hi,
>>
>> I am deploying a Rails website using Capistrano to a VPS running the
>> site through Apache and Mongrel.
>>
>> 'cap deploy' works fine, and mongrels are restarted. When I execute
>> 'cap deploy:rollback', however, my script fails to restart the
>> mongrels giving this error:
>> !!! PID file tmp/pids/mongrel.5000.pid already exists.  Mongrel could
>> be running already.  Check your log/mongrel.5000.log for errors.
>>
>> I have a hunch that the error is related to the fact that I link / 
>> tmp/
>> pids into the shared directory upon each release in the
>> "after_update" task, but I'm not sure how I end up with this problem,
>> as the old dir I'm rolling back also has the link setup. The mongrels
>> are stopped, but for some reason they don't clean up their PID files,
>> and thus they fail to start again.
>>
>> The full output of the rollback looks like this:
>> http://pastie.caboo.se/94602
>>
>> My deploy.rb looks like this:
>> http://pastie.caboo.se/94600
>>
>> Any help on this issue would be appreciated - thanks!
>>
>> Best regards,
>> Casper Fabricius
>> ________________________________________
>
> Casper,
>
> I'd rewrite the default rollback task to:
> 1. Stop mongrel cluster
> 2. Rollback
> 3. Restart mongrel cluster
>
> Between 2-3 you might also want to ensure that the new (old)  
> "current" directory links the pid directory correctly.
>
>   ~Wayne
>
> s///g
> Wayne E. Seguin
> Sr. Systems Architect & Systems Administrator
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070911/e459260c/attachment.html 

From neongrau at gmail.com  Tue Sep 11 04:39:20 2007
From: neongrau at gmail.com (Ralf Vitasek)
Date: Tue, 11 Sep 2007 10:39:20 +0200
Subject: [Mongrel] mongrel_rails service on windows incompatible with	recent
 ruby builds?
References: 7adba8e60709100809sa1f19c8g4466f1703db48e1@mail.gmail.com
Message-ID: <46E65438.5040307@gmail.com>

hello luis!

thank you very much for your reply.

>/ From mongrel_service.log, could you try running this command line?
/> ruby.exe c:\ruby\bin\mongrel_rails start -e development -p 3000 -a
> 0.0.0.0 -l log/mongrel.log -P log/mongrel.pid -c c:/informer_rails -t
> 0 -r public -n 1024

as i said before starting via "mongrel_rails" works with every ruby version.
but i tested again to be correct. so YES this works!


> It seems it's trying to run ruby.exe from the PATH, maybe you don't
> have it in your environment variables?

ruby is in PATH, as i said before the problem is that ruby.exe starts
but just dies right away when started as a service with the latest patchlevel


> Latest version (0.3.2) properly uses C:\ruby\bin\ruby.exe as ruby executable.
> For get that fixes, you need to remove and reinstall the service.

as i said, already tried uninstalling and reinstalling everything before
(3 times already, please don't ask me to try again, i hardly doubt it'll change anything).


> Sorry the troubles, and please let us know if that help you :-)

no, thanks for trying to walk me through but sorry, that didn't really help :-(



let me summarize the problem again:

the service dies right away, without creating any logs in the rails log dir
when using the latest patchlevel ruby version with the win32ole fix 
( http://rubyforge.org/tracker/?func=detail&atid=1698&aid=7553&group_id=426 ).

running the app via "mongrel_rails start" works with EVERY version.
i just need to copy over the zipped ruby version into c:\ruby to see the problem
1.8.5p35 -> service works
copy 1.8.5p52 into c:\ruby -> doesnt work anymore
copy 1.8.5p35 back into c:\ruby -> works again
copy 1.8.6p36 into c:\ruby -> doesnt work anymore
copy 1.8.6 into c:\ruby -> work again


please try this ruby release:
ftp://ftp.ruby-lang.org/pub/ruby/binaries/mswin32/ruby-1.8.5-p52-i386-mswin32.zip
(this is the latest version with the memory leak fixed)

or in case you're using 1.8.6
ftp://ftp.ruby-lang.org/pub/ruby/binaries/mswin32/ruby-1.8.6-p36-i386-mswin32.zip


and for the record: these are the latest ruby versions that will get the services to run:
ftp://ftp.ruby-lang.org/pub/ruby/binaries/mswin32/ruby-1.8.5-p35-i386-mswin32.zip
or
ftp://ftp.ruby-lang.org/pub/ruby/binaries/mswin32/ruby-1.8.6-i386-mswin32.zip


i hope this clears things more up.

cheers,
ralf


From wayneeseguin at gmail.com  Tue Sep 11 11:08:31 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Tue, 11 Sep 2007 11:08:31 -0400
Subject: [Mongrel] Fails to restart mongrel cluster after Capistrano
	rollback (PIDs not deleted)
In-Reply-To: <C476B4CF-CCEA-45B0-B5DC-C97938D53879@lol.dk>
References: <389A7842-CD02-435F-8B76-D3BE5EB2C4E0@lol.dk>
	<29182826-820D-43A3-A673-9987549D8461@gmail.com>
	<C476B4CF-CCEA-45B0-B5DC-C97938D53879@lol.dk>
Message-ID: <C2A90466-5319-485C-A251-83FA8FC03780@gmail.com>

On Sep 11, 2007, at 03:31 , Casper Fabricius wrote:
> Thanks Wayne,
>
> That does the trick for me! I simply added this:
>
>   task :rollback do
>     stop
>     rollback_code
>     start
>   end
>
> Cheers,
> Casper

Casper,

Excellent! I am glad that worked. Thank you for sharing your solution.

Regards,

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070911/bf156885/attachment-0001.html 

From rogerpack2005 at gmail.com  Tue Sep 11 15:02:17 2007
From: rogerpack2005 at gmail.com (Roger Pack)
Date: Tue, 11 Sep 2007 13:02:17 -0600
Subject: [Mongrel] multi threaded theoretically useful?
Message-ID: <966599840709111202p25c7060dyf91d17f2ec3617d@mail.gmail.com>

So I'd say overall the concensus is that multi-threading might be faster,
depends on where an apps specific latencies end up (notice the might).

Now if I could only come up with a multi-thread safe version of Rails... :)
[ha ha] and pure ruby DB interfaces... :)

Thank you all!
-Roger
Belief is good. http://www.google.com/search?q=free+bible
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070911/dded4681/attachment.html 

From wayneeseguin at gmail.com  Tue Sep 11 15:03:50 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Tue, 11 Sep 2007 15:03:50 -0400
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <966599840709111202p25c7060dyf91d17f2ec3617d@mail.gmail.com>
References: <966599840709111202p25c7060dyf91d17f2ec3617d@mail.gmail.com>
Message-ID: <70E19272-55EA-4620-B58D-77FACE63AAE6@gmail.com>

On Sep 11, 2007, at 15:02 , Roger Pack wrote:
> So I'd say overall the concensus is that multi-threading might be  
> faster, depends on where an apps specific latencies end up (notice  
> the might).
>
> Now if I could only come up with a multi-thread safe version of  
> Rails... :) [ha ha] and pure ruby DB interfaces... :)
>
> Thank you all!
> -Roger

Merb + Sequel :)

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070911/f21441ff/attachment.html 

From work at ashleymoran.me.uk  Tue Sep 11 16:17:55 2007
From: work at ashleymoran.me.uk (Ashley Moran)
Date: Tue, 11 Sep 2007 21:17:55 +0100
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <966599840709111202p25c7060dyf91d17f2ec3617d@mail.gmail.com>
References: <966599840709111202p25c7060dyf91d17f2ec3617d@mail.gmail.com>
Message-ID: <853BADE0-9DA0-4A76-AFCE-582F9CCF0C83@ashleymoran.me.uk>


On Sep 11, 2007, at 8:02 pm, Roger Pack wrote:

> pure ruby DB interfaces... :)


There's a postgres-pr gem, but I don't use it myself.  I tell people  
running Windows they need it, but I always go for the C version  
myself.  If I was in a situation where I had a lot of long-running  
queries, would I be better with the pure Ruby interface from a  
concurrency point of view?

(I'm assuming that as Ruby VMs improve, this will become true in any  
case)

Ashley

From wyhaines at gmail.com  Tue Sep 11 16:40:54 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Tue, 11 Sep 2007 13:40:54 -0700
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <853BADE0-9DA0-4A76-AFCE-582F9CCF0C83@ashleymoran.me.uk>
References: <966599840709111202p25c7060dyf91d17f2ec3617d@mail.gmail.com>
	<853BADE0-9DA0-4A76-AFCE-582F9CCF0C83@ashleymoran.me.uk>
Message-ID: <f4cd26df0709111340g54835ab4r7911108f0f0b8c4f@mail.gmail.com>

On 9/11/07, Ashley Moran <work at ashleymoran.me.uk> wrote:

> There's a postgres-pr gem, but I don't use it myself.  I tell people
> running Windows they need it, but I always go for the C version
> myself.  If I was in a situation where I had a lot of long-running
> queries, would I be better with the pure Ruby interface from a
> concurrency point of view?
>
> (I'm assuming that as Ruby VMs improve, this will become true in any
> case)

It depends.  In _general_, the blocking, C extension based drivers
will still net you better performance, currently, becase they are so
much faster than the pure ruby drivers.  Additionally, because they
are typically just interfaces to the database's own API layer, you
know that they are stable and compatible, while the pure ruby
implementations may not be as stable.

However, if a large number of your queries are very slow, then there
may be room to use some of those wasted cycles by using a
multithreaded deployment.  It's something that one would really have
to benchmark to know for sure for any given use case.


Kirk Haines

From ff at onebeat.com  Tue Sep 11 16:46:51 2007
From: ff at onebeat.com (Farhad Farzaneh)
Date: Tue, 11 Sep 2007 13:46:51 -0700
Subject: [Mongrel] Mongrel instance dies unexpectedly, but cleanly....
Message-ID: <47bbe02f0709111346r7a559d74ha287412d3906fe80@mail.gmail.com>

Hi,

I have a two instance mongrel cluster, running behind apache
mod_proxy_balancer on a linux VPS.  Mongrel_cluster is 1.01, mongrel is
v1.02, rails 1.2.3, ruby 1.8.6. I seem to have enough memory.

About 2 minutes after I start the cluster, one of my instances dies - it
could be either one.  The log file indicates that it received a TERM
signal.  The instance does die cleanly, removing its pid file.  But I didn't
give it a TERM signal, and I don't think I'm running any services that would
do so.

My searches haven't been very fruitful.  Any ideas would be appreciated.

Thanks

Farhad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070911/82052ed2/attachment.html 

From wayneeseguin at gmail.com  Tue Sep 11 17:05:14 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Tue, 11 Sep 2007 17:05:14 -0400
Subject: [Mongrel] Mongrel instance dies unexpectedly, but cleanly....
In-Reply-To: <47bbe02f0709111346r7a559d74ha287412d3906fe80@mail.gmail.com>
References: <47bbe02f0709111346r7a559d74ha287412d3906fe80@mail.gmail.com>
Message-ID: <6F7B4B03-3BBD-431C-835E-9F649E3920F4@gmail.com>

On Sep 11, 2007, at 16:46 , Farhad Farzaneh wrote:
> Hi,
>
> I have a two instance mongrel cluster, running behind apache  
> mod_proxy_balancer on a linux VPS.  Mongrel_cluster is 1.01,  
> mongrel is v1.02, rails 1.2.3, ruby 1.8.6. I seem to have enough  
> memory.
>
> About 2 minutes after I start the cluster, one of my instances dies  
> - it could be either one.  The log file indicates that it received  
> a TERM signal.  The instance does die cleanly, removing its pid  
> file.  But I didn't give it a TERM signal, and I don't think I'm  
> running any services that would do so.
>
> My searches haven't been very fruitful.  Any ideas would be  
> appreciated.
>
> Thanks
>
> Farhad

Farhad,

You're running on a shared server and when you startup the second  
mongrel more than likely the VPS system is sending the second Mongrel  
a kill signal because you've gone over your resource allocation.

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070911/b00e5743/attachment.html 

From luislavena at gmail.com  Tue Sep 11 17:28:30 2007
From: luislavena at gmail.com (Luis Lavena)
Date: Tue, 11 Sep 2007 18:28:30 -0300
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <46E65438.5040307@gmail.com>
References: <46E65438.5040307@gmail.com>
Message-ID: <71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>

On 9/11/07, Ralf Vitasek <neongrau at gmail.com> wrote:
> hello luis!
>
> thank you very much for your reply.
>
> >/ From mongrel_service.log, could you try running this command line?
> /> ruby.exe c:\ruby\bin\mongrel_rails start -e development -p 3000 -a
> > 0.0.0.0 -l log/mongrel.log -P log/mongrel.pid -c c:/informer_rails -t
> > 0 -r public -n 1024
>
> as i said before starting via "mongrel_rails" works with every ruby version.
> but i tested again to be correct. so YES this works!
>
>
> > It seems it's trying to run ruby.exe from the PATH, maybe you don't
> > have it in your environment variables?
>
> ruby is in PATH, as i said before the problem is that ruby.exe starts
> but just dies right away when started as a service with the latest patchlevel
>

So far, everything is ok in the ruby land...

> > Latest version (0.3.2) properly uses C:\ruby\bin\ruby.exe as ruby executable.
> > For get that fixes, you need to remove and reinstall the service.
>
> as i said, already tried uninstalling and reinstalling everything before
> (3 times already, please don't ask me to try again, i hardly doubt it'll change anything).
>

No problem, I'll not ;-)


>
> let me summarize the problem again:
>
> the service dies right away, without creating any logs in the rails log dir
> when using the latest patchlevel ruby version with the win32ole fix
> ( http://rubyforge.org/tracker/?func=detail&atid=1698&aid=7553&group_id=426 ).
>
> running the app via "mongrel_rails start" works with EVERY version.
> i just need to copy over the zipped ruby version into c:\ruby to see the problem
> 1.8.5p35 -> service works
> copy 1.8.5p52 into c:\ruby -> doesnt work anymore
> copy 1.8.5p35 back into c:\ruby -> works again
> copy 1.8.6p36 into c:\ruby -> doesnt work anymore
> copy 1.8.6 into c:\ruby -> work again
>

Mmm, you're using win32ole to connect to ADO driver, right?

Maybe there is a problem due the non-impersonation of the service,
since by default it will run as SYSTEM, and not the user you're logged
in.

This often bring problems when dealing with ActiveX/OLE and the proper
creation of instances of these objects.

>
> please try this ruby release:
> ftp://ftp.ruby-lang.org/pub/ruby/binaries/mswin32/ruby-1.8.5-p52-i386-mswin32.zip
> (this is the latest version with the memory leak fixed)
>
> or in case you're using 1.8.6
> ftp://ftp.ruby-lang.org/pub/ruby/binaries/mswin32/ruby-1.8.6-p36-i386-mswin32.zip
>

I'm using 1.8.5 patchlevel 12, since I self-compiled it. I'll give a
try but doubt will found any problem, mostly since I don't use
Win32OLE fo my rails projects.

> i hope this clears things more up.
>

Maybe you could contact me off the list and give me some insight of
your setup that I could replicate here.

One thing we left out of testing is the RunAs... could you go to the
SCM (Service Control Manager) and change the Account used by your
rails application?

Try setting it as your local/user account and see what happens.

-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

From work at ashleymoran.me.uk  Tue Sep 11 18:17:00 2007
From: work at ashleymoran.me.uk (Ashley Moran)
Date: Tue, 11 Sep 2007 23:17:00 +0100
Subject: [Mongrel] multi threaded theoretically useful?
In-Reply-To: <f4cd26df0709111340g54835ab4r7911108f0f0b8c4f@mail.gmail.com>
References: <966599840709111202p25c7060dyf91d17f2ec3617d@mail.gmail.com>
	<853BADE0-9DA0-4A76-AFCE-582F9CCF0C83@ashleymoran.me.uk>
	<f4cd26df0709111340g54835ab4r7911108f0f0b8c4f@mail.gmail.com>
Message-ID: <F9590072-9714-4854-B6BF-433587103FC6@ashleymoran.me.uk>


On Sep 11, 2007, at 9:40 pm, Kirk Haines wrote:

> However, if a large number of your queries are very slow, then there
> may be room to use some of those wasted cycles by using a
> multithreaded deployment.  It's something that one would really have
> to benchmark to know for sure for any given use case.

How much time do you have to write that blog post you mentioned ;)

From ff at onebeat.com  Tue Sep 11 20:00:30 2007
From: ff at onebeat.com (Farhad Farzaneh)
Date: Tue, 11 Sep 2007 17:00:30 -0700
Subject: [Mongrel] Mongrel instance dies unexpectedly, but cleanly...
Message-ID: <47bbe02f0709111700s17a5286cy456221cb005cc352@mail.gmail.com>

> You're running on a shared server and when you startup the second
> mongrel more than likely the VPS system is sending the second Mongrel
> a kill signal because you've gone over your resource allocation.
>
>   ~Wayne


Thanks - are there other resources that would be contentious other than
memory? I believe I had about 60Meg still available (even w/ 2 mongrels
already running), and each mongrel only takes about 30 MB w/ my app.

I ended up reducing the number of apache mpm's, and then on a lark started
w/ 3 mongrel servers to see if 1 or 2 would die.  They're all still up, but
after any one dies, I need to re-run the experiment while changing only 1
variable ....

thx

F
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070911/521d8946/attachment.html 

From wayneeseguin at gmail.com  Tue Sep 11 21:17:33 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Tue, 11 Sep 2007 21:17:33 -0400
Subject: [Mongrel] Mongrel instance dies unexpectedly, but cleanly...
In-Reply-To: <47bbe02f0709111700s17a5286cy456221cb005cc352@mail.gmail.com>
References: <47bbe02f0709111700s17a5286cy456221cb005cc352@mail.gmail.com>
Message-ID: <1D4012EF-0446-43B0-970B-DE5B7446C72C@gmail.com>

On Sep 11, 2007, at 20:00 , Farhad Farzaneh wrote:
> You're running on a shared server and when you startup the second
> mongrel more than likely the VPS system is sending the second Mongrel
> a kill signal because you've gone over your resource allocation.
>
>   ~Wayne
>
> Thanks - are there other resources that would be contentious other  
> than memory?

I believe that Memory would be the only item that they would beat  
your Mongrels down on. The mongrels will not consume many file handle  
resources being just started up... ?

Good luck,

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070911/155adb49/attachment.html 

From neongrau at gmail.com  Wed Sep 12 02:59:28 2007
From: neongrau at gmail.com (Ralf Vitasek)
Date: Wed, 12 Sep 2007 08:59:28 +0200
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>
References: <46E65438.5040307@gmail.com>
	<71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>
Message-ID: <7adba8e60709112359m179c3481g32195fc51c6b2148@mail.gmail.com>

On 9/11/07, Luis Lavena <luislavena at gmail.com> wrote:
> Mmm, you're using win32ole to connect to ADO driver, right?

yes, i even tried hacking the ADO.rb to use SQLNCLI instead of
SQLOLEDB but same behavior with that one. and i'm tied to the ADO
driver since the ODBC driver won't work too many SQL statements don't
work there and hard crash the my rails app due to weird out of memory
errors already when indexing a model with acts_as_ferret.

> Maybe there is a problem due the non-impersonation of the service,
> since by default it will run as SYSTEM, and not the user you're logged
> in.

forgot to mention i tired that already, makes no difference. same
behavior shows up when starting the mongrel_service.exe with the
console switch so i'm pretty sure it's using the same local user as
when starting the app via mongrel_rails directly (which works).

> I'm using 1.8.5 patchlevel 12, since I self-compiled it. I'll give a
> try but doubt will found any problem, mostly since I don't use
> Win32OLE fo my rails projects.

i'm only using it with the sqlserver adapter, nowhere else. what i
haven't tried yet was if the problem also shows when using a different
db adapter. i'm gonna test this today with ODBC and or a test app with
mysql.


> Maybe you could contact me off the list and give me some insight of
> your setup that I could replicate here.

sure, what do you need to know?
i'm using rails 1.2.3, my DB runs on MS SQL 2005 (Standard).
the problem showed up on Windows XP prof. and Windows Server 2003

i'll do whatever i can to help getting this issue fixed. already took
me 2 weeks to find the cause of my memory leak and finding a fixed
version (sadly ruby-lang.org is still offering the old unfixed
versions on their download page).



cheers,
ralf

From njvack at wisc.edu  Wed Sep 12 17:47:07 2007
From: njvack at wisc.edu (Nathan Vack)
Date: Wed, 12 Sep 2007 16:47:07 -0500
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
Message-ID: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>

Hi,

I'm doing development with Mongrel 1.0.1 on Ruby 1.8.4 / OS X  
10.4.10, and I've found that often when I shut down my server with  
^C, it hangs for a while, gobbling up all CPU. Sometimes it stops  
after a few seconds, sometimes it's run for several minutes at least.  
This happens across all the apps I develop on, but it seems to hang  
longer after they've been heavily used for many hours / days.

It's not a very big deal -- I just kill -9 it... but has anyone else  
seen this? And suggestions as to how I can avoid it?

Thanks!
-Nate

From luislavena at gmail.com  Wed Sep 12 17:57:20 2007
From: luislavena at gmail.com (Luis Lavena)
Date: Wed, 12 Sep 2007 18:57:20 -0300
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
Message-ID: <71166b3b0709121457m704a2a48x63a18831e1d2215e@mail.gmail.com>

On 9/12/07, Nathan Vack <njvack at wisc.edu> wrote:
> Hi,
>
> I'm doing development with Mongrel 1.0.1 on Ruby 1.8.4 / OS X
> 10.4.10, and I've found that often when I shut down my server with
> ^C, it hangs for a while, gobbling up all CPU. Sometimes it stops
> after a few seconds, sometimes it's run for several minutes at least.
> This happens across all the apps I develop on, but it seems to hang
> longer after they've been heavily used for many hours / days.
>
> It's not a very big deal -- I just kill -9 it... but has anyone else
> seen this? And suggestions as to how I can avoid it?
>

What db engine/adapter are you using? Maybe mongrel is trying to close
pending connections left in the limbo (from HTTP serving or db
connections).

-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

From pete at nextengine.com  Wed Sep 12 18:04:11 2007
From: pete at nextengine.com (Pete DeLaurentis)
Date: Wed, 12 Sep 2007 15:04:11 -0700
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
Message-ID: <A13BB365-D5F3-48DD-9D6E-1F2B90E42A7F@nextengine.com>

We see this on all of our Mac's here as well.  We've tried it with  
different versions of Ruby (1.8.4, 1.8.5, and 18.6), and the problem  
happens with all of them.

If we don't forget to turn off mongrel before walking away from the  
computer for a few minutes... it'll consume all system resources and  
we'll need to restart the machine.  Our production servers are Ubuntu  
and run fine, but it's pretty annoying for development, and odd since  
Mac's are some popular for Ruby development.

To answer Luis Lavena's question: we're using MySQL over HTTP.

-Pete

On Sep 12, 2007, at 2:47 PM, Nathan Vack wrote:

> Hi,
>
> I'm doing development with Mongrel 1.0.1 on Ruby 1.8.4 / OS X
> 10.4.10, and I've found that often when I shut down my server with
> ^C, it hangs for a while, gobbling up all CPU. Sometimes it stops
> after a few seconds, sometimes it's run for several minutes at least.
> This happens across all the apps I develop on, but it seems to hang
> longer after they've been heavily used for many hours / days.
>
> It's not a very big deal -- I just kill -9 it... but has anyone else
> seen this? And suggestions as to how I can avoid it?
>
> Thanks!
> -Nate
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From luislavena at gmail.com  Wed Sep 12 18:06:54 2007
From: luislavena at gmail.com (Luis Lavena)
Date: Wed, 12 Sep 2007 19:06:54 -0300
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <A13BB365-D5F3-48DD-9D6E-1F2B90E42A7F@nextengine.com>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<A13BB365-D5F3-48DD-9D6E-1F2B90E42A7F@nextengine.com>
Message-ID: <71166b3b0709121506j6e99f328r6647f41dc6216735@mail.gmail.com>

On 9/12/07, Pete DeLaurentis <pete at nextengine.com> wrote:
> We see this on all of our Mac's here as well.  We've tried it with
> different versions of Ruby (1.8.4, 1.8.5, and 18.6), and the problem
> happens with all of them.
>

That maybe mean that is not ruby fault :-P

> If we don't forget to turn off mongrel before walking away from the
> computer for a few minutes... it'll consume all system resources and
> we'll need to restart the machine.  Our production servers are Ubuntu
> and run fine, but it's pretty annoying for development, and odd since
> Mac's are some popular for Ruby development.
>
> To answer Luis Lavena's question: we're using MySQL over HTTP.
>

You're using the compiled mysql adapter or the one bundled with Rails?
the connection to the mysql server could not be the issue, but maybe
the adapter.

-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

From pete at nextengine.com  Wed Sep 12 18:19:05 2007
From: pete at nextengine.com (Pete DeLaurentis)
Date: Wed, 12 Sep 2007 15:19:05 -0700
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <71166b3b0709121506j6e99f328r6647f41dc6216735@mail.gmail.com>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<A13BB365-D5F3-48DD-9D6E-1F2B90E42A7F@nextengine.com>
	<71166b3b0709121506j6e99f328r6647f41dc6216735@mail.gmail.com>
Message-ID: <5B6786A0-7921-48E2-BDF0-4C4071BE3984@nextengine.com>

Hi Luis,

About this compiled MySQL adapter...

I haven't found much info on it, but from what I have read, it looks  
like it is useful if the database is residing locally on the same  
machine.  However, if you connect to the database via HTTP (which is  
what we're doing), it's not used.

Is this correct?

Thanks,
Pete

On Sep 12, 2007, at 3:06 PM, Luis Lavena wrote:

> On 9/12/07, Pete DeLaurentis <pete at nextengine.com> wrote:
>> We see this on all of our Mac's here as well.  We've tried it with
>> different versions of Ruby (1.8.4, 1.8.5, and 18.6), and the problem
>> happens with all of them.
>>
>
> That maybe mean that is not ruby fault :-P
>
>> If we don't forget to turn off mongrel before walking away from the
>> computer for a few minutes... it'll consume all system resources and
>> we'll need to restart the machine.  Our production servers are Ubuntu
>> and run fine, but it's pretty annoying for development, and odd since
>> Mac's are some popular for Ruby development.
>>
>> To answer Luis Lavena's question: we're using MySQL over HTTP.
>>
>
> You're using the compiled mysql adapter or the one bundled with Rails?
> the connection to the mysql server could not be the issue, but maybe
> the adapter.
>
> -- 
> Luis Lavena
> Multimedia systems
> -
> Leaders are made, they are not born. They are made by hard effort,
> which is the price which all of us must pay to achieve any goal that
> is worthwhile.
> Vince Lombardi
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From luislavena at gmail.com  Wed Sep 12 18:29:21 2007
From: luislavena at gmail.com (Luis Lavena)
Date: Wed, 12 Sep 2007 19:29:21 -0300
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <5B6786A0-7921-48E2-BDF0-4C4071BE3984@nextengine.com>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<A13BB365-D5F3-48DD-9D6E-1F2B90E42A7F@nextengine.com>
	<71166b3b0709121506j6e99f328r6647f41dc6216735@mail.gmail.com>
	<5B6786A0-7921-48E2-BDF0-4C4071BE3984@nextengine.com>
Message-ID: <71166b3b0709121529s2f12d87fk268cf0d083c7eb2c@mail.gmail.com>

On 9/12/07, Pete DeLaurentis <pete at nextengine.com> wrote:
> Hi Luis,
>
> About this compiled MySQL adapter...
>
> I haven't found much info on it, but from what I have read, it looks
> like it is useful if the database is residing locally on the same
> machine.  However, if you connect to the database via HTTP (which is
> what we're doing), it's not used.
>

> Is this correct?

Not exactly.

Native bindings (the compiled ones) offer besides better performance
(since they are a ruby extension in C), a bit of memory-wise use.

Maybe someone could provide better insight information, but AFAIK, it
is the recommended method of running rails+mysql.

Since I don't have a mac, couldn't point you the right tutorial on
getting mysql extension built on OSX, but try searching google for
"mysql rails bindings osx"

(also "gem install mysql osx" as keywords will raise some good results).

-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

From njvack at wisc.edu  Wed Sep 12 18:30:59 2007
From: njvack at wisc.edu (Nathan Vack)
Date: Wed, 12 Sep 2007 17:30:59 -0500
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <71166b3b0709121457m704a2a48x63a18831e1d2215e@mail.gmail.com>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<71166b3b0709121457m704a2a48x63a18831e1d2215e@mail.gmail.com>
Message-ID: <4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>

On Sep 12, 2007, at 4:57 PM, Luis Lavena wrote:

> What db engine/adapter are you using? Maybe mongrel is trying to close
> pending connections left in the limbo (from HTTP serving or db
> connections).

I'm using the compiled mysql gem 2.7, connecting to localhost over  
TCP. Web-side, I'm using mod_proxy and mod_rewrite -- yeah, I'm  
proxying on my development box.

I don't think it's waiting to close HTTP connections, as the service  
never becomes unresponsive, and I'm only running one mongrel. Perhaps  
the database disconnect call sometimes thrashes?

FWIW, I'm on an Intel (Core 2 Duo) mac. It only eats CPU on one core.  
Mysql version is 5.0.21 -- the precompiled one from mysql.org.

And... I doubt *anyone* is connecting to a mysql server over http ;-)

-Nate

From luislavena at gmail.com  Wed Sep 12 18:34:36 2007
From: luislavena at gmail.com (Luis Lavena)
Date: Wed, 12 Sep 2007 19:34:36 -0300
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<71166b3b0709121457m704a2a48x63a18831e1d2215e@mail.gmail.com>
	<4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>
Message-ID: <71166b3b0709121534s55d18dc9m2e34ced65850643a@mail.gmail.com>

On 9/12/07, Nathan Vack <njvack at wisc.edu> wrote:
> On Sep 12, 2007, at 4:57 PM, Luis Lavena wrote:
>
> > What db engine/adapter are you using? Maybe mongrel is trying to close
> > pending connections left in the limbo (from HTTP serving or db
> > connections).
>
> I'm using the compiled mysql gem 2.7, connecting to localhost over
> TCP. Web-side, I'm using mod_proxy and mod_rewrite -- yeah, I'm
> proxying on my development box.
>
> I don't think it's waiting to close HTTP connections, as the service
> never becomes unresponsive, and I'm only running one mongrel. Perhaps
> the database disconnect call sometimes thrashes?
>

Just to be sure, starting mongrel, waiting a few seconds and then
hitting ^C produces the same result?

Changing database.yml to something like sqlite3 produces the same behavior?

> FWIW, I'm on an Intel (Core 2 Duo) mac. It only eats CPU on one core.
> Mysql version is 5.0.21 -- the precompiled one from mysql.org.
>

Oh, too common with Intel Core technology (and single-threaded ruby VM) ;-)

> And... I doubt *anyone* is connecting to a mysql server over http ;-)
>

hehehe, hope not!

-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

From njvack at wisc.edu  Wed Sep 12 18:38:14 2007
From: njvack at wisc.edu (Nathan Vack)
Date: Wed, 12 Sep 2007 17:38:14 -0500
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <71166b3b0709121534s55d18dc9m2e34ced65850643a@mail.gmail.com>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<71166b3b0709121457m704a2a48x63a18831e1d2215e@mail.gmail.com>
	<4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>
	<71166b3b0709121534s55d18dc9m2e34ced65850643a@mail.gmail.com>
Message-ID: <57F742E7-2910-411C-94F0-DFAC3BDEC1C0@wisc.edu>

On Sep 12, 2007, at 5:34 PM, Luis Lavena wrote:

> Just to be sure, starting mongrel, waiting a few seconds and then
> hitting ^C produces the same result?

Yup. It's still responsive enough to print its "** INT signal  
received." messages, interestingly.

> Changing database.yml to something like sqlite3 produces the same  
> behavior?

I'll test this... good idea.

Thanks!
-Nate

From pete at nextengine.com  Wed Sep 12 18:45:41 2007
From: pete at nextengine.com (Pete DeLaurentis)
Date: Wed, 12 Sep 2007 15:45:41 -0700
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<71166b3b0709121457m704a2a48x63a18831e1d2215e@mail.gmail.com>
	<4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>
Message-ID: <7AF5B71A-95EA-4F82-847F-038B71FFD5E0@nextengine.com>

So, here's the article on installing the Mysql gem on OS/X.  I had to  
monkey patch mysql.c, and then it worked after that.

http://blog.maxdunn.com/articles/2006/12/12/installing-mysql-gem-on- 
mac-os-x-10-4

Thanks for the tip Luis.  We'll see if this improves the resource  
drain I've been seeing.

Yeah, that part where I said HTTP, I meant TCP/IP.  With my latest  
project it's all been HTTP, so they're starting to seem a little  
interchangeable.  Shame on me :-)

Thanks,
Pete


On Sep 12, 2007, at 3:30 PM, Nathan Vack wrote:

> On Sep 12, 2007, at 4:57 PM, Luis Lavena wrote:
>
>> What db engine/adapter are you using? Maybe mongrel is trying to  
>> close
>> pending connections left in the limbo (from HTTP serving or db
>> connections).
>
> I'm using the compiled mysql gem 2.7, connecting to localhost over
> TCP. Web-side, I'm using mod_proxy and mod_rewrite -- yeah, I'm
> proxying on my development box.
>
> I don't think it's waiting to close HTTP connections, as the service
> never becomes unresponsive, and I'm only running one mongrel. Perhaps
> the database disconnect call sometimes thrashes?
>
> FWIW, I'm on an Intel (Core 2 Duo) mac. It only eats CPU on one core.
> Mysql version is 5.0.21 -- the precompiled one from mysql.org.
>
> And... I doubt *anyone* is connecting to a mysql server over http ;-)
>
> -Nate
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070912/365237dc/attachment.html 

From ff at onebeat.com  Wed Sep 12 20:09:29 2007
From: ff at onebeat.com (Farhad Farzaneh)
Date: Wed, 12 Sep 2007 17:09:29 -0700
Subject: [Mongrel] : Re:  Mongrel hangs, consumes all CPU on shutdown
Message-ID: <47bbe02f0709121709k391f3a2cv41a4a9e65dc1886e@mail.gmail.com>

> We see this on all of our Mac's here as well.  We've tried it with
> different versions of Ruby (1.8.4, 1.8.5, and 18.6), and the problem
> happens with all of them.
>

We run macs w/ compiled adaptors on local mysql instances and there are no
issues.

F
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070912/ec7efd60/attachment.html 

From pete at nextengine.com  Wed Sep 12 21:11:08 2007
From: pete at nextengine.com (Pete DeLaurentis)
Date: Wed, 12 Sep 2007 18:11:08 -0700
Subject: [Mongrel] : Re:  Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <47bbe02f0709121709k391f3a2cv41a4a9e65dc1886e@mail.gmail.com>
References: <47bbe02f0709121709k391f3a2cv41a4a9e65dc1886e@mail.gmail.com>
Message-ID: <23578545-4F40-4EBD-B4D0-E442BE8F0103@nextengine.com>

Hi,

So I'm now running on the real mysql gem, and speed is noticeably  
improved.

But unfortunately... if I leave it running for around 15 minutes, it  
still eats up enough system resources that I have to physically power  
off the computer.

Any other ideas out there?

Thanks,
Pete

On Sep 12, 2007, at 5:09 PM, Farhad Farzaneh wrote:

> > We see this on all of our Mac's here as well.  We've tried it with
> > different versions of Ruby (1.8.4, 1.8.5, and 18.6), and the problem
> > happens with all of them.
> >
>
> We run macs w/ compiled adaptors on local mysql instances and there  
> are no issues.
>
> F
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From zedshaw at zedshaw.com  Wed Sep 12 21:35:44 2007
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Wed, 12 Sep 2007 21:35:44 -0400
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
Message-ID: <20070912213544.e3cd8fc0.zedshaw@zedshaw.com>

On Wed, 12 Sep 2007 16:47:07 -0500
Nathan Vack <njvack at wisc.edu> wrote:

> Hi,
> 
> I'm doing development with Mongrel 1.0.1 on Ruby 1.8.4 / OS X  
> 10.4.10, and I've found that often when I shut down my server with  
> ^C, it hangs for a while, gobbling up all CPU. Sometimes it stops  
> after a few seconds, sometimes it's run for several minutes at least.  
> This happens across all the apps I develop on, but it seems to hang  
> longer after they've been heavily used for many hours / days.

Your ruby is too old so you probably have a cgi.rb that's vulnerable.  Not even sure if the cgi_multipart_eof_fix works for that Ruby.

Do an upgrade and see if it goes away.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From zedshaw at zedshaw.com  Wed Sep 12 21:39:30 2007
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Wed, 12 Sep 2007 21:39:30 -0400
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <A13BB365-D5F3-48DD-9D6E-1F2B90E42A7F@nextengine.com>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<A13BB365-D5F3-48DD-9D6E-1F2B90E42A7F@nextengine.com>
Message-ID: <20070912213930.c0b48a04.zedshaw@zedshaw.com>

On Wed, 12 Sep 2007 15:04:11 -0700
Pete DeLaurentis <pete at nextengine.com> wrote:

> We see this on all of our Mac's here as well.  We've tried it with  
> different versions of Ruby (1.8.4, 1.8.5, and 18.6), and the problem  
> happens with all of them.
> 
> If we don't forget to turn off mongrel before walking away from the  
> computer for a few minutes... it'll consume all system resources and  
> we'll need to restart the machine.  Our production servers are Ubuntu  
> and run fine, but it's pretty annoying for development, and odd since  
> Mac's are some popular for Ruby development.

Hmm, when I saw it was with 1.8.4 I immediately thought it was to do with the cgi.rb file upload problem.

Here's the description so you can see if it's related (and can google for it):

When you do a file upload the cgi.rb code does this nasty regex backtracking looking for mime boundaries.

In some cases it screws this up--especially with firefox--so that when it gets to the end it needs the EOF to make it stop.

But, when you use a StringIO you don't get an EOF so the cgi.rb code will go into an infinite loop.

Does any of that give you some clues?

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From wayneeseguin at gmail.com  Wed Sep 12 22:10:36 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Wed, 12 Sep 2007 22:10:36 -0400
Subject: [Mongrel] : Re:  Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <23578545-4F40-4EBD-B4D0-E442BE8F0103@nextengine.com>
References: <47bbe02f0709121709k391f3a2cv41a4a9e65dc1886e@mail.gmail.com>
	<23578545-4F40-4EBD-B4D0-E442BE8F0103@nextengine.com>
Message-ID: <05909D5A-D2BD-488A-A328-5EB09363F4E3@gmail.com>

On Sep 12, 2007, at 21:11 , Pete DeLaurentis wrote:
> Hi,
>
> So I'm now running on the real mysql gem, and speed is noticeably
> improved.
>
> But unfortunately... if I leave it running for around 15 minutes, it
> still eats up enough system resources that I have to physically power
> off the computer.
>
> Any other ideas out there?

Turn on debugging and find out exactly what's happening by examining  
the logs.

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070912/f4bcf0fc/attachment-0001.html 

From rogerpack2005 at gmail.com  Wed Sep 12 22:15:36 2007
From: rogerpack2005 at gmail.com (Roger Pack)
Date: Wed, 12 Sep 2007 20:15:36 -0600
Subject: [Mongrel] Mongrel hangs, consumes all CPU on
Message-ID: <966599840709121915i623af468od9fcf48bbee70d3d@mail.gmail.com>

Maybe just monitor it with monit and kill+restart it when it gets out of
control, as a stop gap :)

-Roger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070912/6a56fd3c/attachment.html 

From evan at cloudbur.st  Thu Sep 13 05:40:33 2007
From: evan at cloudbur.st (Evan Weaver)
Date: Thu, 13 Sep 2007 05:40:33 -0400
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <20070912213930.c0b48a04.zedshaw@zedshaw.com>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<A13BB365-D5F3-48DD-9D6E-1F2B90E42A7F@nextengine.com>
	<20070912213930.c0b48a04.zedshaw@zedshaw.com>
Message-ID: <b6f68fc60709130240m39899c39lce68e54127e0b274@mail.gmail.com>

Multipart_fix should work fine with 1.8.4; if it doesn't, there's a
bug I need to look at.

Evan

On 9/12/07, Zed A. Shaw <zedshaw at zedshaw.com> wrote:
> On Wed, 12 Sep 2007 15:04:11 -0700
> Pete DeLaurentis <pete at nextengine.com> wrote:
>
> > We see this on all of our Mac's here as well.  We've tried it with
> > different versions of Ruby (1.8.4, 1.8.5, and 18.6), and the problem
> > happens with all of them.
> >
> > If we don't forget to turn off mongrel before walking away from the
> > computer for a few minutes... it'll consume all system resources and
> > we'll need to restart the machine.  Our production servers are Ubuntu
> > and run fine, but it's pretty annoying for development, and odd since
> > Mac's are some popular for Ruby development.
>
> Hmm, when I saw it was with 1.8.4 I immediately thought it was to do with the cgi.rb file upload problem.
>
> Here's the description so you can see if it's related (and can google for it):
>
> When you do a file upload the cgi.rb code does this nasty regex backtracking looking for mime boundaries.
>
> In some cases it screws this up--especially with firefox--so that when it gets to the end it needs the EOF to make it stop.
>
> But, when you use a StringIO you don't get an EOF so the cgi.rb code will go into an infinite loop.
>
> Does any of that give you some clues?
>
> --
> Zed A. Shaw
> - Hate: http://savingtheinternetwithhate.com/
> - Good: http://www.zedshaw.com/
> - Evil: http://yearofevil.com/
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>


-- 
Evan Weaver
Cloudburst, LLC

From zedshaw at zedshaw.com  Thu Sep 13 06:59:10 2007
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Thu, 13 Sep 2007 06:59:10 -0400
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <b6f68fc60709130240m39899c39lce68e54127e0b274@mail.gmail.com>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<A13BB365-D5F3-48DD-9D6E-1F2B90E42A7F@nextengine.com>
	<20070912213930.c0b48a04.zedshaw@zedshaw.com>
	<b6f68fc60709130240m39899c39lce68e54127e0b274@mail.gmail.com>
Message-ID: <20070913065910.c68b1fd2.zedshaw@zedshaw.com>

On Thu, 13 Sep 2007 05:40:33 -0400
"Evan Weaver" <evan at cloudbur.st> wrote:

> Multipart_fix should work fine with 1.8.4; if it doesn't, there's a
> bug I need to look at.

Yeah, I realized that after I shot off the first response without reading the thread.  This actually might be another place where, on OSX, we get a similar hang though.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From hutch at recursive.ca  Thu Sep 13 08:32:10 2007
From: hutch at recursive.ca (Bob Hutchison)
Date: Thu, 13 Sep 2007 08:32:10 -0400
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<71166b3b0709121457m704a2a48x63a18831e1d2215e@mail.gmail.com>
	<4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>
Message-ID: <35AE334D-292B-4E32-A940-7F9665EEE1AF@recursive.ca>

Hi,

I have exactly the same problem. I'm using OS X 10.4.10, ruby 1.8.5,  
mongrel 1.0.

This has been getting *much* worse recently, it used to be  
occasional, now it seems to be pretty much every time. Maybe co- 
incident with the 10.4.10 update, maybe after.

It does not happen with Merb.

It does not happen with Rails in production mode, only development mode.

It takes a while before it happens, I don't know how long, but we're  
talking about minutes not hours.

I do not use a DB at all, if Rails tries to do something with a DB,  
and it might, it'll be with sqllite.

I have no other information beyond that.

Cheers,
Bob

----
Bob Hutchison                  -- tumblelog at http:// 
www.recursive.ca/so/
Recursive Design Inc.          -- weblog at http://www.recursive.ca/ 
hutch
http://www.recursive.ca/       -- works on http://www.raconteur.info/ 
cms-for-static-content/home/




From evan at cloudbur.st  Thu Sep 13 09:25:58 2007
From: evan at cloudbur.st (Evan Weaver)
Date: Thu, 13 Sep 2007 09:25:58 -0400
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <35AE334D-292B-4E32-A940-7F9665EEE1AF@recursive.ca>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<71166b3b0709121457m704a2a48x63a18831e1d2215e@mail.gmail.com>
	<4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>
	<35AE334D-292B-4E32-A940-7F9665EEE1AF@recursive.ca>
Message-ID: <b6f68fc60709130625n71bf841al5457eac86209891b@mail.gmail.com>

Next time it hangs, you should invade the process using Mauricio's
live inspection scripts and raise an exception. Then you'll get a
traceback and can see where it is stuck.

http://eigenclass.org/hiki.rb?ruby+live+process+introspection

The scripts are for 1.8.6 but you can still manage to write a
traceback to a file on 1.8.5 with some blind eval-ing, if I remember
correctly.

Evan

On 9/13/07, Bob Hutchison <hutch at recursive.ca> wrote:
> Hi,
>
> I have exactly the same problem. I'm using OS X 10.4.10, ruby 1.8.5,
> mongrel 1.0.
>
> This has been getting *much* worse recently, it used to be
> occasional, now it seems to be pretty much every time. Maybe co-
> incident with the 10.4.10 update, maybe after.
>
> It does not happen with Merb.
>
> It does not happen with Rails in production mode, only development mode.
>
> It takes a while before it happens, I don't know how long, but we're
> talking about minutes not hours.
>
> I do not use a DB at all, if Rails tries to do something with a DB,
> and it might, it'll be with sqllite.
>
> I have no other information beyond that.
>
> Cheers,
> Bob
>
> ----
> Bob Hutchison                  -- tumblelog at http://
> www.recursive.ca/so/
> Recursive Design Inc.          -- weblog at http://www.recursive.ca/
> hutch
> http://www.recursive.ca/       -- works on http://www.raconteur.info/
> cms-for-static-content/home/
>
>
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>


-- 
Evan Weaver
Cloudburst, LLC

From jnylund at yahoo.com  Thu Sep 13 11:47:08 2007
From: jnylund at yahoo.com (Joel Nylund)
Date: Thu, 13 Sep 2007 11:47:08 -0400
Subject: [Mongrel] streaming video with mongrel
Message-ID: <67900B3F-1BE8-43E6-BFC8-C1EAC12F8DF4@yahoo.com>

Hi, on my site I have several screen casts (using flash video flv) in  
addition to the normal rails application which is the core service,  
my setup is apache proxy/balancing to 2 mongrels.

I was thinking I should probably not be serving up streaming video  
from mongrel as it will occupy the mongrel processes for long periods  
of time right?

So I was thinking of excluding a directory from the proxy and serving  
videos up from apache.

Does anyone have an example of how to do this and or agree its a good  
idea to not serve the videos from mongrel and offload this work to  
apache?

thanks
Joel


From jgeiger at gmail.com  Thu Sep 13 12:02:15 2007
From: jgeiger at gmail.com (Joey Geiger)
Date: Thu, 13 Sep 2007 11:02:15 -0500
Subject: [Mongrel] streaming video with mongrel
In-Reply-To: <67900B3F-1BE8-43E6-BFC8-C1EAC12F8DF4@yahoo.com>
References: <67900B3F-1BE8-43E6-BFC8-C1EAC12F8DF4@yahoo.com>
Message-ID: <466af3440709130902s5bef811anc92e0118cbaf27f0@mail.gmail.com>

Well, based on your setup, and if you copied one of the examples on
the web already, you should be serving the video from apache anyway
already. If not, there are plenty of examples on the web you can
search for "apache mongrel"

The main one is
http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/
Add in checks for .flv files that exist and it should then be served by apache.

Also, you could store the files in a directory that isn't under the
mongrel proxy and have apache sent that, or use xsendfile.

On 9/13/07, Joel Nylund <jnylund at yahoo.com> wrote:
> Hi, on my site I have several screen casts (using flash video flv) in
> addition to the normal rails application which is the core service,
> my setup is apache proxy/balancing to 2 mongrels.
>
> I was thinking I should probably not be serving up streaming video
> from mongrel as it will occupy the mongrel processes for long periods
> of time right?
>
> So I was thinking of excluding a directory from the proxy and serving
> videos up from apache.
>
> Does anyone have an example of how to do this and or agree its a good
> idea to not serve the videos from mongrel and offload this work to
> apache?
>
> thanks
> Joel
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>

From pete at nextengine.com  Thu Sep 13 12:05:59 2007
From: pete at nextengine.com (Pete DeLaurentis)
Date: Thu, 13 Sep 2007 09:05:59 -0700
Subject: [Mongrel] streaming video with mongrel
In-Reply-To: <67900B3F-1BE8-43E6-BFC8-C1EAC12F8DF4@yahoo.com>
References: <67900B3F-1BE8-43E6-BFC8-C1EAC12F8DF4@yahoo.com>
Message-ID: <28C76DBA-5350-4CAF-9322-B7F3AE7ED243@nextengine.com>

You can use X-Sendfile.  Your mongrel / ruby application simply sends  
back a X-Sendfile header pointing to the local file that you want to  
download to the user.

The web-server then takes over and streams the file to the server,  
and your mongrel process gets to continue without having to wait for  
the download.

I use it with Lighttpd 1.5, and it works very well.  From what I  
hear, this is also possible with Apache.

This lets you have some security of download + still have speed.

-Pete


On Sep 13, 2007, at 8:47 AM, Joel Nylund wrote:

> Hi, on my site I have several screen casts (using flash video flv) in
> addition to the normal rails application which is the core service,
> my setup is apache proxy/balancing to 2 mongrels.
>
> I was thinking I should probably not be serving up streaming video
> from mongrel as it will occupy the mongrel processes for long periods
> of time right?
>
> So I was thinking of excluding a directory from the proxy and serving
> videos up from apache.
>
> Does anyone have an example of how to do this and or agree its a good
> idea to not serve the videos from mongrel and offload this work to
> apache?
>
> thanks
> Joel
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From dreamwords at gmail.com  Thu Sep 13 12:50:03 2007
From: dreamwords at gmail.com (Yan Meng)
Date: Fri, 14 Sep 2007 00:50:03 +0800
Subject: [Mongrel] Apache mod_proxy_balancer hang on high traffic hour
In-Reply-To: <c1f75af70709130942l6ec608base9a8fa5f34330df@mail.gmail.com>
References: <c1f75af70709130942l6ec608base9a8fa5f34330df@mail.gmail.com>
Message-ID: <c1f75af70709130950u1b6eecd1r542e563609d40026@mail.gmail.com>

more info

just 1 server

webserver + appserver + dbserver

On 9/14/07, Yan Meng <dreamwords at gmail.com> wrote:
>
> Hi, Guys
>
> Our team is running a web2.0 finance site in China
>
> http://www.caibangzi.com/
>
> We use APACHE + mod_proxy_balancer + mongrel to deploy our application, it
> workd super in the past few monthes.
>
> However, just  from last week, at high traffic hour (about 40,000 request
> per hour), our site is very very slow, it usually took more than 10 seconds
> to get a page or simply return timeout error.
>
> I telnet to the server and found that when I visit directly to one mongrel
> instance, it can return shortly, however when I visit from apache interface,
> it hang.
>
> I also cannot visit (timeout) the mod_proxy_balancer's frontend to see the
> balance-manager.
>
> However, the server load is just normal :(
>
> We guess maybe the reason is slow action block mongrel, is it the reason?
> Have any of you experienced of this?
>
> Any comments are appreciated.
>
> Our Server:
>
> Dual-Core AMD Opteron(tm) Processor 2214 HE * 2
> 4G RAM
> 20 mongrel instances
> mysql
>
> --
> Best Regards,
>
> -Yan Meng
> http://www.caibangzi.com/




-- 
Best Regards,

-Yan Meng
http://www.caibangzi.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070914/3b7e6372/attachment.html 

From hutch at recursive.ca  Thu Sep 13 15:40:45 2007
From: hutch at recursive.ca (Bob Hutchison)
Date: Thu, 13 Sep 2007 15:40:45 -0400
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <35AE334D-292B-4E32-A940-7F9665EEE1AF@recursive.ca>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<71166b3b0709121457m704a2a48x63a18831e1d2215e@mail.gmail.com>
	<4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>
	<35AE334D-292B-4E32-A940-7F9665EEE1AF@recursive.ca>
Message-ID: <A3E29E6A-AFF9-4280-B774-5735C45BF5B8@recursive.ca>

Hi,

Here is something else that might be related:

<http://giles.tumblr.com/post/11479326>

Cheers,
Bob

On 13-Sep-07, at 8:32 AM, Bob Hutchison wrote:

> Hi,
>
> I have exactly the same problem. I'm using OS X 10.4.10, ruby  
> 1.8.5, mongrel 1.0.
>
> This has been getting *much* worse recently, it used to be  
> occasional, now it seems to be pretty much every time. Maybe co- 
> incident with the 10.4.10 update, maybe after.
>
> It does not happen with Merb.
>
> It does not happen with Rails in production mode, only development  
> mode.
>
> It takes a while before it happens, I don't know how long, but  
> we're talking about minutes not hours.
>
> I do not use a DB at all, if Rails tries to do something with a DB,  
> and it might, it'll be with sqllite.
>
> I have no other information beyond that.
>
> Cheers,
> Bob
>
> ----
> Bob Hutchison                  -- tumblelog at http:// 
> www.recursive.ca/so/
> Recursive Design Inc.          -- weblog at http://www.recursive.ca/ 
> hutch
> http://www.recursive.ca/       -- works on http:// 
> www.raconteur.info/cms-for-static-content/home/
>
>
>

----
Bob Hutchison                  -- tumblelog at http:// 
www.recursive.ca/so/
Recursive Design Inc.          -- weblog at http://www.recursive.ca/ 
hutch
http://www.recursive.ca/       -- works on http://www.raconteur.info/ 
cms-for-static-content/home/




From bill at tribley.org  Thu Sep 13 18:42:04 2007
From: bill at tribley.org (William Tribley)
Date: Thu, 13 Sep 2007 17:42:04 -0500
Subject: [Mongrel] Apache mod_proxy_balancer hang on high traffic  hour
In-Reply-To: <c1f75af70709130950u1b6eecd1r542e563609d40026@mail.gmail.co
 m>
References: <c1f75af70709130942l6ec608base9a8fa5f34330df@mail.gmail.com>
	<c1f75af70709130950u1b6eecd1r542e563609d40026@mail.gmail.com>
Message-ID: <20070913224849.94D1D52408D9@rubyforge.org>

I am a neophyte at this, but if you read the documentation about 
mongrel and load balancing they strongly suggest going to more boxes 
and only running 4 or so Mongrel instances per box. If you only have 
a dual core server, it does not make much sense to jack up the RAM so 
you can stuff more processes in. Performance is all about eliminating 
latency, if you have two lowly cpus handling 20k requests per hour 
yes, I bet you're slow.

Other factors
1. Database should be clustered and running on its own box.
2. Are you properly managing logs? They would be growing at a rapid rate.
3. Any chance of going to a quad or octal CPU configuration?

Best regards,
Bill

At 11:50 AM 9/13/2007, you wrote:
>more info
>
>just 1 server
>
>webserver + appserver + dbserver
>
>On 9/14/07, Yan Meng 
><<mailto:dreamwords at gmail.com>dreamwords at gmail.com > wrote:
>Hi, Guys
>
>Our team is running a web2.0 finance site in China
>
><http://www.caibangzi.com/>http://www.caibangzi.com/
>
>We use APACHE + mod_proxy_balancer + mongrel to deploy our 
>application, it workd super in the past few monthes.
>
>However, just  from last week, at high traffic hour (about 40,000 
>request per hour), our site is very very slow, it usually took more 
>than 10 seconds to get a page or simply return timeout error.
>
>I telnet to the server and found that when I visit directly to one 
>mongrel instance, it can return shortly, however when I visit from 
>apache interface, it hang.
>
>I also cannot visit (timeout) the mod_proxy_balancer's frontend to 
>see the balance-manager.
>
>However, the server load is just normal :(
>
>We guess maybe the reason is slow action block mongrel, is it the 
>reason? Have any of you experienced of this?
>
>Any comments are appreciated.
>
>Our Server:
>
>Dual-Core AMD Opteron(tm) Processor 2214 HE * 2
>4G RAM
>20 mongrel instances
>mysql
>
>--
>Best Regards,
>
>-Yan Meng
><http://www.caibangzi.com/>http://www.caibangzi.com/
>
>
>
>
>--
>Best Regards,
>
>-Yan Meng
><http://www.caibangzi.com/>http://www.caibangzi.com/
>_______________________________________________
>Mongrel-users mailing list
>Mongrel-users at rubyforge.org
>http://rubyforge.org/mailman/listinfo/mongrel-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070913/5e2744e0/attachment.html 

From bill at tribley.org  Thu Sep 13 18:45:41 2007
From: bill at tribley.org (William Tribley)
Date: Thu, 13 Sep 2007 17:45:41 -0500
Subject: [Mongrel] Apache mod_proxy_balancer hang on high traffic  hour
In-Reply-To: <c1f75af70709130950u1b6eecd1r542e563609d40026@mail.gmail.co
 m>
References: <c1f75af70709130942l6ec608base9a8fa5f34330df@mail.gmail.com>
	<c1f75af70709130950u1b6eecd1r542e563609d40026@mail.gmail.com>
Message-ID: <20070913231224.B6A905240A9D@rubyforge.org>

Have you run system profilers to see where the bottlenecks are? You 
can do this when the system is moderately loaded, it will give you an 
idea what problem will hit when the load goes up.

At 11:50 AM 9/13/2007, you wrote:
>more info
>
>just 1 server
>
>webserver + appserver + dbserver
>
>On 9/14/07, Yan Meng 
><<mailto:dreamwords at gmail.com>dreamwords at gmail.com > wrote:
>Hi, Guys
>
>Our team is running a web2.0 finance site in China
>
><http://www.caibangzi.com/>http://www.caibangzi.com/
>
>We use APACHE + mod_proxy_balancer + mongrel to deploy our 
>application, it workd super in the past few monthes.
>
>However, just  from last week, at high traffic hour (about 40,000 
>request per hour), our site is very very slow, it usually took more 
>than 10 seconds to get a page or simply return timeout error.
>
>I telnet to the server and found that when I visit directly to one 
>mongrel instance, it can return shortly, however when I visit from 
>apache interface, it hang.
>
>I also cannot visit (timeout) the mod_proxy_balancer's frontend to 
>see the balance-manager.
>
>However, the server load is just normal :(
>
>We guess maybe the reason is slow action block mongrel, is it the 
>reason? Have any of you experienced of this?
>
>Any comments are appreciated.
>
>Our Server:
>
>Dual-Core AMD Opteron(tm) Processor 2214 HE * 2
>4G RAM
>20 mongrel instances
>mysql
>
>--
>Best Regards,
>
>-Yan Meng
><http://www.caibangzi.com/>http://www.caibangzi.com/
>
>
>
>
>--
>Best Regards,
>
>-Yan Meng
><http://www.caibangzi.com/>http://www.caibangzi.com/
>_______________________________________________
>Mongrel-users mailing list
>Mongrel-users at rubyforge.org
>http://rubyforge.org/mailman/listinfo/mongrel-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070913/99d10b92/attachment.html 

From hutch at recursive.ca  Thu Sep 13 19:55:11 2007
From: hutch at recursive.ca (Bob Hutchison)
Date: Thu, 13 Sep 2007 19:55:11 -0400
Subject: [Mongrel] Mongrel hangs, consumes all CPU on shutdown
In-Reply-To: <A6225B76-4FD2-42C5-991D-BE32F54CD9E0@mac.com>
References: <CD7DEB99-0251-4C20-A0EA-814AE5A3952F@wisc.edu>
	<71166b3b0709121457m704a2a48x63a18831e1d2215e@mail.gmail.com>
	<4059D75A-891D-4802-855B-2B5B958EA50C@wisc.edu>
	<35AE334D-292B-4E32-A940-7F9665EEE1AF@recursive.ca>
	<A3E29E6A-AFF9-4280-B774-5735C45BF5B8@recursive.ca>
	<A6225B76-4FD2-42C5-991D-BE32F54CD9E0@mac.com>
Message-ID: <ED8C3B23-317C-493F-BD6E-CC4919C7FC00@recursive.ca>

Hi,

On 13-Sep-07, at 4:18 PM, Jason Frankovitz wrote:

> Just out of curiosity, these Mac OS systems with the mongrel  
> problems - are these production systems? Or is everyone just having  
> these issues on their development machines (I use a MacBook Pro and  
> it's great)

For me it is only when the Rails environment is 'development'.

>
> I'd be surprised if a lot of folks are using Xserves and Mac Pros  
> for RoR, but stranger things have happened. I certainly wouldn't  
> mind an Xserve, myself...

I would too :-)

Cheers,
Bob

>
> Wondering,
> -Jason
>
>
> On Sep 13, 2007, at 12:40 PM, Bob Hutchison wrote:
>
>> Hi,
>>
>> Here is something else that might be related:
>>
>> <http://giles.tumblr.com/post/11479326>
>>
>> Cheers,
>> Bob
>>
>> On 13-Sep-07, at 8:32 AM, Bob Hutchison wrote:
>>
>>> Hi,
>>>
>>> I have exactly the same problem. I'm using OS X 10.4.10, ruby
>>> 1.8.5, mongrel 1.0.
>>>
>>> This has been getting *much* worse recently, it used to be
>>> occasional, now it seems to be pretty much every time. Maybe co-
>>> incident with the 10.4.10 update, maybe after.
>>>
>>> It does not happen with Merb.
>>>
>>> It does not happen with Rails in production mode, only development
>>> mode.
>>>
>>> It takes a while before it happens, I don't know how long, but
>>> we're talking about minutes not hours.
>>>
>>> I do not use a DB at all, if Rails tries to do something with a DB,
>>> and it might, it'll be with sqllite.
>>>
>>> I have no other information beyond that.
>>>
>>> Cheers,
>>> Bob
>>>
>>> ----
>>> Bob Hutchison                  -- tumblelog at http://
>>> www.recursive.ca/so/
>>> Recursive Design Inc.          -- weblog at http://www.recursive.ca/
>>> hutch
>>> http://www.recursive.ca/       -- works on http://
>>> www.raconteur.info/cms-for-static-content/home/
>>>
>>>
>>>
>>
>> ----
>> Bob Hutchison                  -- tumblelog at http://
>> www.recursive.ca/so/
>> Recursive Design Inc.          -- weblog at http://www.recursive.ca/
>> hutch
>> http://www.recursive.ca/       -- works on http://www.raconteur.info/
>> cms-for-static-content/home/
>>
>>
>>
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>

----
Bob Hutchison                  -- tumblelog at http:// 
www.recursive.ca/so/
Recursive Design Inc.          -- weblog at http://www.recursive.ca/ 
hutch
http://www.recursive.ca/       -- works on http://www.raconteur.info/ 
cms-for-static-content/home/




From neongrau at gmail.com  Fri Sep 14 11:05:27 2007
From: neongrau at gmail.com (Ralf Vitasek)
Date: Fri, 14 Sep 2007 17:05:27 +0200
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>
References: <46E65438.5040307@gmail.com>
	<71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>
Message-ID: <7adba8e60709140805o806f5cex6d93924cb579e672@mail.gmail.com>

hello luis!

in the meantime i've done some more tests and i'm pretty sure the
problem has nothing to do with win32ole.

i switched to ODBC mode and observed the same behavior as with the
win32ole driven ADO driver.

and  by creating a new plain rails app that was using a mysql 5
database the problem showed up the same. meaning as soon as i copy
back one of the previous ruby builds the service will start up fine.

sadly the only message i get after "net start testrails" is to call
NET HELPMSG 3534
for more details :(

have you yet been able to test one of the latest builds?
or is there anything else you want me to check or provide infos?

cheers,
ralf

From jan.svitok at gmail.com  Fri Sep 14 11:34:34 2007
From: jan.svitok at gmail.com (Jano Svitok)
Date: Fri, 14 Sep 2007 17:34:34 +0200
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <7adba8e60709140805o806f5cex6d93924cb579e672@mail.gmail.com>
References: <46E65438.5040307@gmail.com>
	<71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>
	<7adba8e60709140805o806f5cex6d93924cb579e672@mail.gmail.com>
Message-ID: <8d9b3d920709140834v12751c3dyf08113dc624aa9c3@mail.gmail.com>

On 9/14/07, Ralf Vitasek <neongrau at gmail.com> wrote:
> hello luis!
>
> in the meantime i've done some more tests and i'm pretty sure the
> problem has nothing to do with win32ole.
>
> i switched to ODBC mode and observed the same behavior as with the
> win32ole driven ADO driver.
>
> and  by creating a new plain rails app that was using a mysql 5
> database the problem showed up the same. meaning as soon as i copy
> back one of the previous ruby builds the service will start up fine.

I don't remember the whole story here, just offering few tips:

1. add some tracing to mongrel.rb (and possibly to mongrel.bat/cmd,
depending on what's called when staring the service)

2. use ProcExp to see if ruby.exe starts at all.

3. use DbgView to see if there are any debug messages that could help

4. use full path wherever possible, don't rely on proper PATH.

J.

From luislavena at gmail.com  Fri Sep 14 17:00:24 2007
From: luislavena at gmail.com (Luis Lavena)
Date: Fri, 14 Sep 2007 18:00:24 -0300
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <7adba8e60709140805o806f5cex6d93924cb579e672@mail.gmail.com>
References: <46E65438.5040307@gmail.com>
	<71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>
	<7adba8e60709140805o806f5cex6d93924cb579e672@mail.gmail.com>
Message-ID: <71166b3b0709141400p34afbf2fla85a2f456c568b41@mail.gmail.com>

On 9/14/07, Ralf Vitasek <neongrau at gmail.com> wrote:
> hello luis!
>
> in the meantime i've done some more tests and i'm pretty sure the
> problem has nothing to do with win32ole.
>

Good to know, this narrow the problem a bit...

> i switched to ODBC mode and observed the same behavior as with the
> win32ole driven ADO driver.
>
> and  by creating a new plain rails app that was using a mysql 5
> database the problem showed up the same. meaning as soon as i copy
> back one of the previous ruby builds the service will start up fine.
>

If I remember correctly, some versions of ruby works, some others don't....

> sadly the only message i get after "net start testrails" is to call
> NET HELPMSG 3534
> for more details :(
>

Yeah, it's windows, the less informative OS.

> have you yet been able to test one of the latest builds?
> or is there anything else you want me to check or provide infos?
>

With all the info you gather will be able to test it tonight, since I
was overloaded with things outside the office the past days.

-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

From rogerpack2005 at gmail.com  Fri Sep 14 20:04:16 2007
From: rogerpack2005 at gmail.com (Roger Pack)
Date: Fri, 14 Sep 2007 18:04:16 -0600
Subject: [Mongrel] http keep-alive?
Message-ID: <966599840709141704o2652cbbco52414ba49dfd79bd@mail.gmail.com>

I read this in a previous post (
http://rubyforge.org/pipermail/mongrel-users/2006-December/002354.html)
....
First, Mongrel accepts remote clients and creates one Thread for each
request.  Mongrel also enforces a single request/response using
Connect:close headers because Ruby only supports 1024 files (so far).  If
Mongrel doesn't do this then people like yourself can write a simple
"trickle attack" client that hits the Mongrel server, opens a bunch of
continuous connections, and then eat up all available files very quickly.
Basically, a DDoS attack that's very simple to do.
....


Is this still a problem?  If it is, I think it might be sweet if it were
optional (then load balancer's could keep open connections--if only load
balancers can hit it...).  Just a thought :)

-Roger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070914/86b8533c/attachment.html 

From wyhaines at gmail.com  Fri Sep 14 20:16:34 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Fri, 14 Sep 2007 17:16:34 -0700
Subject: [Mongrel] http keep-alive?
In-Reply-To: <966599840709141704o2652cbbco52414ba49dfd79bd@mail.gmail.com>
References: <966599840709141704o2652cbbco52414ba49dfd79bd@mail.gmail.com>
Message-ID: <f4cd26df0709141716p312ddaeaua09f00233b9365d@mail.gmail.com>

On 9/14/07, Roger Pack <rogerpack2005 at gmail.com> wrote:
> I read this in a previous post
> (http://rubyforge.org/pipermail/mongrel-users/2006-December/002354.html)
> ....
> First, Mongrel accepts remote clients and creates one Thread for each
> request.  Mongrel also enforces a single request/response using
> Connect:close headers because Ruby only supports 1024 files (so far).  If
> Mongrel doesn't do this then people like yourself can write a simple
> "trickle attack" client that hits the Mongrel server, opens a bunch of
> continuous connections, and then eat up all available files very quickly.
> Basically, a DDoS attack that's very simple to do.
> ....
>
>
> Is this still a problem?  If it is, I think it might be sweet if it were
> optional (then load balancer's could keep open connections--if only load
> balancers can hit it...).  Just a thought :)

It's still possible, and probably will remain so for quite a while.
Ruby uses a select() loop to manage it's threads.  It's fd_setsize is
1024.  select()'s performance also degrades as the count of handles it
is managing goes up.

With the next version of evented_mongrel I am going to provide a way
for people to specify, if they are on a platform that supports epoll
(Linux 2.6.x), the max number of connections that they want to be able
to handle.  This would, in theory, reduce the threat of the trickle
attack because an evented_mongrel could have many more than 1024
concurrent connections without any problem.

If you read the archives, the subject of keep-alive is somewhat
controversial, though we (the current admin/developer crew on mongrel)
have discussed it at least once.  I think it is something we are
willing to explore further, if I recall the discussion correctly.


Kirk Haines

From luislavena at gmail.com  Fri Sep 14 23:52:20 2007
From: luislavena at gmail.com (Luis Lavena)
Date: Sat, 15 Sep 2007 00:52:20 -0300
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <7adba8e60709140805o806f5cex6d93924cb579e672@mail.gmail.com>
References: <46E65438.5040307@gmail.com>
	<71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>
	<7adba8e60709140805o806f5cex6d93924cb579e672@mail.gmail.com>
Message-ID: <71166b3b0709142052r5424d484t3e7b987b026cb60b@mail.gmail.com>

On 9/14/07, Ralf Vitasek <neongrau at gmail.com> wrote:
> hello luis!
>
> in the meantime i've done some more tests and i'm pretty sure the
> problem has nothing to do with win32ole.
>

You're Right!!!!

It seems that running in console mode also fails (for the versions you
listed with problems).

I'm hooking a simple logger into mongrel_service to trace the problem
inside ruby (and see if it reports something).

I don't know what was "fixed" in the latest patchlevel release of
1.8.5, but it seems something "broke" :-P

-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

From luislavena at gmail.com  Sat Sep 15 00:51:25 2007
From: luislavena at gmail.com (Luis Lavena)
Date: Sat, 15 Sep 2007 01:51:25 -0300
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <71166b3b0709142052r5424d484t3e7b987b026cb60b@mail.gmail.com>
References: <46E65438.5040307@gmail.com>
	<71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>
	<7adba8e60709140805o806f5cex6d93924cb579e672@mail.gmail.com>
	<71166b3b0709142052r5424d484t3e7b987b026cb60b@mail.gmail.com>
Message-ID: <71166b3b0709142151i6dded259ga246474ad43ba4d1@mail.gmail.com>

On 9/15/07, Luis Lavena <luislavena at gmail.com> wrote:
> On 9/14/07, Ralf Vitasek <neongrau at gmail.com> wrote:
> > hello luis!
> >
> > in the meantime i've done some more tests and i'm pretty sure the
> > problem has nothing to do with win32ole.
> >
>
> You're Right!!!!
>
> It seems that running in console mode also fails (for the versions you
> listed with problems).
>
> I'm hooking a simple logger into mongrel_service to trace the problem
> inside ruby (and see if it reports something).
>
> I don't know what was "fixed" in the latest patchlevel release of
> 1.8.5, but it seems something "broke" :-P
>

Ok, let just try something:

Please, download the following file:

http://mmediasys.com/mongrel/mongrel_service-0.3.3-mswin32.gem

Before install it, please remove the 0.3.2 of mongrel_service gem
(Also, manually remove the mongrel_service.exe and the related log
files from ruby\bin folder).

Install it, and reinstall the service (or test with a dummy/empty
rails application).
NOTE: You need to install a new service for the updated
mongrel_service executable get copied in ruby\bin, without it, nothing
will work.

Now, try starting your service with net start and also in console mode.

I've tested with patchlevel 52 and 12 of 1.8.5, with both versions works.

This experimental version includes back ported code from upcoming
project. If this works for you, I'll commit the changes to the SVN and
release the updated version.

Also, check C:\Ruby\bin\services.log

This is still to rough, but there is logged the STDOUT/STDERR pipes for Mongrel.
(Should be inside your application log folder, but didn't have the
time to back port my OptionParser to mongrel_service).

Let me know any problems (and hope success too!).

Regards,

-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

From luislavena at gmail.com  Sat Sep 15 00:56:52 2007
From: luislavena at gmail.com (Luis Lavena)
Date: Sat, 15 Sep 2007 01:56:52 -0300
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <71166b3b0709142151i6dded259ga246474ad43ba4d1@mail.gmail.com>
References: <46E65438.5040307@gmail.com>
	<71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>
	<7adba8e60709140805o806f5cex6d93924cb579e672@mail.gmail.com>
	<71166b3b0709142052r5424d484t3e7b987b026cb60b@mail.gmail.com>
	<71166b3b0709142151i6dded259ga246474ad43ba4d1@mail.gmail.com>
Message-ID: <71166b3b0709142156t70222e6dk54f9415d03f42ab8@mail.gmail.com>

On 9/15/07, Luis Lavena <luislavena at gmail.com> wrote:
>
> http://mmediasys.com/mongrel/mongrel_service-0.3.3-mswin32.gem
>

Damn, my host didn't serve the file as binary... so, please download this:

http://mmediasys.com/mongrel/mongrel_service-0.3.3-mswin32.gem.bin

the .bin is just a dummy extension, rename it to just .gem and will work ;-)

Regards and good weekend everybody.

-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

From neongrau at gmail.com  Sat Sep 15 06:08:42 2007
From: neongrau at gmail.com (Ralf Vitasek)
Date: Sat, 15 Sep 2007 12:08:42 +0200
Subject: [Mongrel] mongrel_rails service on windows incompatible with
	recent ruby builds?
In-Reply-To: <71166b3b0709142156t70222e6dk54f9415d03f42ab8@mail.gmail.com>
References: <46E65438.5040307@gmail.com>
	<71166b3b0709111428v43ad1540m8d89f17b6b4e27bf@mail.gmail.com>
	<7adba8e60709140805o806f5cex6d93924cb579e672@mail.gmail.com>
	<71166b3b0709142052r5424d484t3e7b987b026cb60b@mail.gmail.com>
	<71166b3b0709142151i6dded259ga246474ad43ba4d1@mail.gmail.com>
	<71166b3b0709142156t70222e6dk54f9415d03f42ab8@mail.gmail.com>
Message-ID: <7adba8e60709150308g614884cid5f7e9d3d43e3419@mail.gmail.com>

GREAT JOB!

it works! :)
thank you so much!!!

cheers,
ralf

On 9/15/07, Luis Lavena <luislavena at gmail.com> wrote:
> On 9/15/07, Luis Lavena <luislavena at gmail.com> wrote:
> >
> > http://mmediasys.com/mongrel/mongrel_service-0.3.3-mswin32.gem
> >
>
> Damn, my host didn't serve the file as binary... so, please download this:
>
> http://mmediasys.com/mongrel/mongrel_service-0.3.3-mswin32.gem.bin
>
> the .bin is just a dummy extension, rename it to just .gem and will work ;-)
>
> Regards and good weekend everybody.
>
> --
> Luis Lavena
> Multimedia systems
> -
> Leaders are made, they are not born. They are made by hard effort,
> which is the price which all of us must pay to achieve any goal that
> is worthwhile.
> Vince Lombardi
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>

From benkruger at gmail.com  Sat Sep 15 09:05:06 2007
From: benkruger at gmail.com (Ben Kruger)
Date: Sat, 15 Sep 2007 06:05:06 -0700
Subject: [Mongrel] Apache mod_proxy_balancer hang on high traffic hour
Message-ID: <7eab5da20709150605j2e7419b6mf4e8f3bfa303d0bc@mail.gmail.com>

We are having a similar problem.  For the last 6 months everything has been
fine.  The last couple of weeks as our pageviews continue to rise we are now
facing an unresponsive Mongrel intermittently.

We are running on BSD 6.2 with 4 Mongrels in the cluster.

Comments are appreciated as well!

Thank you,

Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070915/26a77799/attachment.html 

From jason at joyent.com  Sat Sep 15 09:33:58 2007
From: jason at joyent.com (Jason A. Hoffman)
Date: Sat, 15 Sep 2007 06:33:58 -0700
Subject: [Mongrel] Apache mod_proxy_balancer hang on high traffic hour
In-Reply-To: <7eab5da20709150605j2e7419b6mf4e8f3bfa303d0bc@mail.gmail.com>
References: <7eab5da20709150605j2e7419b6mf4e8f3bfa303d0bc@mail.gmail.com>
Message-ID: <44840215-94EF-4BF5-91F1-38DD87E20739@joyent.com>

Apache's mod_proxy_balancer module is a fully blocking module and with  
the default httpd.conf you're going to max out in the 120-160 requests/ 
second range on a decent box. You can tune up its proxying to about a  
1000 req/sec

So yes the net result is that you can really only put a couple of  
mongrels behind apache's proxy engine (about 2   "hello world" rails  
mongrels).

If you can hit mongrels directly and all is fine, but it's clogged up  
via apache ( and you're doing in the hundreds of request/sec), than  
this is the botteneck that you've hit. This at least explains the  
original poster's problem.

-Jason

On Sep 15, 2007, at 6:05 AM, "Ben Kruger" <benkruger at gmail.com> wrote:

> We are having a similar problem.  For the last 6 months everything  
> has been fine.  The last couple of weeks as our pageviews continue  
> to rise we are now facing an unresponsive Mongrel intermittently.
>
> We are running on BSD 6.2 with 4 Mongrels in the cluster.
>
> Comments are appreciated as well!
>
> Thank you,
>
> Ben
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users

From jason at joyent.com  Sat Sep 15 09:33:58 2007
From: jason at joyent.com (Jason A. Hoffman)
Date: Sat, 15 Sep 2007 06:33:58 -0700
Subject: [Mongrel] Apache mod_proxy_balancer hang on high traffic hour
In-Reply-To: <7eab5da20709150605j2e7419b6mf4e8f3bfa303d0bc@mail.gmail.com>
References: <7eab5da20709150605j2e7419b6mf4e8f3bfa303d0bc@mail.gmail.com>
Message-ID: <44840215-94EF-4BF5-91F1-38DD87E20739@joyent.com>

Apache's mod_proxy_balancer module is a fully blocking module and with  
the default httpd.conf you're going to max out in the 120-160 requests/ 
second range on a decent box. You can tune up its proxying to about a  
1000 req/sec

So yes the net result is that you can really only put a couple of  
mongrels behind apache's proxy engine (about 2   "hello world" rails  
mongrels).

If you can hit mongrels directly and all is fine, but it's clogged up  
via apache ( and you're doing in the hundreds of request/sec), than  
this is the botteneck that you've hit. This at least explains the  
original poster's problem.

-Jason

On Sep 15, 2007, at 6:05 AM, "Ben Kruger" <benkruger at gmail.com> wrote:

> We are having a similar problem.  For the last 6 months everything  
> has been fine.  The last couple of weeks as our pageviews continue  
> to rise we are now facing an unresponsive Mongrel intermittently.
>
> We are running on BSD 6.2 with 4 Mongrels in the cluster.
>
> Comments are appreciated as well!
>
> Thank you,
>
> Ben
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users

From wyhaines at gmail.com  Sat Sep 15 17:54:55 2007
From: wyhaines at gmail.com (Kirk Haines)
Date: Sat, 15 Sep 2007 14:54:55 -0700
Subject: [Mongrel] Apache mod_proxy_balancer hang on high traffic hour
In-Reply-To: <c1f75af70709130950u1b6eecd1r542e563609d40026@mail.gmail.com>
References: <c1f75af70709130942l6ec608base9a8fa5f34330df@mail.gmail.com>
	<c1f75af70709130950u1b6eecd1r542e563609d40026@mail.gmail.com>
Message-ID: <f4cd26df0709151454u7ad1263fg134469f177629fec@mail.gmail.com>

On 9/13/07, Yan Meng <dreamwords at gmail.com> wrote:
> more info
>
> just 1 server
>
> webserver + appserver + dbserver
>
> On 9/14/07, Yan Meng <dreamwords at gmail.com > wrote:

> > We use APACHE + mod_proxy_balancer + mongrel to deploy our application, it
> workd super in the past few monthes.
> >
> > However, just  from last week, at high traffic hour (about 40,000 request
> per hour), our site is very very slow, it usually took more than 10 seconds
> to get a page or simply return timeout error.
> >
> > I telnet to the server and found that when I visit directly to one mongrel
> instance, it can return shortly, however when I visit from apache interface,
> it hang.
> >
> > I also cannot visit (timeout) the mod_proxy_balancer's frontend to see the
> balance-manager.

I've been letting this one bounce around in my head for a while.

I think that sentence there, right at the end of what I quoted, is an
important clue.

If you can't get to the mod_proxy_balancer's front end, then Apache is
failing to field the request in the first place.  Start there.  Look
at your Apache config.  Look at your Apache.  Are you using an Apache
that has things built into it, or loaded, which are not needed?  If
so, rebuild without those items, or comment them out of your
configuration.  Look at your allowances for keep alive, and for
MaxClients and related load handling config items.

After you look at that, then come back and look at mongrel and your
application.  Do you know how fast your app actions are?  Do you have
some actions that are very slow?  Are you facing a situation where
some bottleneck is preventing your one server from handling requests
as fast as they are coming in?

40k requests/hour on your single server may or may not be a problem.
It all depends on the application.  That's only about 11
requests/second, which isn't really a lot.

That's about all that can be said without more information and analysis.


Kirk Haines

From _ at whats-your.name  Sun Sep 16 18:33:40 2007
From: _ at whats-your.name (cdr)
Date: Sun, 16 Sep 2007 18:33:40 -0400
Subject: [Mongrel] MIME case-sensitivity
Message-ID: <20070916223340.GB26664@replic.net>

ive got code in my app which adds mimes for .JPG, .PNG, that id like to remove

what do people think about forcing a toLower() on the extname in Mongrel::DirHandler?

From laurent at marzu.org  Mon Sep 17 08:32:43 2007
From: laurent at marzu.org (Laurent Marzullo)
Date: Mon, 17 Sep 2007 14:32:43 +0200
Subject: [Mongrel] mongrel_rails hang and does not respond
Message-ID: <1190032363.46ee73eb94428@ssl0.ovh.net>

Hello,

I've install ror/mongrel etc, etc on a linux server (rhes).

As i'm not root, all package startinf from ruby have been install into

~/gconf/usr

ruby-1.8.6
and all of this gem package:
-r-xr--r--    1 dimnce   dba        168448 Aug  9 14:53 rails-1.2.3.gem
-r-xr--r--    1 dimnce   dba         84480 Aug  9 14:56 rake-0.7.3.gem
-r-xr--r--    1 dimnce   dba        217088 Aug  9 14:58 activesupport-1.4.2.gem
-r-xr--r--    1 dimnce   dba        406016 Aug  9 15:04 activerecord-1.15.3.gem
-r-xr--r--    1 dimnce   dba        607232 Aug  9 15:09 actionpack-1.13.3.gem
-r-xr--r--    1 dimnce   dba         67584 Aug  9 15:13
actionwebservice-1.2.3.gem
-r-xr--r--    1 dimnce   dba         84480 Aug  9 15:13 actionmailer-1.3.3.gem
-rwx------    1 dimnce   dba         84480 Sep 14 14:16 capistrano-2.0.0.gem
-rwx------    1 dimnce   dba         19968 Sep 14 14:16
mongrel_cluster-1.0.2.gem
-rwx------    1 dimnce   dba        159232 Sep 14 14:16 mongrel-1.0.1.gem
-rwx------    1 dimnce   dba        150016 Sep 14 14:18 net-ssh-1.0.10.gem
-rwx------    1 dimnce   dba        649728 Sep 14 14:19 needle-1.3.0.gem
-rwx------    1 dimnce   dba         43520 Sep 14 14:20 net-sftp-1.1.0.gem
-rwx------    1 dimnce   dba         54272 Sep 14 14:21 highline-1.4.0.gem
-rwx------    1 dimnce   dba         41984 Sep 14 14:22 gem_plugin-0.2.2.gem
-rwx------    1 dimnce   dba         30720 Sep 14 14:24 daemons-1.0.7.gem
-rwx------    1 dimnce   dba         20992 Sep 14 14:25 fastthread-1.0.gem
-rwx------    1 dimnce   dba          8192 Sep 14 14:26
cgi_multipart_eof_fix-2.3.gem


I've then compiled and installed httpd-2.2.6. And try to use mongrel with
apache.


In fact, all is running correctly until I start mongrel_rails with the '-d'
option.

if I start mongrel with the following command:
mongrel_rails start -e production -p 3210

my application is running correctly, but if I start mongrel with
mongrel_rails start -e production -p 3210 -d

(note the -d option), my application does not respond to any request.

I've tried to use mongrel directly (without requesting apache) and I've the same
behavior.

Do you have any hint why -d does not run correctly ?

Thanks for your help
Regards.



+--------------------+
+ Laurent Marzullo
+

From rogerpack2005 at gmail.com  Mon Sep 17 11:41:02 2007
From: rogerpack2005 at gmail.com (Roger Pack)
Date: Mon, 17 Sep 2007 09:41:02 -0600
Subject: [Mongrel] Apache mod_proxy_balancer hang on high traffic
Message-ID: <966599840709170841m7baff546r522b87af67f91e87@mail.gmail.com>

Try nginx + evented mongrel instead of apache + normal Mongrel.  Maybe :)
-Roger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070917/c6f6f43a/attachment.html 

From barjunk at attglobal.net  Mon Sep 17 12:37:02 2007
From: barjunk at attglobal.net (barsalou)
Date: Mon, 17 Sep 2007 08:37:02 -0800
Subject: [Mongrel] mongrel_rails hang and does not respond
In-Reply-To: <1190032363.46ee73eb94428@ssl0.ovh.net>
References: <1190032363.46ee73eb94428@ssl0.ovh.net>
Message-ID: <20070917083702.zdfmg6zulsscc00g@lcgalaska.com>

Check to make sure the at least the process started.  Also, do you logs 
have anything to offer?

Mike B.

Quoting Laurent Marzullo <laurent at marzu.org>:

> Hello,
>
> I've install ror/mongrel etc, etc on a linux server (rhes).
>
> As i'm not root, all package startinf from ruby have been install into
>
> ~/gconf/usr
>
> ruby-1.8.6
> and all of this gem package:
> -r-xr--r--    1 dimnce   dba        168448 Aug  9 14:53 rails-1.2.3.gem
> -r-xr--r--    1 dimnce   dba         84480 Aug  9 14:56 rake-0.7.3.gem
> -r-xr--r--    1 dimnce   dba        217088 Aug  9 14:58 
> activesupport-1.4.2.gem
> -r-xr--r--    1 dimnce   dba        406016 Aug  9 15:04 
> activerecord-1.15.3.gem
> -r-xr--r--    1 dimnce   dba        607232 Aug  9 15:09 actionpack-1.13.3.gem
> -r-xr--r--    1 dimnce   dba         67584 Aug  9 15:13
> actionwebservice-1.2.3.gem
> -r-xr--r--    1 dimnce   dba         84480 Aug  9 15:13 
> actionmailer-1.3.3.gem
> -rwx------    1 dimnce   dba         84480 Sep 14 14:16 capistrano-2.0.0.gem
> -rwx------    1 dimnce   dba         19968 Sep 14 14:16
> mongrel_cluster-1.0.2.gem
> -rwx------    1 dimnce   dba        159232 Sep 14 14:16 mongrel-1.0.1.gem
> -rwx------    1 dimnce   dba        150016 Sep 14 14:18 net-ssh-1.0.10.gem
> -rwx------    1 dimnce   dba        649728 Sep 14 14:19 needle-1.3.0.gem
> -rwx------    1 dimnce   dba         43520 Sep 14 14:20 net-sftp-1.1.0.gem
> -rwx------    1 dimnce   dba         54272 Sep 14 14:21 highline-1.4.0.gem
> -rwx------    1 dimnce   dba         41984 Sep 14 14:22 gem_plugin-0.2.2.gem
> -rwx------    1 dimnce   dba         30720 Sep 14 14:24 daemons-1.0.7.gem
> -rwx------    1 dimnce   dba         20992 Sep 14 14:25 fastthread-1.0.gem
> -rwx------    1 dimnce   dba          8192 Sep 14 14:26
> cgi_multipart_eof_fix-2.3.gem
>
>
> I've then compiled and installed httpd-2.2.6. And try to use mongrel with
> apache.
>
>
> In fact, all is running correctly until I start mongrel_rails with the '-d'
> option.
>
> if I start mongrel with the following command:
> mongrel_rails start -e production -p 3210
>
> my application is running correctly, but if I start mongrel with
> mongrel_rails start -e production -p 3210 -d
>
> (note the -d option), my application does not respond to any request.
>
> I've tried to use mongrel directly (without requesting apache) and 
> I've the same
> behavior.
>
> Do you have any hint why -d does not run correctly ?
>
> Thanks for your help
> Regards.
>
>
>
> +--------------------+
> + Laurent Marzullo
> +
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>



----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.


From rogerpack2005 at gmail.com  Mon Sep 17 14:40:54 2007
From: rogerpack2005 at gmail.com (Roger Pack)
Date: Mon, 17 Sep 2007 12:40:54 -0600
Subject: [Mongrel] http keep-alive?
Message-ID: <966599840709171140m22af8b4fxa2ad9b5231dc5dce@mail.gmail.com>

I see--so if I understand correctly, proxies (i.e. nginx) pass 'each
request' straight back to mongrel (a new TCP port), which would enable
malicious users to create lots of keep-alive requests with mongrel itself,
hence the fear of enabling keep-alive's, and mongrel currently limiting the
number of simultaneous requests.
Enabling this feature would only be useful, then, to a proxy which 'shares'
requests to mongrel from several clients.  Hmm. I'm not sure if any proxies
do that (besides swiftiply).  I wonder then if the gain to said type of
proxies would be worth making it an option (or rather, I wonder if the gain
would actually be small so not worth trying for).  Then again a few ms here
and there, right? ha ha.
Thank you!
-Roger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070917/e233f110/attachment.html 

From ferna084 at umn.edu  Mon Sep 17 15:08:15 2007
From: ferna084 at umn.edu (Jose Fernandez)
Date: Mon, 17 Sep 2007 14:08:15 -0500
Subject: [Mongrel] Mongrel clusters are not being started on reboot
Message-ID: <49C3037A-64A3-4550-94DC-6CB482DAD90B@umn.edu>

I've done a clean install of Ubuntu 7.04 server in 2 different  
machines and everytime I reboot the machine or manually run /etc/ 
init.d/mongrel_cluster restart/start the mongrels won't start correctly.

I have to manually start them by using mongrel_rails cluster::start  
from each app folder.   I've been searching around and it looks like  
mongrel has some problems with pids, so whats the final word on this?

- Jose

From barjunk at attglobal.net  Mon Sep 17 15:59:52 2007
From: barjunk at attglobal.net (barsalou)
Date: Mon, 17 Sep 2007 11:59:52 -0800
Subject: [Mongrel] Mongrel clusters are not being started on reboot
In-Reply-To: <49C3037A-64A3-4550-94DC-6CB482DAD90B@umn.edu>
References: <49C3037A-64A3-4550-94DC-6CB482DAD90B@umn.edu>
Message-ID: <20070917115952.ts6yxs84o48cgk8s@lcgalaska.com>

I've found I had to use a script to change to the app directory before 
using the mongrel_rails cluster::start command.

I think there are some patches that you can apply to fix this problem.

Mike B.

Quoting Jose Fernandez <ferna084 at umn.edu>:

> I've done a clean install of Ubuntu 7.04 server in 2 different
> machines and everytime I reboot the machine or manually run /etc/
> init.d/mongrel_cluster restart/start the mongrels won't start correctly.
>
> I have to manually start them by using mongrel_rails cluster::start
> from each app folder.   I've been searching around and it looks like
> mongrel has some problems with pids, so whats the final word on this?
>
> - Jose
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>



----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.


From laurent at marzu.org  Tue Sep 18 07:29:28 2007
From: laurent at marzu.org (Laurent Marzullo)
Date: Tue, 18 Sep 2007 13:29:28 +0200
Subject: [Mongrel] mongrel_rails hang and does not respond
In-Reply-To: <20070917083702.zdfmg6zulsscc00g@lcgalaska.com>
References: <1190032363.46ee73eb94428@ssl0.ovh.net>
	<20070917083702.zdfmg6zulsscc00g@lcgalaska.com>
Message-ID: <1190114968.46efb698c2c25@ssl0.ovh.net>

Hello,

Yes, my process is running:

-bash-2.05b$ mongrel_rails start -e development -p 3210 -l log/mongrel.3210.log
-P tmp/pids/mongrel.3210.pid -B  -d

-bash-2.05b$ /sbin/fuser -n tcp 3210
3210/tcp:            13746

-bash-2.05b$ ps -fp 13746
UID        PID  PPID  C STIME TTY          TIME CMD
dimnce   13746     1  5 12:51 ?        00:00:01
/exec/products/dimensions/9.1/gconf/usr/bin/ruby
/exec/products/dimensions/9.1/gconf/usr/bin/mongrel_rails start -e

-bash-2.05b$

So after waiting approximatively 20minutes, here what my browser print:
--------------------------------------------------------------------------
500 Server Error
The following error occurred: [code=RESOURCE_RECLAIMED] Internal error. Try
again.
Could not open error file
--------------------------------------------------------------------------


And here the different log:
----------- log/mongrel.3210.log ----------------
** Starting Mongrel listening at 0.0.0.0:3210
** Installing debugging prefixed filters. Look in log/mongrel_debug for the
files.
** Starting Rails with development environment...
** Rails loaded.
** Loading any Rails specific GemPlugins
** Signals ready.  TERM => stop.  USR2 => restart.  INT => stop (no restart).
** Rails signals registered.  HUP => reload (without restart).  It might not
work well.
** Mongrel available at 0.0.0.0:3210
** Writing PID file to tmp/pids/mongrel.3210.pid
10.173.13.184 - [Tue, 18 Sep 2007 10:53:37 GMT] "GET /delivery/deliver HTTP/1.1"
You have tried to set a null level to root.


----------- log/mongrel_debug/rails.log ----------------
Tue Sep 18 12:53:38 +0200 2007 REQUEST /delivery/deliver
--- !map:Mongrel::HttpParams
SERVER_NAME: 10.196.129.124
PATH_INFO: /delivery/deliver
HTTP_VIA: 1.1 rsc-ras-cache (NetCache NetApp/6.0.1P1D2)
HTTP_ACCEPT_ENCODING: gzip,deflate
HTTP_USER_AGENT: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.6)
Gecko/20070725 Firefox/2.0.0.6
SCRIPT_NAME: /
SERVER_PROTOCOL: HTTP/1.1
HTTP_ACCEPT_LANGUAGE: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
HTTP_HOST: 10.196.129.124:3210
REMOTE_ADDR: 10.173.13.184
SERVER_SOFTWARE: Mongrel 1.0.1
REQUEST_PATH: /delivery/deliver
HTTP_ACCEPT_CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.7
HTTP_COOKIE: _gconf_session_id=77924946c0ef0f5bdfbe7605487ce5e1;
_session_id=81423d670d8000adb260eb809bc55a26
HTTP_VERSION: HTTP/1.1
REQUEST_URI: /delivery/deliver
SERVER_PORT: "3210"
GATEWAY_INTERFACE: CGI/1.2
HTTP_X_FORWARDED_FOR: 10.173.13.184
HTTP_ACCEPT:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
HTTP_CONNECTION: keep-alive
REQUEST_METHOD: GET


----------- log/mongrel_debug/files.log ----------------
Tue Sep 18 12:53:37 +0200 2007 FILES OPEN BEFORE REQUEST /delivery/deliver
---
/exec/products/dimensions/9.1/gconf/work/devel/gconf/config/../log/development.log:
1
log/mongrel_debug/objects.log: 2
tmp/pids/mongrel.3210.pid: 1
log/mongrel_debug/rails.log: 1
log/mongrel_debug/files.log: 1
log/mongrel_debug/threads.log: 1
log/mongrel_debug/access.log: 1


Any idea, hint, help ?

Thanks all



Selon barsalou <barjunk at attglobal.net>:

> Check to make sure the at least the process started.  Also, do you logs
> have anything to offer?
>
> Mike B.
>
> Quoting Laurent Marzullo <laurent at marzu.org>:
>
> > Hello,
> >
> > I've install ror/mongrel etc, etc on a linux server (rhes).
> >
> > As i'm not root, all package startinf from ruby have been install into
> >
> > ~/gconf/usr
> >
> > ruby-1.8.6
> > and all of this gem package:
> > -r-xr--r--    1 dimnce   dba        168448 Aug  9 14:53 rails-1.2.3.gem
> > -r-xr--r--    1 dimnce   dba         84480 Aug  9 14:56 rake-0.7.3.gem
> > -r-xr--r--    1 dimnce   dba        217088 Aug  9 14:58
> > activesupport-1.4.2.gem
> > -r-xr--r--    1 dimnce   dba        406016 Aug  9 15:04
> > activerecord-1.15.3.gem
> > -r-xr--r--    1 dimnce   dba        607232 Aug  9 15:09
> actionpack-1.13.3.gem
> > -r-xr--r--    1 dimnce   dba         67584 Aug  9 15:13
> > actionwebservice-1.2.3.gem
> > -r-xr--r--    1 dimnce   dba         84480 Aug  9 15:13
> > actionmailer-1.3.3.gem
> > -rwx------    1 dimnce   dba         84480 Sep 14 14:16
> capistrano-2.0.0.gem
> > -rwx------    1 dimnce   dba         19968 Sep 14 14:16
> > mongrel_cluster-1.0.2.gem
> > -rwx------    1 dimnce   dba        159232 Sep 14 14:16 mongrel-1.0.1.gem
> > -rwx------    1 dimnce   dba        150016 Sep 14 14:18 net-ssh-1.0.10.gem
> > -rwx------    1 dimnce   dba        649728 Sep 14 14:19 needle-1.3.0.gem
> > -rwx------    1 dimnce   dba         43520 Sep 14 14:20 net-sftp-1.1.0.gem
> > -rwx------    1 dimnce   dba         54272 Sep 14 14:21 highline-1.4.0.gem
> > -rwx------    1 dimnce   dba         41984 Sep 14 14:22
> gem_plugin-0.2.2.gem
> > -rwx------    1 dimnce   dba         30720 Sep 14 14:24 daemons-1.0.7.gem
> > -rwx------    1 dimnce   dba         20992 Sep 14 14:25 fastthread-1.0.gem
> > -rwx------    1 dimnce   dba          8192 Sep 14 14:26
> > cgi_multipart_eof_fix-2.3.gem
> >
> >
> > I've then compiled and installed httpd-2.2.6. And try to use mongrel with
> > apache.
> >
> >
> > In fact, all is running correctly until I start mongrel_rails with the '-d'
> > option.
> >
> > if I start mongrel with the following command:
> > mongrel_rails start -e production -p 3210
> >
> > my application is running correctly, but if I start mongrel with
> > mongrel_rails start -e production -p 3210 -d
> >
> > (note the -d option), my application does not respond to any request.
> >
> > I've tried to use mongrel directly (without requesting apache) and
> > I've the same
> > behavior.
> >
> > Do you have any hint why -d does not run correctly ?
> >
> > Thanks for your help
> > Regards.
> >
> >
> >
> > +--------------------+
> > + Laurent Marzullo
> > +
> > _______________________________________________
> > Mongrel-users mailing list
> > Mongrel-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mongrel-users
> >
>
>
>
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
>
>


+--------------------+
+ Laurent Marzullo
+

From gbenedict at gmail.com  Tue Sep 18 08:39:50 2007
From: gbenedict at gmail.com (Greg Benedict)
Date: Tue, 18 Sep 2007 08:39:50 -0400
Subject: [Mongrel] Mongrel clusters are not being started on reboot
In-Reply-To: <20070917115952.ts6yxs84o48cgk8s@lcgalaska.com>
References: <49C3037A-64A3-4550-94DC-6CB482DAD90B@umn.edu>
	<20070917115952.ts6yxs84o48cgk8s@lcgalaska.com>
Message-ID: <6b93ca880709180539k32e13718s5c24e3bc0171954f@mail.gmail.com>

Which version of Mongrel Cluster are you using?

Greg

On 9/17/07, barsalou <barjunk at attglobal.net> wrote:
> I've found I had to use a script to change to the app directory before
> using the mongrel_rails cluster::start command.
>
> I think there are some patches that you can apply to fix this problem.
>
> Mike B.
>
> Quoting Jose Fernandez <ferna084 at umn.edu>:
>
> > I've done a clean install of Ubuntu 7.04 server in 2 different
> > machines and everytime I reboot the machine or manually run /etc/
> > init.d/mongrel_cluster restart/start the mongrels won't start correctly.
> >
> > I have to manually start them by using mongrel_rails cluster::start
> > from each app folder.   I've been searching around and it looks like
> > mongrel has some problems with pids, so whats the final word on this?
> >
> > - Jose
> > _______________________________________________
> > Mongrel-users mailing list
> > Mongrel-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mongrel-users
> >
>
>
>
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>


-- 
Thanks,

Greg Benedict

From me at seebq.com  Tue Sep 18 12:07:39 2007
From: me at seebq.com (Charles Brian Quinn)
Date: Tue, 18 Sep 2007 12:07:39 -0400
Subject: [Mongrel] Mongrel Upload Progress progress method returns
	Content-Length: 0
Message-ID: <3a2de0cd0709180907se96ab0clbdfb37126c2bd75c@mail.gmail.com>

Running into a strange bug using the mongrel upload progress plugin
handler, sitting in front of a simple mongrel.

The bug appears on an old Xserve (mac OS X 10.3.9 powerpc), but not on
my macbook pro, or even a generic 686 linux box.

http://pastie.textmate.org/98341

The gist of the pastie above shows that attempts through the browser
and curl are actually hitting the filesController, as evidenced in the
log, but the content returned is nothing - 0 length.

I've turned on mongrel debugging, and nothing jumps out at me, only
difference I've found in the logs is the HTTP_ACCEPT variable.

I'm guessing it's some kind of incompatiability with Zlib or
something.  The ruby on the non-working Xserve is compiled from source
using the newest readline.

Any idea where to start debugging?
-- 
Charles Brian Quinn
self-promotion: www.seebq.com
highgroove studios: www.highgroove.com
slingshot rails business hosting: www.slingshothosting.com
main: 404.394.4935  fax: 678.826.0969

Ruby on Rails Bootcamp at the Big Nerd Ranch
Intensive Ruby on Rails Training:
http://www.bignerdranch.com/classes/ruby.shtml

From ferna084 at umn.edu  Tue Sep 18 12:27:29 2007
From: ferna084 at umn.edu (Jose Fernandez)
Date: Tue, 18 Sep 2007 11:27:29 -0500
Subject: [Mongrel] Mongrel clusters are not being started on reboot
In-Reply-To: <6b93ca880709180539k32e13718s5c24e3bc0171954f@mail.gmail.com>
References: <49C3037A-64A3-4550-94DC-6CB482DAD90B@umn.edu>
	<20070917115952.ts6yxs84o48cgk8s@lcgalaska.com>
	<6b93ca880709180539k32e13718s5c24e3bc0171954f@mail.gmail.com>
Message-ID: <6A4A297C-8C2F-4B95-95BE-6C5A95D84754@umn.edu>

1.0.2

Jose

On Sep 18, 2007, at 7:39 AM, Greg Benedict wrote:

> Which version of Mongrel Cluster are you using?
>
> Greg
>
> On 9/17/07, barsalou <barjunk at attglobal.net> wrote:
>> I've found I had to use a script to change to the app directory  
>> before
>> using the mongrel_rails cluster::start command.
>>
>> I think there are some patches that you can apply to fix this  
>> problem.
>>
>> Mike B.
>>
>> Quoting Jose Fernandez <ferna084 at umn.edu>:
>>
>>> I've done a clean install of Ubuntu 7.04 server in 2 different
>>> machines and everytime I reboot the machine or manually run /etc/
>>> init.d/mongrel_cluster restart/start the mongrels won't start  
>>> correctly.
>>>
>>> I have to manually start them by using mongrel_rails cluster::start
>>> from each app folder.   I've been searching around and it looks like
>>> mongrel has some problems with pids, so whats the final word on  
>>> this?
>>>
>>> - Jose
>>> _______________________________________________
>>> Mongrel-users mailing list
>>> Mongrel-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>>
>>
>>
>>
>> ----------------------------------------------------------------
>> This message was sent using IMP, the Internet Messaging Program.
>>
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>
>
>
> -- 
> Thanks,
>
> Greg Benedict
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From gbenedict at gmail.com  Tue Sep 18 16:52:43 2007
From: gbenedict at gmail.com (Greg Benedict)
Date: Tue, 18 Sep 2007 16:52:43 -0400
Subject: [Mongrel] Mongrel clusters are not being started on reboot
In-Reply-To: <6A4A297C-8C2F-4B95-95BE-6C5A95D84754@umn.edu>
References: <49C3037A-64A3-4550-94DC-6CB482DAD90B@umn.edu>
	<20070917115952.ts6yxs84o48cgk8s@lcgalaska.com>
	<6b93ca880709180539k32e13718s5c24e3bc0171954f@mail.gmail.com>
	<6A4A297C-8C2F-4B95-95BE-6C5A95D84754@umn.edu>
Message-ID: <6b93ca880709181352n78b782b2m7bda70ed0a05e1d5@mail.gmail.com>

Does it show anything in the system error log /var/log/error_log after boot?

On 9/18/07, Jose Fernandez <ferna084 at umn.edu> wrote:
> 1.0.2
>
> Jose
>
> On Sep 18, 2007, at 7:39 AM, Greg Benedict wrote:
>
> > Which version of Mongrel Cluster are you using?
> >
> > Greg
> >
> > On 9/17/07, barsalou <barjunk at attglobal.net> wrote:
> >> I've found I had to use a script to change to the app directory
> >> before
> >> using the mongrel_rails cluster::start command.
> >>
> >> I think there are some patches that you can apply to fix this
> >> problem.
> >>
> >> Mike B.
> >>
> >> Quoting Jose Fernandez <ferna084 at umn.edu>:
> >>
> >>> I've done a clean install of Ubuntu 7.04 server in 2 different
> >>> machines and everytime I reboot the machine or manually run /etc/
> >>> init.d/mongrel_cluster restart/start the mongrels won't start
> >>> correctly.
> >>>
> >>> I have to manually start them by using mongrel_rails cluster::start
> >>> from each app folder.   I've been searching around and it looks like
> >>> mongrel has some problems with pids, so whats the final word on
> >>> this?
> >>>
> >>> - Jose
> >>> _______________________________________________
> >>> Mongrel-users mailing list
> >>> Mongrel-users at rubyforge.org
> >>> http://rubyforge.org/mailman/listinfo/mongrel-users
> >>>
> >>
> >>
> >>
> >> ----------------------------------------------------------------
> >> This message was sent using IMP, the Internet Messaging Program.
> >>
> >> _______________________________________________
> >> Mongrel-users mailing list
> >> Mongrel-users at rubyforge.org
> >> http://rubyforge.org/mailman/listinfo/mongrel-users
> >>
> >
> >
> > --
> > Thanks,
> >
> > Greg Benedict
> > _______________________________________________
> > Mongrel-users mailing list
> > Mongrel-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mongrel-users
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>


-- 
Thanks,

Greg Benedict

From jeremy.burks at gmail.com  Thu Sep 20 15:13:52 2007
From: jeremy.burks at gmail.com (Jeremy Burks)
Date: Thu, 20 Sep 2007 14:13:52 -0500
Subject: [Mongrel] Bug in Configurator.change_privilege?
In-Reply-To: <f4cd26df0707301350o6b19175apb2efae4daa89f49b@mail.gmail.com>
References: <4b06455b0706040601n324e3e75y4a73fc393086d9de@mail.gmail.com>
	<fa870e440707301310ubc32a1dy39c5e36c23e92b5b@mail.gmail.com>
	<f4cd26df0707301350o6b19175apb2efae4daa89f49b@mail.gmail.com>
Message-ID: <fa870e440709201213g640d3f3dk6ebffd8f57ce3a2@mail.gmail.com>

Is there any chance of getting this fixed for 1.0.2?

Thanks
- jeremy

From wayneeseguin at gmail.com  Thu Sep 20 15:17:51 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Thu, 20 Sep 2007 15:17:51 -0400
Subject: [Mongrel] Bug in Configurator.change_privilege?
In-Reply-To: <fa870e440709201213g640d3f3dk6ebffd8f57ce3a2@mail.gmail.com>
References: <4b06455b0706040601n324e3e75y4a73fc393086d9de@mail.gmail.com>
	<fa870e440707301310ubc32a1dy39c5e36c23e92b5b@mail.gmail.com>
	<f4cd26df0707301350o6b19175apb2efae4daa89f49b@mail.gmail.com>
	<fa870e440709201213g640d3f3dk6ebffd8f57ce3a2@mail.gmail.com>
Message-ID: <CB350DCF-67EE-4664-B555-892DBB019D5F@gmail.com>

On Sep 20, 2007, at 15:13 , Jeremy Burks wrote:
> Is there any chance of getting this fixed for 1.0.2?

File a ticket. Submit the patch, we'll see if we *might* be able to  
get it in.

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070920/a66e9240/attachment.html 

From jeremy.burks at gmail.com  Thu Sep 20 15:57:04 2007
From: jeremy.burks at gmail.com (Jeremy Burks)
Date: Thu, 20 Sep 2007 14:57:04 -0500
Subject: [Mongrel] Bug in Configurator.change_privilege?
In-Reply-To: <CB350DCF-67EE-4664-B555-892DBB019D5F@gmail.com>
References: <4b06455b0706040601n324e3e75y4a73fc393086d9de@mail.gmail.com>
	<fa870e440707301310ubc32a1dy39c5e36c23e92b5b@mail.gmail.com>
	<f4cd26df0707301350o6b19175apb2efae4daa89f49b@mail.gmail.com>
	<fa870e440709201213g640d3f3dk6ebffd8f57ce3a2@mail.gmail.com>
	<CB350DCF-67EE-4664-B555-892DBB019D5F@gmail.com>
Message-ID: <fa870e440709201257m40643ae4pa72721218c1ab5d1@mail.gmail.com>

Here is the patch:
http://rubyforge.org/tracker/index.php?func=detail&aid=14116&group_id=1306&atid=5147

On 9/20/07, Wayne E. Seguin <wayneeseguin at gmail.com> wrote:
>
> On Sep 20, 2007, at 15:13 , Jeremy Burks wrote:
> Is there any chance of getting this fixed for 1.0.2?
>
> File a ticket. Submit the patch, we'll see if we *might* be able to get it
> in.
>
>   ~Wayne
>
> s///g
> Wayne E. Seguin
> Sr. Systems Architect & Systems Administrator
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>

From scott.mcnab at gmail.com  Fri Sep 21 01:51:03 2007
From: scott.mcnab at gmail.com (Scott McNab)
Date: Fri, 21 Sep 2007 13:51:03 +0800
Subject: [Mongrel] Bug in Configurator.change_privilege?
In-Reply-To: <fa870e440709201257m40643ae4pa72721218c1ab5d1@mail.gmail.com>
References: <4b06455b0706040601n324e3e75y4a73fc393086d9de@mail.gmail.com>
	<fa870e440707301310ubc32a1dy39c5e36c23e92b5b@mail.gmail.com>
	<f4cd26df0707301350o6b19175apb2efae4daa89f49b@mail.gmail.com>
	<fa870e440709201213g640d3f3dk6ebffd8f57ce3a2@mail.gmail.com>
	<CB350DCF-67EE-4664-B555-892DBB019D5F@gmail.com>
	<fa870e440709201257m40643ae4pa72721218c1ab5d1@mail.gmail.com>
Message-ID: <4b06455b0709202251v4f60a0b7m4078f19c91f68eb3@mail.gmail.com>

Yes I second the vote for this to please be included in the 1.0.2
release, if at all possible.
Thanks
Scott


On 9/21/07, Jeremy Burks <jeremy.burks at gmail.com> wrote:
> Here is the patch:
> http://rubyforge.org/tracker/index.php?func=detail&aid=14116&group_id=1306&atid=5147
>
> On 9/20/07, Wayne E. Seguin <wayneeseguin at gmail.com> wrote:
> >
> > On Sep 20, 2007, at 15:13 , Jeremy Burks wrote:
> > Is there any chance of getting this fixed for 1.0.2?
> >
> > File a ticket. Submit the patch, we'll see if we *might* be able to get it
> > in.
> >
> >   ~Wayne

From wayneeseguin at gmail.com  Fri Sep 21 09:04:06 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Fri, 21 Sep 2007 09:04:06 -0400
Subject: [Mongrel] Bug in Configurator.change_privilege?
In-Reply-To: <4b06455b0709202251v4f60a0b7m4078f19c91f68eb3@mail.gmail.com>
References: <4b06455b0706040601n324e3e75y4a73fc393086d9de@mail.gmail.com>
	<fa870e440707301310ubc32a1dy39c5e36c23e92b5b@mail.gmail.com>
	<f4cd26df0707301350o6b19175apb2efae4daa89f49b@mail.gmail.com>
	<fa870e440709201213g640d3f3dk6ebffd8f57ce3a2@mail.gmail.com>
	<CB350DCF-67EE-4664-B555-892DBB019D5F@gmail.com>
	<fa870e440709201257m40643ae4pa72721218c1ab5d1@mail.gmail.com>
	<4b06455b0709202251v4f60a0b7m4078f19c91f68eb3@mail.gmail.com>
Message-ID: <C5EF11E0-8C83-4FD2-92D6-22FDE3C5E35B@gmail.com>

On Sep 21, 2007, at 01:51 , Scott McNab wrote:
> Yes I second the vote for this to please be included in the 1.0.2
> release, if at all possible.
> Thanks
> Scott

Scott,

It has been applied for the 1.0.2 release. Thank you for your  
contribution.

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070921/0111a76f/attachment.html 

From jeremy.burks at gmail.com  Fri Sep 21 10:29:05 2007
From: jeremy.burks at gmail.com (Jeremy Burks)
Date: Fri, 21 Sep 2007 09:29:05 -0500
Subject: [Mongrel] Bug in Configurator.change_privilege?
In-Reply-To: <C5EF11E0-8C83-4FD2-92D6-22FDE3C5E35B@gmail.com>
References: <4b06455b0706040601n324e3e75y4a73fc393086d9de@mail.gmail.com>
	<fa870e440707301310ubc32a1dy39c5e36c23e92b5b@mail.gmail.com>
	<f4cd26df0707301350o6b19175apb2efae4daa89f49b@mail.gmail.com>
	<fa870e440709201213g640d3f3dk6ebffd8f57ce3a2@mail.gmail.com>
	<CB350DCF-67EE-4664-B555-892DBB019D5F@gmail.com>
	<fa870e440709201257m40643ae4pa72721218c1ab5d1@mail.gmail.com>
	<4b06455b0709202251v4f60a0b7m4078f19c91f68eb3@mail.gmail.com>
	<C5EF11E0-8C83-4FD2-92D6-22FDE3C5E35B@gmail.com>
Message-ID: <fa870e440709210729k31a1b89bnf33f2cec224a443a@mail.gmail.com>

Awesome. Thanks. And thanks to Scott for tracking down the problem.

- jeremy

On 9/21/07, Wayne E. Seguin <wayneeseguin at gmail.com> wrote:
>
> On Sep 21, 2007, at 01:51 , Scott McNab wrote:
> Yes I second the vote for this to please be included in the 1.0.2
> release, if at all possible.
> Thanks
> Scott
>
> Scott,
>
> It has been applied for the 1.0.2 release. Thank you for your contribution.
>
>   ~Wayne
>
> s///g
> Wayne E. Seguin
> Sr. Systems Architect & Systems Administrator
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>

From me at seebq.com  Fri Sep 21 18:03:37 2007
From: me at seebq.com (Charles Brian Quinn)
Date: Fri, 21 Sep 2007 18:03:37 -0400
Subject: [Mongrel] [PATCH] Apache Documentation Updates
Message-ID: <3a2de0cd0709211503p59c2ab54uf11f3074736e4162@mail.gmail.com>

Hello mongrelians,

I've had several submissions sent to me regarding the apache
documentation on the mongrel site:

http://mongrel.rubyforge.org/docs/apache.html

I have assembled quite a bit of updates and changes, but if you know
of anything that has been useful to you, or that is missing or can be
reworded or added (or missing attributes!) please let me know.

cheers,
-- 
Charles Brian Quinn
self-promotion: www.seebq.com
highgroove studios: www.highgroove.com
slingshot rails business hosting: www.slingshothosting.com
main: 404.394.4935  fax: 678.826.0969

Ruby on Rails Bootcamp at the Big Nerd Ranch
Intensive Ruby on Rails Training:
http://www.bignerdranch.com/classes/ruby.shtml
-------------- next part --------------
A non-text attachment was scrubbed...
Name: updated_apache_docs.patch
Type: application/octet-stream
Size: 5906 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070921/aa34c7b1/attachment-0001.obj 

From zedshaw at zedshaw.com  Fri Sep 21 22:26:07 2007
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Fri, 21 Sep 2007 22:26:07 -0400
Subject: [Mongrel] [PATCH] Apache Documentation Updates
In-Reply-To: <3a2de0cd0709211503p59c2ab54uf11f3074736e4162@mail.gmail.com>
References: <3a2de0cd0709211503p59c2ab54uf11f3074736e4162@mail.gmail.com>
Message-ID: <20070921222607.50ae29f8.zedshaw@zedshaw.com>

On Fri, 21 Sep 2007 18:03:37 -0400
"Charles Brian Quinn" <me at seebq.com> wrote:

> Hello mongrelians,
> 
> I've had several submissions sent to me regarding the apache
> documentation on the mongrel site:
> 
> http://mongrel.rubyforge.org/docs/apache.html
> 
> I have assembled quite a bit of updates and changes, but if you know
> of anything that has been useful to you, or that is missing or can be
> reworded or added (or missing attributes!) please let me know.

Hey hey Charles, how you been?

Go ahead and either shoot me the patch or toss it on the rubyforge patch queue.  If you're interested in just getting access and getting setup to push these out yourself I can do that too.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From wayneeseguin at gmail.com  Sat Sep 22 01:08:47 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Sat, 22 Sep 2007 01:08:47 -0400
Subject: [Mongrel] [PATCH] Apache Documentation Updates
In-Reply-To: <3a2de0cd0709211503p59c2ab54uf11f3074736e4162@mail.gmail.com>
References: <3a2de0cd0709211503p59c2ab54uf11f3074736e4162@mail.gmail.com>
Message-ID: <C3155DFB-D4F1-4247-BD34-1ED6AC4E32F9@gmail.com>

On Sep 21, 2007, at 18:03 , Charles Brian Quinn wrote:
> Hello mongrelians,
>
> I've had several submissions sent to me regarding the apache
> documentation on the mongrel site:
>
> http://mongrel.rubyforge.org/docs/apache.html
>
> I have assembled quite a bit of updates and changes, but if you know
> of anything that has been useful to you, or that is missing or can be
> reworded or added (or missing attributes!) please let me know.
>
> cheers,
> --  
> Charles Brian Quinn

Hello Charles!

We're going to be pushing 1.0.2 very soon and it would be nice to  
push that with i. Please post the patch to the tracker and we'll make  
sure it gets in. Thank you!!!

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070922/3b0d7180/attachment.html 

From rogerpack2005 at gmail.com  Sat Sep 22 13:35:32 2007
From: rogerpack2005 at gmail.com (Roger Pack)
Date: Sat, 22 Sep 2007 11:35:32 -0600
Subject: [Mongrel] sockets reading wrong data
Message-ID: <966599840709221035t654f6a23qbe9fe3630fa9c45f@mail.gmail.com>

I am aware of a problem with multi-threaded ruby apps, in which one socket
reads from another's input/output or what not.  Zed reports this as a
generic problem in Ruby multi-threaded apps, and I've seen the same in my
own programs.
I have, however, never heard of this becoming a problem for mongrel.  Is
this true?  Is there something mongrel does to get 'around' this, that would
explain why it doesn't occur there?  Or is it just that it doesn't run all
that many threads per process, so it avoids it?

Just wondering for my own bemusement.
Thank you.
-Roger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070922/a7939e6e/attachment.html 

From ry at tinyclouds.org  Sat Sep 22 13:59:57 2007
From: ry at tinyclouds.org (ry dahl)
Date: Sat, 22 Sep 2007 19:59:57 +0200
Subject: [Mongrel] http helpers c extension
Message-ID: <21ee31950709221059x613b96c0yb5df6d7314fd7b64@mail.gmail.com>

hi,

I rewrote escape, unescape, and query_parse in a C extension.
http://s3.amazonaws.com/four.livejournal/20070922/http_helpers.tar.bz2

Here are some sample benchmarks (the benchmark script is included in
the package)

      user     system      total        real
escape: Single long
Mongrel:  1.680000   0.020000   1.700000 (  1.837793)
HttpHlp:  0.030000   0.010000   0.040000 (  0.036590)

escape: Many small
Mongrel:  2.580000   0.020000   2.600000 (  2.751985)
HttpHlp:  0.290000   0.010000   0.300000 (  0.353808)

unescape: Single long
Mongrel:  2.490000   0.030000   2.520000 (  2.659217)
HttpHlp:  0.030000   0.010000   0.040000 (  0.238297)

unescape: Many small
Mongrel:  3.400000   0.030000   3.430000 (  3.952430)
HttpHlp:  0.280000   0.000000   0.280000 (  0.344116)

query_parse: one long query
Merb:  0.350000   0.010000   0.360000 (  0.518884)
HHlp:  0.020000   0.000000   0.020000 (  0.023698)

query_parse: many short query strings
Merb:  5.970000   0.060000   6.030000 (  7.105239)
HHlp:  0.710000   0.010000   0.720000 (  0.893846)

query_parse: deeply nested query
Merb:  0.000000   0.000000   0.000000 (  0.000242)
HHlp:  0.000000   0.000000   0.000000 (  0.000047)


My query_parse implements a different behavior than Mongrel has.
Mongrel interprets "q[a]=b" to be { 'q[a]' => 'b' }. Instead the C
extension mimic's Merb's behavior, creating nested hashes.

I think mongrel should change the behavior of it's query_parse
function and use a C implementation. HttpHelpers.query_parse also uses
a ragel state machine, so it shouldn't be increadably hard to put this
query_parse directly into http11.

Is there a development branch of mongrel? What should I patch against?

What do people feel about how query_parse should behave? Should it
happen automatically as Mongrel parses the rest of the query? Should
it by default use the Merb hash syntax?




ry

From zedshaw at zedshaw.com  Sat Sep 22 14:45:35 2007
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Sat, 22 Sep 2007 14:45:35 -0400
Subject: [Mongrel] streaming upload progress plug-in
In-Reply-To: <21ee31950707071758v5fc59e59i2768cc0aa891634e@mail.gmail.com>
References: <21ee31950707071758v5fc59e59i2768cc0aa891634e@mail.gmail.com>
Message-ID: <20070922144535.24f6863a.zedshaw@zedshaw.com>

On Sun, 8 Jul 2007 02:58:45 +0200
"ry dahl" <ry at tinyclouds.org> wrote:

> I'm working on new methods of doing Ajax calls through streaming
> responses. I've created a little javascript library (which depends on
> prototype.js) that can interpret a stream of JSON objects separated by
> semicolons.

Hey Ry, I've been missing your contributions lately.  Let me reply to each of them individually.

With this one though, while it's cool, it's so damn lame that we have to do an HTTP server request to find out how much data the browser has sent over a socket.

Have you investigated any of the flash or flex based uploaders?

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From zedshaw at zedshaw.com  Sat Sep 22 14:49:47 2007
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Sat, 22 Sep 2007 14:49:47 -0400
Subject: [Mongrel] SqueezeBox, a new Mongrel handler
In-Reply-To: <21ee31950708131106t4c966542v203170a69ea2d443@mail.gmail.com>
References: <21ee31950708131106t4c966542v203170a69ea2d443@mail.gmail.com>
Message-ID: <20070922144947.d82e1d86.zedshaw@zedshaw.com>

On Mon, 13 Aug 2007 20:06:51 +0200
"ry dahl" <ry at tinyclouds.org> wrote:

> SqueezeBox is a simple file system based routing framework for erubis
> templates.
> 
> It aims for a couple of niceties as well:

Looks like fun.  Is there anything in particular you're doing with it, or just for fun.
 
> Instead of putting your logic inside your template, like you would
> with PHP, SqueezeBox recognizes a separate logic file to be called
> before it loads the template. This allows you room to load
> ActiveRecord models or do calculations. These logic files return to
> the webserver the last line of their execution. If render() is the
> last line, it will load the template associated with that logic file.
> If a string containing, say, JSON is returned, then the JSON will be
> the body of the response and the template ignored. In fact, any object
> that contains a read method can be returned and the SqueezeBox will
> stream the response. (This cute behavior is stolen from Merb.)

Ok, now you say this is a handler.  I haven't checked it quite yet, but what's the possibility of a rails project installing this for the parts of a site that need to be quick.

I believe this was the original intent with merb, but then it became its own rails-clone.  I'm still looking for something tiny that can piggy-back on rails.

Kind of like Master Blaster in that really bad Mad Max movie with Tina Turner.

> Please try it out! Thanks!

I'll play with it today.  Might actually need something like this for the Utu project.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From zedshaw at zedshaw.com  Sat Sep 22 14:51:05 2007
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Sat, 22 Sep 2007 14:51:05 -0400
Subject: [Mongrel] Cookies for Mongrels
In-Reply-To: <21ee31950708170854q5f18d25dia096294c66e55ee8@mail.gmail.com>
References: <21ee31950708170854q5f18d25dia096294c66e55ee8@mail.gmail.com>
Message-ID: <20070922145105.d9a55eae.zedshaw@zedshaw.com>

On Fri, 17 Aug 2007 17:54:43 +0200
"ry dahl" <ry at tinyclouds.org> wrote:

> This class provides a CookieJar object for Mongrels. Just initialize
> with response and request objects and then treat like a hash. Does not
> depend on cgi.rb

So far looks good.  I guess it'd have to be something that a framework would switch to instead of the cgi.rb cookies.  Right?

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From zedshaw at zedshaw.com  Sat Sep 22 14:57:43 2007
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Sat, 22 Sep 2007 14:57:43 -0400
Subject: [Mongrel] http helpers c extension
In-Reply-To: <21ee31950709221059x613b96c0yb5df6d7314fd7b64@mail.gmail.com>
References: <21ee31950709221059x613b96c0yb5df6d7314fd7b64@mail.gmail.com>
Message-ID: <20070922145743.030cfbd2.zedshaw@zedshaw.com>

On Sat, 22 Sep 2007 19:59:57 +0200
"ry dahl" <ry at tinyclouds.org> wrote:

> hi,
> 
> I rewrote escape, unescape, and query_parse in a C extension.
> http://s3.amazonaws.com/four.livejournal/20070922/http_helpers.tar.bz2

I'll take a look at this during the weekend and check the code.  So far the benchmarks are good so I'll talk with the mongrel team and see what they think.  Are you actively using this?
 
> 
> My query_parse implements a different behavior than Mongrel has.
> Mongrel interprets "q[a]=b" to be { 'q[a]' => 'b' }. Instead the C
> extension mimic's Merb's behavior, creating nested hashes.

I think that works for Merb and your stuff, but Rails expects the cgi.rb flavor (which isn't correct, your's is).  Have you tried it with rails yet?

> I think mongrel should change the behavior of it's query_parse
> function and use a C implementation. HttpHelpers.query_parse also uses
> a ragel state machine, so it shouldn't be increadably hard to put this
> query_parse directly into http11.

That's entirely true, but let me double check your C.  If I find a buffer overflow you owe me a pizza. :-)

> 
> Is there a development branch of mongrel? What should I patch against?

Go join the development list.  I'm starting to slowly come out of hiding and will be working to direct the mongrel team volunteers as they try to push out 1.0.2.  There's no development branch, but we could try to work this in.

http://rubyforge.org/mailman/listinfo/mongrel-development

> What do people feel about how query_parse should behave? Should it
> happen automatically as Mongrel parses the rest of the query? Should
> it by default use the Merb hash syntax?

Now, you did this with Ragel, what about merging the parsing into the existing http11 parser to do this all inline and on the fly?  That way, when it comes out it's C coma it's got a fully cooked response.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From ry at tinyclouds.org  Sat Sep 22 15:17:23 2007
From: ry at tinyclouds.org (ry dahl)
Date: Sat, 22 Sep 2007 21:17:23 +0200
Subject: [Mongrel] http helpers c extension
In-Reply-To: <20070922145743.030cfbd2.zedshaw@zedshaw.com>
References: <21ee31950709221059x613b96c0yb5df6d7314fd7b64@mail.gmail.com>
	<20070922145743.030cfbd2.zedshaw@zedshaw.com>
Message-ID: <21ee31950709221217v559a9805rd89c23dbffe77280@mail.gmail.com>

Hi Zed,
this isn't ready to just be inserted into Mongrel as it is. I just
wrote it for fun/to see if I could make Merb faster/to learn ragel. I
thought I would clean it up and reformat it if y'all were interested -
it has several crashing problems and it doesn't work with cookies yet.

> I'm starting to slowly come out of hiding and
> will be working to direct the mongrel team volunteers as they try to push out 1.0.2.

glad to hear it


ry

From dom at pazap.com  Sat Sep 22 23:22:56 2007
From: dom at pazap.com (Dominic Son)
Date: Sat, 22 Sep 2007 20:22:56 -0700
Subject: [Mongrel] can't seem to activate the '-d' option..
Message-ID: <beb075620709222022ld2cb526o4e1ada34a98e6791@mail.gmail.com>

hi. apologies if this has been asked before...

my configs:
ruby -v 1.8.5
mongrel 1.0.1
rails 1.2.3

i do a mongrel_rails start -e production -p 4000 -d
it'll say:
'** Ruby version is not up-to-date; loading cgi_multipart_eof_fix'

but that shouldn't be the problems, since i see that all the time and
my clusters work..
( oh yeah, does it matter i have other clusters running, i doubt it..)

however, this single mongrel process is not running in the background
no pids written, nothing..

(it will work without the -d option )


- Dominic Son

The ability to simplify means to eliminate the unnecessary so that the
necessary may speak.
-Hofstadter's Law

From armin at personifi.com  Sun Sep 23 02:30:54 2007
From: armin at personifi.com (armin roehrl)
Date: Sun, 23 Sep 2007 08:30:54 +0200
Subject: [Mongrel] performance observation on redhat
Message-ID: <20070923063054.43DE710A3D2@ws6-4.us4.outblaze.com>

Hi,

    I made an interesting observation using webservers (not just mongrel) under red hat enterprise
linux ES release 4 (Nahant Update 5). Maybe this is helpful or somebody with deeper networking
expertise can comment on this.

Once client said that 1-2% of the response of our server were unacceptably slow (really huge 3s-21s). 
So I did more  ab and httperf tests and notice that very few requests do take a very long time. Being clueless
I first thought that the ruby garbage collector or mongrel is causing the effect, so after looking
at a similar setup using erlang's yaws or nginx alone, I noticed that sometimes this can happen,
especially when increasing the number of concurrent connects to a large number (e.g. 250-500).
I did the same on OS-X and did not notice these outliers.

After a lot of painful searching by luck we found one cure: inet_peer_threshold was too small.
Chaging it to a much larger value made the problem go away.

echo 500000 > /proc/sys/net/ipv4/inet_peer_threshold

There is a trade-off here. A too small value causes too many delays from inet-peer-storage cleaning 
and a too big value makes life well for some limited time, but when it hits you, it becomes really expensive.

Did you ever see this?

Thanks,
  -Armin

From wayneeseguin at gmail.com  Sun Sep 23 09:12:19 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Sun, 23 Sep 2007 09:12:19 -0400
Subject: [Mongrel] can't seem to activate the '-d' option..
In-Reply-To: <beb075620709222022ld2cb526o4e1ada34a98e6791@mail.gmail.com>
References: <beb075620709222022ld2cb526o4e1ada34a98e6791@mail.gmail.com>
Message-ID: <7B82E816-37D5-4EB9-9920-8EC420E6E74F@gmail.com>

On Sep 22, 2007, at 23:22 , Dominic Son wrote:
> hi. apologies if this has been asked before...
>
> my configs:
> ruby -v 1.8.5
> mongrel 1.0.1
> rails 1.2.3
>
> i do a mongrel_rails start -e production -p 4000 -d
> it'll say:
> '** Ruby version is not up-to-date; loading cgi_multipart_eof_fix'
>
> but that shouldn't be the problems, since i see that all the time and
> my clusters work..
> ( oh yeah, does it matter i have other clusters running, i doubt it..)
>
> however, this single mongrel process is not running in the background
> no pids written, nothing..
>
> (it will work without the -d option )
>
>
> - Dominic Son

Dominic,

Start that mongrel up in debug mode and check the debug logs for that  
process.

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070923/8dbfa910/attachment.html 

From wayneeseguin at gmail.com  Sun Sep 23 09:14:06 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Sun, 23 Sep 2007 09:14:06 -0400
Subject: [Mongrel] performance observation on redhat
In-Reply-To: <20070923063054.43DE710A3D2@ws6-4.us4.outblaze.com>
References: <20070923063054.43DE710A3D2@ws6-4.us4.outblaze.com>
Message-ID: <0451432B-F007-443E-9831-F9B26B028F1A@gmail.com>

On Sep 23, 2007, at 02:30 , armin roehrl wrote:
> Hi,
>
>     I made an interesting observation using webservers (not just  
> mongrel) under red hat enterprise
> linux ES release 4 (Nahant Update 5). Maybe this is helpful or  
> somebody with deeper networking
> expertise can comment on this.
>
> Once client said that 1-2% of the response of our server were  
> unacceptably slow (really huge 3s-21s).
> So I did more  ab and httperf tests and notice that very few  
> requests do take a very long time. Being clueless
> I first thought that the ruby garbage collector or mongrel is  
> causing the effect, so after looking
> at a similar setup using erlang's yaws or nginx alone, I noticed  
> that sometimes this can happen,
> especially when increasing the number of concurrent connects to a  
> large number (e.g. 250-500).
> I did the same on OS-X and did not notice these outliers.
>
> After a lot of painful searching by luck we found one cure:  
> inet_peer_threshold was too small.
> Chaging it to a much larger value made the problem go away.
>
> echo 500000 > /proc/sys/net/ipv4/inet_peer_threshold
>
> There is a trade-off here. A too small value causes too many delays  
> from inet-peer-storage cleaning
> and a too big value makes life well for some limited time, but when  
> it hits you, it becomes really expensive.
>
> Did you ever see this?
>
> Thanks,
>   -Armin

Armin,

We might put this in the documentation, will discuss with the dev team.

Thank you for this.

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070923/e0d86058/attachment.html 

From wayneeseguin at gmail.com  Sun Sep 23 09:16:56 2007
From: wayneeseguin at gmail.com (Wayne E. Seguin)
Date: Sun, 23 Sep 2007 09:16:56 -0400
Subject: [Mongrel] performance observation on redhat
In-Reply-To: <20070923063054.43DE710A3D2@ws6-4.us4.outblaze.com>
References: <20070923063054.43DE710A3D2@ws6-4.us4.outblaze.com>
Message-ID: <C36209E3-69F0-4CCA-831F-DFC34A1E65E5@gmail.com>

On Sep 23, 2007, at 02:30 , armin roehrl wrote:
> Did you ever see this?

We run RedShit EL 4 in production also and yes we have come across  
this issue, never tracked it down till now though ( thank you :) ).

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070923/32b93800/attachment-0001.html 

From twscannell at gmail.com  Sun Sep 23 12:19:28 2007
From: twscannell at gmail.com (TW Scannell)
Date: Sun, 23 Sep 2007 09:19:28 -0700
Subject: [Mongrel] Newbie confusion.
Message-ID: <12f0ca280709230919l52f91aaega0c5edb5f3167123@mail.gmail.com>

Hello,

I am new to Rails and the Linux world coming from 12 years of advanced VB
programming. I am writing to try and clear up some confusion on my part
about installing Mongrel.

Please don't take this as a rant, I do appreciate the effort that y'all put
into the community, but as a newbie, I was, and still am confused.
I want to learn about Mongrel, so it found
http://mongrel.rubyforge.org/docs/started.html and followed the simple
instructions:

$ sudo gem install mongrel$ cd myrailsapp$ mongrel_rails start -d

Which runs Mongrel in the background. You can stop it with:

$ mongrel_rails stop


What I found was far more than that, and to be honest, I have no idea
whether or not I installed correctly.
When I ran sudo gem install mongrel, I was presented with a series of
options that I had no idea of how to respond.

Install required dependency daemons? [Yn]  y
Install required dependency fastthread? [Yn]  y
Install required dependency gem_plugin? [Yn]  y
Install required dependency cgi_multipart_eof_fix? [Yn]  y

After the install, the command "mongrel_rails start -d" was not recognized

$ mongrel_rails start -d
bash: mongrel_rails: command not found

soo... I gave up and did what I had been doing, which is script/server.
Lo-and-behold, mongrel starts up... ???

$ script/server
=> Booting Mongrel (use 'script/server webrick' to force WEBrick)
=> Rails application starting on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
** Starting Mongrel listening at 0.0.0.0:3000
** Starting Rails with development environment...
** Rails loaded.
** Loading any Rails specific GemPlugins
** Signals ready.  TERM => stop.  USR2 => restart.  INT => stop (no restart).
** Rails signals registered.  HUP => reload (without restart).  It
might not work well.
** Mongrel available at 0.0.0.0:3000
** Use CTRL-C to stop.

I guess that that the correct cmd would be "script/server -d ???

OK, it's not the end of the world, but it sure threw me for a
loop.Like I said, I appreciate all that you do, but maybe some of this
could be put into the "Getting Started" page.

Thanks
TW Scannell
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070923/8b48d2bd/attachment.html 

From barjunk at attglobal.net  Sun Sep 23 14:41:35 2007
From: barjunk at attglobal.net (barsalou)
Date: Sun, 23 Sep 2007 10:41:35 -0800
Subject: [Mongrel] performance observation on redhat
In-Reply-To: <0451432B-F007-443E-9831-F9B26B028F1A@gmail.com>
References: <20070923063054.43DE710A3D2@ws6-4.us4.outblaze.com>
	<0451432B-F007-443E-9831-F9B26B028F1A@gmail.com>
Message-ID: <20070923104135.e7c28jtlno4w8wcc@lcgalaska.com>

Armin,

Since your goofing around with that, how about some of the other 
settings like maxtime and ttl values?

Seems like you have deleayd when some garbage collection operations are 
taking place, so maybe tweeking that a little more will give you the 
performance your looking for.

I've never used this feature, but thought that might interesting as well.

Mike B.

Quoting "Wayne E. Seguin" <wayneeseguin at gmail.com>:

> On Sep 23, 2007, at 02:30 , armin roehrl wrote:
>> Hi,
>>
>>     I made an interesting observation using webservers (not just  
>> mongrel) under red hat enterprise
>> linux ES release 4 (Nahant Update 5). Maybe this is helpful or  
>> somebody with deeper networking
>> expertise can comment on this.
>>
>> Once client said that 1-2% of the response of our server were  
>> unacceptably slow (really huge 3s-21s).
>> So I did more  ab and httperf tests and notice that very few  
>> requests do take a very long time. Being clueless
>> I first thought that the ruby garbage collector or mongrel is  
>> causing the effect, so after looking
>> at a similar setup using erlang's yaws or nginx alone, I noticed  
>> that sometimes this can happen,
>> especially when increasing the number of concurrent connects to a  
>> large number (e.g. 250-500).
>> I did the same on OS-X and did not notice these outliers.
>>
>> After a lot of painful searching by luck we found one cure:  
>> inet_peer_threshold was too small.
>> Chaging it to a much larger value made the problem go away.
>>
>> echo 500000 > /proc/sys/net/ipv4/inet_peer_threshold
>>
>> There is a trade-off here. A too small value causes too many delays  
>> from inet-peer-storage cleaning
>> and a too big value makes life well for some limited time, but when  
>> it hits you, it becomes really expensive.
>>
>> Did you ever see this?
>>
>> Thanks,
>>   -Armin
>
> Armin,
>
> We might put this in the documentation, will discuss with the dev team.
>
> Thank you for this.
>
>   ~Wayne
>
> s///g
> Wayne E. Seguin
> Sr. Systems Architect & Systems Administrator
>
>



----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.


From j.edwards at itransact.com  Sun Sep 23 15:20:48 2007
From: j.edwards at itransact.com (Jason Edwards)
Date: Sun, 23 Sep 2007 13:20:48 -0600
Subject: [Mongrel] Newbie confusion.
In-Reply-To: <12f0ca280709230919l52f91aaega0c5edb5f3167123@mail.gmail.com>
References: <12f0ca280709230919l52f91aaega0c5edb5f3167123@mail.gmail.com>
Message-ID: <4192AA21-4937-4985-B0FF-6571D5B12736@itransact.com>

TW,

Welcome to the Linux and Ruby world -- it's a wonderful place :-)

The most likely explanation for this is, 'mongrel_rails' wasn't found  
in your path.  'mongrel_rails' is sometimes installed in '/usr/local/ 
bin' which some distros don't include by default.

Next time you run into a situation where a command is not found, try:

user at host:~$ locate <command-name>  # e.g. locate mongrel_rails
-- or --
user at host:~$ find / -name <command-name>  # e.g. find / -name  
mongrel_rails

Which results in something like:

/usr/local/bin/mongrel_rails
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.1/bin/mongrel_rails

Now check your path like this:

user at host:~$ echo $PATH
/bin:/sbin:/usr/bin:/usr/local/bin:/usr/sbin

In this case, 'mongrel_rails' would not be found, so I would need to  
add '/usr/local/bin' to the $PATH environment variable, like so:

user at host:~$ export PATH=$PATH:/usr/local/bin

Hope this helps,

Jason


On Sep 23, 2007, at 10:19 AM, TW Scannell wrote:

> Hello,
>
> I am new to Rails and the Linux world coming from 12 years of  
> advanced VB programming. I am writing to try and clear up some  
> confusion on my part about installing Mongrel.
>
> Please don't take this as a rant, I do appreciate the effort that  
> y'all put into the community, but as a newbie, I was, and still am  
> confused.
> I want to learn about Mongrel, so it found http:// 
> mongrel.rubyforge.org/docs/started.html and followed the simple  
> instructions:
>
> $ sudo gem install mongrel
> $ cd myrailsapp
> $ mongrel_rails start -d
> Which runs Mongrel in the background. You can stop it with:
>
> $ mongrel_rails stop
>
> What I found was far more than that, and to be honest, I have no  
> idea whether or not I installed correctly.
> When I ran sudo gem install mongrel, I was presented with a series  
> of options that I had no idea of how to respond.
>
> Install required dependency daemons? [Yn]  y
> Install required dependency fastthread? [Yn]  y
> Install required dependency gem_plugin? [Yn]  y
> Install required dependency cgi_multipart_eof_fix? [Yn]  y
>
> After the install, the command "mongrel_rails start -d" was not  
> recognized
>
> $ mongrel_rails start -d
> bash: mongrel_rails: command not found
>
> soo...
> I gave up and did what I had been doing, which is script/server.
> Lo-and-behold, mongrel starts up... ???
>
>
> $ script/server
> => Booting Mongrel (use 'script/server webrick' to force WEBrick)
>
> => Rails application starting on http://0.0.0.0:3000
>
> => Call with -d to detach
> => Ctrl-C to shutdown server
>
> ** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
> ** Starting Mongrel listening at
> 0.0.0.0:3000
> ** Starting Rails with development environment...
>
> ** Rails loaded.
> ** Loading any Rails specific GemPlugins
>
> ** Signals ready.  TERM => stop.  USR2 => restart.  INT => stop (no  
> restart).
> ** Rails signals registered.  HUP => reload (without restart).  It  
> might not work well.
>
> ** Mongrel available at 0.0.0.0:3000
>
> ** Use CTRL-C to stop.
>
> I guess that that the correct cmd would be "script/server -d ???
>
> OK, it's not the end of the world, but it sure threw me for a loop.
>
> Like I said, I appreciate all that you do, but maybe some of this  
> could be put into the "Getting Started" page.
>
>
> Thanks
>
> TW Scannell
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070923/5a781762/attachment.html 

From me at seebq.com  Wed Sep 26 16:43:56 2007
From: me at seebq.com (Charles Brian Quinn)
Date: Wed, 26 Sep 2007 16:43:56 -0400
Subject: [Mongrel] Mongrel Upload Progress progress method returns
	Content-Length: 0
In-Reply-To: <3a2de0cd0709180907se96ab0clbdfb37126c2bd75c@mail.gmail.com>
References: <3a2de0cd0709180907se96ab0clbdfb37126c2bd75c@mail.gmail.com>
Message-ID: <3a2de0cd0709261343y11ee33f0n652758985393d7f8@mail.gmail.com>

Semi-Solution / Workaround / FYI:

The Apache server had HTTP KeepAlive's turned off:

KeepAlive Off

This was causing the HTTP_CONNECT: close

instead of keep-alives to allow the uploads to show progress.

Hitting the mongrel directly on it's port confirmed this.

On 9/18/07, Charles Brian Quinn <me at seebq.com> wrote:
> Running into a strange bug using the mongrel upload progress plugin
> handler, sitting in front of a simple mongrel.
>
> The bug appears on an old Xserve (mac OS X 10.3.9 powerpc), but not on
> my macbook pro, or even a generic 686 linux box.
>
> http://pastie.textmate.org/98341
>
> The gist of the pastie above shows that attempts through the browser
> and curl are actually hitting the filesController, as evidenced in the
> log, but the content returned is nothing - 0 length.
>
> I've turned on mongrel debugging, and nothing jumps out at me, only
> difference I've found in the logs is the HTTP_ACCEPT variable.
>
> I'm guessing it's some kind of incompatiability with Zlib or
> something.  The ruby on the non-working Xserve is compiled from source
> using the newest readline.
>
> Any idea where to start debugging?
> --
> Charles Brian Quinn
> self-promotion: www.seebq.com
> highgroove studios: www.highgroove.com
> slingshot rails business hosting: www.slingshothosting.com
> main: 404.394.4935  fax: 678.826.0969
>
> Ruby on Rails Bootcamp at the Big Nerd Ranch
> Intensive Ruby on Rails Training:
> http://www.bignerdranch.com/classes/ruby.shtml
>


-- 
Charles Brian Quinn
self-promotion: www.seebq.com
highgroove studios: www.highgroove.com
slingshot rails business hosting: www.slingshothosting.com
main: 404.394.4935  fax: 678.826.0969

Ruby on Rails Bootcamp at the Big Nerd Ranch
Intensive Ruby on Rails Training:
http://www.bignerdranch.com/classes/ruby.shtml