From charles.nutter at sun.com  Fri Jun  1 04:52:34 2007
From: charles.nutter at sun.com (Charles Oliver Nutter)
Date: Fri, 01 Jun 2007 03:52:34 -0500
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu consumption
In-Reply-To: <d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
Message-ID: <465FDE52.2040807@sun.com>

Jim Weirich wrote:
> On 5/27/07, Ryan Davis <ryand-ruby at zenspider.com> wrote:
>> On May 24, 2007, at 16:19 , Eric Hodel wrote:
>>
>>> Jim and I were talking about trying incremental upload all the time,
>>> rather than bulk at RailsConf...
>> How crazy would it be to incorporate librsync into ruby and/or rubygems?
> 
> I don't know how crazy.  Is it commonly available (on all platforms)?

It's not available on windows without some user intervention, but of 
course if it were shipped with Ruby that may resolve that.

Either way, we'd need to find some library to wrap in JRuby. I'm sure 
there's several rsync libs out there, but it's a bit of hassle (and 
another dependency).

- Charlie

From jim at weirichhouse.org  Fri Jun  1 08:24:53 2007
From: jim at weirichhouse.org (Jim Weirich)
Date: Fri, 01 Jun 2007 08:24:53 -0400
Subject: [Rubygems-developers] New Beta Gem with Better Memory Footprint
	(was: Bulk updates)
In-Reply-To: <37C4EA5E-1C1F-48DF-BE66-2E696005651B@zenspider.com>
References: <d56373190705300328r542e2e6fyf9f54a81845b2173@mail.gmail.com>	<4A9E73EE-3E55-42E5-B245-88B72BF01FDC@segment7.net>	<d56373190705300804r5204dcb5g664f2ad24d284519@mail.gmail.com>	<3538E783-B1EF-4087-9810-A13D43F53DAA@segment7.net>
	<37C4EA5E-1C1F-48DF-BE66-2E696005651B@zenspider.com>
Message-ID: <46601015.7070604@weirichhouse.org>

Ryan Davis wrote:
 > I just committed a change that seems to cut memory use by about 40ish
 > %. There was a phase that was munging the output to deal with an old
 > yaml bug and was creating quite a bit of memory. Removing it dropped
 > my memory consumption from 112meg to 73meg. We could drop a tad more
 > if we nuked the part of the cache for the source we were currently
 > bulk loading, but it only equated to about 10 meg in my experiments.

I've taken Ryan's changes (removing a copy step in the bulk download) 
and added a few more for a beta release of RubyGems.

Here are the changes:

* Bulk downloads no longer copy the YAML string before loading it.
* Bulk downloads don't kick in until more than 500 gems need resyncing
   (the old limit was 50).
* Users may set the bulk download threshhold in the command line (-B)
   or in the .gemrc file.

Here are the memory stats using Ruby 1.8.5 on Mac OSX:

Old Version Bulk:       107Mb Real,  146Mb Virtual
New Version Bulk:        98Mb Real,  130Mb Virtual
New Version Incremental  36Mb Real,   63Mb Virtual

It looks like the bulk download memory footprint is rather modest. I
don't know if it will be enough for those who are currently feeling
the hurt.

However, by specifying a bulk download threshhold of 100000, you can
force gems to use the incremental method all the time. The downside to
using incremental downloads is that it is *much* slower for large
updates (however, if bulk downloads make you swap, then who knows).

On my machine, a bulk update takes about 20 seconds. An
incremental update takes about 0.2 seconds per out-of-date gem. If you
clobber the cache and update everything (currently 6600 some odd
gems), the incremental update takes about 20 minutes. My calculations
(based on my home network connection, the current size of the gem
index and many other things too numerous to mention) puts the break
even point at around 100 gems.

I would like those who are experiencing memory problems to give the
beta version a try and report back. You can get the beta version using
(using sudo if necessary on your system):

    gem update --system --source http://onestepback.org/betagems

Of course, if you are having problems due to memory, the above might
not work. You can download the update explicitly using the following
URL:

    http://onestepback.org/betagems/gems/rubygems-update-0.9.4.1.gem

and then install locally.

To change the threshhold value in the beta, use the -B option. For
example, the following command will incrementally synchronize up to a
million out of date gems:

    gem list -r -B 1000000

I have one more idea on memory footprint issues, but would like to get
reports back on the beta before proceeding.

Thanks to everyone for their feedback.

-- 
-- Jim Weirich      jim at weirichhouse.org     http://onestepback.org
-- In theory, practice and theory are the same.
-- In practice, they are different.

From ryand-ruby at zenspider.com  Fri Jun  1 11:36:27 2007
From: ryand-ruby at zenspider.com (Ryan Davis)
Date: Fri, 1 Jun 2007 11:36:27 -0400
Subject: [Rubygems-developers] New Beta Gem with Better Memory Footprint
	(was: Bulk updates)
In-Reply-To: <46601015.7070604@weirichhouse.org>
References: <d56373190705300328r542e2e6fyf9f54a81845b2173@mail.gmail.com>	<4A9E73EE-3E55-42E5-B245-88B72BF01FDC@segment7.net>	<d56373190705300804r5204dcb5g664f2ad24d284519@mail.gmail.com>	<3538E783-B1EF-4087-9810-A13D43F53DAA@segment7.net>
	<37C4EA5E-1C1F-48DF-BE66-2E696005651B@zenspider.com>
	<46601015.7070604@weirichhouse.org>
Message-ID: <FEB58F06-63C2-470E-B6C5-CA2F91508AD1@zenspider.com>


On Jun 1, 2007, at 08:24 , Jim Weirich wrote:

> I've taken Ryan's changes (removing a copy step in the bulk download)
> and added a few more for a beta release of RubyGems.
>
> Here are the changes:
>
> * Bulk downloads no longer copy the YAML string before loading it.
> * Bulk downloads don't kick in until more than 500 gems need resyncing
>    (the old limit was 50).
> * Users may set the bulk download threshhold in the command line (-B)
>    or in the .gemrc file.

Is this going to thrash rubyforge.org with the number of connections?

> Here are the memory stats using Ruby 1.8.5 on Mac OSX:
>
> Old Version Bulk:       107Mb Real,  146Mb Virtual
> New Version Bulk:        98Mb Real,  130Mb Virtual
> New Version Incremental  36Mb Real,   63Mb Virtual

For the record, I got much better numbers than this. I'm not sure  
what the differentiating factor is. I had 120Mb -> 73Mb for the first  
two steps.

I'm not sure how/why the incremental would be that much better. I  
proposed sticking with bulk but letting go of the previous cache for  
that server so it can be GC'd. I think this would be a good balance  
between # of socket connections and memory consumption.

Also Jim, I just committed a change to properly skip those files with  
the bad names. Please scoop that up as well.


From jim.weirich at gmail.com  Fri Jun  1 13:24:59 2007
From: jim.weirich at gmail.com (Jim Weirich)
Date: Fri, 1 Jun 2007 13:24:59 -0400
Subject: [Rubygems-developers] New Beta Gem with Better Memory Footprint
	(was: Bulk updates)
In-Reply-To: <FEB58F06-63C2-470E-B6C5-CA2F91508AD1@zenspider.com>
References: <d56373190705300328r542e2e6fyf9f54a81845b2173@mail.gmail.com>
	<4A9E73EE-3E55-42E5-B245-88B72BF01FDC@segment7.net>
	<d56373190705300804r5204dcb5g664f2ad24d284519@mail.gmail.com>
	<3538E783-B1EF-4087-9810-A13D43F53DAA@segment7.net>
	<37C4EA5E-1C1F-48DF-BE66-2E696005651B@zenspider.com>
	<46601015.7070604@weirichhouse.org>
	<FEB58F06-63C2-470E-B6C5-CA2F91508AD1@zenspider.com>
Message-ID: <d4dd9080706011024x35a64972nf125f12d39a4233e@mail.gmail.com>

On 6/1/07, Ryan Davis <ryand-ruby at zenspider.com> wrote:
>
> On Jun 1, 2007, at 08:24 , Jim Weirich wrote:
>
> > I've taken Ryan's changes (removing a copy step in the bulk download)
> > and added a few more for a beta release of RubyGems.
> >
> > Here are the changes:
> >
> > * Bulk downloads no longer copy the YAML string before loading it.
> > * Bulk downloads don't kick in until more than 500 gems need resyncing
> >    (the old limit was 50).
> > * Users may set the bulk download threshhold in the command line (-B)
> >    or in the .gemrc file.
>
> Is this going to thrash rubyforge.org with the number of connections?

I don't recall making the connections concurrent ... although I
thought about it.  If it is non-concurrent, then at any given time you
don't have any more connections, its just that the lifetime of the
connections are shorter.

> For the record, I got much better numbers than this. I'm not sure
> what the differentiating factor is. I had 120Mb -> 73Mb for the first
> two steps.

I think memory footprint measurements are still a black art.  FWIW, I
added a --bench command to gem that will pause it just before it exits
so you can get the final number from activity monitor.

> I'm not sure how/why the incremental would be that much better. I
> proposed sticking with bulk but letting go of the previous cache for
> that server so it can be GC'd. I think this would be a good balance
> between # of socket connections and memory consumption.

Although not in the beta, I did try this and didn't see any marked
difference.  Of course, I could have done something wrong, so please
double check it.

> Also Jim, I just committed a change to properly skip those files with
> the bad names. Please scoop that up as well.

Will do.  Thanks.

-- 
-- Jim Weirich    jim at weirichhouse.org     http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

From djberg96 at gmail.com  Fri Jun  1 15:42:35 2007
From: djberg96 at gmail.com (Daniel Berger)
Date: Fri, 1 Jun 2007 13:42:35 -0600
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu consumption
In-Reply-To: <465FDE52.2040807@sun.com>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
Message-ID: <6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>

On 6/1/07, Charles Oliver Nutter <charles.nutter at sun.com> wrote:
> Jim Weirich wrote:
> > On 5/27/07, Ryan Davis <ryand-ruby at zenspider.com> wrote:
> >> On May 24, 2007, at 16:19 , Eric Hodel wrote:
> >>
> >>> Jim and I were talking about trying incremental upload all the time,
> >>> rather than bulk at RailsConf...
> >> How crazy would it be to incorporate librsync into ruby and/or rubygems?
> >
> > I don't know how crazy.  Is it commonly available (on all platforms)?
>
> It's not available on windows without some user intervention, but of
> course if it were shipped with Ruby that may resolve that.
>
> Either way, we'd need to find some library to wrap in JRuby. I'm sure
> there's several rsync libs out there, but it's a bit of hassle (and
> another dependency).

It's not on Solaris 10, either, though there's a Blastwave package for it.

I looked on CPAN for a pure Perl version, but all I could find were
command line wrappers. On the Python front I did come across this:

http://www.vdesmedt.com/~vds2212/rsync.html

I don't know Python very well, however, so if there's an ex-pythonista
out there who's willing to give porting that a shot, it could prove to
be very handy. :)

Regards,

Dan

From nicksieger at gmail.com  Fri Jun  1 17:18:43 2007
From: nicksieger at gmail.com (Nick Sieger)
Date: Fri, 1 Jun 2007 16:18:43 -0500
Subject: [Rubygems-developers] Gem not found w/ no source cache file
Message-ID: <f0b861390706011418m23755702ua35a83241d4b579b@mail.gmail.com>

Some of you may know about this longstanding issue (at least since
0.9.2), where a clean, initial install with no pre-existing
source_cache file causes no gems to be found:

http://rubyforge.org/tracker/index.php?func=detail&aid=9679&group_id=126&atid=575

$ sudo rm /usr/lib/ruby/gems/1.8/source_cache
Password:
$ sudo gem install rake
Bulk updating Gem source index for: http://gems.rubyforge.org
ERROR:  While executing gem ... (Gem::GemNotFoundException)
    Could not find rake (> 0) in any repository

I found a simple solution to this, and the patch is attached.  It's
also in the tracker issue, but I don't know how often tracker is
checked.  Explanation in the tracker.

Cheers,
/Nick

From nicksieger at gmail.com  Fri Jun  1 17:21:41 2007
From: nicksieger at gmail.com (Nick Sieger)
Date: Fri, 1 Jun 2007 16:21:41 -0500
Subject: [Rubygems-developers] Gem not found w/ no source cache file
In-Reply-To: <f0b861390706011418m23755702ua35a83241d4b579b@mail.gmail.com>
References: <f0b861390706011418m23755702ua35a83241d4b579b@mail.gmail.com>
Message-ID: <f0b861390706011421o4ab08661lee32d44e71da063f@mail.gmail.com>

On 6/1/07, Nick Sieger <nicksieger at gmail.com> wrote:

> I found a simple solution to this, and the patch is attached.  It's
> also in the tracker issue, but I don't know how often tracker is
> checked.  Explanation in the tracker.

Err, it's attached now :P

/Nick
-------------- next part --------------
A non-text attachment was scrubbed...
Name: si_cache.patch
Type: text/x-patch
Size: 399 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/rubygems-developers/attachments/20070601/a96ba1de/attachment.bin 

From self at mattmower.com  Fri Jun  1 17:33:09 2007
From: self at mattmower.com (Matt Mower)
Date: Fri, 1 Jun 2007 22:33:09 +0100
Subject: [Rubygems-developers] New Beta Gem with Better Memory Footprint
	(was: Bulk updates)
In-Reply-To: <46601015.7070604@weirichhouse.org>
References: <d56373190705300328r542e2e6fyf9f54a81845b2173@mail.gmail.com>
	<4A9E73EE-3E55-42E5-B245-88B72BF01FDC@segment7.net>
	<d56373190705300804r5204dcb5g664f2ad24d284519@mail.gmail.com>
	<3538E783-B1EF-4087-9810-A13D43F53DAA@segment7.net>
	<37C4EA5E-1C1F-48DF-BE66-2E696005651B@zenspider.com>
	<46601015.7070604@weirichhouse.org>
Message-ID: <d56373190706011433p4dcd964ay17b9f8204c21b47b@mail.gmail.com>

On 01/06/07, Jim Weirich <jim at weirichhouse.org> wrote:
> I have one more idea on memory footprint issues, but would like to get
> reports back on the beta before proceeding.
>

I updated to 0.9.4.1 on my 96mb VPS and tried using 'gem update' again.

It began a bulk update but the ruby process quickly hit 75mb RSS and
then appeared to descend into swap-death as before.

Then I ran an incremental update using -B. It updated about 6,600 gems
and, as you expected, it was pretty slow. Nevertheless the process
grew very slowly from 15 to about 28mb and completed successfully this
time!

I'm grateful to you and Ryan both for the effort you are putting into
this problem.

Regards,

Matt

-- 
Matt Mower :: http://matt.blogs.it/

From Daniel.Berger at qwest.com  Fri Jun  1 17:31:16 2007
From: Daniel.Berger at qwest.com (Berger, Daniel)
Date: Fri, 1 Jun 2007 16:31:16 -0500
Subject: [Rubygems-developers] Gem not found w/ no source cache file
In-Reply-To: <f0b861390706011421o4ab08661lee32d44e71da063f@mail.gmail.com>
Message-ID: <7524A45A1A5B264FA4809E2156496CFBE72C4D@ITOMAE2KM01.AD.QINTRA.COM>

> -----Original Message-----
> From: rubygems-developers-bounces at rubyforge.org 
> [mailto:rubygems-developers-bounces at rubyforge.org] On Behalf 
> Of Nick Sieger
> Sent: Friday, June 01, 2007 3:22 PM
> To: rubygems-developers at rubyforge.org
> Subject: Re: [Rubygems-developers] Gem not found w/ no source 
> cache file
> 
> 
> On 6/1/07, Nick Sieger <nicksieger at gmail.com> wrote:
> 
> > I found a simple solution to this, and the patch is attached.  It's 
> > also in the tracker issue, but I don't know how often tracker is 
> > checked.  Explanation in the tracker.
> 
> Err, it's attached now :P

Just in case, I recommend binary mode:

File.open(fn, "wb") {|f| f << Marshal.dump({}) }

Regards,

Dan


This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly 
prohibited and may be unlawful.  If you have received this communication 
in error, please immediately notify the sender by reply e-mail and destroy 
all copies of the communication and any attachments.

From jim.weirich at gmail.com  Fri Jun  1 18:50:58 2007
From: jim.weirich at gmail.com (Jim Weirich)
Date: Fri, 1 Jun 2007 18:50:58 -0400
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu consumption
In-Reply-To: <6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
	<6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
Message-ID: <d4dd9080706011550o7ac48ae3iafc039f49e9378d4@mail.gmail.com>

On 6/1/07, Daniel Berger <djberg96 at gmail.com> wrote:
> I looked on CPAN for a pure Perl version, but all I could find were
> command line wrappers. On the Python front I did come across this:
>
> http://www.vdesmedt.com/~vds2212/rsync.html
>
> I don't know Python very well, however, so if there's an ex-pythonista
> out there who's willing to give porting that a shot, it could prove to
> be very handy. :)

If a pure ruby client can be found/written, that would go a long way
to making this approach viable.

Currently a static web page server is all that is needed to make a
full fledged gem server.  If we add support for rsync, I would still
like to maintain compatibility with regular static servers.

-- 
-- Jim Weirich    jim at weirichhouse.org     http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

From jos at catnook.com  Fri Jun  1 19:32:40 2007
From: jos at catnook.com (Jos Backus)
Date: Fri, 1 Jun 2007 16:32:40 -0700
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu	consumption
In-Reply-To: <6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
	<6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
Message-ID: <20070601233240.GB97713@lizzy.catnook.local>

On Fri, Jun 01, 2007 at 01:42:35PM -0600, Daniel Berger wrote:
> It's not on Solaris 10, either, though there's a Blastwave package for it.
> 
> I looked on CPAN for a pure Perl version, but all I could find were
> command line wrappers. On the Python front I did come across this:
> 
> http://www.vdesmedt.com/~vds2212/rsync.html
> 
> I don't know Python very well, however, so if there's an ex-pythonista
> out there who's willing to give porting that a shot, it could prove to
> be very handy. :)
 
This is like xcopy with rsync-like options processing. You want (a Ruby
version of) Pysync instead:

    http://freshmeat.net/projects/pysync/

-- 
Jos Backus
jos at catnook.com

From jim.weirich at gmail.com  Sat Jun  2 00:35:02 2007
From: jim.weirich at gmail.com (Jim Weirich)
Date: Sat, 2 Jun 2007 00:35:02 -0400
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu consumption
In-Reply-To: <20070601233240.GB97713@lizzy.catnook.local>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
	<6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
	<20070601233240.GB97713@lizzy.catnook.local>
Message-ID: <d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>

On 6/1/07, Jos Backus <jos at catnook.com> wrote:
> This is like xcopy with rsync-like options processing. You want (a Ruby
> version of) Pysync instead:
>
>     http://freshmeat.net/projects/pysync/

Can someone educate me on the advantages of rsync?

-- 
-- Jim Weirich    jim at weirichhouse.org     http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

From jim.weirich at gmail.com  Sat Jun  2 01:02:03 2007
From: jim.weirich at gmail.com (Jim Weirich)
Date: Sat, 2 Jun 2007 01:02:03 -0400
Subject: [Rubygems-developers] Gem not found w/ no source cache file
In-Reply-To: <f0b861390706011418m23755702ua35a83241d4b579b@mail.gmail.com>
References: <f0b861390706011418m23755702ua35a83241d4b579b@mail.gmail.com>
Message-ID: <d4dd9080706012202p4339e4cu95153415f89d4edc@mail.gmail.com>

On 6/1/07, Nick Sieger <nicksieger at gmail.com> wrote:
> I found a simple solution to this, and the patch is attached.  It's
> also in the tracker issue, but I don't know how often tracker is
> checked.  Explanation in the tracker.

Thanks.  I just noticed this bug during my memory testing.  I was
about to investigate when this patch showed up.  What timing.

I've applied the patch and it will be included in the next version
(with binary mode Dan :).

-- 
-- Jim Weirich    jim at weirichhouse.org     http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

From jos at catnook.com  Sat Jun  2 03:53:54 2007
From: jos at catnook.com (Jos Backus)
Date: Sat, 2 Jun 2007 00:53:54 -0700
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu	consumption
In-Reply-To: <d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
	<6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
	<20070601233240.GB97713@lizzy.catnook.local>
	<d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>
Message-ID: <20070602075354.GA84122@lizzy.catnook.local>

On Sat, Jun 02, 2007 at 12:35:02AM -0400, Jim Weirich wrote:
>  On 6/1/07, Jos Backus <jos at catnook.com> wrote:
> > This is like xcopy with rsync-like options processing. You want (a Ruby
> > version of) Pysync instead:
> >
> >     http://freshmeat.net/projects/pysync/
> 
>  Can someone educate me on the advantages of rsync?

In the context of Rubygems: I don't know what they are, if any. But pysync.rb
encodes the basic algorithm and should not be too hard to convert to Ruby if
needed. I was merely supplying a pointer to an rsync implementation that could
function as a starting point.

-- 
Jos Backus
jos at catnook.com

From ryand-ruby at zenspider.com  Sat Jun  2 06:07:00 2007
From: ryand-ruby at zenspider.com (Ryan Davis)
Date: Sat, 2 Jun 2007 06:07:00 -0400
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu consumption
In-Reply-To: <d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
	<6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
	<20070601233240.GB97713@lizzy.catnook.local>
	<d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>
Message-ID: <83DF2DC9-D8B6-48DD-B7F3-136EFB1890C7@zenspider.com>


On Jun 2, 2007, at 00:35 , Jim Weirich wrote:

> Can someone educate me on the advantages of rsync?

Throw out all our code and yet get bulk and incremental syncing all  
in one. triple win.


From rubygems at freeze.org  Sat Jun  2 08:59:41 2007
From: rubygems at freeze.org (Jim Freeze)
Date: Sat, 2 Jun 2007 07:59:41 -0500
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu consumption
In-Reply-To: <d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
	<6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
	<20070601233240.GB97713@lizzy.catnook.local>
	<d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>
Message-ID: <70C24745-9AEF-4FA8-BA31-F6BA0AE33DA0@freeze.org>

On Jun 1, 2007, at 11:35 PM, Jim Weirich wrote:

> On 6/1/07, Jos Backus <jos at catnook.com> wrote:
>> This is like xcopy with rsync-like options processing. You want (a  
>> Ruby
>> version of) Pysync instead:
>>
>>     http://freshmeat.net/projects/pysync/
>
> Can someone educate me on the advantages of rsync?

rsync is not nearly as well known and used at it should be.
IMO, it could essentially replace cp and scp.

Rsync does a fast hash on small sections of a file to test for changes
and only copies the sections of a file that have changed.

So, if you had a 100MB binary file on two computers and changed
only 100 bytes on one of them (in the middle of the file), rsync would
be able to transfer that 100 bytes, and some additional overhead, to
synchronize the files. It would not have to transfer the 100MB.

Also, it has multiple options, like compression and BW limiting to
support slow pipes.

Once you start using rsync, you wonder how you ever got along without  
it. :)

Jim



>
> -- 
> -- Jim Weirich    jim at weirichhouse.org     http://onestepback.org
> -----------------------------------------------------------------
> "Beware of bugs in the above code; I have only proved it correct,
> not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
> _______________________________________________
> Rubygems-developers mailing list
> Rubygems-developers at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rubygems-developers


From darix at web.de  Sat Jun  2 09:27:40 2007
From: darix at web.de (Marcus Rueckert)
Date: Sat, 2 Jun 2007 15:27:40 +0200
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu consumption
In-Reply-To: <70C24745-9AEF-4FA8-BA31-F6BA0AE33DA0@freeze.org>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
	<6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
	<20070601233240.GB97713@lizzy.catnook.local>
	<d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>
	<70C24745-9AEF-4FA8-BA31-F6BA0AE33DA0@freeze.org>
Message-ID: <20070602132740.GC3490@pixel.global-banlist.de>

On 2007-06-02 07:59:41 -0500, Jim Freeze wrote:
> On Jun 1, 2007, at 11:35 PM, Jim Weirich wrote:
> 
> > On 6/1/07, Jos Backus <jos at catnook.com> wrote:
> >> This is like xcopy with rsync-like options processing. You want (a  
> >> Ruby
> >> version of) Pysync instead:
> >>
> >>     http://freshmeat.net/projects/pysync/
> >
> > Can someone educate me on the advantages of rsync?
> 
> rsync is not nearly as well known and used at it should be.
> IMO, it could essentially replace cp and scp.
> 
> Rsync does a fast hash on small sections of a file to test for changes
> and only copies the sections of a file that have changed.
> 
> So, if you had a 100MB binary file on two computers and changed
> only 100 bytes on one of them (in the middle of the file), rsync would
> be able to transfer that 100 bytes, and some additional overhead, to
> synchronize the files. It would not have to transfer the 100MB.
> 
> Also, it has multiple options, like compression and BW limiting to
> support slow pipes.
> 
> Once you start using rsync, you wonder how you ever got along without  
> it. :)

zsync might be more interesting in the end. it shares many features with
rsync but doesnt require a special server. a simple .zsync file and a
plain http server are enough:

http://zsync.moria.org.uk/

    darix

-- 
           openSUSE - SUSE Linux is my linux
               openSUSE is good for you
                   www.opensuse.org

From rubygems at freeze.org  Sat Jun  2 10:36:12 2007
From: rubygems at freeze.org (Jim Freeze)
Date: Sat, 2 Jun 2007 09:36:12 -0500
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu consumption
In-Reply-To: <20070602132740.GC3490@pixel.global-banlist.de>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
	<6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
	<20070601233240.GB97713@lizzy.catnook.local>
	<d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>
	<70C24745-9AEF-4FA8-BA31-F6BA0AE33DA0@freeze.org>
	<20070602132740.GC3490@pixel.global-banlist.de>
Message-ID: <C877B270-D90F-4F87-A88A-B572CC1E5192@freeze.org>

On Jun 2, 2007, at 8:27 AM, Marcus Rueckert wrote:

> On 2007-06-02 07:59:41 -0500, Jim Freeze wrote:
>> On Jun 1, 2007, at 11:35 PM, Jim Weirich wrote:
>>
>>> On 6/1/07, Jos Backus <jos at catnook.com> wrote:
>>>> This is like xcopy with rsync-like options processing. You want (a
>>>> Ruby
>>>> version of) Pysync instead:
>>>>
>>>>     http://freshmeat.net/projects/pysync/
>>>
>>> Can someone educate me on the advantages of rsync?
>
> zsync might be more interesting in the end. it shares many features  
> with
> rsync but doesnt require a special server. a simple .zsync file and a
> plain http server are enough:

Hmm, haven't looked at zsync, but rsync does not require a server  
either.
But, it must be installed on both machines.

Jim

> http://zsync.moria.org.uk/
>
>     darix
>
> -- 
>            openSUSE - SUSE Linux is my linux
>                openSUSE is good for you
>                    www.opensuse.org
> _______________________________________________
> Rubygems-developers mailing list
> Rubygems-developers at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rubygems-developers


From darix at web.de  Sat Jun  2 13:14:45 2007
From: darix at web.de (Marcus Rueckert)
Date: Sat, 2 Jun 2007 19:14:45 +0200
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu consumption
In-Reply-To: <C877B270-D90F-4F87-A88A-B572CC1E5192@freeze.org>
References: <1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
	<6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
	<20070601233240.GB97713@lizzy.catnook.local>
	<d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>
	<70C24745-9AEF-4FA8-BA31-F6BA0AE33DA0@freeze.org>
	<20070602132740.GC3490@pixel.global-banlist.de>
	<C877B270-D90F-4F87-A88A-B572CC1E5192@freeze.org>
Message-ID: <20070602171445.GD3490@pixel.global-banlist.de>

On 2007-06-02 09:36:12 -0500, Jim Freeze wrote:
> Hmm, haven't looked at zsync, but rsync does not require a server  
> either.
> But, it must be installed on both machines.

that means either rsync or rsync over ssh. the first requires rsync
running via inetd. the later ssh login.

so for a mirror, i assume you wont get ssh login, it requires the
rsyncd. that is a special daemon.

no?

    darix

-- 
           openSUSE - SUSE Linux is my linux
               openSUSE is good for you
                   www.opensuse.org

From thewoolleyman at gmail.com  Sat Jun  2 18:54:50 2007
From: thewoolleyman at gmail.com (Chad Woolley)
Date: Sat, 2 Jun 2007 15:54:50 -0700
Subject: [Rubygems-developers] gem update --system,
	heavy ram and cpu consumption
In-Reply-To: <70C24745-9AEF-4FA8-BA31-F6BA0AE33DA0@freeze.org>
References: <4655FA6B.7050609@qwest.com>
	<1996CD69-1EEC-4EA7-B0B6-C43AA3E2DF7C@segment7.net>
	<0B05280D-85B7-4332-85ED-AB4AB9C23DED@zenspider.com>
	<d4dd9080705271723k6037f9ccw14020311aedee428@mail.gmail.com>
	<465FDE52.2040807@sun.com>
	<6037b70c0706011242w105ea674kf43a5470e02ecd12@mail.gmail.com>
	<20070601233240.GB97713@lizzy.catnook.local>
	<d4dd9080706012135x22c031d5ha91c552496fcb6b5@mail.gmail.com>
	<70C24745-9AEF-4FA8-BA31-F6BA0AE33DA0@freeze.org>
Message-ID: <e3af97510706021554q7dc2a9aamea2d4541e21f572a@mail.gmail.com>

On 6/2/07, Jim Freeze <rubygems at freeze.org> wrote:
> rsync is not nearly as well known and used at it should be.
> IMO, it could essentially replace cp and scp.
>
> Also, it has multiple options, like compression and BW limiting to
> support slow pipes.

Don't forget --delete, to clean up obsolete files on the destination.

From jeremy at hinegardner.org  Sun Jun  3 04:23:23 2007
From: jeremy at hinegardner.org (Jeremy Hinegardner)
Date: Sun, 3 Jun 2007 02:23:23 -0600
Subject: [Rubygems-developers] Patch for mkrf/rakefile based extension
	failing to install
Message-ID: <20070603082323.GO18144@hinegardner.org>

Hi,

I'm working on a gem with exensions and decided to try out mkrf.  It
seems to work just fine but gem fails to install.  The bug appears to be
in ExtRakeBuilder.build I have uploaded a patch to rubyforge.  

    http://rubyforge.org/tracker/index.php?func=detail&aid=11313&group_id=126&atid=575

I'm assuming that a gem developer will declare an mkrf extension as:

    spec = Gem::Specification.new do |s|
        #...
        s.extensions << "ext/path/to/mkrf_conf.rb"
        #...
    end

If I am incorrect in my assumptions, please let me know.  I'd like to
plan on releasing all my extension based gems with mkrf.

enjoy,

-jeremy

--
========================================================================
 Jeremy Hinegardner                              jeremy at hinegardner.org 


From alltom at gmail.com  Tue Jun  5 22:29:51 2007
From: alltom at gmail.com (Tom Lieber)
Date: Wed, 6 Jun 2007 11:29:51 +0900
Subject: [Rubygems-developers] 0.9.4 setup.rb on Gentoo hangs
Message-ID: <326364c20706051929l5a0a6f6bub4d7d79221deefbe@mail.gmail.com>

I'm trying to install RubyGems on Gentoo. I first tried with emerge,
but that install hung, so I tried by installing from the the 0.9.4
tarball. This is where it stops:

mkdir -p /usr/lib/ruby/site_ruby/1.8/rbconfig
install datadir.rb /usr/lib/ruby/site_ruby/1.8/rbconfig
<--- lib/rbconfig
<--- lib
  Successfully built RubyGem
  Name: sources
  Version: 0.0.1
  File: sources-0.0.1.gem

When I Ctrl+C, I get this stack trace:

/usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:335:in `initialize': Interrupt
        from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:446:in `new'
        from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:446:in `each_entry'
        from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:442:in `loop'
        from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:442:in `each_entry'
        from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:427:in `each'
        from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:488:in `initialize'
        from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:586:in `new'
        from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:586:in
`open_from_io'
         ... 10 levels...
        from setup.rb:1000:in `exec_install'
        from setup.rb:814:in `invoke'
        from setup.rb:773:in `invoke'
        from setup.rb:1578

What could this be?

Sincerely,

Tom Lieber
http://AllTom.com/
http://GadgetLife.org/

From drbrain at segment7.net  Wed Jun  6 01:43:26 2007
From: drbrain at segment7.net (Eric Hodel)
Date: Tue, 5 Jun 2007 22:43:26 -0700
Subject: [Rubygems-developers] 0.9.4 setup.rb on Gentoo hangs
In-Reply-To: <326364c20706051929l5a0a6f6bub4d7d79221deefbe@mail.gmail.com>
References: <326364c20706051929l5a0a6f6bub4d7d79221deefbe@mail.gmail.com>
Message-ID: <16C539DE-2A1F-4F0E-9B4F-E3D0B843DC8B@segment7.net>

On Jun 5, 2007, at 19:29, Tom Lieber wrote:

> I'm trying to install RubyGems on Gentoo. I first tried with emerge,
> but that install hung, so I tried by installing from the the 0.9.4
> tarball. This is where it stops:
>
> mkdir -p /usr/lib/ruby/site_ruby/1.8/rbconfig
> install datadir.rb /usr/lib/ruby/site_ruby/1.8/rbconfig
> <--- lib/rbconfig
> <--- lib
>   Successfully built RubyGem
>   Name: sources
>   Version: 0.0.1
>   File: sources-0.0.1.gem
>
> When I Ctrl+C, I get this stack trace:
>
> /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:335:in  
> `initialize': Interrupt
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:446:in  
> `new'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:446:in  
> `each_entry'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:442:in  
> `loop'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:442:in  
> `each_entry'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:427:in  
> `each'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:488:in  
> `initialize'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:586:in  
> `new'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:586:in
> `open_from_io'
>          ... 10 levels...
>         from setup.rb:1000:in `exec_install'
>         from setup.rb:814:in `invoke'
>         from setup.rb:773:in `invoke'
>         from setup.rb:1578
>
> What could this be?

What does:

ruby -rrbconfig -e 'p Config::CONFIG["CFLAGS"]'

say?


From alltom at gmail.com  Wed Jun  6 08:21:06 2007
From: alltom at gmail.com (Tom Lieber)
Date: Wed, 6 Jun 2007 21:21:06 +0900
Subject: [Rubygems-developers] 0.9.4 setup.rb on Gentoo hangs
In-Reply-To: <16C539DE-2A1F-4F0E-9B4F-E3D0B843DC8B@segment7.net>
References: <326364c20706051929l5a0a6f6bub4d7d79221deefbe@mail.gmail.com>
	<16C539DE-2A1F-4F0E-9B4F-E3D0B843DC8B@segment7.net>
Message-ID: <326364c20706060521g9688a6fhda9df6dbfdb11e6c@mail.gmail.com>

On 6/6/07, Eric Hodel <drbrain at segment7.net> wrote:
> On Jun 5, 2007, at 19:29, Tom Lieber wrote:
> > I'm trying to install RubyGems on Gentoo. I first tried with emerge,
> > but that install hung, so I tried by installing from the the 0.9.4
> > tarball. This is where it stops:
> >
> > mkdir -p /usr/lib/ruby/site_ruby/1.8/rbconfig
> > install datadir.rb /usr/lib/ruby/site_ruby/1.8/rbconfig
> > <--- lib/rbconfig
> > <--- lib
> >   Successfully built RubyGem
> >   Name: sources
> >   Version: 0.0.1
> >   File: sources-0.0.1.gem
> >
> > When I Ctrl+C, I get this stack trace:
> >
> > /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:335:in
> > `initialize': Interrupt
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:446:in
> > `new'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:446:in
> > `each_entry'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:442:in
> > `loop'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:442:in
> > `each_entry'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:427:in
> > `each'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:488:in
> > `initialize'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:586:in
> > `new'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:586:in
> > `open_from_io'
> >          ... 10 levels...
> >         from setup.rb:1000:in `exec_install'
> >         from setup.rb:814:in `invoke'
> >         from setup.rb:773:in `invoke'
> >         from setup.rb:1578
> >
> > What could this be?
>
> What does:
>
> ruby -rrbconfig -e 'p Config::CONFIG["CFLAGS"]'
>
> say?

-Os -march=pentium4 -pipe  -fPIC

Sincerely,

Tom Lieber
http://AllTom.com/
http://GadgetLife.org/

From pclouds at gentoo.org  Wed Jun  6 08:52:25 2007
From: pclouds at gentoo.org (=?UTF-8?Q?Nguy=E1=BB=85n_Th=C3=A1i_Ng=E1=BB=8Dc_Duy?=)
Date: Wed, 6 Jun 2007 08:52:25 -0400
Subject: [Rubygems-developers] 0.9.4 setup.rb on Gentoo hangs
In-Reply-To: <326364c20706051929l5a0a6f6bub4d7d79221deefbe@mail.gmail.com>
References: <326364c20706051929l5a0a6f6bub4d7d79221deefbe@mail.gmail.com>
Message-ID: <fcaeb9bf0706060552v4ab51736v3187562526653504@mail.gmail.com>

On 6/5/07, Tom Lieber <alltom at gmail.com> wrote:
> I'm trying to install RubyGems on Gentoo. I first tried with emerge,
> but that install hung, so I tried by installing from the the 0.9.4
> tarball. This is where it stops:
>
> mkdir -p /usr/lib/ruby/site_ruby/1.8/rbconfig
> install datadir.rb /usr/lib/ruby/site_ruby/1.8/rbconfig
> <--- lib/rbconfig
> <--- lib
>   Successfully built RubyGem
>   Name: sources
>   Version: 0.0.1
>   File: sources-0.0.1.gem
>
> When I Ctrl+C, I get this stack trace:
>
> /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:335:in `initialize': Interrupt
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:446:in `new'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:446:in `each_entry'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:442:in `loop'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:442:in `each_entry'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:427:in `each'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:488:in `initialize'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:586:in `new'
>         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:586:in
> `open_from_io'
>          ... 10 levels...
>         from setup.rb:1000:in `exec_install'
>         from setup.rb:814:in `invoke'
>         from setup.rb:773:in `invoke'
>         from setup.rb:1578
>
> What could this be?

I didn't read through but you might ran into this bug:

https://bugs.gentoo.org/show_bug.cgi?id=161566
-- 
Duy

From alltom at gmail.com  Wed Jun  6 10:03:31 2007
From: alltom at gmail.com (Tom Lieber)
Date: Wed, 6 Jun 2007 23:03:31 +0900
Subject: [Rubygems-developers] 0.9.4 setup.rb on Gentoo hangs
In-Reply-To: <fcaeb9bf0706060550o2555bb25td369c5593732c0a8@mail.gmail.com>
References: <326364c20706051929l5a0a6f6bub4d7d79221deefbe@mail.gmail.com>
	<fcaeb9bf0706060550o2555bb25td369c5593732c0a8@mail.gmail.com>
Message-ID: <326364c20706060703q7d3f574hd3a8cbb2c8161e02@mail.gmail.com>

On 6/6/07, Nguyen Thai Ngoc Duy <pclouds at gmail.com> wrote:
> On 6/5/07, Tom Lieber <alltom at gmail.com> wrote:
> > I'm trying to install RubyGems on Gentoo. I first tried with emerge,
> > but that install hung, so I tried by installing from the the 0.9.4
> > tarball. This is where it stops:
> >
> > mkdir -p /usr/lib/ruby/site_ruby/1.8/rbconfig
> > install datadir.rb /usr/lib/ruby/site_ruby/1.8/rbconfig
> > <--- lib/rbconfig
> > <--- lib
> >   Successfully built RubyGem
> >   Name: sources
> >   Version: 0.0.1
> >   File: sources-0.0.1.gem
> >
> > When I Ctrl+C, I get this stack trace:
> >
> > /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:335:in `initialize': Interrupt
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:446:in `new'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:446:in `each_entry'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:442:in `loop'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:442:in `each_entry'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:427:in `each'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:488:in `initialize'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:586:in `new'
> >         from /usr/lib/ruby/site_ruby/1.8/rubygems/package.rb:586:in
> > `open_from_io'
> >          ... 10 levels...
> >         from setup.rb:1000:in `exec_install'
> >         from setup.rb:814:in `invoke'
> >         from setup.rb:773:in `invoke'
> >         from setup.rb:1578
> >
> > What could this be?
>
> I didn't read through but you might ran into this bug:
>
> https://bugs.gentoo.org/show_bug.cgi?id=161566

That looks like it. Thanks.

Sincerely,

Tom Lieber
http://AllTom.com/
http://GadgetLife.org/

From drbrain at segment7.net  Wed Jun  6 15:58:53 2007
From: drbrain at segment7.net (Eric Hodel)
Date: Wed, 6 Jun 2007 12:58:53 -0700
Subject: [Rubygems-developers] 0.9.4 setup.rb on Gentoo hangs
In-Reply-To: <326364c20706060521g9688a6fhda9df6dbfdb11e6c@mail.gmail.com>
References: <326364c20706051929l5a0a6f6bub4d7d79221deefbe@mail.gmail.com>
	<16C539DE-2A1F-4F0E-9B4F-E3D0B843DC8B@segment7.net>
	<326364c20706060521g9688a6fhda9df6dbfdb11e6c@mail.gmail.com>
Message-ID: <6704117D-AB0B-408F-88EB-F9E25EAA289D@segment7.net>

On Jun 6, 2007, at 05:21, Tom Lieber wrote:
> On 6/6/07, Eric Hodel <drbrain at segment7.net> wrote:
>> On Jun 5, 2007, at 19:29, Tom Lieber wrote:
>> > What could this be?
>>
>> What does:
>>
>> ruby -rrbconfig -e 'p Config::CONFIG["CFLAGS"]'
>>
>> say?
>
> -Os -march=pentium4 -pipe  -fPIC

Ok.  Making sure you weren't compiling with something that added - 
fomit-frame-pointer, which I've also seen cause problems in the tar- 
handling code.

From randy.j.parker at gmail.com  Thu Jun  7 13:27:23 2007
From: randy.j.parker at gmail.com (Randy Parker)
Date: Thu, 7 Jun 2007 13:27:23 -0400
Subject: [Rubygems-developers] Installing Rubygems in User's Local Directory
	to fix Gem on Memory-Limited Systems
Message-ID: <735c8b860706071027n599aa573p72efedb6fcce4db3@mail.gmail.com>

These two bugs currently prevent all memory-restricted shared hosting users
from using "gem install" or "gem update".
*
*[ support-Bugs-11406 ] "gem install" takes more memory than some hosting
companies allow

[ rubygems-Bugs-10491 ] "gem update" fails with SEG FAULT while trying to
update it's gem list from the server

Several threads on this list deal with the problem:
http://rubyforge.org/pipermail/rubygems-developers/2007-May/002789.html
http://rubyforge.org/pipermail/rubygems-developers/2007-May/002798.html
http://rubyforge.org/pipermail/rubygems-developers/2007-May/002803.html

Jim produced a beta gem that reduces the problem - rubygems-update-0.9.4.2:
http://rubyforge.org/pipermail/rubygems-developers/2007-June/002811.html

I don't think we yet know what fraction of the victims can use gem again
with this improvement.  It depends on what the hoster's memory limits are,
and how they are implemented.  Some hosters sell accounts will with less
than 48MB.  I'm trying to test three hosters myself.  In the long run,
several other approaches have been discussed, perhaps using some kind of
rsync.

The only way to use the fix is to "gem update --system --source
http://onestepback.org/betagems".  This requires the shared hosting gem
environment to be configured so that the "gem system" dirs are in the user's
local account.  I'm having trouble with that.

shell-prompt$ gem env
RubyGems Environment:
  - VERSION: 0.9.2 (0.9.2)
  - INSTALLATION DIRECTORY: /users/home/randy/gems
  - GEM PATH:
     - /users/home/randy/gems
     - /usr/local/lib/ruby/gems/1.8
  - REMOTE SOURCES:
     - http://gems.rubyforge.org


shell-prompt$ env | grep -e GEM -e RUBY
GEM_HOME=/users/home/randy/gems
GEM_PATH=/users/home/randy/gems:/usr/local/lib/ruby/gems/1.8
RUBYLIB=/users/home/randy/gems/lib/ruby/site_ruby/1.8:/usr/local/lib/ruby/site_ruby/1.8

shell-prompt$ gem update --system --source http://onestepback.org/betagems
Updating RubyGems...
Attempting remote update of rubygems-update
Successfully installed rubygems-update-0.9.4.2
Updating version of RubyGems to 0.9.4.2
Installing RubyGems 0.9.4.2
---> bin
<--- bin
---> lib
---> lib/rbconfig
<--- lib/rbconfig
---> lib/rubygems
---> lib/rubygems/digest
<--- lib/rubygems/digest
---> lib/rubygems/commands
<--- lib/rubygems/commands
<--- lib/rubygems
<--- lib
---> bin
updating shebang: gem
updating shebang: gemri
updating shebang: gemlock
updating shebang: index_gem_repository.rb
updating shebang: gem_server
updating shebang: gemwhich
updating shebang: gem_mirror
updating shebang: update_rubygems
<--- bin
---> lib
---> lib/rbconfig
<--- lib/rbconfig
---> lib/rubygems
---> lib/rubygems/digest
<--- lib/rubygems/digest
---> lib/rubygems/commands
<--- lib/rubygems/commands
<--- lib/rubygems
<--- lib
rm -f InstalledFiles
---> bin
mkdir -p /usr/local/bin/
install gem /usr/local/bin/
setup.rb:633:in `initialize': Permission denied - /usr/local/bin/gem
(Errno::EACCES)
        from setup.rb:633:in `open'
        from setup.rb:633:in `install'
        from setup.rb:1377:in `install_files'
        from setup.rb:1376:in `each'
        from setup.rb:1376:in `install_files'
        from setup.rb:1346:in `install_dir_bin'
        from setup.rb:1532:in `__send__'
        from setup.rb:1532:in `traverse'
         ... 6 levels...
        from setup.rb:1000:in `exec_install'
        from setup.rb:814:in `invoke'
        from setup.rb:773:in `invoke'
        from setup.rb:1578
RubyGems system software updated


So I also tried

gem update --system --source
http://onestepback.org/betagems--install-dir=$GEM_HOME

with the exact same effect.  After some searching, I found
[rubygems #8470] gem update --system should take the prefix at which
rubygems is installed into accountwhich advises

To workaround, switch to the downloaded rubygem update (e.g.
GEM_HOME/gems/rubygems-update-0.9.2/), then run the install sequence
again:

   ruby setup.rb config --prefix=$GEM_HOME
   ruby setup.rb setup
   ruby setup.rb install

I did this, and even added

   ruby setup.rb install --prefix=$GEM_HOME

But in all cases the install step insisted on trying (and naturally failing)
to install to the hosting computer's system dir, with the same line as
included above:

setup.rb:633:in `initialize': Permission denied - /usr/local/bin/gem
(Errno::EACCES)

Why is it attempting to install to the system when I'm telling it not to?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070607/8ee5851b/attachment-0001.html 

From drbrain at segment7.net  Thu Jun  7 14:49:47 2007
From: drbrain at segment7.net (Eric Hodel)
Date: Thu, 7 Jun 2007 11:49:47 -0700
Subject: [Rubygems-developers] Installing Rubygems in User's Local
	Directory to fix Gem on Memory-Limited Systems
In-Reply-To: <735c8b860706071027n599aa573p72efedb6fcce4db3@mail.gmail.com>
References: <735c8b860706071027n599aa573p72efedb6fcce4db3@mail.gmail.com>
Message-ID: <B751A3BA-78FD-4CD4-8E67-AA95B9B638B6@segment7.net>

On Jun 7, 2007, at 10:27, Randy Parker wrote:

> [...]
>
> After some searching, I found [rubygems #8470] gem update --system  
> should take the prefix at which rubygems is installed into  
> accountwhich advises
>
> To workaround, switch to the downloaded rubygem update ( e.g.
> GEM_HOME/gems/rubygems-update-0.9.2/), then run the install sequence
> again:
>
> But in all cases the install step insisted on trying (and naturally  
> failing) to install to the hosting computer's system dir, with the  
> same line as included above:
>
> setup.rb:633:in `initialize': Permission denied - /usr/local/bin/ 
> gem (Errno::EACCES)
>
> Why is it attempting to install to the system when I'm telling it  
> not to?

Because the bug hasn't been fixed yet, and nobody's looked into  
fixing it.

Note that the bug may be in setup.rb, which will take a bit of  
figuring out.  Its quite the monster.

--
Poor workers blame their tools. Good workers build better tools. The  
best
workers get their tools to do the work for them. -- Syndicate Wars



From randy.j.parker at gmail.com  Thu Jun  7 15:30:17 2007
From: randy.j.parker at gmail.com (Randy Parker)
Date: Thu, 7 Jun 2007 15:30:17 -0400
Subject: [Rubygems-developers] Installing Rubygems in User's Local
	Directory to fix Gem on Memory-Limited Systems
In-Reply-To: <B751A3BA-78FD-4CD4-8E67-AA95B9B638B6@segment7.net>
References: <735c8b860706071027n599aa573p72efedb6fcce4db3@mail.gmail.com>
	<B751A3BA-78FD-4CD4-8E67-AA95B9B638B6@segment7.net>
Message-ID: <735c8b860706071230q301ea03cycfdb93f0c024f6e@mail.gmail.com>

Which bug is not fixed yet?  I see that [rubygems #8470] is still open on   http://rubyforge.org/tracker/?atid=575&group_id=126&func=browse 

But my understanding of #8470 is that the the local
dir is not _automatically_ used for installation.
Do you mean that the most recent note posted to the bug
report, which proposes a workaround by re-running
the install sequence 

  ruby setup.rb config --prefix=$GEM_HOME
  ruby setup.rb setup
  ruby setup.rb install

Is also known NOT to work?  (it ain't just me?)

If that is true, then is it accurate to say that there is no way
to run "gem update --system" for a "non-standard installation directory"
(that is, one that doesn't require root access, as described by   http://rubygems.org/read/chapter/15#page101 )

If there is no workaround, then because "gem install" / "gem update" require rubygems-update-0.9.4.2 to run on memory-constrained hosting containers,
and the only way to install it is with root permission, then it must be that
nobody can use "gem" unless they have a dedicated system (or a hosting
container that is as big as a dedicated system).

Surely I'm missing something?




On 6/7/07, Eric Hodel < drbrain at segment7.net> wrote:
On Jun 7, 2007, at 10:27, Randy Parker wrote: 

> [...]
>
> After some searching, I found [rubygems #8470] gem update --system
> should take the prefix at which rubygems is installed into
> accountwhich advises
>
> To workaround, switch to the downloaded rubygem update (  e.g.
> GEM_HOME/gems/rubygems-update-0.9.2/), then run the install sequence
> again:
>
> But in all cases the install step insisted on trying (and naturally
> failing) to install to the hosting computer's system dir, with the 
> same line as included above:
>
> setup.rb:633:in `initialize': Permission denied - /usr/local/bin/
> gem (Errno::EACCES)
>
> Why is it attempting to install to the system when I'm telling it 
> not to?

Because the bug hasn't been fixed yet, and nobody's looked into
fixing it.

Note that the bug may be in setup.rb, which will take a bit of
figuring out.  Its quite the monster.
 
--
Poor workers blame their tools. Good workers build better tools. The
best
workers get their tools to do the work for them. -- Syndicate Wars


_______________________________________________
Rubygems-developers mailing list 
Rubygems-developers at rubyforge.org
http://rubyforge.org/mailman/listinfo/rubygems-developers 

 

From drbrain at segment7.net  Thu Jun  7 20:55:52 2007
From: drbrain at segment7.net (Eric Hodel)
Date: Thu, 7 Jun 2007 17:55:52 -0700
Subject: [Rubygems-developers] Installing Rubygems in User's Local
	Directory to fix Gem on Memory-Limited Systems
In-Reply-To: <735c8b860706071230q301ea03cycfdb93f0c024f6e@mail.gmail.com>
References: <735c8b860706071027n599aa573p72efedb6fcce4db3@mail.gmail.com>
	<B751A3BA-78FD-4CD4-8E67-AA95B9B638B6@segment7.net>
	<735c8b860706071230q301ea03cycfdb93f0c024f6e@mail.gmail.com>
Message-ID: <F5555EA5-66C2-4339-ADD2-DBE360DC5E3E@segment7.net>

Please don't top post.

On Jun 7, 2007, at 12:30, Randy Parker wrote:

> Which bug is not fixed yet?  I see that [rubygems #8470] is still  
> open on   http://rubyforge.org/tracker/? 
> atid=575&group_id=126&func=browse
>
> But my understanding of #8470 is that the the local
> dir is not _automatically_ used for installation.

There may be two bugs, but they both seem to be in the same category.

> Do you mean that the most recent note posted to the bug
> report, which proposes a workaround by re-running
> the install sequence
>
>   ruby setup.rb config --prefix=$GEM_HOME
>   ruby setup.rb setup
>   ruby setup.rb install
>
> Is also known NOT to work?  (it ain't just me?)

I've not tried it, but I'm going to take your word for it.  I'm a bit  
busy right now to investigate in detail.

Can you poke around in setup.rb to see if its ignoring --prefix?

> If that is true, then is it accurate to say that there is no way
> to run "gem update --system" for a "non-standard installation  
> directory"
> (that is, one that doesn't require root access, as described by    
> http://rubygems.org/read/chapter/15#page101 )
>
> If there is no workaround, then because "gem install" / "gem  
> update" require rubygems-update-0.9.4.2 to run on memory- 
> constrained hosting containers,
> and the only way to install it is with root permission, then it  
> must be that
> nobody can use "gem" unless they have a dedicated system (or a hosting
> container that is as big as a dedicated system).
>
> Surely I'm missing something?

You can install your own ruby in your homedir, then it'll work.

Right now it appears to be using Ruby's prefix over the one you  
want.  If you have your own ruby, you'll be able to write to where  
setup.rb is trying to put things.

--
Poor workers blame their tools. Good workers build better tools. The  
best
workers get their tools to do the work for them. -- Syndicate Wars



From randy.j.parker at gmail.com  Fri Jun  8 13:53:03 2007
From: randy.j.parker at gmail.com (Randy Parker)
Date: Fri, 8 Jun 2007 13:53:03 -0400
Subject: [Rubygems-developers] Installing Rubygems in User's Local
	Directory to fix Gem on Memory-Limited Systems
In-Reply-To: <F5555EA5-66C2-4339-ADD2-DBE360DC5E3E@segment7.net>
References: <735c8b860706071027n599aa573p72efedb6fcce4db3@mail.gmail.com>
	<B751A3BA-78FD-4CD4-8E67-AA95B9B638B6@segment7.net>
	<735c8b860706071230q301ea03cycfdb93f0c024f6e@mail.gmail.com>
	<F5555EA5-66C2-4339-ADD2-DBE360DC5E3E@segment7.net>
Message-ID: <735c8b860706081053v25cc3ca1i6d2ce84af5344c53@mail.gmail.com>

On 6/7/07, Eric Hodel <drbrain at segment7.net> wrote:
>
> Please don't top post.
>
> On Jun 7, 2007, at 12:30, Randy Parker wrote:
>
> > Which bug is not fixed yet?  I see that [rubygems #8470] is still
> > open on   http://rubyforge.org/tracker/?
> > atid=575&group_id=126&func=browse
> >
> > But my understanding of #8470 is that the the local
> > dir is not _automatically_ used for installation.
>
> There may be two bugs, but they both seem to be in the same category.
>
> > Do you mean that the most recent note posted to the bug
> > report, which proposes a workaround by re-running
> > the install sequence
> >
> >   ruby setup.rb config --prefix=$GEM_HOME
> >   ruby setup.rb setup
> >   ruby setup.rb install
> >
> > Is also known NOT to work?  (it ain't just me?)
>
> I've not tried it, but I'm going to take your word for it.  I'm a bit
> busy right now to investigate in detail.
>
> Can you poke around in setup.rb to see if its ignoring --prefix?
>
> > If that is true, then is it accurate to say that there is no way
> > to run "gem update --system" for a "non-standard installation
> > directory"
> > (that is, one that doesn't require root access, as described by
> > http://rubygems.org/read/chapter/15#page101 )
> >
> > If there is no workaround, then because "gem install" / "gem
> > update" require rubygems-update-0.9.4.2 to run on memory-
> > constrained hosting containers,
> > and the only way to install it is with root permission, then it
> > must be that
> > nobody can use "gem" unless they have a dedicated system (or a hosting
> > container that is as big as a dedicated system).
> >
> > Surely I'm missing something?
>
> You can install your own ruby in your homedir, then it'll work.
>
> Right now it appears to be using Ruby's prefix over the one you
> want.  If you have your own ruby, you'll be able to write to where
> setup.rb is trying to put things.
>


I did not fix setup.rb.  Instead, I installed ruby in ~/ruby.

The Joyent / Textdrive shared host "kientz" is supposed to restrict memory
in my Solaris container to 100MB.  It looks like that is not enough to use
gem.

My other shared hosting accounts are Textdrive's FreeBSD, restricted to
48MB, and Site5's Linux, restricted to 50MB.  I think it is safe to assume
that they will continue to fail with 0.9.4.2 as well.

[kientz:~/ruby] randy$ gem env
RubyGems Environment:
  - VERSION: 0.9.4.2 (0.9.4.2)
  - INSTALLATION DIRECTORY: /users/home/randy/ruby/lib/ruby/gems/1.8
  - GEM PATH:
     - /users/home/randy/ruby/lib/ruby
     - /users/home/randy/ruby/lib/site_ruby/1.8
     - /users/home/randy/ruby/lib/ruby/gems/1.8
  - REMOTE SOURCES:
     - http://gems.rubyforge.org


[kientz:~/ruby] randy$ gem list

*** LOCAL GEMS ***

rubygems-update (0.9.4.2)
    RubyGems Update GEM

sources (0.0.1)
    This package provides download sources for remote gem installation


[kientz:~/ruby] randy$ gem install rails
Bulk updating Gem source index for: http://gems.rubyforge.org
/users/home/randy/ruby/lib/ruby/1.8/yaml.rb:133:in `node_import': failed to
allocate memory (NoMemoryError)
        from /users/home/randy/ruby/lib/ruby/1.8/yaml.rb:133:in `load'
        from /users/home/randy/ruby/lib/ruby/1.8/yaml.rb:133:in `load'
        from
/users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:272:in
`convert_specs'
        from
/users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:295:in
`fetch_bulk_index'
        from
/users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:252:in
`update'
        from
/users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_info_cache_entry.rb:26:in
`refresh'
        from
/users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_info_cache.rb:111:in
`refresh'
        from
/users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_info_cache.rb:104:in
`each'
         ... 11 levels...
        from
/users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/command_manager.rb:121:in
`process_args'
        from
/users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/command_manager.rb:92:in
`run'
        from
/users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/gem_runner.rb:31:in
`run'
        from /users/home/randy/ruby/bin/gem:23

The memory allocation failure is independent of the target gem size, it
results from decompressing the gem index yaml file.  (I realize the rubygems
developers already know this; I'm just pointing this out to anyone else who
might suggest I retry with a smaller gem target than 'rails')
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070608/84bd152f/attachment.html 

From randy.j.parker at gmail.com  Fri Jun  8 14:18:06 2007
From: randy.j.parker at gmail.com (Randy Parker)
Date: Fri, 8 Jun 2007 14:18:06 -0400
Subject: [Rubygems-developers] Installing Rubygems in User's Local
	Directory to fix Gem on Memory-Limited Systems
In-Reply-To: <735c8b860706081053v25cc3ca1i6d2ce84af5344c53@mail.gmail.com>
References: <735c8b860706071027n599aa573p72efedb6fcce4db3@mail.gmail.com>
	<B751A3BA-78FD-4CD4-8E67-AA95B9B638B6@segment7.net>
	<735c8b860706071230q301ea03cycfdb93f0c024f6e@mail.gmail.com>
	<F5555EA5-66C2-4339-ADD2-DBE360DC5E3E@segment7.net>
	<735c8b860706081053v25cc3ca1i6d2ce84af5344c53@mail.gmail.com>
Message-ID: <735c8b860706081118y4e284716w1b9b411dc6e39087@mail.gmail.com>

Following the naive "gem install rails", I tried Jim's incremental update:

gem list -r -B 1000000

this took 8 minutes on Joyent's dual 3GHz AMD running a load average of 0.4,
and sucking only 1.8% of the cpu.  More importantly, it completed
successfully, and a subsequent "gem install rails" also worked!!

On 6/8/07, Randy Parker <randy.j.parker at gmail.com> wrote:
>
> On 6/7/07, Eric Hodel <drbrain at segment7.net> wrote:
> >
> > Please don't top post.
> >
> > On Jun 7, 2007, at 12:30, Randy Parker wrote:
> >
> > > Which bug is not fixed yet?  I see that [rubygems #8470] is still
> > > open on   http://rubyforge.org/tracker/ ?
> > > atid=575&group_id=126&func=browse
> > >
> > > But my understanding of #8470 is that the the local
> > > dir is not _automatically_ used for installation.
> >
> > There may be two bugs, but they both seem to be in the same category.
> >
> > > Do you mean that the most recent note posted to the bug
> > > report, which proposes a workaround by re-running
> > > the install sequence
> > >
> > >   ruby setup.rb config --prefix=$GEM_HOME
> > >   ruby setup.rb setup
> > >   ruby setup.rb install
> > >
> > > Is also known NOT to work?  (it ain't just me?)
> >
> > I've not tried it, but I'm going to take your word for it.  I'm a bit
> > busy right now to investigate in detail.
> >
> > Can you poke around in setup.rb to see if its ignoring --prefix?
> >
> > > If that is true, then is it accurate to say that there is no way
> > > to run "gem update --system" for a "non-standard installation
> > > directory"
> > > (that is, one that doesn't require root access, as described by
> > > http://rubygems.org/read/chapter/15#page101 )
> > >
> > > If there is no workaround, then because "gem install" / "gem
> > > update" require rubygems-update-0.9.4.2 to run on memory-
> > > constrained hosting containers,
> > > and the only way to install it is with root permission, then it
> > > must be that
> > > nobody can use "gem" unless they have a dedicated system (or a hosting
> > > container that is as big as a dedicated system).
> > >
> > > Surely I'm missing something?
> >
> > You can install your own ruby in your homedir, then it'll work.
> >
> > Right now it appears to be using Ruby's prefix over the one you
> > want.  If you have your own ruby, you'll be able to write to where
> > setup.rb is trying to put things.
> >
>
>
> I did not fix setup.rb.  Instead, I installed ruby in ~/ruby.
>
> The Joyent / Textdrive shared host "kientz" is supposed to restrict memory
> in my Solaris container to 100MB.  It looks like that is not enough to use
> gem.
>
> My other shared hosting accounts are Textdrive's FreeBSD, restricted to
> 48MB, and Site5's Linux, restricted to 50MB.  I think it is safe to assume
> that they will continue to fail with 0.9.4.2 as well.
>
> [kientz:~/ruby] randy$ gem env
> RubyGems Environment:
>   - VERSION: 0.9.4.2 (0.9.4.2)
>   - INSTALLATION DIRECTORY: /users/home/randy/ruby/lib/ruby/gems/1.8
>   - GEM PATH:
>      - /users/home/randy/ruby/lib/ruby
>      - /users/home/randy/ruby/lib/site_ruby/1.8
>      - /users/home/randy/ruby/lib/ruby/gems/1.8
>   - REMOTE SOURCES:
>      - http://gems.rubyforge.org
>
>
> [kientz:~/ruby] randy$ gem list
>
> *** LOCAL GEMS ***
>
> rubygems-update (0.9.4.2)
>     RubyGems Update GEM
>
> sources (0.0.1)
>     This package provides download sources for remote gem installation
>
>
> [kientz:~/ruby] randy$ gem install rails
> Bulk updating Gem source index for: http://gems.rubyforge.org
> /users/home/randy/ruby/lib/ruby/1.8/yaml.rb:133:in `node_import': failed
> to allocate memory (NoMemoryError)
>         from /users/home/randy/ruby/lib/ruby/1.8/yaml.rb:133:in `load'
>         from /users/home/randy/ruby/lib/ruby/1.8/yaml.rb:133:in `load'
>         from
> /users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:272:in
> `convert_specs'
>         from
> /users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:295:in
> `fetch_bulk_index'
>         from
> /users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:252:in
> `update'
>         from
> /users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_info_cache_entry.rb:26:in
> `refresh'
>         from
> /users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_info_cache.rb:111:in
> `refresh'
>         from
> /users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/source_info_cache.rb:104:in
> `each'
>          ... 11 levels...
>         from
> /users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/command_manager.rb:121:in
> `process_args'
>         from
> /users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/command_manager.rb:92:in
> `run'
>         from
> /users/home/randy/ruby/lib/ruby/site_ruby/1.8/rubygems/gem_runner.rb:31:in
> `run'
>         from /users/home/randy/ruby/bin/gem:23
>
> The memory allocation failure is independent of the target gem size, it
> results from decompressing the gem index yaml file.  (I realize the rubygems
> developers already know this; I'm just pointing this out to anyone else who
> might suggest I retry with a smaller gem target than 'rails')
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070608/65823154/attachment-0001.html 

From jim.weirich at gmail.com  Fri Jun  8 14:23:29 2007
From: jim.weirich at gmail.com (Jim Weirich)
Date: Fri, 8 Jun 2007 14:23:29 -0400
Subject: [Rubygems-developers] Installing Rubygems in User's Local
	Directory to fix Gem on Memory-Limited Systems
In-Reply-To: <735c8b860706081118y4e284716w1b9b411dc6e39087@mail.gmail.com>
References: <735c8b860706071027n599aa573p72efedb6fcce4db3@mail.gmail.com>
	<B751A3BA-78FD-4CD4-8E67-AA95B9B638B6@segment7.net>
	<735c8b860706071230q301ea03cycfdb93f0c024f6e@mail.gmail.com>
	<F5555EA5-66C2-4339-ADD2-DBE360DC5E3E@segment7.net>
	<735c8b860706081053v25cc3ca1i6d2ce84af5344c53@mail.gmail.com>
	<735c8b860706081118y4e284716w1b9b411dc6e39087@mail.gmail.com>
Message-ID: <B974C97F-97C6-45EE-AD7D-63D2AA4A683E@gmail.com>


On Jun 8, 2007, at 2:18 PM, Randy Parker wrote:

> Following the naive "gem install rails", I tried Jim's incremental  
> update:
>
> gem list -r -B 1000000
>
> this took 8 minutes on Joyent's dual 3GHz AMD running a load  
> average of 0.4, and sucking only 1.8% of the cpu.  More  
> importantly, it completed successfully, and a subsequent "gem  
> install rails" also worked!!

Great!  Thanks for the feedback.

-- Jim Weirich


From ryand-ruby at zenspider.com  Fri Jun  8 15:16:26 2007
From: ryand-ruby at zenspider.com (Ryan Davis)
Date: Fri, 8 Jun 2007 12:16:26 -0700
Subject: [Rubygems-developers] Installing Rubygems in User's Local
	Directory to fix Gem on Memory-Limited Systems
In-Reply-To: <735c8b860706081053v25cc3ca1i6d2ce84af5344c53@mail.gmail.com>
References: <735c8b860706071027n599aa573p72efedb6fcce4db3@mail.gmail.com>
	<B751A3BA-78FD-4CD4-8E67-AA95B9B638B6@segment7.net>
	<735c8b860706071230q301ea03cycfdb93f0c024f6e@mail.gmail.com>
	<F5555EA5-66C2-4339-ADD2-DBE360DC5E3E@segment7.net>
	<735c8b860706081053v25cc3ca1i6d2ce84af5344c53@mail.gmail.com>
Message-ID: <DE785D82-A0A9-4F51-B66C-73D8038EC280@zenspider.com>


On Jun 8, 2007, at 10:53 , Randy Parker wrote:

> The Joyent / Textdrive shared host "kientz" is supposed to restrict  
> memory in my Solaris container to 100MB.  It looks like that is not  
> enough to use gem.
>
> My other shared hosting accounts are Textdrive's FreeBSD,  
> restricted to 48MB, and Site5's Linux, restricted to 50MB.  I think  
> it is safe to assume that they will continue to fail with 0.9.4.2  
> as well.

Maybe it is me, but I'm not sure this is a problem with rubygems. I  
don't feel it is trying to be a works-everywhere type of platform.  
These TxD slices you're using have less RAM than most cell phones  
these days. No really. It'll be hard for you to run a rails app on  
them regardless.

That said, I think you're looking at the problem incorrectly. You're  
not supposed to DO anything on those slices. They're too thin to be  
practical for much of anything but a single task. In your case, you  
should be rsyncing in a rails app directory that includes vendor/ 
rails and all the gems it would be using frozen in vendor as well  
(see rake rails:freeze:gems and other solutions in various blogs).  
You should be 100% self-reliant and self-contained. As I understand  
it, the latest capistrano can deploy via rsync.


From drnicwilliams at gmail.com  Fri Jun  8 15:26:23 2007
From: drnicwilliams at gmail.com (Nic Williams)
Date: Fri, 8 Jun 2007 21:26:23 +0200
Subject: [Rubygems-developers] Installing Rubygems in User's Local
	Directory to fix Gem on Memory-Limited Systems
In-Reply-To: <DE785D82-A0A9-4F51-B66C-73D8038EC280@zenspider.com>
References: <735c8b860706071027n599aa573p72efedb6fcce4db3@mail.gmail.com>
	<B751A3BA-78FD-4CD4-8E67-AA95B9B638B6@segment7.net>
	<735c8b860706071230q301ea03cycfdb93f0c024f6e@mail.gmail.com>
	<F5555EA5-66C2-4339-ADD2-DBE360DC5E3E@segment7.net>
	<735c8b860706081053v25cc3ca1i6d2ce84af5344c53@mail.gmail.com>
	<DE785D82-A0A9-4F51-B66C-73D8038EC280@zenspider.com>
Message-ID: <44b555bb0706081226m2fa76adajff50418c81b178b2@mail.gmail.com>

And for other gems, choose a "vendor everything" strategy - such as the list
given here - http://errtheblog.com/post/2120

Nic

On 6/8/07, Ryan Davis <ryand-ruby at zenspider.com> wrote:
>
>
> On Jun 8, 2007, at 10:53 , Randy Parker wrote:
>
> > The Joyent / Textdrive shared host "kientz" is supposed to restrict
> > memory in my Solaris container to 100MB.  It looks like that is not
> > enough to use gem.
> >
> > My other shared hosting accounts are Textdrive's FreeBSD,
> > restricted to 48MB, and Site5's Linux, restricted to 50MB.  I think
> > it is safe to assume that they will continue to fail with 0.9.4.2
> > as well.
>
> Maybe it is me, but I'm not sure this is a problem with rubygems. I
> don't feel it is trying to be a works-everywhere type of platform.
> These TxD slices you're using have less RAM than most cell phones
> these days. No really. It'll be hard for you to run a rails app on
> them regardless.
>
> That said, I think you're looking at the problem incorrectly. You're
> not supposed to DO anything on those slices. They're too thin to be
> practical for much of anything but a single task. In your case, you
> should be rsyncing in a rails app directory that includes vendor/
> rails and all the gems it would be using frozen in vendor as well
> (see rake rails:freeze:gems and other solutions in various blogs).
> You should be 100% self-reliant and self-contained. As I understand
> it, the latest capistrano can deploy via rsync.
>
> _______________________________________________
> Rubygems-developers mailing list
> Rubygems-developers at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rubygems-developers
>



-- 
Dr Nic Williams
http://drnicacademy.com - Ruby/Rails training around the world
http://www.drnicwilliams.com - Ruby/Rails/Javascript/Web2.0
skype: nicwilliams
(p) +61 7 3102 3237 (Finds me anywhere in the world, via Skype)
(m) +4673 681 5093 (Swedish mobile)
(f) +61 7 3305 7572 (sends fax to my email)
Bj?rnsonsgatan 153, 16 844 Bromma, Sweden
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070608/08a7d6ba/attachment.html 

From todd.fisher at gmail.com  Tue Jun 12 10:38:45 2007
From: todd.fisher at gmail.com (Todd Fisher)
Date: Tue, 12 Jun 2007 10:38:45 -0400
Subject: [Rubygems-developers] gem install and gem uninstall missing
	--install-dir
Message-ID: <e9a952090706120738g7467eff3h2c939e4cd06b941d@mail.gmail.com>

Hi,

gem install has the --install-dir flag, but uninstall does not.  This means
if I wanted to create a gem repository in a non-standard directory such as
/opt/gems  I can not uninstall gems from that alternate folder.
Trying to set the GEM_HOME to an empty gem folder causes the gem command to
fail with this stack trace.


 GEM_HOME=/opt/rhg/gems/ gem uninstall package

/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`gem_original_require': no such file to load -- sources (LoadError)

        from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`require'

        from /usr/lib/ruby/site_ruby/1.8/rubygems/source_info_cache.rb:6

        from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`gem_original_require'

        from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`require'

        from /usr/lib/ruby/site_ruby/1.8/rubygems/remote_installer.rb:12

        from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`gem_original_require'

        from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`require'

        from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:112:in `manage_gems'

        from /usr/bin/gem:10


Thanks in advance for any suggestions!

Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070612/a73a8954/attachment.html 

From drbrain at segment7.net  Tue Jun 12 18:58:41 2007
From: drbrain at segment7.net (Eric Hodel)
Date: Tue, 12 Jun 2007 15:58:41 -0700
Subject: [Rubygems-developers] gem install and gem uninstall missing
	--install-dir
In-Reply-To: <e9a952090706120738g7467eff3h2c939e4cd06b941d@mail.gmail.com>
References: <e9a952090706120738g7467eff3h2c939e4cd06b941d@mail.gmail.com>
Message-ID: <1D9EB714-9CC4-4982-9CD0-9D016E2EDCAA@segment7.net>

On Jun 12, 2007, at 07:38, Todd Fisher wrote:

> gem install has the --install-dir flag, but uninstall does not.   
> This means if I wanted to create a gem repository in a non-standard  
> directory such as /opt/gems  I can not uninstall gems from that  
> alternate folder.
>
> Trying to set the GEM_HOME to an empty gem folder causes the gem  
> command to fail with this stack trace.

Can you file bugs in the tracker for each of these?

--
Poor workers blame their tools. Good workers build better tools. The  
best
workers get their tools to do the work for them. -- Syndicate Wars



From jim.weirich at gmail.com  Wed Jun 13 22:08:25 2007
From: jim.weirich at gmail.com (Jim Weirich)
Date: Wed, 13 Jun 2007 22:08:25 -0400
Subject: [Rubygems-developers] gem install and gem uninstall missing
	--install-dir
In-Reply-To: <e9a952090706120738g7467eff3h2c939e4cd06b941d@mail.gmail.com>
References: <e9a952090706120738g7467eff3h2c939e4cd06b941d@mail.gmail.com>
Message-ID: <BC16A643-112E-48CE-AF28-59369933FE66@weirichhouse.org>


On Jun 12, 2007, at 10:38 AM, Todd Fisher wrote:

> Hi,
>
> gem install has the --install-dir flag, but uninstall does not.   
> This means if I wanted to create a gem repository in a non-standard  
> directory such as /opt/gems  I can not uninstall gems from that  
> alternate folder.

Have you tried this?  What /should/ happen is that RubyGems detects  
the gem in the gem path and uninstalls it from there.  I don't think  
we should need a the equivalend of the --install-dir flag for uninstall.

If it is not working as described, then please do file a bug report  
as Eric has suggested.

>
> Trying to set the GEM_HOME to an empty gem folder causes the gem  
> command to fail with this stack trace.

Yep, this one sounds like a real bug (and should be filed).

Thanks for the feedback.

-- Jim Weirich
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070613/ac8aca80/attachment.html 

From uwe at datek.no  Sat Jun 16 18:33:02 2007
From: uwe at datek.no (Uwe Kubosch)
Date: Sun, 17 Jun 2007 00:33:02 +0200
Subject: [Rubygems-developers] Installing startup scripthat
Message-ID: <1182033182.3863.9.camel@pippin.datek.no>

Hi all!

I have made a gem with an executable, and I would like to start a daemon
at boot using /etc/init.d/<my script>

What is the best way to install such a file?

I would like update and uninstall to update and remove the file also.

Any help is greatly appreciated.


Uwe Kubosch
Norway

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070617/cd4f857d/attachment-0001.html 

From darix at web.de  Sun Jun 17 11:10:38 2007
From: darix at web.de (Marcus Rueckert)
Date: Sun, 17 Jun 2007 17:10:38 +0200
Subject: [Rubygems-developers] Installing startup scripthat
In-Reply-To: <1182033182.3863.9.camel@pippin.datek.no>
References: <1182033182.3863.9.camel@pippin.datek.no>
Message-ID: <20070617151038.GR3490@pixel.global-banlist.de>

On 2007-06-17 00:33:02 +0200, Uwe Kubosch wrote:
> I have made a gem with an executable, and I would like to start a daemon
> at boot using /etc/init.d/<my script>
> 
> What is the best way to install such a file?
> 
> I would like update and uninstall to update and remove the file also.
> 
> Any help is greatly appreciated.

i would only provide it as sample script in your gem directory. the
differences between various operating system/linux distros/init systems
are too big.

last but not least i dont think you can install something to this
directory anyway.

    darix

-- 
           openSUSE - SUSE Linux is my linux
               openSUSE is good for you
                   www.opensuse.org

From drbrain at segment7.net  Mon Jun 18 03:53:49 2007
From: drbrain at segment7.net (Eric Hodel)
Date: Mon, 18 Jun 2007 00:53:49 -0700
Subject: [Rubygems-developers] Installing startup scripthat
In-Reply-To: <20070617151038.GR3490@pixel.global-banlist.de>
References: <1182033182.3863.9.camel@pippin.datek.no>
	<20070617151038.GR3490@pixel.global-banlist.de>
Message-ID: <59286D3B-E789-4767-A5B8-A5FD7A2E973F@segment7.net>

On Jun 17, 2007, at 08:10, Marcus Rueckert wrote:
> On 2007-06-17 00:33:02 +0200, Uwe Kubosch wrote:
>> I have made a gem with an executable, and I would like to start a  
>> daemon
>> at boot using /etc/init.d/<my script>
>>
>> What is the best way to install such a file?
>>
>> I would like update and uninstall to update and remove the file also.
>>
>> Any help is greatly appreciated.
>
> i would only provide it as sample script in your gem directory. the
> differences between various operating system/linux distros/init  
> systems
> are too big.

Agreed.  For ar_mailer, I have a *BSD rc.d script in share/, and a  
note in the readme about where to find it.

--
Poor workers blame their tools. Good workers build better tools. The  
best
workers get their tools to do the work for them. -- Syndicate Wars



From todd.fisher at gmail.com  Tue Jun 19 09:45:40 2007
From: todd.fisher at gmail.com (Todd Fisher)
Date: Tue, 19 Jun 2007 09:45:40 -0400
Subject: [Rubygems-developers] gem install --include-dependencies and
	--install-dir
Message-ID: <e9a952090706190645n33bfa04t3ec41687fcbf3f02@mail.gmail.com>

Hi,

  Attempting to use --include-dependencies with --install-dir fails to pick
up the required gem dependencies for the gem being installed.

For example, try the following:

gem install rails-1.1.6.gem  --include-dependencies
--install-dir=sample/vendor/gems/

I've also tried using the following:

gem install rails --include-dependencies --install-dir=sample/vendor/gems/
--remote  # this works!

However, if I try and pick up a specific version it fails to also install
the dependencies

gem install rails --version 1.1.6 --include-dependencies
--install-dir=sample/vendor/gems/ --remote  # this doesn't work!


This is with the latest gem 0.9.4.

Thanks, for any suggestions or work around!


-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070619/f3694698/attachment.html 

From wes.sheldahl at gmail.com  Tue Jun 19 12:31:18 2007
From: wes.sheldahl at gmail.com (Wes Sheldahl)
Date: Tue, 19 Jun 2007 12:31:18 -0400
Subject: [Rubygems-developers] Installing startup scripthat
In-Reply-To: <59286D3B-E789-4767-A5B8-A5FD7A2E973F@segment7.net>
References: <1182033182.3863.9.camel@pippin.datek.no>
	<20070617151038.GR3490@pixel.global-banlist.de>
	<59286D3B-E789-4767-A5B8-A5FD7A2E973F@segment7.net>
Message-ID: <c0a118d70706190931k2752c773oed312317fff059df@mail.gmail.com>

On 6/18/07, Eric Hodel <drbrain at segment7.net> wrote:
>
> On Jun 17, 2007, at 08:10, Marcus Rueckert wrote:
> > On 2007-06-17 00:33:02 +0200, Uwe Kubosch wrote:
> >> I have made a gem with an executable, and I would like to start a
> >> daemon
> >> at boot using /etc/init.d/<my script>
> >>
> >> What is the best way to install such a file?
> >>
> >> I would like update and uninstall to update and remove the file also.
> >>
> >> Any help is greatly appreciated.
> >
> > i would only provide it as sample script in your gem directory. the
> > differences between various operating system/linux distros/init
> > systems
> > are too big.
>
> Agreed.  For ar_mailer, I have a *BSD rc.d script in share/, and a
> note in the readme about where to find it.



Right. From that point, an OS specific package (like an rpm, .deb package or
*BSD port) could add a post-install script to either just copy the provided
script to /etc/init.d (or wherever), or add its own startup script and copy
that to the right location. That would need to be up to whoever is making
the RPM or whatever. If that packager doesn't do anything extra, then users
just need to read the README and install it themselves.


-- 
Wes Sheldahl
wes.sheldahl at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070619/1d198b23/attachment.html 

From drbrain at segment7.net  Tue Jun 19 16:41:50 2007
From: drbrain at segment7.net (Eric Hodel)
Date: Tue, 19 Jun 2007 13:41:50 -0700
Subject: [Rubygems-developers] gem install --include-dependencies and
	--install-dir
In-Reply-To: <e9a952090706190645n33bfa04t3ec41687fcbf3f02@mail.gmail.com>
References: <e9a952090706190645n33bfa04t3ec41687fcbf3f02@mail.gmail.com>
Message-ID: <01DA41DC-DE22-4848-9D4C-CC4526CEF75A@segment7.net>

On Jun 19, 2007, at 06:45, Todd Fisher wrote:

> Thanks, for any suggestions or work around!

Please file a bug in the bug tracker.

--
Poor workers blame their tools. Good workers build better tools. The  
best
workers get their tools to do the work for them. -- Syndicate Wars



From jim.weirich at gmail.com  Wed Jun 20 01:10:10 2007
From: jim.weirich at gmail.com (Jim Weirich)
Date: Wed, 20 Jun 2007 01:10:10 -0400
Subject: [Rubygems-developers] ZLib bites us again.
Message-ID: <DDF63A6C-CD70-43EC-BFD4-9C90AEBCC3AE@weirichhouse.org>

Version 0.9.3 (IIRC) set the ZLib code in package.rb to always read  
from a memory string rather than a FileIO object.  This was to avoid  
buffer errors on Windows boxen.  Unfortunately, it seems that we  
still get zlib problems.  Jamis Buck reported a problem with the  
capistrano gem on windows XP.  After some deep diving, he came up  
with with a workaround version of zipped_stream that doesn't blow up  
with a buffer error ... at least for this time around.  Essentially,  
Jamis's fix replaces GZipReader with some Ruby code that does the  
equivalent directly with ZLib calls.

I've made a beta version of RubyGems with his patch (you can get it  
with: gem update --system --source http://onestepback.org/betagems).

Jamis points out that this is cargo cult programming at its finest.   
I.e. we really don't know *why* it fixes it.  All we know is that it  
does fix it in this one instance.  We really need someone to dive  
into ZLib and see if we can get the windows version fixed.

I've attached Jamis's original email.  He has a slightly longer  
version we may want to consider if this fix seems to be worthwhile.   
Thank you for your time.

?

-- 
-- Jim Weirich
-- jim at weirichhouse.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070620/608e6ee8/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Jamis.rtf
Type: text/rtf
Size: 2127 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/rubygems-developers/attachments/20070620/608e6ee8/attachment.rtf 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070620/608e6ee8/attachment-0001.html 

From djberg96 at gmail.com  Wed Jun 20 08:34:38 2007
From: djberg96 at gmail.com (Daniel Berger)
Date: Wed, 20 Jun 2007 06:34:38 -0600
Subject: [Rubygems-developers] ZLib bites us again.
In-Reply-To: <DDF63A6C-CD70-43EC-BFD4-9C90AEBCC3AE@weirichhouse.org>
References: <DDF63A6C-CD70-43EC-BFD4-9C90AEBCC3AE@weirichhouse.org>
Message-ID: <46791EDE.2020507@gmail.com>

Jim Weirich wrote:
> Version 0.9.3 (IIRC) set the ZLib code in package.rb to always read from 
> a memory string rather than a FileIO object.  This was to avoid buffer 
> errors on Windows boxen.  Unfortunately, it seems that we still get zlib 
> problems.  Jamis Buck reported a problem with the capistrano gem on 
> windows XP.  After some deep diving, he came up with with a workaround 
> version of zipped_stream that doesn't blow up with a buffer error ... at 
> least for this time around.  Essentially, Jamis's fix replaces 
> GZipReader with some Ruby code that does the equivalent directly with 
> ZLib calls.

Given the colossal number of warnings that spew from building the zlib 
bindings, I'm inclined to fix those first before proceeding to 
investigate the MS Windows issues. But, I'll see what I can do.

Can you provide a link to Jamis' post? I must have missed it.

Thanks,

Dan

From jim.weirich at gmail.com  Sat Jun 23 13:55:55 2007
From: jim.weirich at gmail.com (Jim Weirich)
Date: Sat, 23 Jun 2007 13:55:55 -0400
Subject: [Rubygems-developers] ZLib bites us again.
In-Reply-To: <46791EDE.2020507@gmail.com>
References: <DDF63A6C-CD70-43EC-BFD4-9C90AEBCC3AE@weirichhouse.org>
	<46791EDE.2020507@gmail.com>
Message-ID: <B2136888-DA9B-4232-A88B-A8F6EEC07880@weirichhouse.org>

Sorry ... I'm catching up on some outstanding emails:

On Jun 20, 2007, at 8:34 AM, Daniel Berger wrote:
> Can you provide a link to Jamis' post? I must have missed it.

It was included as an attachment in the original message.  Perhaps  
the mailing list ate the attachment, so I will include it here directly:

--><--snip--><--
From: Jamis Buck

Here's my work-around for the Zlib issue. Just replace the  
zipped_stream method in package.rb with the following:

     def zipped_stream(entry)
       entry.read(10) # skip the gzip header
       zis = Zlib::Inflate.new(-Zlib::MAX_WBITS)
       is = StringIO.new(zis.inflate(entry.read))
     ensure
       zis.finish if zis
     end

A slightly more robust gzip reader looks something like this (sorry  
about the magic numbers, it was just a proof of concept. if you want  
to use this more robust approach, let me know and I can do it up  
right with symbolic constants):

   require 'zlib'

   File.open("out.gz", "rb") do |f|
     head = f.read(10)

     abort "bad magic" if head[0] != 0x1f || head[1] != 0x8b
     abort "bad compression method #{head[2]}" if head[2] != 8
     abort "multipart gzip is unsupported" if head[3] & 0x2 != 0
     abort "encrypted gzip is unsupported" if head[3] & 0x20 != 0
     abort "unknown flags #{head[3]}" if head[3] & 0xc0 != 0

     if head[3] & 0x4 != 0
       len = f.read(2).unpack("n").first
       f.read(len + 2)
     end

     if head[3] & 0x8 != 0
       loop { break if f.read(1) == 0 }
     end

     if head[3] & 0x10 != 0
       loop { break if f.read(1) == 0 }
     end

     zis = Zlib::Inflate.new(-Zlib::MAX_WBITS)
     result = zis.inflate(f.read)

     puts "got #{result.length} bytes"
   end

- Jamis

--><--snip--><--

-- 
-- Jim Weirich
-- jim at weirichhouse.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/rubygems-developers/attachments/20070623/b097ec73/attachment-0001.html 

From alexey.verkhovsky at gmail.com  Mon Jun 25 12:43:15 2007
From: alexey.verkhovsky at gmail.com (Alexey Verkhovsky)
Date: Mon, 25 Jun 2007 10:43:15 -0600
Subject: [Rubygems-developers] Platform selection in gem install
Message-ID: <3945c4270706250943n1a2598c3y55fe0acd9b82e870@mail.gmail.com>

Hi all,

The issue of platform-specific gems was discussed here before, and we
also had some conversations during the last RailsConf about what a
solution could look like in the ideal world, although it doesn't look
like anyone has bandwidth to make it happen.

However, I'd like to contribute a 5% solution that solves half of my
own problems. Namely, replace

  echo "1" | gem install mongrel

with

  gem install mongrel --platform linux-i386

Would a patch that does it be welcome?

--
Alex Verkhovsky
RubyWorks

From ryand-ruby at zenspider.com  Mon Jun 25 14:09:12 2007
From: ryand-ruby at zenspider.com (Ryan Davis)
Date: Mon, 25 Jun 2007 11:09:12 -0700
Subject: [Rubygems-developers] Platform selection in gem install
In-Reply-To: <3945c4270706250943n1a2598c3y55fe0acd9b82e870@mail.gmail.com>
References: <3945c4270706250943n1a2598c3y55fe0acd9b82e870@mail.gmail.com>
Message-ID: <554E7EC8-74DB-4227-8EE6-B1696D3A62FA@zenspider.com>


On Jun 25, 2007, at 09:43 , Alexey Verkhovsky wrote:

> Namely, replace
>
>   echo "1" | gem install mongrel
>
> with
>
>   gem install mongrel --platform linux-i386
>
> Would a patch that does it be welcome?

I can't speak for everyone... but...

	YES!



From uwe at datek.no  Wed Jun 27 02:59:29 2007
From: uwe at datek.no (Uwe Kubosch)
Date: Wed, 27 Jun 2007 08:59:29 +0200
Subject: [Rubygems-developers] Platform selection in gem install
In-Reply-To: <3945c4270706250943n1a2598c3y55fe0acd9b82e870@mail.gmail.com>
References: <3945c4270706250943n1a2598c3y55fe0acd9b82e870@mail.gmail.com>
Message-ID: <1182927569.3873.1.camel@pippin.datek.no>

On Mon, 2007-06-25 at 10:43 -0600, Alexey Verkhovsky wrote:
> Hi all,
> 
> The issue of platform-specific gems was discussed here before, and we
> also had some conversations during the last RailsConf about what a
> solution could look like in the ideal world, although it doesn't look
> like anyone has bandwidth to make it happen.
> 
> However, I'd like to contribute a 5% solution that solves half of my
> own problems. Namely, replace
> 
>   echo "1" | gem install mongrel
> 
> with
> 
>   gem install mongrel --platform linux-i386
> 
> Would a patch that does it be welcome?

YES!  PLEASE!


Uwe

From phurley at gmail.com  Wed Jun 27 07:46:45 2007
From: phurley at gmail.com (Patrick Hurley)
Date: Wed, 27 Jun 2007 07:46:45 -0400
Subject: [Rubygems-developers] Platform selection in gem install
In-Reply-To: <3945c4270706250943n1a2598c3y55fe0acd9b82e870@mail.gmail.com>
References: <3945c4270706250943n1a2598c3y55fe0acd9b82e870@mail.gmail.com>
Message-ID: <554ac39c0706270446p5049b8c3sfa8b164ee1a39e6b@mail.gmail.com>

On 6/25/07, Alexey Verkhovsky <alexey.verkhovsky at gmail.com> wrote:
>   gem install mongrel --platform linux-i386
>
> Would a patch that does it be welcome?

Yes, I might suggest allowing a prioritized list:

--platforms ruby,linux-i686,linux-i386

and/or wild cards or partial matching:

--platforms ruby,linux

But even without I would certainly use it :-)
pth

From drbrain at segment7.net  Wed Jun 27 19:31:49 2007
From: drbrain at segment7.net (Eric Hodel)
Date: Wed, 27 Jun 2007 16:31:49 -0700
Subject: [Rubygems-developers] Platform selection in gem install
In-Reply-To: <3945c4270706250943n1a2598c3y55fe0acd9b82e870@mail.gmail.com>
References: <3945c4270706250943n1a2598c3y55fe0acd9b82e870@mail.gmail.com>
Message-ID: <ADC9C018-E68E-4AF0-AB16-E3B601E57E59@segment7.net>

On Jun 25, 2007, at 09:43, Alexey Verkhovsky wrote:
> The issue of platform-specific gems was discussed here before, and we
> also had some conversations during the last RailsConf about what a
> solution could look like in the ideal world, although it doesn't look
> like anyone has bandwidth to make it happen.
>
> However, I'd like to contribute a 5% solution that solves half of my
> own problems. Namely, replace
>
>   echo "1" | gem install mongrel
>
> with
>
>   gem install mongrel --platform linux-i386
>
> Would a patch that does it be welcome?

Read the bugs and mailing list archive to see what's already been  
proposed.

--
Poor workers blame their tools. Good workers build better tools. The  
best
workers get their tools to do the work for them. -- Syndicate Wars



From alexey.verkhovsky at gmail.com  Wed Jun 27 21:28:51 2007
From: alexey.verkhovsky at gmail.com (Alexey Verkhovsky)
Date: Wed, 27 Jun 2007 19:28:51 -0600
Subject: [Rubygems-developers] Platform selection in gem install
In-Reply-To: <ADC9C018-E68E-4AF0-AB16-E3B601E57E59@segment7.net>
References: <3945c4270706250943n1a2598c3y55fe0acd9b82e870@mail.gmail.com>
	<ADC9C018-E68E-4AF0-AB16-E3B601E57E59@segment7.net>
Message-ID: <3945c4270706271828p372110eerfe37f15516baa9ba@mail.gmail.com>

On 6/27/07, Eric Hodel <drbrain at segment7.net> wrote:
> Read the bugs and mailing list archive to see what's already been proposed.

If you are talking about a discussion on this list that was back in
April, Chad Fowler showed me that.

I think that a command-line option is a good first step towards a full
solution. If you agree, I will submit a patch.

--
Alex