From charlesmbowman at gmail.com  Mon Jul  9 10:09:08 2007
From: charlesmbowman at gmail.com (Charlie Bowman)
Date: Mon, 9 Jul 2007 10:09:08 -0400
Subject: [Backgroundrb-devel] limiting processor usage
Message-ID: <298c4d2a0707090709v232326bfh4cd88ced9e093a16@mail.gmail.com>

I have a very intensive background process that keeps taking up loads of the
cpu.  If I have 2 of these going at once, I can't even ssh to the box much
less get a page to load.  Is there something I can do to limit this.  I dont
mind if the process takes longer to run, I just dont want to freeze my box
every time it runs.  Thanks for any advice!


Charlie
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070709/c5b09e31/attachment-0001.html 

From ruby at geoffgarside.co.uk  Mon Jul  9 10:17:08 2007
From: ruby at geoffgarside.co.uk (Geoff Garside)
Date: Mon, 9 Jul 2007 15:17:08 +0100
Subject: [Backgroundrb-devel] limiting processor usage
In-Reply-To: <298c4d2a0707090709v232326bfh4cd88ced9e093a16@mail.gmail.com>
References: <298c4d2a0707090709v232326bfh4cd88ced9e093a16@mail.gmail.com>
Message-ID: <B12ADBD3-4AC4-4413-8377-217CA60F84CA@geoffgarside.co.uk>

On 9 Jul 2007, at 15:09, Charlie Bowman wrote:

> I have a very intensive background process that keeps taking up  
> loads of the cpu.  If I have 2 of these going at once, I can't even  
> ssh to the box much less get a page to load.  Is there something I  
> can do to limit this.  I dont mind if the process takes longer to  
> run, I just dont want to freeze my box every time it runs.  Thanks  
> for any advice!
>
>
> Charlie
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

You could use a login class to limit the amount of CPU the user  
running the background process can use. Check the login.conf manpage  
for your OS (assuming Unix-like derivative) for details on how to set  
this up.

Geoff

From eden at mojiti.com  Tue Jul 10 00:30:11 2007
From: eden at mojiti.com (Eden Li)
Date: Tue, 10 Jul 2007 12:30:11 +0800
Subject: [Backgroundrb-devel] limiting processor usage
In-Reply-To: <298c4d2a0707090709v232326bfh4cd88ced9e093a16@mail.gmail.com>
References: <298c4d2a0707090709v232326bfh4cd88ced9e093a16@mail.gmail.com>
Message-ID: <1dd361e10707092130x4d924025y24e43bcdfb22d0d8@mail.gmail.com>

You could also try adjusting the niceness of the process via nice(1):

nice -n 19 ./script/backgroundrb start

which makes the kernel assign a lower priority to backgroundrb
processes.  You'll have to experiment to find the right nice number,
it ranges [-20, 19] where -20 is the meanest (least nice, highest
chance to get run).

You can change the niceness of a process that's already running via renice(1).

On 7/9/07, Charlie Bowman <charlesmbowman at gmail.com> wrote:
> I have a very intensive background process that keeps taking up loads of the
> cpu.  If I have 2 of these going at once, I can't even ssh to the box much
> less get a page to load.  Is there something I can do to limit this.  I dont
> mind if the process takes longer to run, I just dont want to freeze my box
> every time it runs.  Thanks for any advice!
>
>
> Charlie
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>

From jodi at nnovation.ca  Tue Jul 10 14:48:11 2007
From: jodi at nnovation.ca (Jodi Showers)
Date: Tue, 10 Jul 2007 14:48:11 -0400
Subject: [Backgroundrb-devel] backgroundrb.ppid
Message-ID: <440E1BD0-C3C3-48D5-811F-D9F96CB9FBC4@nNovation.ca>

Howdy,

I'm trying to use monit with backgroundrb,

but it seems that backgroundrb.ppid isn't holding the id of the  
backgroundrb task.

  jodi at soup/var/www/the-soup/current $ cat log/backgroundrb.ppid
23182
jodi at soup/var/www/the-soup/current $ ps -efl
1 S deploy   23184     1  0  75   0 - 27056 stext  11:27 ?         
00:00:04 backgroundrb
1 S deploy   23186 23184  0  76   0 - 14444 -      11:27 ?         
00:00:00 backgroundrb_logger
1 S deploy   23187 23184  0  76   0 - 14768 -      11:27 ?         
00:00:00 backgroundrb_results
0 R jodi     23799 23755  0  76   0 -   676 -      14:06 pts/0     
00:00:00 grep backgroundrb


Does this make sense?

Since I'm starting backgroundrb with capistrano, the only thing I can  
think of is that it's the pid of the starting process?

Short of the above, my current work around is to `touch` a ping file  
during worker execution, restarting backgroundrb if the ping file is  
too old.

Cheers,
Jodi
General Partner
The nNovation Group inc.
www.nnovation.ca/blog




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070710/603d5d92/attachment.html 

From dgreig at magian.com  Tue Jul 10 21:16:01 2007
From: dgreig at magian.com (Daniel Greig)
Date: Wed, 11 Jul 2007 11:16:01 +1000
Subject: [Backgroundrb-devel] plugin not loading for model? (acts_as_ferret)
Message-ID: <46942F51.3050006@magian.com>

Hi all,

I'm using the older win32 happy backgroundrb in combination with the 
'acts_as_ferret' install as a plugin.
Since I've installed acts_as_ferret, I haven't been able to get 
backgroundrb to start. It keeps failing at the 'acts_as_ferret' method 
call in one of my models.
Here's a code snippet of the model:

class Book < ActiveRecord::Base  
  acts_as_ferret :fields => [:title, :description, :subject, :author]

etc......

It keeps failing when it gets to the acts_as_ferret call in the model:

./script/backgroundrb/../../config/../vendor/rails/activerecord/lib/active_record/base.rb:1248:in 
`method_missing': undefined method `acts_as_ferret' for Book:Class 
(NoMethodError)
from ./script/backgroundrb/../../config/../app/models/book.rb:2
from c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in 
`require'
....

I'm guessing that backgroundrb isn't loading other plugins whent the 
rails environment loads, but I'm not really sure. I've tried to require 
acts_as_ferret in the backgroundrb start script (but I'm probably doing 
it wrong).
Is anyone able to help with how to get this working, or am I completely 
off track?

thanks,
Dan




From sbwoodside at yahoo.com  Wed Jul 11 01:12:29 2007
From: sbwoodside at yahoo.com (S. Woodside)
Date: Wed, 11 Jul 2007 01:12:29 -0400
Subject: [Backgroundrb-devel] Problem with scheduled task mysteriously
	stopping
Message-ID: <30F637F6-6A0C-4F25-B686-AC9F606727AF@yahoo.com>

Hi there,

I have a scheduled task which I started from Rails, set to run every  
5 minutes, it takes probably about 20-30 seconds to execute (it's  
aggregating some feeds). It ran for about 5 hours and then  
mysteriously stopped -- the logs show no errors.

This kind of sucks because it's like a silent failure, has anyone  
else had this problem?

--simon

--
http://simonwoodside.com



From ruby at geoffgarside.co.uk  Wed Jul 11 04:43:15 2007
From: ruby at geoffgarside.co.uk (Geoff Garside)
Date: Wed, 11 Jul 2007 09:43:15 +0100
Subject: [Backgroundrb-devel] plugin not loading for model?
	(acts_as_ferret)
In-Reply-To: <46942F51.3050006@magian.com>
References: <46942F51.3050006@magian.com>
Message-ID: <9A97C921-C802-4DF4-9887-792AA5E0CFAF@geoffgarside.co.uk>

On 11 Jul 2007, at 02:16, Daniel Greig wrote:

> Hi all,
>
> I'm using the older win32 happy backgroundrb in combination with the
> 'acts_as_ferret' install as a plugin.
> Since I've installed acts_as_ferret, I haven't been able to get
> backgroundrb to start. It keeps failing at the 'acts_as_ferret' method
> call in one of my models.
> Here's a code snippet of the model:
>
> class Book < ActiveRecord::Base
>   acts_as_ferret :fields => [:title, :description, :subject, :author]

You'll need the ':remote => true' option on the acts_as_ferret call.

> etc......
>
> It keeps failing when it gets to the acts_as_ferret call in the model:
>
> ./script/backgroundrb/../../config/../vendor/rails/activerecord/lib/ 
> active_record/base.rb:1248:in
> `method_missing': undefined method `acts_as_ferret' for Book:Class
> (NoMethodError)
> from ./script/backgroundrb/../../config/../app/models/book.rb:2
> from c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
> `require'
> ....
>
> I'm guessing that backgroundrb isn't loading other plugins whent the
> rails environment loads, but I'm not really sure. I've tried to  
> require
> acts_as_ferret in the backgroundrb start script (but I'm probably  
> doing
> it wrong).
> Is anyone able to help with how to get this working, or am I  
> completely
> off track?

You might need to tweak your backgroundrb rails middle man to make  
sure its loading in the plugins, I would expect it to be doing so  
though, but I've not used the earlier versions of backgroundrb.

>
> thanks,
> Dan
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

Hope that helps,

Geoff


From wil at 3cglabs.com  Wed Jul 11 11:43:54 2007
From: wil at 3cglabs.com (Wil Chung)
Date: Wed, 11 Jul 2007 10:43:54 -0500
Subject: [Backgroundrb-devel] Problem with scheduled task mysteriously
	stopping
In-Reply-To: <30F637F6-6A0C-4F25-B686-AC9F606727AF@yahoo.com>
References: <30F637F6-6A0C-4F25-B686-AC9F606727AF@yahoo.com>
Message-ID: <4cdd0d160707110843t5c6cca3s6e42ebc1b5e179d5@mail.gmail.com>

>From what I've read in the threads, yes.

I'm only aware of one place that eats up exceptions.
http://rubyforge.org/pipermail/backgroundrb-devel/2007-April/000875.html

Other than that, you might have to comb through and look for other places
that are eating exceptions.

Good luck.  Hope this helps.

Wilhelm

-- 
http://www.3cglabs.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070711/7fdf8273/attachment.html 

From ezmobius at gmail.com  Thu Jul 12 14:00:07 2007
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Thu, 12 Jul 2007 11:00:07 -0700
Subject: [Backgroundrb-devel] Backgroundrb needs a new maintainer.
Message-ID: <6DF3C08C-CFE7-461D-9FEF-168BB4110F71@gmail.com>

Hello All-

	Is there anyone out there that wants to take over as maintainer of  
the backgroundrb project? I feel bad because I know lots of folks use  
it but I absolutely do not have time to work on it anymore :(.

	I'm looking for a new maintainer, I would give you all the access to  
the project you need and try to help you come up to speed on the  
codebase.

	If anyone is interested please let me know

Thanks
-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)





From gethemant at gmail.com  Fri Jul 13 19:38:20 2007
From: gethemant at gmail.com (hemant)
Date: Sat, 14 Jul 2007 05:08:20 +0530
Subject: [Backgroundrb-devel] [Rails] Re: Backgroundrb needs a new
	maintainer.
In-Reply-To: <46977C6A.8040408@bouton.name>
References: <6DF3C08C-CFE7-461D-9FEF-168BB4110F71@gmail.com>
	<46977C6A.8040408@bouton.name>
Message-ID: <fb283d5e0707131638r3fce9ba8t3523b38f4c62d1a7@mail.gmail.com>

On 7/13/07, Lionel Bouton <lionel-subscription at bouton.name> wrote:
>
> Ezra Zygmuntowicz wrote:
> > Hello All-
> >
> >       Is there anyone out there that wants to take over as maintainer of
> > the backgroundrb project? I feel bad because I know lots of folks use
> > it but I absolutely do not have time to work on it anymore :(.
> >
> >       I'm looking for a new maintainer, I would give you all the access to
> > the project you need and try to help you come up to speed on the
> > codebase.
> >
>
> I'm not available right now (I'm working on stompserver right now which
> is more urgent for me), but I'll need backgroundrb later for one of my
> projects so if nobody volunteers in the following months, eventually
> I'll make sure backgroundrb is maintained.
>

Ezra,

Although I am a bit busy, but i would be interested in maintaining
backgroundrb. I am already using it in couple of my projects and
generally know my way around the backgroundrb code.


-- 
gnufied

From ezmobius at gmail.com  Fri Jul 13 19:44:18 2007
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Fri, 13 Jul 2007 16:44:18 -0700
Subject: [Backgroundrb-devel] [Rails] Re: Backgroundrb needs a new
	maintainer.
In-Reply-To: <fb283d5e0707131638r3fce9ba8t3523b38f4c62d1a7@mail.gmail.com>
References: <6DF3C08C-CFE7-461D-9FEF-168BB4110F71@gmail.com>
	<46977C6A.8040408@bouton.name>
	<fb283d5e0707131638r3fce9ba8t3523b38f4c62d1a7@mail.gmail.com>
Message-ID: <39026747-B290-46D3-8896-8186C916FAA6@gmail.com>


On Jul 13, 2007, at 4:38 PM, hemant wrote:

> On 7/13/07, Lionel Bouton <lionel-subscription at bouton.name> wrote:
>>
>> Ezra Zygmuntowicz wrote:
>>> Hello All-
>>>
>>>       Is there anyone out there that wants to take over as  
>>> maintainer of
>>> the backgroundrb project? I feel bad because I know lots of folks  
>>> use
>>> it but I absolutely do not have time to work on it anymore :(.
>>>
>>>       I'm looking for a new maintainer, I would give you all the  
>>> access to
>>> the project you need and try to help you come up to speed on the
>>> codebase.
>>>
>>
>> I'm not available right now (I'm working on stompserver right now  
>> which
>> is more urgent for me), but I'll need backgroundrb later for one  
>> of my
>> projects so if nobody volunteers in the following months, eventually
>> I'll make sure backgroundrb is maintained.
>>
>
> Ezra,
>
> Although I am a bit busy, but i would be interested in maintaining
> backgroundrb. I am already using it in couple of my projects and
> generally know my way around the backgroundrb code.
>
>
> -- 
> gnufied


	Thanks guys. Brett Rogers has agreed to step up as well. Maybe you  
both can work together? His email is brett.a.rogers at gmail.com and he  
has commit rights now. If you would like commit rights please sign up  
for an account at the trac and let me know what email address you  
used and I'll give you commit.

Cheers-

-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From lebreeze at gmail.com  Tue Jul 17 04:57:21 2007
From: lebreeze at gmail.com (Levent Ali)
Date: Tue, 17 Jul 2007 09:57:21 +0100
Subject: [Backgroundrb-devel] Backgroundrb scheduler seems to stop running
Message-ID: <76685bc50707170157p69d5a56bkd8501e28d741cb11@mail.gmail.com>

I have two scheduled tasks in my rails app:

background_schedules.yml

mail_sender_1:
  :class: :mail_sender_worker
  :job_key: :job_key_mail_sender_1
  :worker_method: :do_work
  :trigger_args:
    :start: <%= Time.now + 5.seconds %>
    :end: <%= 1.years.from_now %>
    :repeat_interval: <%= 60.minutes %>

cleanup_1:
  :class: :cleanup_worker
  :job_key: :job_key_cleanup_1
  :worker_method: :do_work
  :trigger_args:
    :start: <%= Time.now + 30.minutes %>
    :end: <%= 1.years.from_now %>
    :repeat_interval: <%= 60.minutes %>

Every morning between midnight and 2am they stop being triggered...

Everything else seems to run fine...

I have another background rb worker that is triggered when an external
app makes a call via an api... that one keeps running..
There are no errors in any of the logs, just that my scheduled tasks
never run again, unless I stop and start backgroundrb

my log:
20070716-22:19:16 (26124) start sending mails
20070716-22:19:17 (26124) finished sending mails
20070716-22:49:11 (26124) do_work in CleanupWorker called
20070716-22:49:11 (26124) started cleanup
20070716-22:49:11 (26124) finished cleanup
20070716-23:19:16 (26124) do_work in MailSenderWorker called
20070716-23:19:16 (26124) start sending mails
20070716-23:19:17 (26124) finished sending mails
20070716-23:49:11 (26124) do_work in CleanupWorker called
20070716-23:49:11 (26124) started cleanup
20070716-23:49:11 (26124) finished cleanup
20070717-08:19:45 (26124) do_work in ProcessVacancyWorker called
20070717-08:19:45 (26124) start processing vacancy
20070717-08:20:59 (26124) finished processing vacancy

As you can see the mail and cleanup workers last ran before midnight...
they should run every hour but there is a gap till 08:19 when the api
was called...

I added the end parameter to my schedules yaml but it made no
difference...

Any ideas?

thanks
levent

From ben.haynor at gmail.com  Wed Jul 18 16:07:18 2007
From: ben.haynor at gmail.com (Ben Haynor)
Date: Wed, 18 Jul 2007 15:07:18 -0500
Subject: [Backgroundrb-devel] RMagick progress monitor with backgroundrb -
	possible?
Message-ID: <ffd27e50707181307tbcfc128ha9caf1daab268c8@mail.gmail.com>

I'm looking for a way to give the user a progress monitor on large
image manipulations with RMagick.  I first looked at this method -
http://www.imagemagick.org/RMagick/doc/imageattrs.html#monitor - to do
it in a controller action, but since I can't call multiple renders in
an action, I would need to store the progress in the database and call
another action from javascript to look it up, which seems like
overkill(although maybe there is a better way I haven't thought of?).
So, I was wondering if it is possible to do it in a more
straightforward way with backgroundrb?

Thanks,
Ben

From wil at 3cglabs.com  Wed Jul 18 17:06:55 2007
From: wil at 3cglabs.com (Wil Chung)
Date: Wed, 18 Jul 2007 16:06:55 -0500
Subject: [Backgroundrb-devel] RMagick progress monitor with backgroundrb
	- possible?
In-Reply-To: <ffd27e50707181307tbcfc128ha9caf1daab268c8@mail.gmail.com>
References: <ffd27e50707181307tbcfc128ha9caf1daab268c8@mail.gmail.com>
Message-ID: <4cdd0d160707181406r1537e25an4a2fe69193307a52@mail.gmail.com>

If monitor() does what I think it does, then it's probably possible.

You can store the progress in the worker status, and then call an action
like "check_status" that asks for the worker's status.  If you were using
rails, you would then return an RJS template (or render a partial) to update
the page without reloading.  To call this action, you can use
periodically_remote (or something like that), or write your own javascript
that calls the action periodically with javascript's setInterval().

Wil

On 7/18/07, Ben Haynor <ben.haynor at gmail.com> wrote:
>
> I'm looking for a way to give the user a progress monitor on large
> image manipulations with RMagick.  I first looked at this method -
> http://www.imagemagick.org/RMagick/doc/imageattrs.html#monitor - to do
> it in a controller action, but since I can't call multiple renders in
> an action, I would need to store the progress in the database and call
> another action from javascript to look it up, which seems like
> overkill(although maybe there is a better way I haven't thought of?).
> So, I was wondering if it is possible to do it in a more
> straightforward way with backgroundrb?
>
> Thanks,
> Ben
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
http://webjazz.blogspot.com
http://www.mobtropolis.com
http://www.3cglabs.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070718/528a8e41/attachment.html 

From levent at leventali.com  Thu Jul 19 12:50:29 2007
From: levent at leventali.com (Levent Ali)
Date: Thu, 19 Jul 2007 17:50:29 +0100
Subject: [Backgroundrb-devel] Scheduler skipping schedules
Message-ID: <76685bc50707190950je33227dpd911073702b543a5@mail.gmail.com>

There is a line in the backgroundrb codebase:

sleep 0.1  # TODO this is dangerous; could skip over some jobs

Could this explain why my scheduler stops triggering?

It only seems to work for about 12 - 24 hours before schedules are
being skipped...

Once I changed the times to using cron trigger in
backgroundrb_schedules.yml and set the times to every 37th and 59th
minute of the hour, instead of the previous repeat_interval of
60.minutes it seemed to improve but it still stops being triggered
within a day and doesn't seem to start again unless I stop and start
background rb...

Sometimes this is both my schedules, sometimes just one of them...
Other tasks that are directly triggered still work...

This is really driving us a bit nuts as we can't get to the bottom of
it and nobody else seems to have any ideas?

I did see a post on an Exception not being caught and took care of
that but it didn't seem to make any difference... The code is
thoroughly tested and I really doubt it's the application code unless
the sheer volume of data could affect it?

From gethemant at gmail.com  Thu Jul 19 17:55:44 2007
From: gethemant at gmail.com (hemant)
Date: Fri, 20 Jul 2007 03:25:44 +0530
Subject: [Backgroundrb-devel] Scheduler skipping schedules
In-Reply-To: <76685bc50707190950je33227dpd911073702b543a5@mail.gmail.com>
References: <76685bc50707190950je33227dpd911073702b543a5@mail.gmail.com>
Message-ID: <fb283d5e0707191455w170e502bu522cb1b61e0dac3f@mail.gmail.com>

On 7/19/07, Levent Ali <levent at leventali.com> wrote:
> There is a line in the backgroundrb codebase:
>
> sleep 0.1  # TODO this is dangerous; could skip over some jobs
>
> Could this explain why my scheduler stops triggering?
>
> It only seems to work for about 12 - 24 hours before schedules are
> being skipped...
>
> Once I changed the times to using cron trigger in
> backgroundrb_schedules.yml and set the times to every 37th and 59th
> minute of the hour, instead of the previous repeat_interval of
> 60.minutes it seemed to improve but it still stops being triggered
> within a day and doesn't seem to start again unless I stop and start
> background rb...
>
> Sometimes this is both my schedules, sometimes just one of them...
> Other tasks that are directly triggered still work...
>
> This is really driving us a bit nuts as we can't get to the bottom of
> it and nobody else seems to have any ideas?
>
> I did see a post on an Exception not being caught and took care of
> that but it didn't seem to make any difference... The code is
> thoroughly tested and I really doubt it's the application code unless
> the sheer volume of data could affect it?

Hi Levent,

We are working on a new core that hopefully fixes these issues with
scheduling(I have myself observed problems with scheduling and
difficult to reproduce cases), in meanwhile I would see if this
problem can be fixed with a workaround.

From sbwoodside at yahoo.com  Thu Jul 19 22:10:45 2007
From: sbwoodside at yahoo.com (S. Woodside)
Date: Thu, 19 Jul 2007 22:10:45 -0400
Subject: [Backgroundrb-devel] Problem with scheduled task mysteriously
	stopping
In-Reply-To: <4cdd0d160707110843t5c6cca3s6e42ebc1b5e179d5@mail.gmail.com>
References: <30F637F6-6A0C-4F25-B686-AC9F606727AF@yahoo.com>
	<4cdd0d160707110843t5c6cca3s6e42ebc1b5e179d5@mail.gmail.com>
Message-ID: <89E0C5D9-78C0-4557-AE13-87DD42325C45@yahoo.com>

Well, I restarted mine and it's been running fine for a week or so.

It would be very nice if errors & exceptions were automatically  
logged without having to do extra work.

--simon

--
http://simonwoodside.com


On Jul 11, 2007, at 11:43 AM, Wil Chung wrote:

> >From what I've read in the threads, yes.
>
> I'm only aware of one place that eats up exceptions.
> http://rubyforge.org/pipermail/backgroundrb-devel/2007-April/ 
> 000875.html
>
> Other than that, you might have to comb through and look for other  
> places that are eating exceptions.
>
> Good luck.  Hope this helps.
>
> Wilhelm
>
> -- 
> http://www.3cglabs.com
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel


From julien.biard at citycampus.com  Fri Jul 20 08:35:30 2007
From: julien.biard at citycampus.com (Julien Biard)
Date: Fri, 20 Jul 2007 14:35:30 +0200
Subject: [Backgroundrb-devel] Backgroundrb and timeout connections
Message-ID: <46A0AC12.8020506@citycampus.com>

Hi

We have a RoR application which needs to get the contents of RSS feeds. 
The application opens HTTP connections, but the problem is that we have 
frequently Timeout::Error because of feeds are not available. It seems 
it'not possible to reduce the time of timeout in Ruby... So, I'm 
wondering if there is way to solve this with Backgroundrb.

Best regards,


Julien

From gethemant at gmail.com  Fri Jul 20 10:37:25 2007
From: gethemant at gmail.com (hemant)
Date: Fri, 20 Jul 2007 20:07:25 +0530
Subject: [Backgroundrb-devel] Backgroundrb and timeout connections
In-Reply-To: <46A0AC12.8020506@citycampus.com>
References: <46A0AC12.8020506@citycampus.com>
Message-ID: <fb283d5e0707200737h2d9cac28t3c6c7d919a404683@mail.gmail.com>

On 7/20/07, Julien Biard <julien.biard at citycampus.com> wrote:
> Hi
>
> We have a RoR application which needs to get the contents of RSS feeds.
> The application opens HTTP connections, but the problem is that we have
> frequently Timeout::Error because of feeds are not available. It seems
> it'not possible to reduce the time of timeout in Ruby... So, I'm
> wondering if there is way to solve this with Backgroundrb.
>
> Best regards,

I am afraid BackgrounDRb can't help you there, but you can handle
timeout errors like this.

class FeedWorker
  def do_work
    yahoo_url = "http://finance.yahoo.com/rss/headline?s="
    symbol_list = ['IBM','GOOG','FUCK']

    symbol_list.each do|sym|
      temp_url = yahoo_url + sym
      begin
        get_feeds(temp_url,sym)
      rescue Timeout::Error
        next
      rescue
        puts $!
        puts $!.backtrace
        next
      end
    end
  end

  def get_feeds(url,temp_sym)
    open(url) do |http|
      t_response = http.read
      puts t_response
    end
  end
end

If you really want to reduce timeout period for attempting fetch of a
page, then you call the method get_feeds in a block like this:

begin
  timeout(3) do
    get_feeds(temp_url,sym)
  end
rescue Timeout::Error

end

I hope this solves your problem.

-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://blog.gnufied.org

From gethemant at gmail.com  Fri Jul 20 10:38:34 2007
From: gethemant at gmail.com (hemant)
Date: Fri, 20 Jul 2007 20:08:34 +0530
Subject: [Backgroundrb-devel] Problem with scheduled task mysteriously
	stopping
In-Reply-To: <89E0C5D9-78C0-4557-AE13-87DD42325C45@yahoo.com>
References: <30F637F6-6A0C-4F25-B686-AC9F606727AF@yahoo.com>
	<4cdd0d160707110843t5c6cca3s6e42ebc1b5e179d5@mail.gmail.com>
	<89E0C5D9-78C0-4557-AE13-87DD42325C45@yahoo.com>
Message-ID: <fb283d5e0707200738y43963d26ye4440594bb90c0f2@mail.gmail.com>

On 7/20/07, S. Woodside <sbwoodside at yahoo.com> wrote:
> Well, I restarted mine and it's been running fine for a week or so.
>
> It would be very nice if errors & exceptions were automatically
> logged without having to do extra work.
>
> --simon
>
> --
> http://simonwoodside.com
>
>
> On Jul 11, 2007, at 11:43 AM, Wil Chung wrote:
>
> > >From what I've read in the threads, yes.
> >
> > I'm only aware of one place that eats up exceptions.
> > http://rubyforge.org/pipermail/backgroundrb-devel/2007-April/
> > 000875.html
> >
> > Other than that, you might have to comb through and look for other
> > places that are eating exceptions.
> >
> > Good luck.  Hope this helps.
> >
> > Wilhelm
> >
> > --

These fixes has been commited to trunk.

thanks

-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

From jonathan.wallace at gmail.com  Thu Jul 26 13:21:31 2007
From: jonathan.wallace at gmail.com (Jonathan Wallace)
Date: Thu, 26 Jul 2007 13:21:31 -0400
Subject: [Backgroundrb-devel] Behaviour of pool_size setting
In-Reply-To: <8bca3aa10705160627g3e8e6bb7w8047a49bdbeed8d4@mail.gmail.com>
References: <4649FDD2.10804@wms-network.de> <20070516124054.GB7613@waste.org>
	<8bca3aa10705160627g3e8e6bb7w8047a49bdbeed8d4@mail.gmail.com>
Message-ID: <fd0631d50707261021t828d59er904f7f427aedaa84@mail.gmail.com>

On 5/16/07, Mason Hale <masonhale at gmail.com> wrote:
>
> I resolved a similar situation by pre-allocating a fixed-size pool of
> workers.
> These workers are long-running and they individually poll the database
> for new work.


I'd prefer to not have the workers polling the database, especially if the
amount of workers is large.  Since responsiveness is one of the goals of my
app, I'd prefer to let users queries slow down the db. :)

My previous implementation had a single long-running worker that would spawn
> additional workers as needed to handle the work in the queue. But my
> experience was that creating a worker from a worker leads to socket not
> found and connection reset by peer errors, and poor reliability in general.
> So I scrapped that in favor of the fixed-size pool above and it has been
> much more reliable.


Has anyone successfully implemented this technique in a high reliability
site?

I do think there is potential to create a fixed-size pool of workers, and
> another long-running 'queue' manager worker, and implement it such that the
> workers request work from the queue manager worker rather than the database.
> This way the queue manager could get  a block of work and parcel it out with
> fewer queries to the database. In addition the workers could check in with
> the manager when they are done, making it possible for the manager to
> compile statistics on the jobs. For me, using the database to manage the
> queue -- and the inherent multi-threading issues -- was the more expedient
> route.


How about using the result hash as way to push information from the "queue
managing" worker to other workers?  Has anyone tried something like this?
Is it reliable?  Or are there race conditions in with the queue-managing
worker writing to the hash and the other workers reading from their key?

-- 
Jonathan Wallace
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070726/b8b43560/attachment-0001.html 

From masonhale at gmail.com  Thu Jul 26 15:24:25 2007
From: masonhale at gmail.com (Mason Hale)
Date: Thu, 26 Jul 2007 14:24:25 -0500
Subject: [Backgroundrb-devel] Behaviour of pool_size setting
In-Reply-To: <fd0631d50707261011w372a37a8i40f80e6053769f8a@mail.gmail.com>
References: <4649FDD2.10804@wms-network.de> <20070516124054.GB7613@waste.org>
	<8bca3aa10705160627g3e8e6bb7w8047a49bdbeed8d4@mail.gmail.com>
	<fd0631d50707261011w372a37a8i40f80e6053769f8a@mail.gmail.com>
Message-ID: <8bca3aa10707261224t49d854c8uecc585edbed61935@mail.gmail.com>

>  How about using the result hash as way to push information from the "queue
> managing" worker to other workers?  Has anyone tried something like this?
> Is it reliable?  Or are there race conditions in with the queue-managing
> worker writing to the hash and the other workers reading from their key?

I highly suggest avoiding use of the result hash. The current
implementation suffers from some threading/locking issues that result
in very weird behavior when accessed concurrently.

See:
http://rubyforge.org/pipermail/backgroundrb-devel/2007-January/000639.html

Still I do think it would be possible to either fix the results hash
issue (which would be great!) or manage the multi-threaded access
issues in a queue manager worker yourself.

For my immediate needs, I punted and just store the queue of things to
be worked on in the db, and let the db server deal with the
concurrency issues. I agree it is a less than ideal solution - but
given my options and other priorities it was a quick easy fix to a
hairy problem. Perhaps a lightweight db like SQLite could be used just
to manage the queue -- and thereby offload the traffic from the main
db, while also saving the need to deal with the concurrency headaches.
(I've never used SQLite, so I'm not sure).

Mason

From jonathan.wallace at gmail.com  Thu Jul 26 16:02:39 2007
From: jonathan.wallace at gmail.com (Jonathan Wallace)
Date: Thu, 26 Jul 2007 16:02:39 -0400
Subject: [Backgroundrb-devel] Behaviour of pool_size setting
In-Reply-To: <8bca3aa10707261224t49d854c8uecc585edbed61935@mail.gmail.com>
References: <4649FDD2.10804@wms-network.de> <20070516124054.GB7613@waste.org>
	<8bca3aa10705160627g3e8e6bb7w8047a49bdbeed8d4@mail.gmail.com>
	<fd0631d50707261011w372a37a8i40f80e6053769f8a@mail.gmail.com>
	<8bca3aa10707261224t49d854c8uecc585edbed61935@mail.gmail.com>
Message-ID: <fd0631d50707261302r40a02568m7f29a099708f7149@mail.gmail.com>

On 7/26/07, Mason Hale <masonhale at gmail.com> wrote:
> >  How about using the result hash as way to push information from the "queue
> > managing" worker to other workers?  Has anyone tried something like this?
> > Is it reliable?  Or are there race conditions in with the queue-managing
> > worker writing to the hash and the other workers reading from their key?
>
> I highly suggest avoiding use of the result hash. The current
> implementation suffers from some threading/locking issues that result
> in very weird behavior when accessed concurrently.
>
> See:
> http://rubyforge.org/pipermail/backgroundrb-devel/2007-January/000639.html

Sorry, I hadn't read far enough back in the archives.  I'm not sure I
follow all the particulars but would this issue include where each
worker writes to its own key in the results hash?  I.e., multiple
writers to the hash but no two writers accessing the same key?

> Still I do think it would be possible to either fix the results hash
> issue (which would be great!) or manage the multi-threaded access
> issues in a queue manager worker yourself.

I'm going to try to run the test you detailed in that thread with the
question I present above to see what happens.  Depending on the
results, I may attempt a dive into the backgroundrb code.

As for multi-threaded (process) access in a queue manager worker, I've
already ruled that out unless absolutely necessary.  I see no need to
introduce concurrency programming (outside of use of bdrb) in my app
just yet.   Succinctly put, deadlocks would suck.

> For my immediate needs, I punted and just store the queue of things to
> be worked on in the db, and let the db server deal with the
> concurrency issues. I agree it is a less than ideal solution - but
> given my options and other priorities it was a quick easy fix to a

I'm currently storing the jobs to be completed in the db already to
ensure that no jobs are ever missed due to crashing by the bdrb
process, a worker or a server.  If each worker is idempotent, then it
doesn't matter if a job that didn't quite complete is re-run on a bdrb
restart / server restart.  Also, since I want a log of jobs completed,
it makes sense to t

As stated before, my concern is to limit the amount queries to the db
if at all possible. The reason for this being the future possibility
of multiple dedicated backgroundrb servers.  It seems unreasonable to
have a bunch of bdrb servers polling the db for jobs.  Do you find
that db caching eliminates the majority of my concern here for you?

> hairy problem. Perhaps a lightweight db like SQLite could be used just
> to manage the queue -- and thereby offload the traffic from the main
> db, while also saving the need to deal with the concurrency headaches.
> (I've never used SQLite, so I'm not sure).

Ha!  It sounds like you're thinking of ruby queue[0].  I thought of
using that for my current issue and foregoing bdrb altogether but I
don't think I like the idea of using ruby queue to execute
./script/runner statements.  It seems somewhat dirty for some reason.
Plus, I don't see any easy way to acquire the progress of the ruby
queue clients as they are running.

On another note, I remember reading in the archives about the problems
with workers spawning workers[1] .  Has anyone tried having a worker
call a method in a traditional rails class that spawns another worker?
 This is another task on my list of things to try.

0. http://codeforpeople.com/lib/ruby/rq/rq-3.1.0/ , found via
http://www.forbiddenweb.org/topic/270/index.html
1. among other threads,
http://rubyforge.org/pipermail/backgroundrb-devel/2007-February/000755.html
-- 
Jonathan Wallace

From raasdnil at gmail.com  Fri Jul 27 03:59:43 2007
From: raasdnil at gmail.com (Mikel Lindsaar)
Date: Fri, 27 Jul 2007 17:59:43 +1000
Subject: [Backgroundrb-devel] RMagick progress monitor with
Message-ID: <57a815bf0707270059y383d70d1i4b82fb45ec67b1f8@mail.gmail.com>

Hey Ben,

I had the same situation, though not with RMagick.

I made progress bars running inside of active scaffold.  But you can
make it work on anything.

Screen cast showing it running and short tutorial are at

http://www.blognow.com.au/q

Screencast:
http://www.blognow.com.au/q/64767/Screen_cast_of_multiple_progress_bar_tracker.html


Regards

Mikel

>
> I'm looking for a way to give the user a progress monitor on large
> image manipulations with RMagick.  I first looked at this method -
> http://www.imagemagick.org/RMagick/doc/imageattrs.html#monitor - to do
> it in a controller action, but since I can't call multiple renders in
> an action, I would need to store the progress in the database and call
> another action from javascript to look it up, which seems like
> overkill(although maybe there is a better way I haven't thought of?).
> So, I was wondering if it is possible to do it in a more
> straightforward way with backgroundrb?
>
> Thanks,
> Ben
>

From gethemant at gmail.com  Fri Jul 27 09:24:54 2007
From: gethemant at gmail.com (hemant)
Date: Fri, 27 Jul 2007 18:54:54 +0530
Subject: [Backgroundrb-devel] Behaviour of pool_size setting
In-Reply-To: <fd0631d50707261302r40a02568m7f29a099708f7149@mail.gmail.com>
References: <4649FDD2.10804@wms-network.de> <20070516124054.GB7613@waste.org>
	<8bca3aa10705160627g3e8e6bb7w8047a49bdbeed8d4@mail.gmail.com>
	<fd0631d50707261011w372a37a8i40f80e6053769f8a@mail.gmail.com>
	<8bca3aa10707261224t49d854c8uecc585edbed61935@mail.gmail.com>
	<fd0631d50707261302r40a02568m7f29a099708f7149@mail.gmail.com>
Message-ID: <fb283d5e0707270624w2b5cbb41t48895e2fcc2c342e@mail.gmail.com>

On 7/27/07, Jonathan Wallace <jonathan.wallace at gmail.com> wrote:
> On 7/26/07, Mason Hale <masonhale at gmail.com> wrote:
> > >  How about using the result hash as way to push information from the "queue
> > > managing" worker to other workers?  Has anyone tried something like this?
> > > Is it reliable?  Or are there race conditions in with the queue-managing
> > > worker writing to the hash and the other workers reading from their key?
> >
> > I highly suggest avoiding use of the result hash. The current
> > implementation suffers from some threading/locking issues that result
> > in very weird behavior when accessed concurrently.
> >
> > See:
> > http://rubyforge.org/pipermail/backgroundrb-devel/2007-January/000639.html
>
> Sorry, I hadn't read far enough back in the archives.  I'm not sure I
> follow all the particulars but would this issue include where each
> worker writes to its own key in the results hash?  I.e., multiple
> writers to the hash but no two writers accessing the same key?
>
> > Still I do think it would be possible to either fix the results hash
> > issue (which would be great!) or manage the multi-threaded access
> > issues in a queue manager worker yourself.
>
> I'm going to try to run the test you detailed in that thread with the
> question I present above to see what happens.  Depending on the
> results, I may attempt a dive into the backgroundrb code.
>
> As for multi-threaded (process) access in a queue manager worker, I've
> already ruled that out unless absolutely necessary.  I see no need to
> introduce concurrency programming (outside of use of bdrb) in my app
> just yet.   Succinctly put, deadlocks would suck.
>
> > For my immediate needs, I punted and just store the queue of things to
> > be worked on in the db, and let the db server deal with the
> > concurrency issues. I agree it is a less than ideal solution - but
> > given my options and other priorities it was a quick easy fix to a
>
> I'm currently storing the jobs to be completed in the db already to
> ensure that no jobs are ever missed due to crashing by the bdrb
> process, a worker or a server.  If each worker is idempotent, then it
> doesn't matter if a job that didn't quite complete is re-run on a bdrb
> restart / server restart.  Also, since I want a log of jobs completed,
> it makes sense to t
>
> As stated before, my concern is to limit the amount queries to the db
> if at all possible. The reason for this being the future possibility
> of multiple dedicated backgroundrb servers.  It seems unreasonable to
> have a bunch of bdrb servers polling the db for jobs.  Do you find
> that db caching eliminates the majority of my concern here for you?
>
> > hairy problem. Perhaps a lightweight db like SQLite could be used just
> > to manage the queue -- and thereby offload the traffic from the main
> > db, while also saving the need to deal with the concurrency headaches.
> > (I've never used SQLite, so I'm not sure).
>
> Ha!  It sounds like you're thinking of ruby queue[0].  I thought of
> using that for my current issue and foregoing bdrb altogether but I
> don't think I like the idea of using ruby queue to execute
> ./script/runner statements.  It seems somewhat dirty for some reason.
> Plus, I don't see any easy way to acquire the progress of the ruby
> queue clients as they are running.
>
> On another note, I remember reading in the archives about the problems
> with workers spawning workers[1] .  Has anyone tried having a worker
> call a method in a traditional rails class that spawns another worker?
>  This is another task on my list of things to try.

Jonathan,

Calling something in rails that invokes another, doesn't sound right.
Its not possible also in current scheme of things.



>
> 0. http://codeforpeople.com/lib/ruby/rq/rq-3.1.0/ , found via
> http://www.forbiddenweb.org/topic/270/index.html
> 1. among other threads,
> http://rubyforge.org/pipermail/backgroundrb-devel/2007-February/000755.html
> --
> Jonathan Wallace
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://blog.gnufied.org

From dazuma at alumni.caltech.edu  Sun Jul 29 14:29:39 2007
From: dazuma at alumni.caltech.edu (Daniel Azuma)
Date: Sun, 29 Jul 2007 11:29:39 -0700
Subject: [Backgroundrb-devel] Server dying with perpetual "Connection reset
	by peer"
In-Reply-To: <mailman.45.1179837138.4652.backgroundrb-devel@rubyforge.org>
References: <mailman.45.1179837138.4652.backgroundrb-devel@rubyforge.org>
Message-ID: <BFEEA9CC-A01A-4371-92DC-0380A0209291@alumni.caltech.edu>

I'm at a loss to explain a very strange error I'm getting.

This seems to happen on our production system where we have a  
backgroundrb server sharing a host with a Rails app running on  
Mongrel. The first one or two, occasionally three, calls to a worker  
(direct MiddleMan calls from Rails, not scheduled) succeed, but  
afterwards, the server fails to initialize new workers. The following  
exception appears in the server logs, and I have to restart the  
server to get it to work again (for one or two requests before it  
dies again).

This appears to be the same issue that Peer Allan observed back in  
May-- in that case the workers were triggered on a schedule, but in  
my case it is triggered via a remote MiddleMan.

Does anyone have any idea what could cause this? Sample stack trace  
below:

Connection reset by peer - (DRb::DRbConnError)
/usr/local/lib/ruby/1.8/drb/drb.rb:566:in `read'
/usr/local/lib/ruby/1.8/drb/drb.rb:566:in `load'
/usr/local/lib/ruby/1.8/drb/drb.rb:632:in `recv_reply'
/usr/local/lib/ruby/1.8/drb/drb.rb:921:in `recv_reply'
/usr/local/lib/ruby/1.8/drb/drb.rb:1195:in `send_message'
/usr/local/lib/ruby/1.8/drb/drb.rb:1086:in `method_missing'
/usr/local/lib/ruby/1.8/drb/drb.rb:1170:in `open'
/usr/local/lib/ruby/1.8/drb/drb.rb:1085:in `method_missing'
/usr/local/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend'
/usr/local/lib/ruby/1.8/drb/drb.rb:1084:in `method_missing'
/usr/local/lib/ruby/1.8/drb/drb.rb:1072:in `respond_to?'
/usr/local/lib/ruby/gems/1.8/gems/slave-1.2.1/lib/slave.rb:454:in  
`initialize'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
middleman.rb:210:in `new'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
middleman.rb:210:in `new_worker'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:36:in `dispatch'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:22:in `initialize'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:22:in `new'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:22:in `dispatch'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
middleman.rb:199:in `new_worker'
/usr/local/lib/ruby/1.8/drb/drb.rb:1555:in `__send__'
/usr/local/lib/ruby/1.8/drb/drb.rb:1555:in `perform_without_block'
/usr/local/lib/ruby/1.8/drb/drb.rb:1515:in `perform'
/usr/local/lib/ruby/1.8/drb/drb.rb:1589:in `main_loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1585:in `loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1585:in `main_loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1581:in `start'
/usr/local/lib/ruby/1.8/drb/drb.rb:1581:in `main_loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1430:in `run'
/usr/local/lib/ruby/1.8/drb/drb.rb:1427:in `start'
/usr/local/lib/ruby/1.8/drb/drb.rb:1427:in `run'
/usr/local/lib/ruby/1.8/drb/drb.rb:1347:in `initialize'
/usr/local/lib/ruby/1.8/drb/drb.rb:1627:in `new'
/usr/local/lib/ruby/1.8/drb/drb.rb:1627:in `start_service'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb_server.rb:315:in `run'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
application.rb:194:in `call'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
application.rb:194:in `start_proc'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
daemonize.rb:192:in `call'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
daemonize.rb:192:in `call_as_daemon'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
application.rb:198:in `start_proc'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
application.rb:234:in `start'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
controller.rb:69:in `run'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons.rb:185:in  
`run_proc'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
cmdline.rb:105:in `call'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons/ 
cmdline.rb:105:in `catch_exceptions'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.7/lib/daemons.rb:184:in  
`run_proc'
/path/to/my/app/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb_server.rb:301:in `run'

Thanks,
Daniel Azuma


From masonhale at gmail.com  Mon Jul 30 11:06:28 2007
From: masonhale at gmail.com (Mason Hale)
Date: Mon, 30 Jul 2007 10:06:28 -0500
Subject: [Backgroundrb-devel] Server dying with perpetual "Connection
	reset by peer"
In-Reply-To: <BFEEA9CC-A01A-4371-92DC-0380A0209291@alumni.caltech.edu>
References: <mailman.45.1179837138.4652.backgroundrb-devel@rubyforge.org>
	<BFEEA9CC-A01A-4371-92DC-0380A0209291@alumni.caltech.edu>
Message-ID: <8bca3aa10707300806j749adef6t3304340d729d4a30@mail.gmail.com>

On 7/29/07, Daniel Azuma <dazuma at alumni.caltech.edu> wrote:

> This appears to be the same issue that Peer Allan observed back in
> May-- in that case the workers were triggered on a schedule, but in
> my case it is triggered via a remote MiddleMan.


By "triggered via a remote MiddleMan" do you mean that one backgroundrb
worker is spawning additional workers? If so, then that could be the
problem. I had a lot of connection reset by peer errors when trying to spawn
workers from other workers.

If not, are you using Unix sockets? I've seen cases where the process names
become very long, due to the class name and job key being concatenated into
the process name. If the name is too long, then backgroundrb may not be able
to locate a matching socket file to make the connection. If that might be
the case, try shortening your class names and/or setting explicit, short
jobkeys for your worker processes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070730/d6afb87f/attachment.html 

From dazuma at alumni.caltech.edu  Mon Jul 30 17:26:49 2007
From: dazuma at alumni.caltech.edu (Daniel Azuma)
Date: Mon, 30 Jul 2007 14:26:49 -0700
Subject: [Backgroundrb-devel] Server dying with perpetual "Connection
	reset by peer"
In-Reply-To: <8bca3aa10707300806j749adef6t3304340d729d4a30@mail.gmail.com>
References: <mailman.45.1179837138.4652.backgroundrb-devel@rubyforge.org>
	<BFEEA9CC-A01A-4371-92DC-0380A0209291@alumni.caltech.edu>
	<8bca3aa10707300806j749adef6t3304340d729d4a30@mail.gmail.com>
Message-ID: <9833DF93-D73A-435A-952E-7647E9B03F6B@alumni.caltech.edu>


On 30 Jul, 2007, at 08:06, Mason Hale wrote:

> On 7/29/07, Daniel Azuma <dazuma at alumni.caltech.edu> wrote:
>> This appears to be the same issue that Peer Allan observed back in
>> May-- in that case the workers were triggered on a schedule, but in
>> my case it is triggered via a remote MiddleMan.
>
> By "triggered via a remote MiddleMan" do you mean that one  
> backgroundrb worker is spawning additional workers? If so, then  
> that could be the problem. I had a lot of connection reset by peer  
> errors when trying to spawn workers from other workers.

Sorry, I didn't give a very clear description. All I meant was that  
I'm not using the scheduler (which other people seem to be having  
trouble with). I'm simply using MiddleMan from rails and spawning  
workers that way, not trying to spawn workers from other workers. I  
merely want to use backgroundrb to detach certain jobs from the HTTP  
request and run them asynchronously.

> If not, are you using Unix sockets? I've seen cases where the  
> process names become very long, due to the class name and job key  
> being concatenated into the process name. If the name is too long,  
> then backgroundrb may not be able to locate a matching socket file  
> to make the connection. If that might be the case, try shortening  
> your class names and/or setting explicit, short jobkeys for your  
> worker processes.

I've tried both the drbunix and druby protocols, if that's what  
you're asking, and have had the same result with both. (I assume the  
difference between them is that the former uses unix sockets whereas  
the latter uses ip?) However, I'll look into the process names  
anyway. Thanks.

Daniel