From rawdlite at googlemail.com  Fri Dec  1 11:42:15 2006
From: rawdlite at googlemail.com (tom roth)
Date: Fri, 1 Dec 2006 17:42:15 +0100
Subject: [Backgroundrb-devel] passing arguments to worker method
Message-ID: <d58e51190612010842p2abe4082y421b34b180276304@mail.gmail.com>

Hello all,

i just started using backgroundrb and i really like it a lot so far.
I finally succeeded in setting up version 0.21 and the examples from the
readme run fine.
Now i do not want to create and delete my worker anytime i want it to do
something but call repeatedly a method  with arguments.
There is a arg_method defined in the readme so i guess this is a feasible
way to procede.
Unfortunatly i could not find the syntax to pass args to such a method.
I hope this is not too stupid a question and you excuse my BASE.

Also i would like to know how to determine the version of backgroundrb. I
found it rather irritating to come across a lot of outdated documentation
and it took  me a while  to understand  that my rubyforge install was
outdated. Maybe you could consider writing a version string in
backgroundrb.log ?

regards

tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061201/4ce42629/attachment.html 

From skaar at waste.org  Fri Dec  1 11:59:31 2006
From: skaar at waste.org (skaar)
Date: Fri, 1 Dec 2006 10:59:31 -0600
Subject: [Backgroundrb-devel] passing arguments to worker method
In-Reply-To: <d58e51190612010842p2abe4082y421b34b180276304@mail.gmail.com>
References: <d58e51190612010842p2abe4082y421b34b180276304@mail.gmail.com>
Message-ID: <20061201165931.GA25824@waste.org>

* tom roth (rawdlite at googlemail.com) [061201 10:33]:
>    Hello all,
> 
>    i just started using backgroundrb and i really like it a lot so far.
>    I finally succeeded in setting up version 0.21 and the examples from the
>    readme run fine.
>    Now i do not want to create and delete my worker anytime i want it to do
>    something but call repeatedly a method  with arguments.
>    There is a arg_method defined in the readme so i guess this is a feasible
>    way to procede.
>    Unfortunatly i could not find the syntax to pass args to such a method.
>    I hope this is not too stupid a question and you excuse my BASE.

for worker methods you define yourself, there is no special syntax, you
would define it as you would with any ruby method.

If you are using the scheduler for periodic execution, you just have to
make sure that the arguments matches the one in the method definition.

/skaar

>    Also i would like to know how to determine the version of backgroundrb. I
>    found it rather irritating to come across a lot of outdated documentation
>    and it took  me a while  to understand  that my rubyforge install was
>    outdated. Maybe you could consider writing a version string in
>    backgroundrb.log ?

If you add this to trac, I'll add this to the next release. If you
really want to know the version right now, you can issue:

  > BackgrounDRb::VERSION

in the console.

/skaar

-- 
----------------------------------------------------------------------
|\|\             where in the       |          s_u_b_s_t_r_u_c_t_i_o_n 
| | >===========  W.A.S.T.E.        |                  genarratologies 
|/|/    (_)     is the wisdom       |                  skaar at waste.org
----------------------------------------------------------------------

From rawdlite at googlemail.com  Sat Dec  2 08:15:59 2006
From: rawdlite at googlemail.com (tom roth)
Date: Sat, 2 Dec 2006 14:15:59 +0100
Subject: [Backgroundrb-devel] passing arguments to worker method
In-Reply-To: <20061201165931.GA25824@waste.org>
References: <d58e51190612010842p2abe4082y421b34b180276304@mail.gmail.com>
	<20061201165931.GA25824@waste.org>
Message-ID: <d58e51190612020515q7d7280b8w2771843c57f06f0e@mail.gmail.com>

On 01/12/06, skaar <skaar at waste.org> wrote:
>
>
>
> for worker methods you define yourself, there is no special syntax, you
> would define it as you would with any ruby method.


Thanks a lot, this helped me.
Actualy i had already tried it, but my worker was not running and i got
confused by the error message.
So this leads me to the next question:
What is the best way to determine whether a worker for a given job_key is
running ?

If you are using the scheduler for periodic execution, you just have to
> make sure that the arguments matches the one in the method definition.
>
> /skaar
>
> >    Also i would like to know how to determine the version of
> backgroundrb. I
> >    found it rather irritating to come across a lot of outdated
> documentation
> >    and it took  me a while  to understand  that my rubyforge install was
> >    outdated. Maybe you could consider writing a version string in
> >    backgroundrb.log ?
>
> If you add this to trac, I'll add this to the next release. If you
> really want to know the version right now, you can issue:
>
>   > BackgrounDRb::VERSION
>
> in the console.


cool, i added it to trac then.

regards

tom



/skaar
>
> --
> ----------------------------------------------------------------------
> |\|\             where in the       |          s_u_b_s_t_r_u_c_t_i_o_n
> | | >===========  W.A.S.T.E.        |                  genarratologies
> |/|/    (_)     is the wisdom       |                   skaar at waste.org
> ----------------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061202/e364a3a2/attachment.html 

From james at imajes.info  Sun Dec  3 13:38:02 2006
From: james at imajes.info (James Cox)
Date: Sun, 3 Dec 2006 18:38:02 +0000
Subject: [Backgroundrb-devel] worker is not executed
Message-ID: <924D4046-043A-43F0-9115-F3E2B4844C7A@imajes.info>

Hey,

I've got a problem where my worker is not being properly executed....  
it used to work in an earlier version of brb, but i realized my  
vendor/plugins was out of whack, upgraded to the newer schedule- 
oriented release (0.2.1). I can start brb, and run the new_worker  
middleman, but ntohing happens.

When putting brb in 'run' mode, i see this:

undefined method `wait' for nil:NilClass - (NoMethodError)
/Users/james/Sites/client/backend/trunk/vendor/plugins/backgroundrb/ 
server/lib/backgroundrb/middleman.rb:217:in `new_worker'
/Users/james/Sites/client/backend/trunk/vendor/plugins/backgroundrb/ 
server/lib/backgroundrb/thread_pool.rb:36:in `dispatch'
/Users/james/Sites/client/backend/trunk/vendor/plugins/backgroundrb/ 
server/lib/backgroundrb/thread_pool.rb:22:in `initialize'
/Users/james/Sites/client/backend/trunk/vendor/plugins/backgroundrb/ 
server/lib/backgroundrb/thread_pool.rb:22:in `new'
/Users/james/Sites/client/backend/trunk/vendor/plugins/backgroundrb/ 
server/lib/backgroundrb/thread_pool.rb:22:in `dispatch'
/Users/james/Sites/client/backend/trunk/vendor/plugins/backgroundrb/ 
server/lib/backgroundrb/middleman.rb:199:in `new_worker'
<snip>
/Users/james/Sites/client/backend/trunk/vendor/plugins/backgroundrb/ 
server/lib/backgroundrb_server.rb:301:in `run'
script/backgroundrb:29

but i also get a job_key -- But, looking at the code i'm not  
convinced this is a serious error.

still, i've tried to execute a logger statement, first line in my  
do_work method and it doesn't activate.

Any thoughts?

Best,

James Cox

From rawdlite at googlemail.com  Mon Dec  4 10:02:28 2006
From: rawdlite at googlemail.com (tom roth)
Date: Mon, 4 Dec 2006 16:02:28 +0100
Subject: [Backgroundrb-devel] no method error when starting backgroundrb
Message-ID: <d58e51190612040702mf0db205u99a1ab6fa5b637f2@mail.gmail.com>

I just installed backgroundrb on our server. When tryin to start via
script/backgroundrb i get a no method error.

 ./script/backgroundrb start
/home/tom/ralf_dev/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:34:in
`format_message': undefined method `strftime' for "2006-12-04T15:53:
23.419225 ":String (NoMethodError)
        from /usr/lib/ruby/1.8/logger.rb:320:in `add'
        from /usr/lib/ruby/1.8/logger.rb:372:in `info'
        from
/home/tom/ralf_dev/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:168:in
`setup'
        from
/home/tom/ralf_dev/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:293:in
`run'
        from ./script/backgroundrb:29

system is linux 2.6.17.7 debian.
it works on my mac.
Any ideas ?


regards tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061204/3eca061d/attachment-0001.html 

From rawdlite at googlemail.com  Mon Dec  4 10:36:32 2006
From: rawdlite at googlemail.com (tom roth)
Date: Mon, 4 Dec 2006 16:36:32 +0100
Subject: [Backgroundrb-devel] no method error when starting backgroundrb
In-Reply-To: <d58e51190612040702mf0db205u99a1ab6fa5b637f2@mail.gmail.com>
References: <d58e51190612040702mf0db205u99a1ab6fa5b637f2@mail.gmail.com>
Message-ID: <d58e51190612040736q33f9d090h384d83d7eceea522@mail.gmail.com>

I guess i just found the answer to this in the mail archive.
I need to update my ruby version which was 1.8.2

regards tom

On 04/12/06, tom roth <rawdlite at googlemail.com> wrote:
>
> I just installed backgroundrb on our server. When tryin to start via
> script/backgroundrb i get a no method error.
>
>  ./script/backgroundrb start
> /home/tom/ralf_dev/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:34:in
> `format_message': undefined method `strftime' for "2006-12-04T15:53:
> 23.419225 ":String (NoMethodError)
>         from /usr/lib/ruby/1.8/logger.rb:320:in `add'
>         from /usr/lib/ruby/1.8/logger.rb:372:in `info'
>         from
> /home/tom/ralf_dev/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:168:in
> `setup'
>         from
> /home/tom/ralf_dev/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:293:in
> `run'
>         from ./script/backgroundrb:29
>
> system is linux 2.6.17.7 debian.
> it works on my mac.
> Any ideas ?
>
>
> regards tom
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061204/0d894173/attachment.html 

From sweiss at stylesight.com  Mon Dec  4 14:35:14 2006
From: sweiss at stylesight.com (Stephen Weiss)
Date: Mon, 4 Dec 2006 14:35:14 -0500
Subject: [Backgroundrb-devel] Question about acls
Message-ID: <19FE4DC7-30BA-437C-B006-BD2A42BC9A73@stylesight.com>

Hi,

I'm not much of a developer but I've been using backgroundrb for a  
while now and with the release of 0.2.1 it looks like I can finally  
upgrade from the old version (0.2.0 had some weird issues when jobs  
just wouldn't run, seems to have cleared up now - Thanks!).

Anyhow, while I'm doing this, I would like to tackle this problem.   
Most tasks I run in the background involve image processing so  
they're rather processor-intensive, and it's not very good to have  
them running on the same machine that's running rails - it slows  
things down a lot.  I would like to keep running rails on my main  
server (server1) but move backgroundrb to a secondary server (server2).

I looked through the source trying to figure out how to do this and I  
pieced together this configuration file:

--
:port: 22222
:timer_sleep: 60
:pool_size: 15
:load_rails: true
:rails_env: development
:environment: development
:host: localhost
:uri: druby://localhost:22222/
:database_yml: config/database.yml
:protocol: druby
:acl:
   :deny: all
   :allow: localhost 127.0.0.1 server1
   :order: deny,allow
autostart:
   1:
     job_key: chat_notifier
     class: chat_notification_worker

Which runs on server2.  (I know, some of the options probably don't  
belong there but when it's set up on a standalone server it works,  
so...)

I'm using the druby protocol, which I'm pretty sure works over TCP  
instead of UDP, which seems to be what I want.

This all runs fine, except that it seems like my acl is completely  
ignored or maybe it's just not written properly.  If I try to connect  
just using telnet to test whether or not the port is open, I can  
connect to the IPv6 localhost (::1) port 22222 from server2 just  
fine.  However, I can't connect to IPv4 localhost (127.0.0.1) port  
22222 from server2, and I definitely can't connect to server2 port  
22222 from server1.

I can run `script/backgroundrb console` OK from server2 as long  
as :uri is set to "druby://localhost:22222/" and :host is set to  
"localhost".  If I set :host to "server2" then backgroundrb runs, but  
the console fails with:

/usr/local/lib/ruby/1.8/drb/drb.rb:736:in `open': druby://localhost: 
22222 - #<Errno::ECONNREFUSED: Connection refused - connect(2)>  
(DRb::DRbConnError)
         from /usr/local/lib/ruby/1.8/drb/drb.rb:729:in `each'
         from /usr/local/lib/ruby/1.8/drb/drb.rb:729:in `open'
         from /usr/local/lib/ruby/1.8/drb/drb.rb:1189:in `initialize'
         from /usr/local/lib/ruby/1.8/drb/drb.rb:1169:in `new'
         from /usr/local/lib/ruby/1.8/drb/drb.rb:1169:in `open'
         from /usr/local/lib/ruby/1.8/drb/drb.rb:1085:in  
`method_missing'
         from /usr/local/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend'
         from /usr/local/lib/ruby/1.8/drb/drb.rb:1084:in  
`method_missing'
          ... 8 levels...
         from /web/trunk/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb/console.rb:9:in `backgroundrb_console_start'
         from /web/trunk/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb/console.rb:39:in `init'
         from /web/trunk/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb_server.rb:296:in `run'
         from script/backgroundrb:29

I thought maybe since it was pointing to localhost, and Mac OS X  
(this is running on OS X Server 10.4, both machines) has ::1 set up  
as the first resolved address for localhost, maybe backgroundrb was  
being confused into going only over IPv6, but using the IPv4 address  
completely doesn't work?  I've tried various combinations of  
localhost, server2, and server2's ip address in both the :host  
and :uri options with no success, unless both point to localhost, and  
no matter what I do, I can only connect via IPv6 localhost, despite  
the fact that my ACL allows connections from 127.0.0.1 and from  
server1 as well.  Setting the acl to :allow => "all" did not change a  
thing.

I know next to nothing about drb so if I'm missing something  
blatantly obvious I apologize.  Would really like to get this set up  
soon, even a cheap hack that'll work for the next week or so would be  
a godsend.  TIA for any help!

--
Steve

From jasonsydes at gmail.com  Mon Dec  4 19:55:54 2006
From: jasonsydes at gmail.com (Jason Sydes)
Date: Mon, 4 Dec 2006 16:55:54 -0800
Subject: [Backgroundrb-devel] Question about acls
In-Reply-To: <19FE4DC7-30BA-437C-B006-BD2A42BC9A73@stylesight.com>
References: <19FE4DC7-30BA-437C-B006-BD2A42BC9A73@stylesight.com>
Message-ID: <bb42e3ea0612041655x24924e34i3996591675c1e6ff@mail.gmail.com>

(Sorry if this ends up being a double post, sent from the wrong address
initially.)

Hi Steve,

Not sure if this will help, but a few things to note:

1) I've had to turn completely off my Mac OS X firewall to get DRb processes
to communicate to each other (if I don't I get the DRb::DRbConnError
exception that you also got).  I might've screwed something up, but that's
at least worth a shot.

2) I'm pretty sure (?) that you don't explicitly set the :uri in the config
file.  Instead, it looks like it's built from :host and :port.

3) Also, not sure if you're already doing this, but you probably want
separate config files for server1 and server2.  The Middleman proxy object
setup *and* the backgroundrb server setup both use the same config file.
The following isn't tested, but something like this might work (note that
you should probably just replace 'server1' and 'server2' with 'localhost'
since they're the same thing in your devel env)

# server 1
:host: server2
:port: 22222
:timer_sleep: 60
:pool_size: 15
:load_rails: true
:rails_env: development
:environment: development
:database_yml: config/database.yml
:protocol: druby

# server 2
:host: localhost
:port: 22222
:timer_sleep: 60
:pool_size: 15
:load_rails: true
:rails_env: development
:environment: development
:database_yml: config/database.yml
:protocol: druby
:acl:
  :deny: all
  :allow: localhost 127.0.0.1 server1
  :order: deny,allow
autostart:
  1:
    job_key: chat_notifier
    class: chat_notification_worker


Hope that helps,
Jason



On 12/4/06, Stephen Weiss <sweiss at stylesight.com> wrote:
>
> Hi,
>
> I'm not much of a developer but I've been using backgroundrb for a
> while now and with the release of 0.2.1 it looks like I can finally
> upgrade from the old version (0.2.0 had some weird issues when jobs
> just wouldn't run, seems to have cleared up now - Thanks!).
>
> Anyhow, while I'm doing this, I would like to tackle this problem.
> Most tasks I run in the background involve image processing so
> they're rather processor-intensive, and it's not very good to have
> them running on the same machine that's running rails - it slows
> things down a lot.  I would like to keep running rails on my main
> server (server1) but move backgroundrb to a secondary server (server2).
>
> I looked through the source trying to figure out how to do this and I
> pieced together this configuration file:
>
> --
> :port: 22222
> :timer_sleep: 60
> :pool_size: 15
> :load_rails: true
> :rails_env: development
> :environment: development
> :host: localhost
> :uri: druby://localhost:22222/
> :database_yml: config/database.yml
> :protocol: druby
> :acl:
>    :deny: all
>    :allow: localhost 127.0.0.1 server1
>    :order: deny,allow
> autostart:
>    1:
>      job_key: chat_notifier
>      class: chat_notification_worker
>
> Which runs on server2.  (I know, some of the options probably don't
> belong there but when it's set up on a standalone server it works,
> so...)
>
> I'm using the druby protocol, which I'm pretty sure works over TCP
> instead of UDP, which seems to be what I want.
>
> This all runs fine, except that it seems like my acl is completely
> ignored or maybe it's just not written properly.  If I try to connect
> just using telnet to test whether or not the port is open, I can
> connect to the IPv6 localhost (::1) port 22222 from server2 just
> fine.  However, I can't connect to IPv4 localhost (127.0.0.1) port
> 22222 from server2, and I definitely can't connect to server2 port
> 22222 from server1.
>
> I can run `script/backgroundrb console` OK from server2 as long
> as :uri is set to "druby://localhost:22222/" and :host is set to
> "localhost".  If I set :host to "server2" then backgroundrb runs, but
> the console fails with:
>
> /usr/local/lib/ruby/1.8/drb/drb.rb:736:in `open': druby://localhost:
> 22222 - #<Errno::ECONNREFUSED: Connection refused - connect(2)>
> (DRb::DRbConnError)
>          from /usr/local/lib/ruby/1.8/drb/drb.rb:729:in `each'
>          from /usr/local/lib/ruby/1.8/drb/drb.rb:729:in `open'
>          from /usr/local/lib/ruby/1.8/drb/drb.rb:1189:in `initialize'
>          from /usr/local/lib/ruby/1.8/drb/drb.rb:1169:in `new'
>          from /usr/local/lib/ruby/1.8/drb/drb.rb:1169:in `open'
>          from /usr/local/lib/ruby/1.8/drb/drb.rb:1085:in
> `method_missing'
>          from /usr/local/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend'
>          from /usr/local/lib/ruby/1.8/drb/drb.rb:1084:in
> `method_missing'
>           ... 8 levels...
>          from /web/trunk/vendor/plugins/backgroundrb/server/lib/
> backgroundrb/console.rb:9:in `backgroundrb_console_start'
>          from /web/trunk/vendor/plugins/backgroundrb/server/lib/
> backgroundrb/console.rb:39:in `init'
>          from /web/trunk/vendor/plugins/backgroundrb/server/lib/
> backgroundrb_server.rb:296:in `run'
>          from script/backgroundrb:29
>
> I thought maybe since it was pointing to localhost, and Mac OS X
> (this is running on OS X Server 10.4, both machines) has ::1 set up
> as the first resolved address for localhost, maybe backgroundrb was
> being confused into going only over IPv6, but using the IPv4 address
> completely doesn't work?  I've tried various combinations of
> localhost, server2, and server2's ip address in both the :host
> and :uri options with no success, unless both point to localhost, and
> no matter what I do, I can only connect via IPv6 localhost, despite
> the fact that my ACL allows connections from 127.0.0.1 and from
> server1 as well.  Setting the acl to :allow => "all" did not change a
> thing.
>
> I know next to nothing about drb so if I'm missing something
> blatantly obvious I apologize.  Would really like to get this set up
> soon, even a cheap hack that'll work for the next week or so would be
> a godsend.  TIA for any help!
>
> --
> Steve
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061204/0fcd1fdb/attachment.html 

From rawdlite at googlemail.com  Tue Dec  5 09:00:59 2006
From: rawdlite at googlemail.com (tom roth)
Date: Tue, 5 Dec 2006 15:00:59 +0100
Subject: [Backgroundrb-devel] worker method not running in background
Message-ID: <d58e51190612050600x7469845q2700898d77e1d41a@mail.gmail.com>

When i call a worker method my browsers hangs until the task is finished.
The same code works nicely in the background when inside the do_work method.
Does this mean i can only start a task in background via the do_work method
?
Or am i doing something wrong here ?

regards tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061205/43bae8ef/attachment.html 

From sweiss at stylesight.com  Tue Dec  5 09:44:31 2006
From: sweiss at stylesight.com (Stephen Weiss)
Date: Tue, 5 Dec 2006 09:44:31 -0500
Subject: [Backgroundrb-devel] Question about acls
In-Reply-To: <bb42e3ea0612041645i64da65b1t624685f66040a262@mail.gmail.com>
References: <19FE4DC7-30BA-437C-B006-BD2A42BC9A73@stylesight.com>
	<bb42e3ea0612041645i64da65b1t624685f66040a262@mail.gmail.com>
Message-ID: <C35B8358-C54B-4DED-BF50-8CC302110384@stylesight.com>

On Dec 4, 2006, at 7:45 PM, Jason Sydes wrote:

> Hi Stephen,
> Not sure if this will help, but a few things to note:

Thanks for the reply!
>
> 1) I've had to turn completely off my Mac OS X firewall to get DRb  
> processes to communicate to each other (if I don't I get the  
> DRb::DRbConnError exception that you also got).  I might've screwed  
> something up, but that's at least worth a shot.
>

My firewalls are totally off, I have a PIX for that.  The pix doesn't  
affect any traffic between the two machines (they're directly linked  
for file sharing, connections between the two servers go over this  
direct link).

> 2) I'm pretty sure (?) that you don't explicitly set the :uri in  
> the config file.  Instead, it looks like it's built from :host  
> and :port.

Yeah I wasn't sure about this either, but it doesn't seem to like it  
makes a difference.  If I take out the line for uri, it works just  
the same.

>
> 3) Also, not sure if you're already doing this, but you probably  
> want separate config files for server1 and server2.  The Middleman  
> proxy object setup *and* the backgroundrb server setup both use the  
> same config file.  The following isn't tested, but something like  
> this might work (note that you should probably just replace  
> 'server1' and 'server2' with 'localhost' since they're the same  
> thing in your devel env)

Yes, I understood this part.  There is a separate configuration file  
for server1.  However, I don't think the config file for server1 has  
much to do with the problem at hand - server2 is not allowing  
connections from ipv4 localhost, let alone remote servers, so the  
remote server(1)'s configuration is a moot point.  First I'll get it  
to the point where I can connect from server1 in the first place -  
then I'll worry about how server1 is configured.

That being said, my configuration for server1, I think, would be  
something like:

---
port: 22222
timer_sleep: 60
load_rails: true
environment: development
host: server2
uri: druby://server2:22222/
database_yml: config/database.yml
pool_size: 10

(again, maybe not the uri line)

And yes, the database.yml's are both set up correctly.  But honestly  
I hadn't even worried about it, without connectivity it won't help much.

I think I might try a different port, or just building a straight-up  
druby connection and see if that works, to eliminate any other  
trouble.  I still feel like it's something with the ACL though....   
Again thanks for any help!

--
Steve

From jasonsydes at gmail.com  Tue Dec  5 13:13:44 2006
From: jasonsydes at gmail.com (Jason Sydes)
Date: Tue, 5 Dec 2006 10:13:44 -0800
Subject: [Backgroundrb-devel] worker method not running in background
In-Reply-To: <d58e51190612050600x7469845q2700898d77e1d41a@mail.gmail.com>
References: <d58e51190612050600x7469845q2700898d77e1d41a@mail.gmail.com>
Message-ID: <bb42e3ea0612051013p69c19b70r5eeb7aeb49ce08f1@mail.gmail.com>

On 12/5/06, tom roth <rawdlite at googlemail.com> wrote:
>
> When i call a worker method my browsers hangs until the task is finished.
> The same code works nicely in the background when inside the do_work method.
> Does this mean i can only start a task in background via the do_work
> method ?
> Or am i doing something wrong here ?


Hi Tom,
You need to call your_method() via work_thread() so that your_method is
wrapped in a (background) ruby thread.  Something like:

your_worker.work_thread(:method => :your_method)
or maybe
your_worker.work_thread(:method => :your_method, :args => some_args)

If you try calling do_work directly (instead of via new_worker()) I'd expect
your browser to hang as well.

Hope that helps,
Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061205/d92e38f3/attachment.html 

From sweiss at stylesight.com  Tue Dec  5 15:18:06 2006
From: sweiss at stylesight.com (Stephen Weiss)
Date: Tue, 5 Dec 2006 15:18:06 -0500
Subject: [Backgroundrb-devel] Question about acls
Message-ID: <6482957C-8DD0-46AE-9D78-B71EC69D3705@stylesight.com>

So, I had some success.  Basically, I totally eliminated the acl by  
commenting out the line:

self.setup_drb_acl

in method 'setup' in backgroundrb_server.rb.

Then, the connection opens up (I have a firewall and this machine is  
totally blocked from the outside, I'm really not worried about  
leaving this open).  Important to note here is that you can't just  
remove the ACL from your configuration - you have to edit the source,  
since backgroundrb_server.rb will install its own ACL otherwise.

However, staying with port 22222 also proved to be a problem - even  
though I was able to connect, no data seemed to go through on port  
22222.  Switching to port 22223 completely solved this problem.

Now my program is very happy.  I can start backgroundrb on server2,  
and start rails on server1.  Then, when I do script/console and type  
MiddleMan.jobs on server1, I get a response.  If I search for a  
worker that isn't there, I get the expected error message, labelled  
from server2.

So, 2 basic things I'm taking away from this - port 22222 has some  
issue, at least on my machines - and it seems that applying any ACL  
blocks remote access, regardless of what's actually in the ACL.  One  
more thing - if I did set the ACL to :allow => "127.0.0.1" instead  
of :allow => "localhost 127.0.0.1", connections through 127.0.0.1  
worked, while as expected connections through ::1 failed.  No matter  
what, you have to completely disable the acl to get a connection from  
a remote machine.

Sorry if this is all documented a million times somewhere, I couldn't  
find it.  Thanks again for such useful software, it's almost perfect  
now!


One more question - is there an issue now with calling one worker's  
method from another worker?  I have a chat notification worker that  
works fine if it's called from rails or from the backgroundrb  
console, but if it's called from another worker nothing seems to  
happen (like, the method never gets called, there's no error or  
anything).  It's not a deal-breaker, but it did work in ver 1...

--
Steve

From skaar at waste.org  Tue Dec  5 15:23:18 2006
From: skaar at waste.org (skaar)
Date: Tue, 5 Dec 2006 14:23:18 -0600
Subject: [Backgroundrb-devel] Question about acls
In-Reply-To: <C35B8358-C54B-4DED-BF50-8CC302110384@stylesight.com>
References: <19FE4DC7-30BA-437C-B006-BD2A42BC9A73@stylesight.com>
	<bb42e3ea0612041645i64da65b1t624685f66040a262@mail.gmail.com>
	<C35B8358-C54B-4DED-BF50-8CC302110384@stylesight.com>
Message-ID: <20061205202317.GA20047@waste.org>

> port: 22222
> timer_sleep: 60
> load_rails: true
> environment: development
> host: server2
> uri: druby://server2:22222/
> database_yml: config/database.yml
> pool_size: 10

what if you do
   :port: 22222
   :host: server2
   :protocol: druby

and just not specify :uri: and let backgroundrb control that
and then:
   :acl:
     :order: deny allow
     :deny: all
     :allow: server1 server2

(you might want to try the IP of server1

/skaar

-- 
----------------------------------------------------------------------
|\|\             where in the       |          s_u_b_s_t_r_u_c_t_i_o_n 
| | >===========  W.A.S.T.E.        |                  genarratologies 
|/|/    (_)     is the wisdom       |                  skaar at waste.org
----------------------------------------------------------------------

From gethemant at gmail.com  Wed Dec  6 15:38:15 2006
From: gethemant at gmail.com (hemant)
Date: Thu, 7 Dec 2006 02:08:15 +0530
Subject: [Backgroundrb-devel] Testing your workers
Message-ID: <fb283d5e0612061238o3f70fafx54a0914f657b0cd9@mail.gmail.com>

This has been asked before, never well attended perhaps.

Any thoughts on testing your workers?

Current implementation is broken. An empty test case like this:

require File.dirname(__FILE__) + '/../test_helper'
require "#{RAILS_ROOT}/lib/workers/market_summary_worker"
require "#{RAILS_ROOT}/vendor/plugins/backgroundrb/lib/backgroundrb.rb"
require 'drb'

class MarketSummaryWorkerTest < Test::Unit::TestCase

  # Replace this with your real tests.
  def test_truth
    assert MarketSummaryWorker.included_modules.include?(DRbUndumped)
  end
end

Throws error:
------------
/usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:123:in
`const_missing': uninitialized constant Worker (NameError)
        from /home/hemant/ubac_webfront/config/../lib/workers/market_summary_worker.rb:9
        from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in
`require'
        from /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:147:in
`require'
        from test/unit/market_summary_worker_test.rb:2
----------------

So, I added couple of more files to the require and:
-------------------
require File.dirname(__FILE__) + '/../test_helper'

BACKGROUNDRB_CODE = File.dirname(__FILE__) +
'/../../vendor/plugins/backgroundrb'

$LOAD_PATH << File.join(BACKGROUNDRB_CODE + '/lib')
$LOAD_PATH << File.join(BACKGROUNDRB_CODE + '/server/lib')
$LOAD_PATH << File.join(BACKGROUNDRB_CODE + '/server/lib/backgroundrb')

require 'backgroundrb_server'
require "backgroundrb"
require "worker_rails"
require File.dirname(__FILE__) + '/../../lib/workers/add_analyst_worker'

require 'drb'
----------------


Now the error I get is:

---------------------
./test/unit/../../vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:158:in
`register_worker_class': You have a nil object when you didn't expect
it! (NoMethodError)
You might have expected an instance of Array.
The error occured while evaluating nil.include? from
./test/unit/../../vendor/plugins/backgroundrb/server/lib/backgroundrb/worker.rb:17:in
`register'
        from ./test/unit/../../lib/workers/add_analyst_worker.rb:117
        from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in
`require'
        from /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:147:in
`require'
        from test/unit/add_analyst_worker_test.rb:12
----------------------

So, I would imagine testing support is quite lame here.  Can anyone
point me, in the direction, what else I should require in the testcase
so that at least an empty test case executes?






-- 
There was only one Road; that it was like a great river: its springs
were at every doorstep, and every path was its tributary.

From taylorsingletary at gmail.com  Thu Dec  7 11:14:11 2006
From: taylorsingletary at gmail.com (D. Taylor Singletary)
Date: Thu, 7 Dec 2006 08:14:11 -0800
Subject: [Backgroundrb-devel] backgrounDRb problems with results and
	MiddleMan
Message-ID: <75d04d500612070814u5111585ek85b23e69436b845d@mail.gmail.com>

Hi there,

I've been experimenting with implementing BackgroundDRb in my rails app but
am running into some problems.

The primary problem I'm getting is in cleaning up worker threads, or getting
any kind of response out of worker threads.

If try using the response function/hash in my Worker the worker will not
execute reporting that response is an unknown variable or function. (My
worker descends from BackgrounDRb::Rails )

undefined local variable or method `results' for
#<ScheduleGeneratorWorker:0x1461f94> - (NameError)
./script/backgroundrb/../../config/../lib/workers/schedule_generator_worker.rb:9:in
`do_work'

This makes it hard to decide when to delete the workers, if I can't get any
status out of them.

My second issue is that I'm trying to create an after_filter to a controller
that looks for any "keys" saved to the session and then load the worker
class to query status (assuming that "response" actually worked..). When I
try to re-substantiate the worker using MiddleMan.worker(worker_key) I get
an error reporting that worker is not a method of MiddleMan.


NoMethodError (undefined method `worker' for
#<BackgrounDRb::MiddleMan:0x146377c>):
    /app/controllers/workbench_controller.rb:31:in `get_rid_of_workers'


All the rdoc documentation I've read for BackgroundDRb say that this is the
way to control the response and load a worker.

What am I doing wrong? Has anyone else had issues like this? I can get the
thread to start if I take out all of my references to results and my filter,
but then I have no way of monitoring it or stopping it.

Thanks for any help. A previous version of this message was posted to the
Rails mailing list.

D. Taylor Singletary
http://www.realitytechnicians.com





-- 
D. Taylor Singletary
Reality Technician
"Look at my hair; like the design." -- David Byrne
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061207/8f329a1e/attachment.html 

From ezmobius at gmail.com  Thu Dec  7 13:49:15 2006
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Thu, 7 Dec 2006 10:49:15 -0800
Subject: [Backgroundrb-devel] backgrounDRb problems with results and
	MiddleMan
In-Reply-To: <75d04d500612070814u5111585ek85b23e69436b845d@mail.gmail.com>
References: <75d04d500612070814u5111585ek85b23e69436b845d@mail.gmail.com>
Message-ID: <1A17537F-B8D4-426F-9065-F2FC1939C4E0@gmail.com>

Hey There-

	I think you may be using the old version of the plugin. Make sure  
you install it from here:

http://svn.devjavu.com/backgroundrb

	Try that instead. You will need to remove all traces of the old  
plugin before you install this one.

-Ezra

On Dec 7, 2006, at 8:14 AM, D. Taylor Singletary wrote:

> Hi there,
>
> I've been experimenting with implementing BackgroundDRb in my rails  
> app but am running into some problems.
>
> The primary problem I'm getting is in cleaning up worker threads,  
> or getting any kind of response out of worker threads.
>
> If try using the response function/hash in my Worker the worker  
> will not execute reporting that response is an unknown variable or  
> function. (My worker descends from BackgrounDRb::Rails )
>
> undefined local variable or method `results' for  
> #<ScheduleGeneratorWorker:0x1461f94> - (NameError)
> ./script/backgroundrb/../../config/../lib/workers/ 
> schedule_generator_worker.rb:9:in `do_work'
>
> This makes it hard to decide when to delete the workers, if I can't  
> get any status out of them.
>
> My second issue is that I'm trying to create an after_filter to a  
> controller that looks for any "keys" saved to the session and then  
> load the worker class to query status (assuming that "response"  
> actually worked..). When I try to re-substantiate the worker using  
> MiddleMan.worker(worker_key) I get an error reporting that worker  
> is not a method of MiddleMan.
>
>
> NoMethodError (undefined method `worker' for  
> #<BackgrounDRb::MiddleMan:0x146377c>):
>     /app/controllers/workbench_controller.rb:31:in  
> `get_rid_of_workers'
>
>
> All the rdoc documentation I've read for BackgroundDRb say that  
> this is the way to control the response and load a worker.
>
> What am I doing wrong? Has anyone else had issues like this? I can  
> get the thread to start if I take out all of my references to  
> results and my filter, but then I have no way of monitoring it or  
> stopping it.
>
> Thanks for any help. A previous version of this message was posted  
> to the Rails mailing list.
>
> D. Taylor Singletary
> http://www.realitytechnicians.com
>
>
>
>
>
> -- 
> D. Taylor Singletary
> Reality Technician
> "Look at my hair; like the design." -- David Byrne
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From taylorsingletary at gmail.com  Thu Dec  7 16:06:51 2006
From: taylorsingletary at gmail.com (D. Taylor Singletary)
Date: Thu, 7 Dec 2006 13:06:51 -0800
Subject: [Backgroundrb-devel] backgrounDRb problems with results and
	MiddleMan
In-Reply-To: <1A17537F-B8D4-426F-9065-F2FC1939C4E0@gmail.com>
References: <75d04d500612070814u5111585ek85b23e69436b845d@mail.gmail.com>
	<1A17537F-B8D4-426F-9065-F2FC1939C4E0@gmail.com>
Message-ID: <75d04d500612071306u66a7afb6oa2fd61baea2c3227@mail.gmail.com>

Thanks for the response. Looks like that was exactly it. Thank you. Working
like a dream now.

On 12/7/06, Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:
>
> Hey There-
>
>         I think you may be using the old version of the plugin. Make sure
> you install it from here:
>
> http://svn.devjavu.com/backgroundrb
>
>         Try that instead. You will need to remove all traces of the old
> plugin before you install this one.
>
> -Ezra
>
> On Dec 7, 2006, at 8:14 AM, D. Taylor Singletary wrote:
>
> > Hi there,
> >
> > I've been experimenting with implementing BackgroundDRb in my rails
> > app but am running into some problems.
> >
> > The primary problem I'm getting is in cleaning up worker threads,
> > or getting any kind of response out of worker threads.
> >
> > If try using the response function/hash in my Worker the worker
> > will not execute reporting that response is an unknown variable or
> > function. (My worker descends from BackgrounDRb::Rails )
> >
> > undefined local variable or method `results' for
> > #<ScheduleGeneratorWorker:0x1461f94> - (NameError)
> > ./script/backgroundrb/../../config/../lib/workers/
> > schedule_generator_worker.rb:9:in `do_work'
> >
> > This makes it hard to decide when to delete the workers, if I can't
> > get any status out of them.
> >
> > My second issue is that I'm trying to create an after_filter to a
> > controller that looks for any "keys" saved to the session and then
> > load the worker class to query status (assuming that "response"
> > actually worked..). When I try to re-substantiate the worker using
> > MiddleMan.worker(worker_key) I get an error reporting that worker
> > is not a method of MiddleMan.
> >
> >
> > NoMethodError (undefined method `worker' for
> > #<BackgrounDRb::MiddleMan:0x146377c>):
> >     /app/controllers/workbench_controller.rb:31:in
> > `get_rid_of_workers'
> >
> >
> > All the rdoc documentation I've read for BackgroundDRb say that
> > this is the way to control the response and load a worker.
> >
> > What am I doing wrong? Has anyone else had issues like this? I can
> > get the thread to start if I take out all of my references to
> > results and my filter, but then I have no way of monitoring it or
> > stopping it.
> >
> > Thanks for any help. A previous version of this message was posted
> > to the Rails mailing list.
> >
> > D. Taylor Singletary
> > http://www.realitytechnicians.com
> >
> >
> >
> >
> >
> > --
> > D. Taylor Singletary
> > Reality Technician
> > "Look at my hair; like the design." -- David Byrne
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
> -- Ezra Zygmuntowicz
> -- Lead Rails Evangelist
> -- ez at engineyard.com
> -- Engine Yard, Serious Rails Hosting
> -- (866) 518-YARD (9273)
>
>
>


-- 
D. Taylor Singletary
Reality Technician
"Look at my hair; like the design." -- David Byrne
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061207/4c0e3d57/attachment-0001.html 

From brian at swivel.com  Thu Dec  7 21:59:44 2006
From: brian at swivel.com (Brian Mulloy)
Date: Thu, 7 Dec 2006 18:59:44 -0800
Subject: [Backgroundrb-devel] Thread Pool Size?
Message-ID: <415516050612071859y38b247a0p4942374c9fd9cf05@mail.gmail.com>

Hi All,

It might be lack of sleep, but I am struggling to accurately limit our pool
size.   It seems like I can specify it on the server with the -s command
line option and also on the client via the YAML pool_size.  Is that right?
Which one wins?

Our problem is that we are getting about 40 threads on each backgroundrb box
and it's flooring our db and each bgrb box.
We want around 8.

Is there a way to put a hard ceiling on the server side thread pool?

Here is our setup:
5 app boxes, app01 - app05, with 8 mongrel instances each (this is where the
40 comes from, I think)
Each app box points to a load balancer in front of two backgroundrb boxes,
crawler01 and crawler02
this is the backgroundrb.yml on each app box
:host: backgroundrb
:port: 2000
:rails_env: production
:pool_size: 1

We have to keep killing the bgrb.  But we're ok, because all the state of
the workers is stored in a record in the db.  That also allows us to use the
load balancer in front of the bgrb (crawler) tier.

BTW, we our site is www.swivel.com and you see our use of the progress bar
pattern and bgrb (although you won't see much progress until we solve this.


long live bgrb

Brian Mulloy
CEO & Cofounder
www.swivel.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061207/2c8b9511/attachment.html 

From ezmobius at gmail.com  Thu Dec  7 23:35:05 2006
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Thu, 7 Dec 2006 20:35:05 -0800
Subject: [Backgroundrb-devel] Thread Pool Size?
In-Reply-To: <415516050612071859y38b247a0p4942374c9fd9cf05@mail.gmail.com>
References: <415516050612071859y38b247a0p4942374c9fd9cf05@mail.gmail.com>
Message-ID: <A661AA50-89B3-472A-93E1-16F14EAF571F@brainspl.at>

Hey Brian-

On Dec 7, 2006, at 6:59 PM, Brian Mulloy wrote:

> Hi All,
>
> It might be lack of sleep, but I am struggling to accurately limit  
> our pool size.   It seems like I can specify it on the server with  
> the -s command line option and also on the client via the YAML  
> pool_size.  Is that right?  Which one wins?
>
> Our problem is that we are getting about 40 threads on each  
> backgroundrb box and it's flooring our db and each bgrb box.
> We want around 8.
>
> Is there a way to put a hard ceiling on the server side thread pool?
>
> Here is our setup:
> 5 app boxes, app01 - app05, with 8 mongrel instances each (this is  
> where the 40 comes from, I think)
> Each app box points to a load balancer in front of two backgroundrb  
> boxes, crawler01 and crawler02
> this is the backgroundrb.yml on each app box
> :host: backgroundrb
> :port: 2000
> :rails_env: production
> :pool_size: 1
>

	You are using the latest release of backgroundrb? 0.2.1? The pool  
size works like this.. It is supposed to only allow the number you  
specify of workers to be running at one time. Any more requests for  
new workers when the pool is full get queued up and then they are run  
when another worker leaves the pool and dies. What exactly is  
happening to you? Are you sending a bunch of requests and they all  
run at once even though you have set the limit?

	It may be that we need to add some tighter control to how many  
active jobs there are. Maybe a way for new_worker to return an error  
that you can rescue and retry on a different drb server? The current  
thread pool should limit the actively running workers but writing  
properly threaded code is hard so I wouldn't be too surprised if I  
made a mistake.

  	I can imagine a way to be more strict about the pool limit but it  
will take some doing I imagine. Can you explain how your bdrb stuff  
works? Are you starting a new worker for each time you do something?  
Or do you have a certain number of workers that always run and keep  
running in a loop accepting jobs? The reason I ask is that the way  
that the new backgroundrb works its better to have a number of named  
workers that always run in a loop accepting jobs from a queue. So  
instead of calling new_worker all the time you set your workers to  
autostart at server start time and then just call methods on the  
daemon style workers. This way you can start exactly how many workers  
you want and give each one a named job key. Then you can either have  
the workers running in a loop where they wake up every second or  
whatever interval you set and do whatever jobs they need to do,

	Since you say that you are already keeping the state of the workers  
in the database then maybe an approach like this would be better  
suited. The basic idea is something like this:

An example worker that pulls events off the db and executes them,  
then saves them out and loops again

class RSSWorker < BackgrounDRb::Worker::RailsBase
   require 'rss'
   require 'net/http'

   def do_work(args)
     @args = args
     mainloop
   end

   def mainloop
     loop {
       sleep @args[:sleep]
       RssUrls.find_all_by_pending(true).each do |rss|
         rss.processing=true
         rss.save
         Net::HTTP.start(rss.host) do |http|
           response = http.get(rss.path)
           raise response.code unless response.code == "200"
           rss_parser = RSS::Parser.new(response.body)
           rss.output = rss_parser.parse
         end
         rss.completed=true
         rss.save
       end
     }
   end

end
RSSWorker.register

	Something like that may work better although I am not certain  
exactly what your workers need to do. You coudl also do the same  
thing with long lived workers but create a method other then do_work  
which you can call from rails via the work_thread method.


Cheers-
-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From brian at swivel.com  Fri Dec  8 03:11:32 2006
From: brian at swivel.com (Brian Mulloy)
Date: Fri, 8 Dec 2006 00:11:32 -0800
Subject: [Backgroundrb-devel] Thread Pool Size?
In-Reply-To: <A661AA50-89B3-472A-93E1-16F14EAF571F@brainspl.at>
References: <415516050612071859y38b247a0p4942374c9fd9cf05@mail.gmail.com>
	<A661AA50-89B3-472A-93E1-16F14EAF571F@brainspl.at>
Message-ID: <415516050612080011o2bf543fcp8e158d4b4c277d82@mail.gmail.com>

>         You are using the latest release of backgroundrb?


module BackgrounDRb
  VERSION = "0.2.0"
end

doh, I am going to upgrade and then give it another go.

these are excellent thoughts, Ezra, thanks.


0.2.1? The pool
> size works like this.. It is supposed to only allow the number you
> specify of workers to be running at one time.



Any more requests for
> new workers when the pool is full get queued up and then they are run
> when another worker leaves the pool and dies.



What exactly is
> happening to you? Are you sending a bunch of requests and they all
> run at once even though you have set the limit?
>
>         It may be that we need to add some tighter control to how many
> active jobs there are. Maybe a way for new_worker to return an error
> that you can rescue and retry on a different drb server? The current
> thread pool should limit the actively running workers but writing
> properly threaded code is hard so I wouldn't be too surprised if I
> made a mistake.
>
>         I can imagine a way to be more strict about the pool limit but it
> will take some doing I imagine. Can you explain how your bdrb stuff
> works? Are you starting a new worker for each time you do something?
> Or do you have a certain number of workers that always run and keep
> running in a loop accepting jobs? The reason I ask is that the way
> that the new backgroundrb works its better to have a number of named
> workers that always run in a loop accepting jobs from a queue. So
> instead of calling new_worker all the time you set your workers to
> autostart at server start time and then just call methods on the
> daemon style workers. This way you can start exactly how many workers
> you want and give each one a named job key. Then you can either have
> the workers running in a loop where they wake up every second or
> whatever interval you set and do whatever jobs they need to do,
>
>         Since you say that you are already keeping the state of the
> workers
> in the database then maybe an approach like this would be better
> suited. The basic idea is something like this:
>
> An example worker that pulls events off the db and executes them,
> then saves them out and loops again
>
> class RSSWorker < BackgrounDRb::Worker::RailsBase
>    require 'rss'
>    require 'net/http'
>
>    def do_work(args)
>      @args = args
>      mainloop
>    end
>
>    def mainloop
>      loop {
>        sleep @args[:sleep]
>        RssUrls.find_all_by_pending(true).each do |rss|
>          rss.processing=true
>          rss.save
>          Net::HTTP.start(rss.host) do |http|
>            response = http.get(rss.path)
>            raise response.code unless response.code == "200"
>            rss_parser = RSS::Parser.new(response.body)
>            rss.output = rss_parser.parse
>          end
>          rss.completed=true
>          rss.save
>        end
>      }
>    end
>
> end
> RSSWorker.register
>
>         Something like that may work better although I am not certain
> exactly what your workers need to do. You coudl also do the same
> thing with long lived workers but create a method other then do_work
> which you can call from rails via the work_thread method.
>
>
> Cheers-
> -- Ezra Zygmuntowicz
> -- Lead Rails Evangelist
> -- ez at engineyard.com
> -- Engine Yard, Serious Rails Hosting
> -- (866) 518-YARD (9273)
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061208/9d575aab/attachment.html 

From masonhale at gmail.com  Fri Dec  8 18:31:49 2006
From: masonhale at gmail.com (Mason Hale)
Date: Fri, 8 Dec 2006 17:31:49 -0600
Subject: [Backgroundrb-devel] trouble using ActiveRecord classes
Message-ID: <8bca3aa10612081531r22588c61tfc4fcefdbf5f8885@mail.gmail.com>

I've been trying unsuccessfully for the better part of today to write a
worker that can use by ActiveRecord model classes.

The bizarre thing is that worker just stops at the point accessing the
database connection. An exception isn't raising, it just dies.

My worker class is below, I've interspersed log messages throughout to try
to determine where it is dying.


===== publish_worker.rb ========

require 'publication' # loads my Publication < ActiveRecord::Base model

class PublishWorker < BackgrounDRb::Worker::RailsBase
  attr_accessor :publication_id

  def do_work(args)
    logger.debug("PublishWorker args: #{args.inspect}")

    @publication_id = args[:publication_id]
    logger.debug("Publication id: #{@publication_id}")

    end_date = args[:end_date] || Time.now.utc
    logger.debug("End date: #{end_date}")

    begin
      logger.debug("In begin block")
      logger.debug("Publication.table_name: #{Publication.table_name}")
      logger.debug("Publication.connected?: #{Publication.connected?}")
      logger.debug("Publication.connection: #{Publication.connection.inspect
}")
      logger.debug("Publication.count: #{Publication.count}")
    rescue
      logger.debug("Got error: #{$!}")
    end

    pub = Publication.find(@publication_id)
    pub.publish_for_date(end_date)
  end

end
PublishWorker.register

=============================

In the log/backgroundrb.log file I see:

20061208-17:23:38 (3611) Starting WorkerLogger
20061208-17:23:38 (3612) In ResultsWorker
20061208-17:23:54 (3612) PublishWorker args: {:publication_id=>"1"}
20061208-17:23:54 (3612) Publication id: 1
20061208-17:23:54 (3612) End date: Fri Dec 08 23:23:54 UTC 2006
20061208-17:23:54 (3612) In begin block
20061208-17:23:54 (3612) Publication.table_name: publication
20061208-17:23:54 (3612) Publication.connected?: false

Which means it never logged the  "Publication.connection.inspect" message,
and also did not log an error.

It's weird that I can get access to my model class but can't access the db
connection.
By the way, I'm working with a PostgreSQL database and its associated
adapter.

Any suggestions or pointers on where to look to track down the issue are
greatly
appreciated.

thanks in advance,
Mason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061208/2e2f249c/attachment.html 

From brian at swivel.com  Fri Dec  8 22:59:15 2006
From: brian at swivel.com (Brian Mulloy)
Date: Fri, 8 Dec 2006 19:59:15 -0800
Subject: [Backgroundrb-devel] Thread Pool Size?
In-Reply-To: <415516050612080011o2bf543fcp8e158d4b4c277d82@mail.gmail.com>
References: <415516050612071859y38b247a0p4942374c9fd9cf05@mail.gmail.com>
	<A661AA50-89B3-472A-93E1-16F14EAF571F@brainspl.at>
	<415516050612080011o2bf543fcp8e158d4b4c277d82@mail.gmail.com>
Message-ID: <415516050612081959s7dd5a60j6e8baeff47e3f7f6@mail.gmail.com>

Hi Ezra,

I did the upgrade to 0.2.1.  I am going to adjust how I am using the workers
based on your suggestions.

However, here is a unit test I wrote, that for better or worse has the
behavior I was hoping for.

The test creates new workers and asserts after each worker that the # of
worker threads is less than I specified with the -s option on backgroundrb
start.

class FooWorker < BackgrounDRb::Worker::RailsBase
  def do_work(args)
    sleep 1
  end
end
FooWorker.register

def test_middle_man_processes
  # thread pool size
  max_bdrb_threads = 2

  # stop bdrb just in case
  %x(RAILS_ENV=test; #{RAILS_ROOT}/script/backgroundrb stop -- -c
backgroundrb_test)

  # start bdrb int test with thread pool size
  %x(RAILS_ENV=test; #{RAILS_ROOT}/script/backgroundrb start -- -c
backgroundrb_test -s #{max_bdrb_threads})

  # give bdrb time to start
  sleep(5)

  # shouldn't have to do this, but we do
  orig_env=ENV['RAILS_ENV']
  ENV['RAILS_ENV'] = 'test'

  10.times do
     MiddleMan.new_worker :class => :foo_worker
     worker_count = %x(pstree | grep -v grep | grep #{:foo_worker} | wc
-l).gsub("\n", '').to_i
     puts "worker_count: #{worker_count} max_bdrb_threads:
#{max_bdrb_threads}"
     # assert that thread pool size is less than what we specified
     assert worker_count <= max_bdrb_threads, "expected #{max_bdrb_threads}
workers, was #{worker_count}"
  end

  # shouldn't have to do this, but we do
  %x(RAILS_ENV=test; #{RAILS_ROOT}/script/backgroundrb stop -- -c
backgroundrb_test)
  ENV['RAILS_ENV'] = orig_env
end

thanks again for the help.  i'm going back to your other suggestions now.

On 12/8/06, Brian Mulloy <brian at swivel.com> wrote:
>
>
>         You are using the latest release of backgroundrb?
>
>
> module BackgrounDRb
>   VERSION = "0.2.0"
> end
>
> doh, I am going to upgrade and then give it another go.
>
> these are excellent thoughts, Ezra, thanks.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061208/33913125/attachment-0001.html 

From rawdlite at googlemail.com  Mon Dec 11 11:48:46 2006
From: rawdlite at googlemail.com (tom roth)
Date: Mon, 11 Dec 2006 17:48:46 +0100
Subject: [Backgroundrb-devel] scheduled only called once
Message-ID: <d58e51190612110848r2d6c280fo7ceb1b185992ee92@mail.gmail.com>

when working with the scheduler i found that using

:trigger_args:  0 * * * * * *

in  config/backgroundrb_schedules.yml the worker gets only called once.

:trigger_args:  "0 * * * * * *"  works as expected.

Maybe this should  be changed in the README ?


regards tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061211/802224e2/attachment.html 

From rawdlite at googlemail.com  Mon Dec 11 12:02:47 2006
From: rawdlite at googlemail.com (tom roth)
Date: Mon, 11 Dec 2006 18:02:47 +0100
Subject: [Backgroundrb-devel] scheduled only called once
In-Reply-To: <d58e51190612110848r2d6c280fo7ceb1b185992ee92@mail.gmail.com>
References: <d58e51190612110848r2d6c280fo7ceb1b185992ee92@mail.gmail.com>
Message-ID: <d58e51190612110902s23cba3cbma5020bda05801bde@mail.gmail.com>

I have to correct myself, it was

:worker_method: :do_work

that did the trick, without this line the worker was only called once.
If this is the designed behaviour, i found the Readme section a bit
confusing. Maybe this helps another newby.

regards tom

On 11/12/06, tom roth <rawdlite at googlemail.com> wrote:
>
> when working with the scheduler i found that using
>
> :trigger_args:  0 * * * * * *
>
> in  config/backgroundrb_schedules.yml the worker gets only called once.
>
> :trigger_args:  "0 * * * * * *"  works as expected.
>
> Maybe this should  be changed in the README ?
>
>
> regards tom
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061211/333bf597/attachment.html 

From sugihara at ing.co.jp  Wed Dec 13 02:54:32 2006
From: sugihara at ing.co.jp (Kenichi Sugihara)
Date: Wed, 13 Dec 2006 16:54:32 +0900
Subject: [Backgroundrb-devel] Unintended thread forked in autostart
Message-ID: <457FB1B8.6030300@ing.co.jp>


Hello all,

I've got the problem about the following simple application using
autostart and repeat_every.

A thread(?) is forked for processing task at first, but at sencond time
two threads seem to be forked.

My simple application code is...

- In worker class
  class FooWorker < BackgrounDRb:Rails
  repeat_every 2.minutes
  first_run Time.now

  def do_work
     puts Time.now.to_s + "Start processing....."
     do_some_task()
     puts Time.now.to_s + "Finish processing"
  end

- In background.yml

 autostart:
   1:
     class: foo_worker
     arg: ''

Result is....
  --- First time---
  Start processing
  Finish processing

  --- Second time ---
  Start processing
  Start processing
  Finish processing
  Finish processing

regards.

-- 
--------------------------------------
Kenichi Sugihara
--------------------------------------

From ezmobius at gmail.com  Wed Dec 13 13:47:21 2006
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Wed, 13 Dec 2006 10:47:21 -0800
Subject: [Backgroundrb-devel] Unintended thread forked in autostart
In-Reply-To: <457FB1B8.6030300@ing.co.jp>
References: <457FB1B8.6030300@ing.co.jp>
Message-ID: <38CCE9BC-C710-49D8-92EE-CBA11284B392@brainspl.at>


On Dec 12, 2006, at 11:54 PM, Kenichi Sugihara wrote:

>
> Hello all,
>
> I've got the problem about the following simple application using
> autostart and repeat_every.
>
> A thread(?) is forked for processing task at first, but at sencond  
> time
> two threads seem to be forked.
>
> My simple application code is...
>
> - In worker class
>   class FooWorker < BackgrounDRb:Rails
>   repeat_every 2.minutes
>   first_run Time.now
>
>   def do_work
>      puts Time.now.to_s + "Start processing....."
>      do_some_task()
>      puts Time.now.to_s + "Finish processing"
>   end
>
> - In background.yml
>
>  autostart:
>    1:
>      class: foo_worker
>      arg: ''
>
> Result is....
>   --- First time---
>   Start processing
>   Finish processing
>
>   --- Second time ---
>   Start processing
>   Start processing
>   Finish processing
>   Finish processing
>
> regards


Hey There-

	The older version of backgroundrb had a problem with the  
repeat_every directive. SO it doesn't work as expected. If you really  
need the timing functionality I suggest you upgrade to the 0.2.1  
version of the plugin. You can get it here.

http://svn.devjavu.com/backgroundrb/tags/release-0.2.1/

http://svn.devjavu.com/backgroundrb/tags/release-0.2.1/README

Cheers-
-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From joshua at zaadz.com  Wed Dec 13 13:55:10 2006
From: joshua at zaadz.com (Joshua Warchol)
Date: Wed, 13 Dec 2006 13:55:10 -0500
Subject: [Backgroundrb-devel] Prime Time
In-Reply-To: <38CCE9BC-C710-49D8-92EE-CBA11284B392@brainspl.at>
References: <457FB1B8.6030300@ing.co.jp>
	<38CCE9BC-C710-49D8-92EE-CBA11284B392@brainspl.at>
Message-ID: <45804C8E.5000108@zaadz.com>

Is the latest release of the new code production ready? I'm finally 
working seriously on deploying workers in our site and right now we're 
on the old branch.

-- 
Joshua Warchol
Code Poet, Zaadz Inc.
http://joshua.zaadz.com/


From ezmobius at gmail.com  Wed Dec 13 15:13:14 2006
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Wed, 13 Dec 2006 12:13:14 -0800
Subject: [Backgroundrb-devel] Prime Time
In-Reply-To: <45804C8E.5000108@zaadz.com>
References: <457FB1B8.6030300@ing.co.jp>
	<38CCE9BC-C710-49D8-92EE-CBA11284B392@brainspl.at>
	<45804C8E.5000108@zaadz.com>
Message-ID: <3DF94C9F-B1B1-44EC-BC03-F5823DBE9941@brainspl.at>


On Dec 13, 2006, at 10:55 AM, Joshua Warchol wrote:

> Is the latest release of the new code production ready? I'm finally
> working seriously on deploying workers in our site and right now we're
> on the old branch.
>
> --  
> Joshua Warchol
> Code Poet, Zaadz Inc.
> http://joshua.zaadz.com/
>


Hey Josh-

	I guess that depends on what kind of workers you are using. If the  
old plugin is working for you then there may not be a good reason  
right now to upgrade. There are many production sites still running  
the old version perfectly well.

	So tell me a bit about what you are using it for? If its lighter  
weight duty then the old branch is great. If it requires a ton of  
processing power then the new multi process model might be better.

Thanks
-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From joshua at zaadz.com  Wed Dec 13 15:33:50 2006
From: joshua at zaadz.com (Joshua Warchol)
Date: Wed, 13 Dec 2006 15:33:50 -0500
Subject: [Backgroundrb-devel] Prime Time
In-Reply-To: <3DF94C9F-B1B1-44EC-BC03-F5823DBE9941@brainspl.at>
References: <457FB1B8.6030300@ing.co.jp>
	<38CCE9BC-C710-49D8-92EE-CBA11284B392@brainspl.at>
	<45804C8E.5000108@zaadz.com>
	<3DF94C9F-B1B1-44EC-BC03-F5823DBE9941@brainspl.at>
Message-ID: <458063AE.90004@zaadz.com>

Ezra Zygmuntowicz wrote:
> So tell me a bit about what you are using it for? If its lighter 
> weight duty then the old branch is great. If it requires a ton of 
> processing power then the new multi process model might be better.
Well, basically the key worker in this situation is going to be creating 
delivery records in a "notifications" table as part of sending 
newsletters. So it's a long running (1-10 minutes) job of creating 
thousands of rows in the table.

I think it can be done just fine in the older code, so I'm probably 
going to stick there and cheer on the new code from the sideline for a 
bit. I have played with the new hotness a bit and it's looking great.

Josh

-- 
Joshua Warchol
Code Poet, Zaadz Inc.
http://joshua.zaadz.com/


From masonhale at gmail.com  Thu Dec 14 12:51:29 2006
From: masonhale at gmail.com (Mason Hale)
Date: Thu, 14 Dec 2006 11:51:29 -0600
Subject: [Backgroundrb-devel] trouble using ActiveRecord classes
In-Reply-To: <8bca3aa10612081531r22588c61tfc4fcefdbf5f8885@mail.gmail.com>
References: <8bca3aa10612081531r22588c61tfc4fcefdbf5f8885@mail.gmail.com>
Message-ID: <8bca3aa10612140951k3ef3bc76p60bee0a9b6f1cdee@mail.gmail.com>

Following up on my previous post: I tracked this problem down to an issue
with the way native code libraries are loaded by Ruby on Mac OS.

This affected the rails script/console in the past, and still apparently
affects DRb. The underlying issue is at a lower
layer than BackgrounDRb and so it not related to BackgrounDRb after all.

More info about this particular issue can be found here:
http://dev.rubyonrails.org/ticket/1283

My workaround is to use the 'pure-ruby' postgres driver on Mac OS X. The
native driver still works on other platforms.
Long-term fix is to patch DRb (similar to the wa IRB was patched to fix the
above issue), or wait on Apple to fix the apparent bug in their .bundle
loading routines.

best regards,

Mason


On 12/8/06, Mason Hale <masonhale at gmail.com> wrote:
>
> I've been trying unsuccessfully for the better part of today to write a
> worker that can use by ActiveRecord model classes.
>
> The bizarre thing is that worker just stops at the point accessing the
> database connection. An exception isn't raising, it just dies.
>
> My worker class is below, I've interspersed log messages throughout to try
> to determine where it is dying.
>
>
> ===== publish_worker.rb ========
>
> require 'publication' # loads my Publication < ActiveRecord::Base model
>
> class PublishWorker < BackgrounDRb::Worker::RailsBase
>   attr_accessor :publication_id
>
>   def do_work(args)
>     logger.debug("PublishWorker args: #{args.inspect}")
>
>     @publication_id = args[:publication_id]
>     logger.debug("Publication id: #{@publication_id}")
>
>     end_date = args[:end_date] || Time.now.utc
>     logger.debug("End date: #{end_date}")
>
>     begin
>       logger.debug ("In begin block")
>       logger.debug("Publication.table_name: #{Publication.table_name}")
>       logger.debug("Publication.connected?: #{Publication.connected?}")
>       logger.debug(" Publication.connection: #{
> Publication.connection.inspect}")
>       logger.debug("Publication.count: #{Publication.count}")
>     rescue
>       logger.debug("Got error: #{$!}")
>     end
>
>     pub = Publication.find(@publication_id)
>     pub.publish_for_date(end_date)
>   end
>
> end
> PublishWorker.register
>
> =============================
>
> In the log/backgroundrb.log file I see:
>
> 20061208-17:23:38 (3611) Starting WorkerLogger
> 20061208-17:23:38 (3612) In ResultsWorker
> 20061208-17:23:54 (3612) PublishWorker args: {:publication_id=>"1"}
> 20061208-17:23:54 (3612) Publication id: 1
> 20061208-17:23:54 (3612) End date: Fri Dec 08 23:23:54 UTC 2006
> 20061208-17:23:54 (3612) In begin block
> 20061208-17:23:54 (3612) Publication.table_name: publication
> 20061208-17:23:54 (3612) Publication.connected ?: false
>
> Which means it never logged the  "Publication.connection.inspect" message,
> and also did not log an error.
>
> It's weird that I can get access to my model class but can't access the db
> connection.
> By the way, I'm working with a PostgreSQL database and its associated
> adapter.
>
> Any suggestions or pointers on where to look to track down the issue are
> greatly
> appreciated.
>
> thanks in advance,
> Mason
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061214/f85526fe/attachment.html 

From eden.li at gmail.com  Thu Dec 14 12:55:47 2006
From: eden.li at gmail.com (Eden Li)
Date: Fri, 15 Dec 2006 01:55:47 +0800
Subject: [Backgroundrb-devel] Is there a way to make backgroundrb 0.2.1 fail
	silently on win32?
Message-ID: <565dbdd40612140955w396cfc2bn7ec2f090467a82f0@mail.gmail.com>

After upgrading to backgroundrb 0.2.x, Rails barfs lstat errors left and
right when using the console or webrick or mongrel on win32.  We run our
site on a Linux server, so I'm not trying to get backgroundrb to work on a
win32 machine.  I just want to quietly stop loading itself if it detects
that it's on a win32 machine so that the rest of Rails can continue to load.

Most of our developers are working on win32 machines and they're focused on
other parts of our site, so it would be nice for them to be able to continue
working on their bits of code even though backgroundrb is sitting dormant in
the vendor/plugins directory.

I tried searching for a way to disable certain plugins in
config/environment.rb, but it appears this idea was batted down by DHH:
http://dev.rubyonrails.org/ticket/5468

Is there something I can do to monkey patch my svn:externals'd link to
backgroundrb 0.2.1 to prevent it from loading itself if PLATFORM =~
/win32/i?

I could unlink backgroundrb from the devjavu svn repository and do the patch
myself on the code, but I'd prefer not to store backgroundrb in my own SVN
repo.

Any ideas?

Thanks in advance,
eden
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061215/3e15eb45/attachment-0001.html 

From masonhale at gmail.com  Thu Dec 14 13:06:14 2006
From: masonhale at gmail.com (Mason Hale)
Date: Thu, 14 Dec 2006 12:06:14 -0600
Subject: [Backgroundrb-devel] feature request: config option to toggle
	scheduled jobs based on environment
Message-ID: <8bca3aa10612141006i46a2bf76v2a6f62e83a1d9387@mail.gmail.com>

I have BackgrounDRb configured to handle two cases of long-running tasks:
(1) on-demand user triggered requests for some action, and (2) scheduled
background running of tasks on a predetermined schedule.

In my particular case, we have a publication task that can take several
minutes to perform. Users can setup a regular publication schedule, or can
hit a 'publish now' button to start a process immediately.

The challenge I have is that in development mode, I want to be able to fire
off the 'publish now' tasks (for testing), but do NOT want the scheduled
background tasks to run automatically.

I want to do this in a way that works with multiple developers and that does
not complicate deployment.

The ideal for me would be some flag I could set that tells BackgroundRB not
to run it's scheduled tasks, and have that flag automatically set based on
the current RAILS_ENV.
This would be similar to the way ActionMailer works, where I have a line:

   config.action_mailer.perform_deliveries = false

in my config/environments/development.rb file and a  line:

  config.action_mailer.perform_deliveries = true

in my config/environments/production.rb file.

So something like:

   config.backgroundrb.enable_scheduler = false
or
   config.backgroundrb.run_scheduled_tasks = false

would be awesome.

I can probably hack something into my copy of the BackgrounDRb code to make
this work for my needs, but I wanted to through it out there as a feature
request as well.

best regards,
Mason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061214/7ed7809d/attachment.html 

From ezmobius at gmail.com  Thu Dec 14 14:38:53 2006
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Thu, 14 Dec 2006 11:38:53 -0800
Subject: [Backgroundrb-devel] Is there a way to make backgroundrb 0.2.1
	fail silently on win32?
In-Reply-To: <565dbdd40612140955w396cfc2bn7ec2f090467a82f0@mail.gmail.com>
References: <565dbdd40612140955w396cfc2bn7ec2f090467a82f0@mail.gmail.com>
Message-ID: <A0796474-8D34-4BAA-899F-165BAEA06623@brainspl.at>


On Dec 14, 2006, at 9:55 AM, Eden Li wrote:

> After upgrading to backgroundrb 0.2.x, Rails barfs lstat errors  
> left and right when using the console or webrick or mongrel on  
> win32.  We run our site on a Linux server, so I'm not trying to get  
> backgroundrb to work on a win32 machine.  I just want to quietly  
> stop loading itself if it detects that it's on a win32 machine so  
> that the rest of Rails can continue to load.
>
> Most of our developers are working on win32 machines and they're  
> focused on other parts of our site, so it would be nice for them to  
> be able to continue working on their bits of code even though  
> backgroundrb is sitting dormant in the vendor/plugins directory.
>
> I tried searching for a way to disable certain plugins in config/ 
> environment.rb, but it appears this idea was batted down by DHH:  
> http://dev.rubyonrails.org/ticket/5468
>
> Is there something I can do to monkey patch my svn:externals'd link  
> to backgroundrb 0.2.1 to prevent it from loading itself if PLATFORM  
> =~ /win32/i?
>
> I could unlink backgroundrb from the devjavu svn repository and do  
> the patch myself on the code, but I'd prefer not to store  
> backgroundrb in my own SVN repo.
>
> Any ideas?
>
> Thanks in advance,
> eden
>

	I don't know of any way to do conditional svn:externals sorry. Maybe  
for now you could remove backgroundrb from your svn repo and set  
svn:ignore on it. Then just have it in there locally but not checking  
it in. Then you could also keep a copy of backgroundrb on the server  
and have your deploy cap task symlink it into the app after deploy.

Cheers-

-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From ezmobius at gmail.com  Thu Dec 14 14:41:41 2006
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Thu, 14 Dec 2006 11:41:41 -0800
Subject: [Backgroundrb-devel] trouble using ActiveRecord classes
In-Reply-To: <8bca3aa10612140951k3ef3bc76p60bee0a9b6f1cdee@mail.gmail.com>
References: <8bca3aa10612081531r22588c61tfc4fcefdbf5f8885@mail.gmail.com>
	<8bca3aa10612140951k3ef3bc76p60bee0a9b6f1cdee@mail.gmail.com>
Message-ID: <8EDC2DF1-7005-4FEC-8541-01D88494B3F9@brainspl.at>


Mason-

	Thank you for following up on this. Interesting problem I haven't  
seen that one before.

Cheers-
-Ezra


On Dec 14, 2006, at 9:51 AM, Mason Hale wrote:

> Following up on my previous post: I tracked this problem down to an  
> issue with the way native code libraries are loaded by Ruby on Mac OS.
>
> This affected the rails script/console in the past, and still  
> apparently affects DRb. The underlying issue is at a lower
> layer than BackgrounDRb and so it not related to BackgrounDRb after  
> all.
>
> More info about this particular issue can be found here:
> http://dev.rubyonrails.org/ticket/1283
>
> My workaround is to use the 'pure-ruby' postgres driver on Mac OS  
> X. The native driver still works on other platforms.
> Long-term fix is to patch DRb (similar to the wa IRB was patched to  
> fix the above issue), or wait on Apple to fix the apparent bug in  
> their .bundle loading routines.
>
> best regards,
>
> Mason
>
>
> On 12/8/06, Mason Hale <masonhale at gmail.com> wrote: I've been  
> trying unsuccessfully for the better part of today to write a  
> worker that can use by ActiveRecord model classes.
>
> The bizarre thing is that worker just stops at the point accessing  
> the database connection. An exception isn't raising, it just dies.
>
> My worker class is below, I've interspersed log messages throughout  
> to try to determine where it is dying.
>
>
> ===== publish_worker.rb ========
>
> require 'publication' # loads my Publication < ActiveRecord::Base  
> model
>
> class PublishWorker < BackgrounDRb::Worker::RailsBase
>   attr_accessor :publication_id
>
>   def do_work(args)
>     logger.debug("PublishWorker args: #{ args.inspect}")
>
>     @publication_id = args[:publication_id]
>     logger.debug("Publication id: #{@publication_id}")
>
>     end_date = args[:end_date] || Time.now.utc
>     logger.debug("End date: #{end_date}")
>
>     begin
>       logger.debug ("In begin block")
>       logger.debug("Publication.table_name: # 
> {Publication.table_name}")
>       logger.debug("Publication.connected?: # 
> {Publication.connected?}")
>       logger.debug (" Publication.connection: # 
> {Publication.connection.inspect}")
>       logger.debug("Publication.count: #{Publication.count}")
>     rescue
>       logger.debug("Got error: #{$!}")
>     end
>
>     pub = Publication.find(@publication_id)
>     pub.publish_for_date(end_date)
>   end
>
> end
> PublishWorker.register
>
> =============================
>
> In the log/ backgroundrb.log file I see:
>
> 20061208-17:23:38 (3611) Starting WorkerLogger
> 20061208-17:23:38 (3612) In ResultsWorker
> 20061208-17:23:54 (3612) PublishWorker args: {:publication_id=>"1"}
> 20061208-17:23:54 (3612) Publication id: 1
> 20061208-17:23:54 (3612) End date: Fri Dec 08 23:23:54 UTC 2006
> 20061208-17:23:54 (3612) In begin block
> 20061208-17:23:54 (3612) Publication.table_name: publication
> 20061208-17:23:54 (3612) Publication.connected ?: false
>
> Which means it never logged the  "Publication.connection.inspect"  
> message,
> and also did not log an error.
>
> It's weird that I can get access to my model class but can't access  
> the db connection.
> By the way, I'm working with a PostgreSQL database and its  
> associated adapter.
>
> Any suggestions or pointers on where to look to track down the  
> issue are greatly
> appreciated.
>
> thanks in advance,
> Mason
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From skaar at waste.org  Thu Dec 14 15:08:46 2006
From: skaar at waste.org (skaar)
Date: Thu, 14 Dec 2006 14:08:46 -0600
Subject: [Backgroundrb-devel] Is there a way to make backgroundrb 0.2.1
	fail silently on win32?
In-Reply-To: <A0796474-8D34-4BAA-899F-165BAEA06623@brainspl.at>
References: <565dbdd40612140955w396cfc2bn7ec2f090467a82f0@mail.gmail.com>
	<A0796474-8D34-4BAA-899F-165BAEA06623@brainspl.at>
Message-ID: <20061214200846.GA10715@waste.org>

> 	I don't know of any way to do conditional svn:externals sorry. Maybe  
> for now you could remove backgroundrb from your svn repo and set  
> svn:ignore on it. Then just have it in there locally but not checking  
> it in. Then you could also keep a copy of backgroundrb on the server  
> and have your deploy cap task symlink it into the app after deploy.

since all Rails loads is the lib/ files - I think we can make it
conditional on platform - if you submit a ticket we can look at it for
the next release.

/skaar
-- 
----------------------------------------------------------------------
|\|\             where in the       |          s_u_b_s_t_r_u_c_t_i_o_n 
| | >===========  W.A.S.T.E.        |                  genarratologies 
|/|/    (_)     is the wisdom       |                  skaar at waste.org
----------------------------------------------------------------------

From george at initiatecommerce.com  Thu Dec 14 23:21:21 2006
From: george at initiatecommerce.com (george .)
Date: Thu, 14 Dec 2006 23:21:21 -0500
Subject: [Backgroundrb-devel] can Timeout::timeout(...) be used in a Worker ?
Message-ID: <C53D9620-604F-46F6-8D11-C09E7C32B282@initiatecommerce.com>

Hi ,

I was wondering if anyone else has tried to use the Timeout::timeout  
method in a backgroundrb Worker and had Timeout raise an exception  
after the allotted amount of time .
I ask because i have written a test script to make sure Timeout would  
work .  I have tested the script by directly feeding it to ruby with :
$ ruby test_script.rb

and piping the script to the rails console:
$ script/console < test_script.rb

Both of which work as expected .

I am using the latest release of backgroundrb: 0.2.1 (Released  
November 28, 2006 - r162) .

If it is not possible to use TImeout::timeout in a Worker currently,  
is there another means to accomplish the same goal as Timeout::timeout ?

thanks ,


george .

From caifara.subscribe at gmail.com  Fri Dec 15 03:40:21 2006
From: caifara.subscribe at gmail.com (Ivo Dancet)
Date: Fri, 15 Dec 2006 09:40:21 +0100
Subject: [Backgroundrb-devel] :job_key acting weird?
Message-ID: <0D52A101-0C11-4167-A00C-00D699DD9621@gmail.com>

Hi all

I'm experiencing something strange with a :job_key, it took me a very  
long time to find out a worker wasn't doing it's job because of the  
name of my job_key. The first one I tried was :make_kohier, and with  
that name (and lot's of other ones) I can't get the worker to work. I  
do get my key in return, but the job doesn't start (the worker only  
had a logger.info method in it, no error there). As I can't see any  
results of this logger, and I can't delete the worker, the worker  
doesn't exist.
I couldn't find the fault until I started changing the name of the  
job_key. I can't see any rules in it, some examples :job_key  
=> :sdlkfskdsld doesn't work, :job_key => :skj does. :udjzkzis  
and :ssdlfdo don't work, :ssdlfdk does work.
Just now I discovered that these keys sometimes work, sometimes  
not... Anyway, I only get errors if I run script/backgroundrb run,  
otherwise, there is nothing informing me of some error. Here is what  
I get in backgroundrb run

[system: Mac OS X Tiger]

You have a nil object when you didn't expect it!
The error occured while evaluating nil.wait - (NoMethodError)
/Users/caifara/Desktop/polders/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb/middleman.rb:217:in `new_worker'
/Users/caifara/Desktop/polders/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb/thread_pool.rb:36:in `dispatch'
/Users/caifara/Desktop/polders/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb/thread_pool.rb:22:in `dispatch'
/Users/caifara/Desktop/polders/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb/middleman.rb:199:in `new_worker'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/1.8/drb/drb.rb:1512:in `perform'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/1.8/drb/drb.rb:1427:in `run'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/1.8/drb/drb.rb:1424:in `run'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/1.8/drb/drb.rb:1344:in `initialize'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/1.8/drb/drb.rb:1624:in `start_service'
/Users/caifara/Desktop/polders/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb_server.rb:315:in `run'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
application.rb:186:in `start_proc'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
application.rb:196:in `start_proc'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
application.rb:226:in `start'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
controller.rb:72:in `run'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons.rb:179:in  
`run_proc'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/cmdline.rb: 
94:in `catch_exceptions'
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/ 
powerpc/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons.rb:178:in  
`run_proc'
/Users/caifara/Desktop/polders/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb_server.rb:301:in `run'
script/backgroundrb:29

Am I missing something here?
Thanks for reading
Ivo

From eden.li at gmail.com  Fri Dec 15 04:00:27 2006
From: eden.li at gmail.com (Eden Li)
Date: Fri, 15 Dec 2006 17:00:27 +0800
Subject: [Backgroundrb-devel] Is there a way to make backgroundrb 0.2.1
	fail silently on win32?
In-Reply-To: <20061214200846.GA10715@waste.org>
References: <565dbdd40612140955w396cfc2bn7ec2f090467a82f0@mail.gmail.com>
	<A0796474-8D34-4BAA-899F-165BAEA06623@brainspl.at>
	<20061214200846.GA10715@waste.org>
Message-ID: <565dbdd40612150100p65f54c43ja9f15662f8a0f94a@mail.gmail.com>

>       I don't know of any way to do conditional svn:externals sorry. Maybe
> for now you could remove backgroundrb from your svn repo and set
> svn:ignore on it. Then just have it in there locally but not checking
> it in. Then you could also keep a copy of backgroundrb on the server
> and have your deploy cap task symlink it into the app after deploy.

This is a possibility, but it requires me to have to go onto my server
and maintain a separate directory of files that aren't versioned along
with the rest of the project.  It's not that bad, but I figured out a
way to get the desired effect without resorting to this.

Rails puts the app's lib/ ahead of all plugins in the library search
path, so I created a file that shares the same name of the file that
gets loaded in backgroundrb's init.rb (middleman_rails_init.rb) and
placed it under my lib directory.  This file will exit silently if it
finds itself being loaded on win32 otherwise it'll continue loading
the appropriate middleman_rails_init from backgroundrb.

> since all Rails loads is the lib/ files - I think we can make it
> conditional on platform - if you submit a ticket we can look at it for
> the next release.

I've filed a ticket and attached a patch for init.rb:
http://backgroundrb.devjavu.com/projects/backgroundrb/ticket/42

From me at seebq.com  Fri Dec 15 10:44:53 2006
From: me at seebq.com (Charles Brian Quinn)
Date: Fri, 15 Dec 2006 10:44:53 -0500
Subject: [Backgroundrb-devel] can Timeout::timeout(...) be used in a
	Worker ?
In-Reply-To: <C53D9620-604F-46F6-8D11-C09E7C32B282@initiatecommerce.com>
References: <C53D9620-604F-46F6-8D11-C09E7C32B282@initiatecommerce.com>
Message-ID: <3a2de0cd0612150744i6c592a96ta166a1d1587e672c@mail.gmail.com>

This doesn't help at all, I know, but I use it in the older version of
backgroundrb with no problems.

Let us know your results, I don't see why it wouldn't work.

On 12/14/06, george. <george at initiatecommerce.com> wrote:
> Hi ,
>
> I was wondering if anyone else has tried to use the Timeout::timeout
> method in a backgroundrb Worker and had Timeout raise an exception
> after the allotted amount of time .
> I ask because i have written a test script to make sure Timeout would
> work .  I have tested the script by directly feeding it to ruby with :
> $ ruby test_script.rb
>
> and piping the script to the rails console:
> $ script/console < test_script.rb
>
> Both of which work as expected .
>
> I am using the latest release of backgroundrb: 0.2.1 (Released
> November 28, 2006 - r162) .
>
> If it is not possible to use TImeout::timeout in a Worker currently,
> is there another means to accomplish the same goal as Timeout::timeout ?
>
> thanks ,
>
>
> george .
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>


-- 
Charles Brian Quinn
self-promotion: www.seebq.com
highgroove studios: www.highgroove.com
slingshot hosting: www.slingshothosting.com

From skaar at waste.org  Fri Dec 15 13:17:37 2006
From: skaar at waste.org (skaar)
Date: Fri, 15 Dec 2006 12:17:37 -0600
Subject: [Backgroundrb-devel] can Timeout::timeout(...) be used in a
	Worker ?
In-Reply-To: <C53D9620-604F-46F6-8D11-C09E7C32B282@initiatecommerce.com>
References: <C53D9620-604F-46F6-8D11-C09E7C32B282@initiatecommerce.com>
Message-ID: <20061215181737.GB13359@waste.org>

What is it that doesn't work with Timeout::timeout?

  class TimeoutWorker < BackgrounDRb::Worker::Base
    require 'timeout'
  
    def do_work(args)
      self.rescued_timeout
      #self.simple_timeout
    end
  
    def simple_timeout
      Timeout::timeout(5) { sleep 10 }
    end
  
    def rescued_timeout
      begin
        self.simple_timeout
      rescue Timeout::Error
        "timed out"
      end
    end
  
  end
  TimeoutWorker.register

There is a number of things to notice with this. You need to make sure
that to handle exceptions in do_work properly. 'do_work' is called
automatically as a backgrounded call, so an exception will never reach
the caller. The work_thread worker method will terminate the worker on
an unhandled exception. Above, I've commented out self.simple_timeout,
since it would always result in the worker getting terminated.

If you want to call

  worker(new_worker(:class => :timeout_worker, :job_key => :timeout)).simple_timeout

directly, you will also need to require 'timeout' on the caller side,
otherwise Timeout::Error will be an unknown symbol for DRb.

The resuced_timout method properly handle the exception in the worker.

/skaar



* george . (george at initiatecommerce.com) [061214 22:12]:
> Hi ,
> 
> I was wondering if anyone else has tried to use the Timeout::timeout  
> method in a backgroundrb Worker and had Timeout raise an exception  
> after the allotted amount of time .
> I ask because i have written a test script to make sure Timeout would  
> work .  I have tested the script by directly feeding it to ruby with :
> $ ruby test_script.rb
> 
> and piping the script to the rails console:
> $ script/console < test_script.rb
> 
> Both of which work as expected .
> 
> I am using the latest release of backgroundrb: 0.2.1 (Released  
> November 28, 2006 - r162) .
> 
> If it is not possible to use TImeout::timeout in a Worker currently,  
> is there another means to accomplish the same goal as Timeout::timeout ?
> 
> thanks ,
> 
> 
> george .
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

-- 
----------------------------------------------------------------------
|\|\             where in the       |          s_u_b_s_t_r_u_c_t_i_o_n 
| | >===========  W.A.S.T.E.        |                  genarratologies 
|/|/    (_)     is the wisdom       |                  skaar at waste.org
----------------------------------------------------------------------

From alevchuk at gmail.com  Fri Dec 15 15:52:05 2006
From: alevchuk at gmail.com (Aleksandr Levchuk)
Date: Fri, 15 Dec 2006 12:52:05 -0800
Subject: [Backgroundrb-devel] Is there a way to make backgroundrb 0.2.1
	fail silently on win32?
In-Reply-To: <565dbdd40612140955w396cfc2bn7ec2f090467a82f0@mail.gmail.com>
References: <565dbdd40612140955w396cfc2bn7ec2f090467a82f0@mail.gmail.com>
Message-ID: <a39d6f080612151252p2ef450a5y1e21f245d39c86e4@mail.gmail.com>

If one has all the calls to BackgrounDRb isolated in a few Rails
models. For example I have app/model/nucleus.rb - which does all the
interactions with workers. One could solve such a  problem by using
Rails mock objects. I would just have to place a mocking model into
app/test/mocks/development/nucleus.rb, and it will use that model
instead when the application is running in the development
environment.

For your case, the developers could run Rails with
--environment=development on win32. On linux you could be run it with
--environment=testing or --environment=production, so that
BackgrounDRb is used.


Question for Ezra: Will BackgrounDRb work on Windows eventually?

On 12/14/06, Eden Li <eden.li at gmail.com> wrote:
> After upgrading to backgroundrb 0.2.x, Rails barfs lstat errors left and
> right when using the console or webrick or mongrel on win32.  We run our
> site on a Linux server, so I'm not trying to get backgroundrb to work on a
> win32 machine.  I just want to quietly stop loading itself if it detects
> that it's on a win32 machine so that the rest of Rails can continue to load.
>
> Most of our developers are working on win32 machines and they're focused on
> other parts of our site, so it would be nice for them to be able to continue
> working on their bits of code even though backgroundrb is sitting dormant in
> the vendor/plugins directory.
>
> I tried searching for a way to disable certain plugins in
> config/environment.rb, but it appears this idea was batted down by DHH:
> http://dev.rubyonrails.org/ticket/5468
>
> Is there something I can do to monkey patch my svn:externals'd link to
> backgroundrb 0.2.1 to prevent it from loading itself if PLATFORM =~
> /win32/i?
>
> I could unlink backgroundrb from the devjavu svn repository and do the patch
> myself on the code, but I'd prefer not to store backgroundrb in my own SVN
> repo.
>
> Any ideas?
>
> Thanks in advance,
> eden
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
>


-- 
--------------------------------------------
Aleksandr Levchuk
University of California, Riverside
1-951-368-0004
--------------------------------------------

From eden.li at gmail.com  Sat Dec 16 07:11:51 2006
From: eden.li at gmail.com (Eden Li)
Date: Sat, 16 Dec 2006 20:11:51 +0800
Subject: [Backgroundrb-devel] Is there a way to make backgroundrb 0.2.1
	fail silently on win32?
In-Reply-To: <a39d6f080612151252p2ef450a5y1e21f245d39c86e4@mail.gmail.com>
References: <565dbdd40612140955w396cfc2bn7ec2f090467a82f0@mail.gmail.com>
	<a39d6f080612151252p2ef450a5y1e21f245d39c86e4@mail.gmail.com>
Message-ID: <565dbdd40612160411n52c672f5v754189cd66d80318@mail.gmail.com>

Interesting solution, and it would probably work if I wasn't running
into exceptions even before the Rails models get loaded fully.

As mentioned in my previous mail, I discovered Rails was trying to run
vendor/plugins/backgroundrb/init.rb which will load
vendor/plugins/backgroundrb/lib/middleman_rails_init.rb which always
throws on win32.  I also explain how I got around it in that mail.

On 12/16/06, Aleksandr Levchuk <alevchuk at gmail.com> wrote:
> If one has all the calls to BackgrounDRb isolated in a few Rails
> models. For example I have app/model/nucleus.rb - which does all the
> interactions with workers. One could solve such a  problem by using
> Rails mock objects. I would just have to place a mocking model into
> app/test/mocks/development/nucleus.rb, and it will use that model
> instead when the application is running in the development
> environment.
>
> For your case, the developers could run Rails with
> --environment=development on win32. On linux you could be run it with
> --environment=testing or --environment=production, so that
> BackgrounDRb is used.

From backgroundrb at gmail.com  Mon Dec 18 17:48:22 2006
From: backgroundrb at gmail.com (Mike Garey)
Date: Mon, 18 Dec 2006 17:48:22 -0500
Subject: [Backgroundrb-devel] Creating another database connection for large
	mysql import?
Message-ID: <f2ac04f0612181448h3ee2dbdcl8809f2b7ceec0a2@mail.gmail.com>

I'm using backgroundrb to periodically download a large file via ftp from a
remote location and then import it into the database.  To perform the
import, I was using the following:

ActiveRecord::Base.connection.execute(%{load data infile ...;})

although on a file with 2.5 million records, this can take 5 minutes, which
seems to tie up my rails application while this executes (even though it's
being run from a backgroundrb process).  I figured this was happening
because I'm using the same database connection that the rest of my rails app
is running on.. I then tried using the following:

Importer.establish_connection(:development)
Importer.connection.execute(%{load data infile ... ;})

where Importer is a dummy class I created just so I could try to establish a
database connection separate from the rest of my rails classes.  However,
this seems to produce the same behavior as when I use ActiveRecord::
Base.connection.execute..

Does anyone have any suggestions on how I might be able to resolve this?

Also, I read a message on the list that an upcoming version of backgroundrb
will have the ability to automatically reload a worker if running in
development mode and the worker file changes.. Just wondering how to enable
this, since I've been stopping/restarting the server to get the file
reloaded.

Thanks for any help, and also a big thanks to Ezra (and any other
contributors) for all the hard work that's been put into backgroundrb, it's
truly appreciated!

Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061218/8c2f1f3c/attachment.html 

From ezmobius at gmail.com  Mon Dec 18 19:06:41 2006
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Mon, 18 Dec 2006 16:06:41 -0800
Subject: [Backgroundrb-devel] Creating another database connection for
	large mysql import?
In-Reply-To: <f2ac04f0612181448h3ee2dbdcl8809f2b7ceec0a2@mail.gmail.com>
References: <f2ac04f0612181448h3ee2dbdcl8809f2b7ceec0a2@mail.gmail.com>
Message-ID: <940299F5-94F3-4E63-90FB-DAFA6C838FA8@brainspl.at>


On Dec 18, 2006, at 2:48 PM, Mike Garey wrote:

> I'm using backgroundrb to periodically download a large file via  
> ftp from a remote location and then import it into the database.   
> To perform the import, I was using the following:
>
> ActiveRecord::Base.connection.execute (%{load data infile ...;})
>
> although on a file with 2.5 million records, this can take 5  
> minutes, which seems to tie up my rails application while this  
> executes (even though it's being run from a backgroundrb process).   
> I figured this was happening because I'm using the same database  
> connection that the rest of my rails app is running on.. I then  
> tried using the following:
>
> Importer.establish_connection(:development)
> Importer.connection.execute(%{load data infile ... ;})
>
> where Importer is a dummy class I created just so I could try to  
> establish a database connection separate from the rest of my rails  
> classes.  However, this seems to produce the same behavior as when  
> I use ActiveRecord:: Base.connection.execute..
>
> Does anyone have any suggestions on how I might be able to resolve  
> this?
>
> Also, I read a message on the list that an upcoming version of  
> backgroundrb will have the ability to automatically reload a worker  
> if running in development mode and the worker file changes.. Just  
> wondering how to enable this, since I've been stopping/restarting  
> the server to get the file reloaded.
>
> Thanks for any help, and also a big thanks to Ezra (and any other  
> contributors) for all the hard work that's been put into  
> backgroundrb, it's truly appreciated!
>
> Mike


Hey Mike-

	Can you show me exactly how you are calling the worker method that  
does the 5 minute task? The do_work method is run asyncronously when  
the worker is first created but other methods you define in your  
worker will block rails if you call them directly.  In the 0.2.1  
release you can use work_thread to call your worker method in a  
thread so it spins off and works in the background while rails can  
continue on its way.

so
worker = MiddleMan.worker(session[:key)
worker.work_thread(:some_method)


Cheers-
-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From backgroundrb at gmail.com  Mon Dec 18 22:56:14 2006
From: backgroundrb at gmail.com (Mike Garey)
Date: Mon, 18 Dec 2006 22:56:14 -0500
Subject: [Backgroundrb-devel] Creating another database connection for
	large mysql import?
In-Reply-To: <940299F5-94F3-4E63-90FB-DAFA6C838FA8@brainspl.at>
References: <f2ac04f0612181448h3ee2dbdcl8809f2b7ceec0a2@mail.gmail.com>
	<940299F5-94F3-4E63-90FB-DAFA6C838FA8@brainspl.at>
Message-ID: <f2ac04f0612181956l1f204dc1ua7ef9ab76e9df6c@mail.gmail.com>

I'm calling my import method from do_work, so this should be run
asynchronously as well right?  I also tried your method, but I receive the
following error:

    MiddleMan.new_worker(:class => :download_worker, :args => file, :job_key
=> :download_worker)
    worker = MiddleMan.worker(:download_worker)
    worker.work_thread(:import_file)

NoMethodError (undefined method `[]' for :import_file:Symbol):
    (drbunix:///tmp/backgroundrbunix_localhost_2000)
(drbunix:///tmp/backgroundrb.99462/download_worker_download_worker_0_0.678415183889324)
/usr/local/www/data-dist/vhosts/mysite/krad/vendor/plugins/backgroundrb/server/lib/backgroundrb/worker.rb:44:in
`work_thread'
    .//app/controllers/products_controller.rb:291:in `import_data_file'

the line that causes the error is:

    worker.work_thread(:import_file)

Are you sure that it isn't the single database connection that may be
causing rails to stall?  Thanks,

Mike



On 12/18/06, Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:
>
>
> On Dec 18, 2006, at 2:48 PM, Mike Garey wrote:
>
> > I'm using backgroundrb to periodically download a large file via
> > ftp from a remote location and then import it into the database.
> > To perform the import, I was using the following:
> >
> > ActiveRecord::Base.connection.execute (%{load data infile ...;})
> >
> > although on a file with 2.5 million records, this can take 5
> > minutes, which seems to tie up my rails application while this
> > executes (even though it's being run from a backgroundrb process).
> > I figured this was happening because I'm using the same database
> > connection that the rest of my rails app is running on.. I then
> > tried using the following:
> >
> > Importer.establish_connection(:development)
> > Importer.connection.execute(%{load data infile ... ;})
> >
> > where Importer is a dummy class I created just so I could try to
> > establish a database connection separate from the rest of my rails
> > classes.  However, this seems to produce the same behavior as when
> > I use ActiveRecord:: Base.connection.execute..
> >
> > Does anyone have any suggestions on how I might be able to resolve
> > this?
> >
> > Also, I read a message on the list that an upcoming version of
> > backgroundrb will have the ability to automatically reload a worker
> > if running in development mode and the worker file changes.. Just
> > wondering how to enable this, since I've been stopping/restarting
> > the server to get the file reloaded.
> >
> > Thanks for any help, and also a big thanks to Ezra (and any other
> > contributors) for all the hard work that's been put into
> > backgroundrb, it's truly appreciated!
> >
> > Mike
>
>
> Hey Mike-
>
>         Can you show me exactly how you are calling the worker method that
> does the 5 minute task? The do_work method is run asyncronously when
> the worker is first created but other methods you define in your
> worker will block rails if you call them directly.  In the 0.2.1
> release you can use work_thread to call your worker method in a
> thread so it spins off and works in the background while rails can
> continue on its way.
>
> so
> worker = MiddleMan.worker(session[:key)
> worker.work_thread(:some_method)
>
>
> Cheers-
> -- Ezra Zygmuntowicz
> -- Lead Rails Evangelist
> -- ez at engineyard.com
> -- Engine Yard, Serious Rails Hosting
> -- (866) 518-YARD (9273)
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061218/d38fc055/attachment-0001.html 

From masonhale at gmail.com  Tue Dec 19 08:01:49 2006
From: masonhale at gmail.com (Mason Hale)
Date: Tue, 19 Dec 2006 07:01:49 -0600
Subject: [Backgroundrb-devel] Creating another database connection for
	large mysql import?
In-Reply-To: <f2ac04f0612181956l1f204dc1ua7ef9ab76e9df6c@mail.gmail.com>
References: <f2ac04f0612181448h3ee2dbdcl8809f2b7ceec0a2@mail.gmail.com>
	<940299F5-94F3-4E63-90FB-DAFA6C838FA8@brainspl.at>
	<f2ac04f0612181956l1f204dc1ua7ef9ab76e9df6c@mail.gmail.com>
Message-ID: <8bca3aa10612190501k2fcac3d8q778047a8acf11215@mail.gmail.com>

With rails, you should get a single db connection per process (by default).
That a database connection in one process could
tie up the database connection in another, separate process doesn't sound
possible.

It sounds more likely to me that the db loading you are doing is resulting
in locks in the database that are blocking other requests from doing their
work.
This would be happening in the db, and would not be related to the number of
connections to the database.

The locking behavior of MySQL varies based on the table type used, in
particular MyISAM tables used table-level locking (which
can easily bite you) while InnoDB tables use row-level locks.
see: http://dev.mysql.com/doc/refman/5.0/en/internal-locking.html

Another possibility is that the import you are doing is monopolizing the
DB's cpu/io capacity and/or is filling the bandwidth between your app server
and db server.

Mason

On 12/18/06, Mike Garey <backgroundrb at gmail.com> wrote:
>
> I'm calling my import method from do_work, so this should be run
> asynchronously as well right?  I also tried your method, but I receive the
> following error:
>
>     MiddleMan.new_worker(:class => :download_worker, :args => file,
> :job_key => :download_worker)
>     worker = MiddleMan.worker(:download_worker)
>     worker.work_thread(:import_file)
>
> NoMethodError (undefined method `[]' for :import_file:Symbol):
>     (drbunix:///tmp/backgroundrbunix_localhost_2000)
> (drbunix:///tmp/backgroundrb.99462/download_worker_download_worker_0_0.678415183889324)
> /usr/local/www/data-dist/vhosts/mysite/krad/vendor/plugins/backgroundrb/server/lib/backgroundrb/worker.rb:44:in
> `work_thread'
>     .//app/controllers/products_controller.rb:291:in `import_data_file'
>
> the line that causes the error is:
>
>     worker.work_thread(:import_file)
>
> Are you sure that it isn't the single database connection that may be
> causing rails to stall?  Thanks,
>
> Mike
>
>
>
> On 12/18/06, Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:
> >
> >
> > On Dec 18, 2006, at 2:48 PM, Mike Garey wrote:
> >
> > > I'm using backgroundrb to periodically download a large file via
> > > ftp from a remote location and then import it into the database.
> > > To perform the import, I was using the following:
> > >
> > > ActiveRecord::Base.connection.execute (%{load data infile ...;})
> > >
> > > although on a file with 2.5 million records, this can take 5
> > > minutes, which seems to tie up my rails application while this
> > > executes (even though it's being run from a backgroundrb process).
> > > I figured this was happening because I'm using the same database
> > > connection that the rest of my rails app is running on.. I then
> > > tried using the following:
> > >
> > > Importer.establish_connection(:development)
> > > Importer.connection.execute(%{load data infile ... ;})
> > >
> > > where Importer is a dummy class I created just so I could try to
> > > establish a database connection separate from the rest of my rails
> > > classes.  However, this seems to produce the same behavior as when
> > > I use ActiveRecord:: Base.connection.execute..
> > >
> > > Does anyone have any suggestions on how I might be able to resolve
> > > this?
> > >
> > > Also, I read a message on the list that an upcoming version of
> > > backgroundrb will have the ability to automatically reload a worker
> > > if running in development mode and the worker file changes.. Just
> > > wondering how to enable this, since I've been stopping/restarting
> > > the server to get the file reloaded.
> > >
> > > Thanks for any help, and also a big thanks to Ezra (and any other
> > > contributors) for all the hard work that's been put into
> > > backgroundrb, it's truly appreciated!
> > >
> > > Mike
> >
> >
> > Hey Mike-
> >
> >         Can you show me exactly how you are calling the worker method
> > that
> > does the 5 minute task? The do_work method is run asyncronously when
> > the worker is first created but other methods you define in your
> > worker will block rails if you call them directly.  In the 0.2.1
> > release you can use work_thread to call your worker method in a
> > thread so it spins off and works in the background while rails can
> > continue on its way.
> >
> > so
> > worker = MiddleMan.worker(session[:key)
> > worker.work_thread(:some_method)
> >
> >
> > Cheers-
> > -- Ezra Zygmuntowicz
> > -- Lead Rails Evangelist
> > -- ez at engineyard.com
> > -- Engine Yard, Serious Rails Hosting
> > -- (866) 518-YARD (9273)
> >
> >
> >
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061219/3ca864b9/attachment.html 

From george at initiatecommerce.com  Tue Dec 19 16:39:34 2006
From: george at initiatecommerce.com (george .)
Date: Tue, 19 Dec 2006 16:39:34 -0500
Subject: [Backgroundrb-devel] can Timeout::timeout(...) be used in a
	Worker ?
In-Reply-To: <3a2de0cd0612150744i6c592a96ta166a1d1587e672c@mail.gmail.com>
References: <C53D9620-604F-46F6-8D11-C09E7C32B282@initiatecommerce.com>
	<3a2de0cd0612150744i6c592a96ta166a1d1587e672c@mail.gmail.com>
Message-ID: <983E399C-0663-4F2F-B862-35A20299B0E7@initiatecommerce.com>

I have been investigating the problem some more since the first  
reply .  I have successfully been able to use sleep() and have  
Timeout work with no problems .  So i began to dig deeper into the  
functions i am executing in the Timeout block .  I have not found  
anything definitive yet but i want to share some more information in  
hopes that it might spark a new approach to troubleshooting the cause .

First, here is some more information about the block of code i am  
executing inside the Timeout .  I am working with FasterCSV and have  
a fairly large (~7mb) tab delimited file which is malformed and  
causes FasterCSV to not be so fast .  I  am attempting to wrap the  
parser.each method in a Timeout block for a reasonable amount of  
time , which should throw an exception if the parsing takes too  
long .  However, this seems to not work when i am running in the  
context of Backgroundrb .  I have verified this in both a Rails +  
Backgroundrb development environment and using "script/backgroundrb  
console" , both of which produce the same results (i.e. they have the  
same issue of not timing out) .

After some time i got the feeling it was not just Backgroundrb which  
might be the issue but rather the combination of Backgroundrb and  
FasterCSV .  Digging into the FasterCSV code i noticed it is using an  
underlying IO object and a few regular expressions to do the bulk of  
the processing work .  I am now wondering if the IO.gets method or  
the regular expressions are not allowing the Timeout thread a chance  
to throw the exception .
An interesting thing i did notice is if i add any   
BackgrounDRb::ServerLogger.logger.debug calls in the FasterCSV  
processing code the Timeout works as expected while in Backgroundrb ,  
but when i take the logger calls out Timeout does not throw the  
exception .

This is currently where i am .  If someone has an idea of something  
else i can try to narrow down the problem please do let me know .

thanks ,


george .

From jerrod at indierockmedia.com  Tue Dec 19 18:33:56 2006
From: jerrod at indierockmedia.com (Jerrod Blavos)
Date: Tue, 19 Dec 2006 18:33:56 -0500
Subject: [Backgroundrb-devel] BackgrounDRb after restarting server
Message-ID: <584FD90D-F7AC-4327-9D0C-B3BBFB480DA4@indierockmedia.com>


i get this:

undefined method `object' for nil:NilClass

when i try to get the status of a worker after backgroundrb has been  
restarted using a job key saved in the DB.  how do i get around this?  
i know the process is nonexistent, but shouldnt it return something  
more friendly if it doesnt exist?  am i forgetting something? Thanks!

-Jerrod





Jerrod Blavos
ph. 919.341.0143
fx.  482.492.5009
jerrod at indierockmedia.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061219/703c9598/attachment.html 

From jerrod at indierockmedia.com  Wed Dec 20 09:54:34 2006
From: jerrod at indierockmedia.com (Jerrod Blavos)
Date: Wed, 20 Dec 2006 09:54:34 -0500
Subject: [Backgroundrb-devel] backgrounDRb + mongrel_upload_progress
Message-ID: <95CA1311-60B6-4B28-A4A6-791B1D5B5F88@indierockmedia.com>

is it possible to use these together?  im asking because in the  
sample docs for mongrel_upload_progress, it has you start by hand a  
druby instance.

This doesnt work very well when moving to production.  as in, i cant  
seem to get that particular process to either stop or restart after a  
new deploy.  I posted to the mongrel-users list about this and have  
not gotten any sort of response.

any ideas, insight or help would be appreciated.

I have also seem the merb file upload examples, but since one of the  
reasons the upload using mongrel_upload_progress is nice is that  
rails does the thread locking which ties up the whole process does  
MERB avoid this problem? would merb be a good way to go? what i have  
is a situation where i know there will be a lot of file uploads, so  
being ready for this is important.

ideas?  tips?

Thanks!

-Jerrod

From carl.lerche at gmail.com  Tue Dec 26 15:49:32 2006
From: carl.lerche at gmail.com (Carl Lerche)
Date: Tue, 26 Dec 2006 12:49:32 -0800
Subject: [Backgroundrb-devel] Subversion authentication?
Message-ID: <ab90312e0612261249h1f0b082s2ccfd358cc72f68b@mail.gmail.com>

Hello,

I've been trying to get the latest copy of BackgrounDRB as per the
documentation here http://backgroundrb.rubyforge.org/. However,
whenever I try to access
http://svn.devjavu.com/backgroundrb/tags/release-0.2.1 I keep an
authentication prompt and if I leave it blank I am declined access.

Is there somewhere else to get the code or am I doing something wrong?

Thanks,
-carl

-- 
EPA Rating: 3000 Lines of Code / Gallon (of coffee)

From ezmobius at gmail.com  Tue Dec 26 17:06:54 2006
From: ezmobius at gmail.com (Ezra Zygmuntowicz)
Date: Tue, 26 Dec 2006 14:06:54 -0800
Subject: [Backgroundrb-devel] Subversion authentication?
In-Reply-To: <ab90312e0612261249h1f0b082s2ccfd358cc72f68b@mail.gmail.com>
References: <ab90312e0612261249h1f0b082s2ccfd358cc72f68b@mail.gmail.com>
Message-ID: <DF6C4C53-7EDB-43C9-B514-FB07A664D854@brainspl.at>

Hey Carl-

	There was some problem with devjavu. I emailed Jeff and he fixed it  
so you shoudl be able to grab it no problem now.

-Ezra

On Dec 26, 2006, at 12:49 PM, Carl Lerche wrote:

> Hello,
>
> I've been trying to get the latest copy of BackgrounDRB as per the
> documentation here http://backgroundrb.rubyforge.org/. However,
> whenever I try to access
> http://svn.devjavu.com/backgroundrb/tags/release-0.2.1 I keep an
> authentication prompt and if I leave it blank I am declined access.
>
> Is there somewhere else to get the code or am I doing something wrong?
>
> Thanks,
> -carl
>
> -- 
> EPA Rating: 3000 Lines of Code / Gallon (of coffee)
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>

-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)



From bryan at osesm.com  Wed Dec 27 12:43:14 2006
From: bryan at osesm.com (Bryan Liles)
Date: Wed, 27 Dec 2006 12:43:14 -0500
Subject: [Backgroundrb-devel] MiddleMan and RSpec partial mocks
Message-ID: <0ED1073B-A502-45A5-82CA-D5ED3AED10E9@osesm.com>

I'm trying to test with RSpec (0.7.5), and I have the following code:

	MiddleMan.should_receive(:new_worker)

Every time I run this code, I get this error:

	undefined method `new_worker' for class `DRb::DRbObject'

How do I get around this?



From ms at pccl.info  Wed Dec 27 04:54:58 2006
From: ms at pccl.info (Mark Somerville)
Date: Wed, 27 Dec 2006 09:54:58 +0000
Subject: [Backgroundrb-devel] Progress update for non-JavaScript users
Message-ID: <200612270954.59138.ms@pccl.info>

Hi there,

I'm running BackgrounDRb for importing some large files into our system. It's 
very nice and we're able to use a simple JavaScript call to periodically 
provide progress messages to the user and, ultimately, success or failure 
messages.

How are people relaying progress and/or results messages back to users who 
don't have JavaScript enabled?

Pretty much the only solution I can think of is to, on each request, check if 
the current user has an import going on and, if so, fetch the status and 
display it somewhere. Is this how most people are doing it? Or am I missing 
something obvious? Any thoughts appreciated.

Thanks a lot,

Mark

From mark at martianrover.com  Wed Dec 27 15:01:54 2006
From: mark at martianrover.com (P. Mark Anderson)
Date: Wed, 27 Dec 2006 16:01:54 -0400
Subject: [Backgroundrb-devel] Progress update for non-JavaScript users
In-Reply-To: <200612270954.59138.ms@pccl.info>
References: <200612270954.59138.ms@pccl.info>
Message-ID: <15D3200E-2D1F-4CEA-BB7A-7F6CA93E148F@martianrover.com>

This is tangential to the topic, but under what circumstances are we  
to bother with the few Javascript impaired?

More on topic, what happens when an iframe has a <meta http- 
equiv="refresh" content="5">?

pmark






On Dec 27, 2006, at 5:54 AM, Mark Somerville wrote:

> Hi there,
>
> I'm running BackgrounDRb for importing some large files into our  
> system. It's
> very nice and we're able to use a simple JavaScript call to  
> periodically
> provide progress messages to the user and, ultimately, success or  
> failure
> messages.
>
> How are people relaying progress and/or results messages back to  
> users who
> don't have JavaScript enabled?
>
> Pretty much the only solution I can think of is to, on each  
> request, check if
> the current user has an import going on and, if so, fetch the  
> status and
> display it somewhere. Is this how most people are doing it? Or am I  
> missing
> something obvious? Any thoughts appreciated.
>
> Thanks a lot,
>
> Mark
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel


From bryan at osesm.com  Wed Dec 27 17:58:51 2006
From: bryan at osesm.com (Bryan Liles)
Date: Wed, 27 Dec 2006 17:58:51 -0500
Subject: [Backgroundrb-devel] calling a MiddleMan from inside a MiddleMan
Message-ID: <878C5C3A-18CD-4D89-A96A-B1CE7D9D9EB9@osesm.com>

Is it possible to call a worker from inside a worker?  Right now, if  
I try, I get a recycled object error.


From eden.li at gmail.com  Wed Dec 27 19:04:48 2006
From: eden.li at gmail.com (Eden Li)
Date: Wed, 27 Dec 2006 16:04:48 -0800
Subject: [Backgroundrb-devel] Progress update for non-JavaScript users
In-Reply-To: <15D3200E-2D1F-4CEA-BB7A-7F6CA93E148F@martianrover.com>
References: <200612270954.59138.ms@pccl.info>
	<15D3200E-2D1F-4CEA-BB7A-7F6CA93E148F@martianrover.com>
Message-ID: <565dbdd40612271604v3438b1f7x1eeaf62cb9468f10@mail.gmail.com>

You'll probably never run into a Javascript-impaired browser, but
you'll find that a lot of your users (especially those who are
security-minded) will be running Javascript-DISabled browsers.  Note
the popularity of the Firefox plugin noscript:
http://www.noscript.net/whats

On 12/27/06, P. Mark Anderson <mark at martianrover.com> wrote:
> This is tangential to the topic, but under what circumstances are we
> to bother with the few Javascript impaired?
>
> More on topic, what happens when an iframe has a <meta http-
> equiv="refresh" content="5">?
>
> pmark
>
>
>
>
>
>
> On Dec 27, 2006, at 5:54 AM, Mark Somerville wrote:
>
> > Hi there,
> >
> > I'm running BackgrounDRb for importing some large files into our
> > system. It's
> > very nice and we're able to use a simple JavaScript call to
> > periodically
> > provide progress messages to the user and, ultimately, success or
> > failure
> > messages.
> >
> > How are people relaying progress and/or results messages back to
> > users who
> > don't have JavaScript enabled?
> >
> > Pretty much the only solution I can think of is to, on each
> > request, check if
> > the current user has an import going on and, if so, fetch the
> > status and
> > display it somewhere. Is this how most people are doing it? Or am I
> > missing
> > something obvious? Any thoughts appreciated.
> >
> > Thanks a lot,
> >
> > Mark
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>

From michael.dauria at gmail.com  Thu Dec 28 00:53:57 2006
From: michael.dauria at gmail.com (Michael D'Auria)
Date: Thu, 28 Dec 2006 00:53:57 -0500
Subject: [Backgroundrb-devel] calling a MiddleMan from inside a MiddleMan
In-Reply-To: <878C5C3A-18CD-4D89-A96A-B1CE7D9D9EB9@osesm.com>
References: <878C5C3A-18CD-4D89-A96A-B1CE7D9D9EB9@osesm.com>
Message-ID: <1907e2ca0612272153m159693d5mfa8cd7cd16ace70c@mail.gmail.com>

Can you explain what you would want to do this for?  Maybe we can come up
with a different way to solve your original problem.

.: Michael :.

On 12/27/06, Bryan Liles <bryan at osesm.com> wrote:
>
> Is it possible to call a worker from inside a worker?  Right now, if
> I try, I get a recycled object error.
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061228/6888687c/attachment.html 

From bryan at osesm.com  Thu Dec 28 07:45:13 2006
From: bryan at osesm.com (Bryan Liles)
Date: Thu, 28 Dec 2006 07:45:13 -0500
Subject: [Backgroundrb-devel] calling a MiddleMan from inside a MiddleMan
In-Reply-To: <1907e2ca0612272153m159693d5mfa8cd7cd16ace70c@mail.gmail.com>
References: <878C5C3A-18CD-4D89-A96A-B1CE7D9D9EB9@osesm.com>
	<1907e2ca0612272153m159693d5mfa8cd7cd16ace70c@mail.gmail.com>
Message-ID: <B6739281-B3B1-4112-8906-2CFB51FB91A6@osesm.com>

On Dec 28, 2006, at 12:53 AM, Michael D'Auria wrote:

> Can you explain what you would want to do this for?  Maybe we can  
> come up with a different way to solve your original problem.
>
> .: Michael :.
>
> On 12/27/06, Bryan Liles <bryan at osesm.com> wrote:
> Is it possible to call a worker from inside a worker?  Right now, if
> I try, I get a recycled object error.
>

I was trying to implement a Queueing system.  I've given up on making  
this work in backgroundrb for right now.  I'm going to use ActiveMQ  
and stomp instead.

I had a worker called RequestWorker, and it was determining what  
needed to be run and then calling ScheduleWorker.  I guess I was  
trying to implement a poor mans queue.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061228/eb7a7797/attachment.html 

From sweiss at stylesight.com  Sun Dec 31 00:49:05 2006
From: sweiss at stylesight.com (Stephen Weiss)
Date: Sun, 31 Dec 2006 00:49:05 -0500
Subject: [Backgroundrb-devel] :job_key acting weird?
Message-ID: <22503D04-6574-4E8B-BC7F-6FCFCE2C12D4@stylesight.com>

Hey,

Just reading through the digests and I found this.  I'm experiencing  
a very similar thing, also on Mac OS X Tiger (Server or normal  
Client, Server is production and 10.4 client is my development/test  
server - same results on both)

I found that the following class names would not start, no matter what:

ChatTesterWorker
ImageToWebProcessorWorker
CTWorker

using :class => :chat_test_worker, :image_to_web_processor_worker,  
or :c_t_worker - all of which produce the correct class names

However, these alternative class names, with the same exact code,  
would work fine:

TestWorker
WebProcessorWorker
DoThisWorker

These are two different classes (I tried ChatTesterWorker as CTWorker  
as well) with different code, both that have the same problem where  
they will fail with particular names.  They just simply don't run -  
they return a job key and then they never even show up in the process  
list.  No errors, nothing logged - even if the only thing in the code  
is a logger.info call.

As far as job keys, I found almost always an explicit job key will  
fail for me... the only one I've gotten to work successfully  
is :chat, although I've only tried a few... I think :notifier did not  
work for me, and :batcher did not work either.

I use ruby 1.8.5 (and I'm sure I'm not accidentally using the version  
that comes with Tiger, I've fully deleted it off the systems), rails  
1.1.6, backgroundrb 2.1.

Another thing that's still happening is that you can't seem to call  
another worker from inside a rails worker.  I have a chat  
notification worker that uses xmpp4r.  It works fine when called  
directly from rails via my User model.  Here is the code for the  
ChatNotificationWorker class:
---
require 'xmpp4r'
class ChatNotificationWorker < BackgrounDRb::Worker::Base
   #  Results worker completely does not work for me as described in  
the README,
   #  but using attr_reader methods still works just fine for getting  
progress and error messages
   attr_reader :error_msg
   def do_work(params)
   @error_msg  = ""
     begin
       @client = Jabber::Client.new(Jabber::JID.new 
('race at my_chat_server/Rails'))
       @myPassword = 'my_password'
       @client.connect
       @client.auth_nonsasl(@myPassword, false)

     rescue
         @error_msg  = $!.to_s
     end
   end
   def notify(params)
       to = params[:address]
       body = params[:message]
       subject = "Automated Message"
       m = Jabber::Message::new(to, body).set_type(:normal).set_id 
(Time.now.to_s).set_subject(subject)
       @client.send(m)
   end
end
ChatNotificationWorker.register
---

And the method from the User model class:
---
def notify_by_chat(message)
     begin
       if self.chat_address && self.chat_address.include?("@")
         MiddleMan.worker(:chat).notify(:address =>  
self.chat_address, :message => message)
       else
         logger.info("no chat address for #{self.name}")
       end
     rescue
       logger.info("The following error was thrown:")
       logger.info($!.to_s)
     end
end
---
There is a separate method to start the chat worker, since autostart  
doesn't seem to function right.  That part works fine.

If, from within rails, I call for instance User.find(7).notify_by_chat 
("message"), everything goes right and I get a chat message as expected.

If, however, from within a different RailsBase worker, I call the  
exact same code, I get nothing:
---
class DoThisWorker < BackgrounDRb::Worker::RailsBase

def do_work(args)
    User.find(7).notify_by_chat("message")
end

end
DoThisWorker.register
---

If I use script/backgroundrb run, I see:
---
/usr/local/lib/ruby/1.8/drb/drb.rb:612: [BUG] Segmentation fault
ruby 1.8.5 (2006-08-25) [powerpc-darwin8.7.2]

undefined method `wait' for nil:NilClass - (NoMethodError)
/web/trunk/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
middleman.rb:217:in `new_worker'
/web/trunk/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:36:in `dispatch'
/web/trunk/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:22:in `initialize'
/web/trunk/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:22:in `new'
/web/trunk/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
thread_pool.rb:22:in `dispatch'
/web/trunk/vendor/plugins/backgroundrb/server/lib/backgroundrb/ 
middleman.rb:199:in `new_worker'
/usr/local/lib/ruby/1.8/drb/drb.rb:1555:in `__send__'
/usr/local/lib/ruby/1.8/drb/drb.rb:1555:in `perform_without_block'
/usr/local/lib/ruby/1.8/drb/drb.rb:1515:in `perform'
/usr/local/lib/ruby/1.8/drb/drb.rb:1589:in `main_loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1585:in `loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1585:in `main_loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1581:in `start'
/usr/local/lib/ruby/1.8/drb/drb.rb:1581:in `main_loop'
/usr/local/lib/ruby/1.8/drb/drb.rb:1430:in `run'
/usr/local/lib/ruby/1.8/drb/drb.rb:1427:in `start'
/usr/local/lib/ruby/1.8/drb/drb.rb:1427:in `run'
/usr/local/lib/ruby/1.8/drb/drb.rb:1347:in `initialize'
/usr/local/lib/ruby/1.8/drb/drb.rb:1627:in `new'
/usr/local/lib/ruby/1.8/drb/drb.rb:1627:in `start_service'
/web/trunk/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb_server.rb:314:in `run'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
application.rb:186:in `call'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
application.rb:186:in `start_proc'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
application.rb:196:in `call'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
application.rb:196:in `start_proc'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
application.rb:226:in `start'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
controller.rb:72:in `run'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons.rb:179:in  
`run_proc'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
cmdline.rb:94:in `call'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/ 
cmdline.rb:94:in `catch_exceptions'
/usr/local/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons.rb:178:in  
`run_proc'
/web/trunk/vendor/plugins/backgroundrb/server/lib/ 
backgroundrb_server.rb:301:in `run'
script/backgroundrb:29
---

Which, again, looks surprisingly similar to the error from Ivo,  
although with the Segmentation Fault on top.  Very discouraging.

If I call it from rails using work_thread like so:

MiddleMan.worker(:chat).work_thread(:method => :notify, :args =>  
{:address => self.chat_address, :message => message})

It also fails with the same segmentation fault and nil.wait error.

This *did* work with version 1, however version 1 leaves all sorts of  
db connections laying around so I'd much rather stick with version  
2.x, which otherwise works great!  However, chat notifications are  
mainly useful for these background jobs, and not so much directly  
from rails, so I really would like to fix this.  Any thoughts as to a  
better way for a worker to send a chat message without having to  
start up its own chat session?  It would seem to make sense to use  
one separate worker to do all the chatting, since it cuts down on  
excess connections.

Any suggestions would be appreciated!  Thanks for all your hard work  
guys, happy new year!

--
Steve

> Hi all
>
> I'm experiencing something strange with a :job_key, it took me a very
> long time to find out a worker wasn't doing it's job because of the
> name of my job_key. The first one I tried was :make_kohier, and with
> that name (and lot's of other ones) I can't get the worker to work. I
> do get my key in return, but the job doesn't start (the worker only
> had a logger.info method in it, no error there). As I can't see any
> results of this logger, and I can't delete the worker, the worker
> doesn't exist.
> I couldn't find the fault until I started changing the name of the
> job_key. I can't see any rules in it, some examples :job_key
> => :sdlkfskdsld doesn't work, :job_key => :skj does. :udjzkzis
> and :ssdlfdo don't work, :ssdlfdk does work.
> Just now I discovered that these keys sometimes work, sometimes
> not... Anyway, I only get errors if I run script/backgroundrb run,
> otherwise, there is nothing informing me of some error. Here is what
> I get in backgroundrb run
>
> [system: Mac OS X Tiger]
>
> You have a nil object when you didn't expect it!
> The error occured while evaluating nil.wait - (NoMethodError)

> {snip}
>

> Am I missing something here?
> Thanks for reading
> Ivo

From bryan at osesm.com  Sun Dec 31 12:05:23 2006
From: bryan at osesm.com (Bryan Liles)
Date: Sun, 31 Dec 2006 12:05:23 -0500
Subject: [Backgroundrb-devel] backgroundrb 0.2.1 doesn't always load rails
	environment
Message-ID: <0FA5B494-EBCC-4484-B551-A5372AF40C03@osesm.com>

I found this stack trace in my logs.  My worker name is MiscWorker,  
and Qualifier is a Rails model.

  uninitialized constant MiscWorker::Qualifier: /Users/bryan/ 
Workspace/sandbox/scraper-trunk/config/../vendor/rails/activerecord/ 
lib/../../activesupport/lib/active_support/dependencies.rb:476:in  
`const_missing'
/Users/bryan/Workspace/sandbox/scraper-trunk/lib/workers/ 
envoyd_worker.rb:11:in `do_work'
/Users/bryan/Workspace/sandbox/scraper-trunk/vendor/plugins/ 
backgroundrb/server/lib/backgroundrb/worker.rb:55:in `work_thread'
/Users/bryan/Workspace/sandbox/scraper-trunk/vendor/plugins/ 
backgroundrb/server/lib/backgroundrb/worker.rb:69:in `work_thread'
/Users/bryan/Workspace/sandbox/scraper-trunk/vendor/plugins/ 
backgroundrb/server/lib/backgroundrb/worker.rb:67:in `work_thread'