<br><br><div class="gmail_quote">On Fri, Apr 11, 2008 at 11:30 PM, Ryan Leavengood <<a href="mailto:leavengood@gmail.com">leavengood@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="Ih2E3d">On Sat, Apr 12, 2008 at 1:57 AM, Raghu Srinivasan<br>
<<a href="mailto:raghu.srinivasan@gmail.com">raghu.srinivasan@gmail.com</a>> wrote:<br>
><br>
> On my site, a user enters an RSS feed to be processed and since this takes<br>
> about 5-10 secs, I pass the process off to a background job and meanwhile do<br>
> some Ajaxy spinners and tap-dancing until the job completes and then<br>
> redirect_to the appropriate page. This works great.<br>
<br>
</div>Tap-dancing, LOL. I love that.<br>
<div class="Ih2E3d"><br>
> Next, is there a way around this? Can I have 2 threads/processes/ports/etc<br>
> for Bdrb so that the batch job doesn't interfere with a live user's<br>
> experience. Or any other workaround for this? Right now if the web user<br>
> comes along when 50 jobs are left and each job takes 10 secs, then he has a<br>
> nearly 10 minute wait, which sucks.<br>
<br>
</div>In my opinion you should always separate scheduled long running<br>
processes from user spawned ones. What I would do in this case is<br>
extract the common RSS processing functionality into a class in your<br>
Rails lib directory, then create two different BackgrounDRb workers<br>
that make use of that class. One would be UserRSSWorker and the other<br>
could be ScheduledRSSWorker. The first should only be used for user<br>
requests (and in addition you should use thread_pool.defer to allow<br>
multiple requests at once, which also might solve your original<br>
problem with one worker) and the other can be set up on your schedule.<br>
Also I am not sure how to do it offhand, but you should try to set up<br>
the ScheduledRSSWorker so that BackgrounDRb instantiates it fresh to<br>
run and then kills it once it is done, since you don't need it sitting<br>
around all day doing nothing.<br>
</blockquote><div><br>The RSS processing is already extracted into a .rb in my lib directory. Right now I have just one worker (called parse_feeds). If I understand you correctly, here's what I need to do: duplicate parse_feeds such that I have two identical workers, once called parse_feeds_web and the other parse_feeds_batch. And in my Rails controller, kick off parse_feeds_web for a live user but use the parse_feeds_batch for the nightly job. This will ensure that parse_feeds_web need not wait for parse_feeds_batch processes to complete. Sounds great. I'll try that.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>
But as I said above in parenthesis in general if you want a worker to<br>
be able to handle many jobs at once, use thread_pool.defer. Be sure to<br>
read the documentation because since this is threaded code there are<br>
things you need to be careful about.<br>
<font color="#888888"></font></blockquote><div><br>I did not know about thread_pool.defer. Thanks. I'll be sure to read and understand it before I mess with threaded stuff. But it will be good in case several users all plug in their feeds at the same time. <br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><font color="#888888"><br>
Ryan<br>
</font></blockquote></div><br>Thanks Ryan!<br>