[Backgroundrb-devel] Best way to run hundreds of concurrent tasks?

[hemant kumar]

Can you paste the worker code? I am not sure, whats causing this.
Which OS btw?



On Sun, 2008-09-14 at 21:58 -0700, Robert Matei wrote:
> BDRB is the latest version from the SVN repository (supposed to mirror
> stable git releases)
> Packet 0.1.13
> using Facebooker to interact with Facebook. 
> 
> 
> On Sun, Sep 14, 2008 at 9:30 PM, hemant kumar <gethemant at gmail.com>
> wrote:
>         
>         On Sun, 2008-09-14 at 19:34 -0700, Robert Matei wrote:
>         > Hi all,
>         >
>         > I'm trying to figure out to run some asynchronous tasks in a
>         Facebook
>         > app. I've got things working, but BRB crashes after a little
>         while,
>         > and I'm not sure if my setup is ideal.
>         >
>         > Here's the scenario:
>         > - whenever a user visits my app, I need to fire off a bunch
>         of API
>         > calls.
>         > - these calls need to start right away, because the user
>         sees a
>         > 'loading' screen until they return.
>         > - the job is not calculation or memory-intensive on my end,
>         but it
>         > does take 15-20 seconds to get all the responses from
>         Facebook.
>         >
>         > I'm on a fairly small server, so it's important to keep
>         memory use low
>         > (and stability high), while still being able to handle
>         thousands of
>         > simultaneous visitors. Right now I'm using a single worker,
>         and each
>         > visitor generates a new thread in the thread pool. After a
>         little
>         > while tasks only get a few calls in before failing with a
>         "Connection
>         > reset by peer" error. Eventually my worker just disappears
>         > altogether.
>         >
>         > What's going on? What kind of setup should I be running?
>         
>         
>         What version of BDRB? What version of packet gem? Which OS?
>         
>         Are you using mechanize or something for interacting with
>         facebook? or
>         just normal RFacebook or something?
>         
>         
>         
>         
> 
>