[Celerity-users] memory leaks and serialize blockage

[jason franklin-stokes]

Hi Tom, 

I can't provide tests at the moment - my hypotheses is based on observation. 

1. I cannot serialize webclient in htmlunit when i have gone through celerity - but i can do it if i dont use celerity. 
2. when i return directly from enable_event_listener (i.e. i dont let code run) the number of loose javascript threads goes down from 60 to 13 (over 200 websites) and 
3. the time needed to reach memory limit is about 2 times longer.

one thing that looks suspicious to me is in the celerity listener.rb

it looks like instances of weblient are being given objects that contain webclient. This is a loop reference for the garbage collector which it should be able to handle..... unless webclient is being used in a different thread - which I think it is in the case of javascript background tasks - which would mean that it cannot get garbage collected as long as the thread is running -  and for some reason the thread wont exit on close all windows (my be this has something to do with the self referencing - I am just guessing here). 

Therefore we end up with a growing number of threads all holding their own webclient objects.

The guys at htmluniit say they have almost entirely sure they have no memory leakage - and that all reports of memory leakage in the past months have been due to users not the htmlunit code. Now this is pretty big leakage and i am sure that they would have discovered a problem as big as this by now. So my guess is that the problem is most probably in the celerity code. 

here is the block of code that i think is causing the problem.

def initialize(webclient)
      @webclient = webclient
      @procs = Hash.new { |h, k| h[k] = [] }
    end

def add_listener(type, &block)
      case type
      when :status
        @webclient.setStatusHandler(self)
      when :alert
        @webclient.setAlertHandler(self)
      when :attachment
        @webclient.setAttachmentHandler(self)
      when :web_window_event
        @webclient.addWebWindowListener(self)
      when :html_parser
        @webclient.setHTMLParserListener(self)
      when :incorrectness
        @webclient.setIncorrectnessListener(self)
      when :confirm
        @webclient.setConfirmHandler(self)
      when :prompt
        @webclient.setPromptHandler(self)
      else
        raise ArgumentError, "unknown listener type #{type.inspect}"
      end

      @procs[type] << block
    end










 




On Dec 10, 2009, at 12:08 AM, Tomasz Kalkosiński wrote:

> On Wednesday 09 of December 2009 23:58:14 jason franklin-stokes wrote:
>> 
>> the method enable_event_listener has 2 side effects. 
>> 
>> 1. it seems to hold on to objects which result in the javascript threads not being able to close. so over time memory useage grows and cannot be collected by the GC
>> 
>> 2. as soon as enable_event_listener has been called it becomes no longer possible to serilaize webclient. 
>> 
>> question. 
>> 
>> what exactly is this method for? is there another way of doing this so that it does hold objects in memory and does not get in the way of serialize?
>> 
>> would be great to get some feedback
> 
> Hello jason
> 
> I use Celerity with spawned 10 threads and I saw memory consuming is really high in here too.
> 
> Can you provide some tests for your assumptions about enable_event_listener? I don't know the real purpose of this method.
> 
> Greetings,
> Tomasz Kalkosiński
> 
>> 
>> thanks
>> 
>> jason.
>> 
>> _______________________________________________
>> Celerity-users mailing list
>> Celerity-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/celerity-users
>> 
> 
> _______________________________________________
> Celerity-users mailing list
> Celerity-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/celerity-users